Networking optimize

Tuning the Linux kernel for more aggressive network throughput


The Linux kernel and the distributions that package it typically provide very conservative defaults to certain network settings that affect networking parameters. These settings can be tuned via the /proc filesystem or using the sysctl program. The latter is often better, as it reads the contents of /etc/sysctl.conf, which allows you to keep settings across reboots.

The following is a snippet from /etc/sysctl.conf that may improve network performance:

net.ipv4.tcp_window_scaling = 1

net.ipv4.tcp_syncookies = 1

net.core.rmem_max = 16777216

net.core.wmem_max = 16777216

net.ipv4.tcp_rmem = 4096 87380 16777216

net.ipv4.tcp_wmem = 4096 65536 16777216

The above isn't to replace what may already exist in /etc/sysctl.conf, but rather to supplement it. The first command enables TCP window scaling, which allows clients to download data at a higher rate by enabling extra bits in TCP packets that are used to increase the window size.

The second command enables TCP SYN cookies, which is often enabled by default and is extremely effective in preventing conditions such as SYN floods that can drain the server of resources used to process incoming connections.

The last four options increase the TCP send and receive buffers, which allow an application to move its data out faster so as to serve other requests. This also improves the client's ability to send data to the server when it gets busy.

By adding these commands to the /etc/sysctl.conf file, you ensure they take effect on every reboot. To enable them immediately without a reboot, use:

# sysctl -p /etc/sysctl.conf

To see all of the currently configured sysctl options, use:

# sysctl -a

This will list all of the configuration keys and their current values. The sysctl.conf file allows you to configure and save new defaults; what you see from this output are the defaults defined in the kernel that are currently effective. To see the value of one particular item, use:

# sysctl -q net.ipv4.tcp_window_scaling

Likewise, to set the value of one item without configuring it in sysctl.conf -- and understanding that it won't be retained across reboots, use:

# sysctl -w net.ipv4.tcp_window_scaling=1

This can be useful for testing the effectiveness of certain settings without committing them to being defaults.

About

Vincent Danen works on the Red Hat Security Response Team and lives in Canada. He has been writing about and developing on Linux for over 10 years and is a veteran Mac user.

25 comments
alexandre
alexandre

put this as last line of sysctl.conf to update established connections to use new configs net.ipv4.route.flush = 1

obinna4god
obinna4god

I read the article, and it was interesting. Hence I am a newbie to linux but want master it and utilize the command. How do i get those commands and make it work as computer reboots. How to I issue the command and make it register or make it permanent to avoid the command wiped out after computer restart

Totohydra
Totohydra

They have an "array"-type of entry. When I try to enter them using... sysctl -w net.ipv4.tcp_rmem=4096 87380 16777216 ... the first value loads but for the others I get... net.ipv4.tcp_rmem = 4096 error: "87380" must be of the form name=value error: "16777216" must be of the form name=value

stomfi
stomfi

I tried it on Feisty Fawn. This is my error: error: "pv4.tcp_window_scaling" is an unknown key What do I do to fix this problem?

BALTHOR
BALTHOR

That's Linux,a bunch of typing that apparently only certain people can do.It's not click and run."Just because it doesn't work for you doesn't mean ---lots of people use Linux---".

DanLM
DanLM

Dog gone it, it didn't work in FreeBSD. sysctl -p is not reconized. [i]disone# sysctl -p /etc/sysctl.conf sysctl: illegal option -- p usage: sysctl [-bdehNnox] name[=value] ... sysctl [-bdehNnox] -a disone# [/i] And when I tried to just type 'sysctl ' and the paramater values. It said oid not reconized. [i] disone# sysctl net.ipv4.tcp_window_scaling=1 sysctl: unknown oid 'net.ipv4.tcp_window_scaling' [/i] Ok, if this wasn't a polically correct post... I think my comment would be. Well sh****************t, that sucks. lol, oops. Dan

apotheon
apotheon

As stated in the article, you would add these lines to the /etc/sysctl.conf file to make the changes work after your next reboot (and additional reboots afterward): [b]net.ipv4.tcp_window_scaling = 1 net.ipv4.tcp_syncookies = 1 net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.ipv4.tcp_rmem = 4096 87380 16777216 net.ipv4.tcp_wmem = 4096 65536 16777216[/b]

Totohydra
Totohydra

sysctl -w net.ipv4.tcp_rmem="4096 87380 16777216" ...will reply with... net.ipv4.tcp_rmem = 4096 87380 16777216 sysctl -w net.ipv4.tcp_rmem="4096 65536 16777216" ...will reply with... net.ipv4.tcp_rmem = 4096 65536 16777216

stomfi
stomfi

Yeah I didna cut the first letters. A newbie mistake I should of known better, but I lost my route, got it back, still not resolving, commented out the last four lines, tried again, it works, and proc shows the changed sysctl factors anyway, so I don't know what I'm doing again, not again! Where's the man page?

vdanen
vdanen

If you would have read the comment, he indicated that it didn't work in *FreeBSD*. Well, of course not silly... FreeBSD isn't Linux (so your comment, well, doesn't really make much sense... then again, neither did the original). The FreeBSD kernel is very different from the Linux kernel.

DanLM
DanLM

let me try that again. say what? dan

cstone
cstone

As noted in the article, sysctl settings are for Linux - and FreeBSD (and the other BSD's) are not Linux.

obinna4god
obinna4god

Find below the things that displayed when I vi into the linux box. As you saying i should include add the line in the file, just after the last option and then save it. Look when I added it, if it is correct then let me know login as: root Last login: Tue Jun 5 11:02:01 2007 from [root@timasbasenet root]# vi /etc/sysctl.conf # Kernel sysctl configuration file for Red Hat Linux # # For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and # sysctl.conf(5) for more details. # Controls IP packet forwarding net.ipv4.ip_forward = 1 # Controls source route verification net.ipv4.conf.default.rp_filter = 1 # Controls the System Request debugging functionality of the kernel kernel.sysrq = 0 # Controls whether core dumps will append the PID to the core filename. # Useful for debugging multi-threaded applications. kernel.core_uses_pid = 1 ~ ~ ~ ~ ~ ~ "/etc/sysctl.conf" 17L, 526C 17,1 All See the edited file, if it is correct let me know [root@timasbasenet root]# vi /etc/sysctl.conf # Kernel sysctl configuration file for Red Hat Linux # # For binary values, 0 is disabled, 1 is enabled. See sysctl(8) and # sysctl.conf(5) for more details. # Controls IP packet forwarding net.ipv4.ip_forward = 1 # Controls source route verification net.ipv4.conf.default.rp_filter = 1 # Controls the System Request debugging functionality of the kernel kernel.sysrq = 0 # Controls whether core dumps will append the PID to the core filename. # Useful for debugging multi-threaded applications. kernel.core_uses_pid = 1 net.ipv4.tcp_window_scaling = 1 net.ipv4.tcp_syncookies = 1 net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.ipv4.tcp_rmem = 4096 87380 16777216 net.ipv4.tcp_wmem = 4096 65536 16777216 ~ ~ ~ ~ ~ ~ "/etc/sysctl.conf" 17L, 526C 17,1 All

DanLM
DanLM

I did not see any reference to FreeBSD in the article, so what can I say. If I had, I wouldn't have tried it. In other words, I missed where he said it wouldn't work in FreeBSD. I also have special arguments in the sysctl.conf in my FreeBSD box for security purposes, so I was supprized again that you couldn't make it work. The command syntax, with the -p function doesnt bother me. The arguments being placed in the config did. Also, FreeBSD does load Linux binaries if you so wish. So, the comment that BSD isn't Linux isn't quite true now is it. Especially sence I do have them loaded. Dan

DanLM
DanLM

I am done appoligising for missing the point about freaken bsd in the article. BSD DOES load linux binaries if so desired. I DO load linux binaries in my bsd install. Both you and twit below are not necessarily correct. If I load Linux binaries, then there is definitely a correlation between BSD and Linux. Just in case you don't beleive me... Chew on this. http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/linuxemu.html Want to be a smart ass and bust my balls, then be ready to accept like criticism on your own statements. Dan

apotheon
apotheon

That looks fine. You might want to separate it from the other lines in the file with a comment that says something about those, perhaps like: # network performance optimization I got from a TR blog net.ipv4.tcp_window_scaling = 1 net.ipv4.tcp_syncookies = 1 net.core.rmem_max = 16777216 net.core.wmem_max = 16777216 net.ipv4.tcp_rmem = 4096 87380 16777216 net.ipv4.tcp_wmem = 4096 65536 16777216 . . . but that's not necessary. The way you pasted it in your post should work just fine. (edit: typo)

tafairch
tafairch

Yes, FreeBSD does load Linux binaries. But the comment that BSD isn't Linux is EXACTLY true. FreeBSD is based off of BSD UNIX - the version of UNIX developed at the University of California, Berkeley. Linux, was written by Linus Torvalds, originally as an alternative to Minix, and while not using the source code, is more like System V UNIX (by AT&T). The fact that you can load Linux Binaries on your FreeBSD machine has ABSOLUTELY NOTHING to do with trying to issue commands that affect the way a Linux kernel operates, on a system that isn't running a Linux kernel.

Kurse
Kurse

Loading Linux libraries in FreeBSD has nothing to with if its Linux or not. You can load Windows libraries also, but its still not Windows. FreeBSD is FreeBSD. It's compatible with Linux, in a vast many ways, Ill give you that, but your argument about it loading Linux libraries makes no sense.

apotheon
apotheon

"[i]While you could probably make sysctl run on FreeBSD, it would do a whole lot of nothing because the FreeBSD kernel is very very different from the Linux kernel and these commands are pretty specific to the Linux kernel.[/i]" The sysctl utility is alive and well on FreeBSD. It just has different options available than on Linux-based OSes.

bblackmoor
bblackmoor

This is useful info. Thanks. (I meant to put this at the bottom of the replies, not as a reply to your deleted comment. Ah, well.)

vdanen
vdanen

The FreeBSD kernel is vastly different from the Linux kernel. Yes, FreeBSD can run many Linux apps via emulation, but it doesn't run the Linux kernel (and why would you want to? If you wanted to run a Linux kernel, you wouldn't be running FreeBSD). While you could probably make sysctl run on FreeBSD, it would do a whole lot of nothing because the FreeBSD kernel is very very different from the Linux kernel and these commands are pretty specific to the Linux kernel.

cstone
cstone

Take a DEEP breath and relax a bit - don't see any 'ball busting' going on.... And, while you may be able to load some linux binaries, I doubt that changes per this article for network parameters would have any impact. Your linked article states: "In a nutshell, the compatibility allows FreeBSD users to run about 90% of all Linux applications without modification. This includes applications such as StarOffice?, the Linux version of Netscape?, Adobe? Acrobat?, RealPlayer?, VMware?, Oracle?, WordPerfect?, Doom, Quake, and more. There are, however, some Linux-specific operating system features that are not supported under FreeBSD." Mainly for running applications - not configuring system settings and parameters....