Monday, July 11, 2016

Enhance any web server performance & scalability on multi core systems

Hi all,
After reading a lot of contributions all over the Net for several years, I found and nicely experienced the following changes to be applied to 2 system files (/etc/security/limits.conf and /etc/sysctl.conf) to boost the performance and the scalability of any web server and/or web application server.
These data were published as of November 2007 by worldwide reknown Intel experts: Annie Foong and Bryan Veal, and in 2014 are still very valuable and must be applied to any multi core systems running any web server.
For the first file (/etc/security/limits.conf), please perform the following command:
gksudo gedit /etc/security/limits.conf
Next, at the end of the file, please add the following lines:
###############################################################################
# Modifications made to enhance the system scalability to its best.           #
# "Performance Scalability of a Multi-Core Web Server", Nov 2007              #
#  Bryan Veal and Annie Foong, Intel Corporation, Page 4/10"                  #
###############################################################################
soft nofile 1000000
hard nofile 1000000
###############################################################################
# End of modifications                                                        #
###############################################################################
Then save the file (and that's it for this first file).

For the second file (/etc/sysctl.conf), please perform the following command:
gksudo gedit /etc/sysctl.conf
Next, at the end of the file, please add the following lines:
###############################################################################
# Modifications made to enhance the system scalability to its best.           #
# "Performance Scalability of a Multi-Core Web Server", Nov 2007              #
#  Bryan Veal and Annie Foong, Intel Corporation, Page 4/10"                  #
###############################################################################
fs.file-max = 5000000
net.core.netdev_max_backlog = 400000
net.core.optmem_max = 10000000
net.core.rmem_default = 10000000
net.core.rmem_max = 10000000
net.core.somaxconn = 100000
net.core.wmem_default = 10000000
net.core.wmem_max = 10000000
net.ipv4.conf.all.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.tcp_congestion_control = bic
net.ipv4.tcp_ecn = 0
net.ipv4.tcp_max_syn_backlog = 12000
net.ipv4.tcp_max_tw_buckets = 2000000
net.ipv4.tcp_mem = 30000000 30000000 30000000
net.ipv4.tcp_rmem = 30000000 30000000 30000000
net.ipv4.tcp_sack = 1
net.ipv4.tcp_syncookies = 0
net.ipv4.tcp_timestamps = 1
net.ipv4.tcp_wmem = 30000000 30000000 30000000
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.ip_forward = 0
net.ipv4.tcp_dsack = 0
net.ipv4.tcp_fack = 0
net.ipv4.tcp_fin_timeout = 30
net.ipv4.tcp_orphan_retries = 0
net.ipv4.tcp_keepalive_time = 120
net.ipv4.tcp_keepalive_probes = 3
net.ipv4.tcp_keepalive_intvl = 10
net.ipv4.tcp_retries2 = 15
net.ipv4.tcp_retries1 = 3
net.ipv4.tcp_synack_retries = 5
net.ipv4.tcp_syn_retries = 5
net.ipv4.tcp_moderate_rcvbuf = 1
kernel.sysrq = 0
kernel.shmmax = 67108864
#
# Optionally, avoid TIME_WAIT states on localhost no-HTTP Keep-Alive tests:
# "error: connect failed: Cannot assign requested address (99)"
#
# On Linux, the 2MSL time is hardcoded to 60 seconds in /include/net/tcp.h:
# define Tcp_TIMEWAIT_LEN (60*HZ).
#
# The option below is safe to use on production systems.
#
net.ipv4.tcp_tw_reuse = 1
#
# WARNING:
# --------
# The option below lets you reduce TIME_WAITs by several orders of magnitude
# but this option is for benchmarks, NOT for production servers (NAT issues).
#
# Enable this option at your own risks if you need to benchmark your server
# using tools such as weighttp (from Lighttpd) or abc (from Trustleap).
#
# net.ipv4.tcp_recycle = 1
###############################################################################
# End of modifications                                                        #
###############################################################################
And save the file (and that's it for this second file).
Then... reboot your system and that's all.
You'll notice a real performance gain and whatever your web server may be (my prefered one by far is the freeware G-WAN from Trustleap), it will definitely scale better under huge loads.

https://community.linuxmint.com/tutorial/view/1714

http://stackoverflow.com/questions/410616/increasing-the-maximum-number-of-tcp-ip-connections-in-linux

http://stackoverflow.com/questions/26177059/refresh-net-core-somaxcomm-or-any-sysctl-property-for-docker-containers

1 comment:

  1. https://blog.wwwjfy.net/2013/08/04/basic-server-tuning

    ReplyDelete