apache tuneup

Apache Tune Up
I was able to do about 4K concurrent connections with this setup.
Using Centos 6.7 64bit Apache httpd-2.2.15-47.el6.centos.x86_64
a Virtual Machine 4x CPU’s and 4GBs of memory, I have 2 disks, one for /var and other for the rest.
Important to understand, this apache servers only one static image we use for the BI team.
It does recieve and log a huge cookie and sends a 43 byte 1×1.gif image.
Atention: It may not work right for dynamic web servers, as I use the mod_cache.
The impressive thing, it uses almos no CPU and memory is doing fine.
I did my benchmark using ab (apache benchmark) with 2 remote server in the sam LAN.
They key thing is to make a change and test it.
Some kernel tuning for the network
Just addo to /etc/sysctl.conf and run sysctl -p
net.core.netdev_max_backlog = 50000
net.ipv4.conf.all.accept_redirects=0
net.ipv4.conf.all.accept_source_route=0
net.ipv4.conf.all.send_redirects=0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.accept_source_route = 0
net.ipv4.conf.default.rp_filter = 1
net.ipv4.conf.default.rp_filter = 1
net.ipv4.icmp_echo_ignore_broadcasts=1
net.ipv4.ip_forward = 0
net.ipv4.ip_forward = 0
net.ipv4.ip_local_port_range = 1024 65535
net.ipv4.tcp_fin_timeout = 1
net.ipv4.tcp_fin_timeout = 1
net.ipv4.tcp_keepalive_intvl = 1
net.ipv4.tcp_keepalive_probes = 1
net.ipv4.tcp_keepalive_time = 3
net.ipv4.tcp_max_orphans = 262144
net.ipv4.tcp_max_syn_backlog = 30000
net.ipv4.tcp_max_tw_buckets = 200000
net.ipv4.tcp_orphan_retries = 1
net.ipv4.tcp_retries2 = 1
net.ipv4.tcp_rfc1337 = 1
net.ipv4.tcp_rmem = 40 65536 16777216
net.ipv4.tcp_synack_retries = 1
net.ipv4.tcp_syncookies = 0
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_syn_retries = 1
net.ipv4.tcp_tw_recycle = 0
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_wmem = 40 87380 8388608
vm.swappiness = 10
/etc/httpd/conf/httpd.conf
ServerTokens OS
ServerRoot “/etc/httpd”
PidFile run/httpd.pid
Timeout 3
KeepAlive On
MaxKeepAliveRequests 600
KeepAliveTimeout 3

StartServers 8
MinSpareServers 4
MaxSpareServers 24
ServerLimit 96
MaxClients 96
ListenBacklog 512
MaxRequestsPerChild 20000

Listen 10.16.83.12:80
ExtendedStatus On
BufferedLogs On
LoadModule include_module modules/mod_include.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule logio_module modules/mod_logio.so
LoadModule env_module modules/mod_env.so
LoadModule ext_filter_module modules/mod_ext_filter.so
LoadModule mime_magic_module modules/mod_mime_magic.so
LoadModule expires_module modules/mod_expires.so
LoadModule deflate_module modules/mod_deflate.so
LoadModule headers_module modules/mod_headers.so
LoadModule usertrack_module modules/mod_usertrack.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule mime_module modules/mod_mime.so
LoadModule status_module modules/mod_status.so
LoadModule info_module modules/mod_info.so
LoadModule vhost_alias_module modules/mod_vhost_alias.so
LoadModule negotiation_module modules/mod_negotiation.so
LoadModule dir_module modules/mod_dir.so
LoadModule actions_module modules/mod_actions.so
LoadModule speling_module modules/mod_speling.so
LoadModule userdir_module modules/mod_userdir.so
LoadModule alias_module modules/mod_alias.so
LoadModule substitute_module modules/mod_substitute.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule cache_module modules/mod_cache.so
LoadModule version_module modules/mod_version.so
CacheEnable mem /
Include conf.d/*.conf
User apache
Group apache
UseCanonicalName On
DocumentRoot “/var/www/html”
TypesConfig /etc/mime.types
DefaultType image/gif
HostnameLookups Off
ErrorLog logs/error_log
LogLevel warn
LogFormat “%h %l %u %t \”%r\” %>s %b \”%{Referer}i\” \”%{User-Agent}i\”” combined
LogFormat “%h %l %u %t \”%r\” %>s %b” common
LogFormat “%{Referer}i -> %U” referer
LogFormat “%{User-agent}i” agent
CustomLog logs/access_log combined
LogFormat “%{%Y-%m-%d %H:%M:%S}t | %{Host}i | %{X-Forwarded-For}i | \”%r\” | \”%{Referer}i\” | \”%{User-Agent}i\” | \”%{Cookie}i\” | %>s | %b” ivc
ServerSignature Off
AddLanguage en .en
AddLanguage pt .pt
AddLanguage pt-BR .pt-br
LanguagePriority pt-br en
ForceLanguagePriority Prefer Fallback
AddDefaultCharset UTF-8
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz
AddType application/x-x509-ca-cert .crt
AddType application/x-pkcs7-crl .crl
AddHandler type-map var
AddType text/html .shtml
AddOutputFilter INCLUDES .shtml
BrowserMatch “Mozilla/2” nokeepalive
BrowserMatch “MSIE 4\.0b2;” nokeepalive downgrade-1.0 force-response-1.0
BrowserMatch “RealPlayer 4\.0” force-response-1.0
BrowserMatch “Java/1\.0” force-response-1.0
BrowserMatch “JDK/1\.0” force-response-1.0
BrowserMatch “Microsoft Data Access Internet Publishing Provider” redirect-carefully
BrowserMatch “MS FrontPage” redirect-carefully
BrowserMatch “^WebDrive” redirect-carefully
BrowserMatch “^WebDAVFS/1.[0123]” redirect-carefully
BrowserMatch “^gnome-vfs/1.0” redirect-carefully
BrowserMatch “^XML Spy” redirect-carefully
BrowserMatch “^Dreamweaver-WebDAV-SCM1″ redirect-carefully
Commands I use to monitor:
Memory/CPU used by Apache proccess
watch ‘top -U apache -bn 1 ‘
Average Process memory use:
ps aux | grep ‘httpd’ | awk ‘{print $6/1024;}’ | awk ‘{avg += ($1 – avg) / NR;} END {print avg ” MB”;}’
Disk I/O:
iostat -x 1
# When stressed -doing about 21216.00 w/s (avgque aobut 15)
Connections/Time_Wait etc:
ss -s
Benchmark result (x2) as I ran 2 at the same time from different servers
Benchmarking 172.16.83.97 (be patient)
Server Software: Apache/2.2.15
Server Hostname: 172.16.83.97
Server Port: 80
Document Path: /asdasd/1×1.gif?utmwv=5.6.7&utms=1&utmn=1364744509&utmhn=bob.bo.com&utme=8(Origem-visita*Tipo-Conteudo*Wall-cooe*Pwall-tipovisitae*6!Paall-cookie*TipoCadastrado)9((direto%271*Capa*04*-*6!04*-)11(2*3!1*1*6!2*1)&utmcs=UTF-8&utmsr=1364×768&utmvp=1348×171&utmsc=24-bit&utmul=pt-br&utmje=1&utmfl=19.0%20r0&utmdt=Jornal%20O%20bo%20%7C%20Not%C3%ADcias%20Online&utmhid=1667722349&utmr=-&utmp=%2F
Document Length: 43 bytes
Concurrency Level: 80
Time taken for tests: 660.634 seconds
Complete requests: 1.500.000
Failed requests: 0
Write errors: 0
Total transferred: 442500000 bytes
HTML transferred: 64500000 bytes
Requests per second: 2270.55 [#/sec] (mean)
Time per request: 35.234 [ms] (mean)
Time per request: 0.440 [ms] (mean, across all concurrent requests)
Transfer rate: 654.11 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 6 81.2 0 7001
Processing: 8 29 23.3 28 3218
Waiting: 8 29 22.9 28 3217
Total: 8 35 84.9 29 7034
Percentage of the requests served within a certain time (ms)
50% 29
66% 31
75% 32
80% 33
90% 35
95% 37
98% 41
99% 70
100% 7034 (longest request)