Archive for January, 2009

Stop http Pipeline from overloading your server, using connlimit iptables

Posted by admin in Uncategorized on January 25, 2009

Have you edited or tweaked your pipeline settings in Firefox?

You can do it by typing about:config in your firefox url tab.

Most blogs and tutorial will tell you to set high values to improve the speed, such as this one: www.mydigitallife.info/2007/10/16/speed-up-your-firefox-by-adjusting-your-http-pipelining/

Many people will go crazy and make values even higher such as:

network.http.pipelining.maxrequests 32
network.http.max-persistent-connections-per-proxy 128
network.http.max-persistent-connections-per-server 128
network.http.max-connections-per-server 256

These settings are very high and will create at least 32 connections to your server.

if you have many images and SSI includes, it could overload your apache webserver.
If you use apache2.2 with worker_mpm it will create 1 thread for each connection, thus you will have 32 new threads forked within just a few seconds.

Of course if you have a quad-core server with lots of ram you should not bother to read this.
But for most cheap vps and single core servers, it can really help.

so how you do it? simple, just use iptables conn_limit module

iptables -I INPUT -p tcp --syn --dport 80 -m connlimit --connlimit-above  8 -j REJECT

you may need to adjust the order or to insert or append like

# to apped to the end of the INPUT chain:
iptables -A input …
or
# to insert at position 10 of the input chain:
iptables -I input 10 …

I have tested this schema and work very well with firefox pipeline freaks.

the server will only then take up to 8 simultaneously connections per IP

to test the established connections you can try with netstat from the server:

netstat -na | grep -i esta | grep -v 127.0.0.1 | sort -n -t. -k2

Any comments, suggestions are welcome …

Update:

This can be not so good to legit users behind a proxy or firewall, because the the IP will be unique to all users behind the proxy/firewall.

in this case, you would increase the limit value.

VN:F [1.8.3_1051]
Rating: 5.5/10 (2 votes cast)
VN:F [1.8.3_1051]
Rating: 0 (from 0 votes)

, ,

1 Comment

Linux script to collect system statistics and send to your email

Posted by admin in Uncategorized on January 8, 2009

This script I use to daily send me the system statistics on my gentoo server.
It will also reformat the output to replace tabs with 5 spaces so it will display nicely on your email client.
On mail.app the fonts Monaco and Inconsolata displays nicely, but the default font does not.

requirements:
- app-admin/sysstat
- net-mail/sendEmail
- app-admin/procinfo
- local postfix server able to deliver emails.
- perl

emerge -va app-admin/sysstat net-mail/sendEmail app-admin/procinfo

I put the script under /etc/cron.daily

 
#!/bin/bash
 
SERVER="mydomain.com"
EMAIL_TO="your_email@gmail.com"
EMAIL_FROM="amin@mydomain.com"
 
# logged in users and what are they running
WHO=`w`
 
# processor stats
MPSTAT=`mpstat`
 
# virtual memory stats
VMSTAT=`vmstat`
 
# Top 20 memory hog applications
PS_MEM=`ps -A -o pid,pcpu,pmem,start_time,state,time,comm | perl -e '($_ = join "",<>) =~ s/(\t)/     /g; print;' |sort -g -k 3 -r | head -20`
 
# Top 10 CPU usage applications
PS_CPU=`ps -A -o pid,pcpu,pmem,start_time,state,time,comm | perl -e '($_ = join "",<>) =~ s/(\t)/     /g; print;' | sort -g -k 2 -r | head -10`
 
#  memory usage in MB
FREE=`free -m`
 
PROCINFO=`procinfo`
 
# iptables status
IPTABLES=`iptables -nL`
 
# established connections
NETSTAT=`netstat -na |grep -i esta |grep -v 127.0.0.1 |sort -n -t. -k2`
 
# line divider
DL="=================================================================================="
 
FINAL="${DL} 
`date`
${DL}
${SERVER} 
${DL} 
${WHO} 
${DL}
${FREE}
${DL}
${MPSTAT}
${DL}
${VMSTAT} 
${DL}
${PROCINFO}
${DL} 
Top 10 CPU processes
${PS_CPU} 
${DL}
Top 20 Memory processes
${PS_MEM} 
${DL}
${IPTABLES}
${DL}
${NETSTAT}
${DL}
"
 
echo "${FINAL}" | 
  perl -e '($_ = join "",<>) =~ s/(\t)/     /g; print;' | 
  sendEmail -f "${EMAIL_FROM}" -u "${SERVER} comparator" -t ${EMAIL_TO}

Resources:

- http://www.cyberciti.biz/tips/how-do-i-find-out-linux-cpu-utilization.html
- http://pagesperso-orange.fr/sebastien.godard/documentation.html
- http://caspian.dotconf.net/menu/Software/SendEmail/

VN:F [1.8.3_1051]
Rating: 0.0/10 (0 votes cast)
VN:F [1.8.3_1051]
Rating: 0 (from 0 votes)

1 Comment