Methods I have seen so far:

1. Checking content-type from file name: this inefficient, a user can just rename a file and you are fooled, or the file can have a different file format and you will not get the expected result.

2. Using Rmagick to check if the file is an image. This is so slow and uses so much Ram. You can try to initialize an rmagick object from an image file, then rescue when the file is not an image.

3. Using mini_magick to check if a file. This method is faster than rmagick. Implemen ted same way as rmagick.

A Better method for OSX and Linux,  is to use the command line tool “file” included in most UNIX operating systems.

It is very fast and very accurate.

1
2
3
4
5
6
7
8
9
10
11
12
13
14

file = "/path/to/file.ext"
if RUBY_PLATFORM.match(/darwin|linux|unix|solaris|bsd/)
 content_type = `file --raw --brief "#{file}"`.chomp
 case
  when content_type.match(/image|png|jpg|jpeg|gif/)
   real_type = "image"
  when content_type.match(/pdf/)
   real_type = "pdf"
  when content_type.match("Microsoft Word|Microsoft Office Document")
   real_type = "doc"
  else # This can go on and on
   real_type = "Unknown"
  end
end

Some examples of content types:

.doc = Microsoft Word document data

.doc = Microsoft Office Document

.pdf = PDF document, version 1.4

.pdf = PDF document, version 1.3

.psd = Adobe Photoshop Image

.png = PNG image data, 3508 x 4961, 8-bit/color RGBA, non-interlaced

.gif = GIF image data, version 89a, 195 x 109

.jpg = JPEG image data, EXIF standard

etc…

I hope this can be useful to someone.

1. XCODE you can download xcode from http://developer.apple.com/tools/xcode/index.html

2. OSX 10.4, 10.5 or 10.6

Procedures:

1. Install Passenger

$ sudo gem install passenger

now check where is passenger installed:

$ passenger-config --root

in my case is: /opt/local/lib/ruby/gems/1.8/gems/passenger-2.2.5

2. Install nginx

If you have nginx from macports, deactivate it in case of conflicts.

You can activate anytime later

$ sudo port deactivate nginx

$ wget http://sysoev.ru/nginx/nginx-0.7.59.tar.gz

$ wget http://sysoev.ru/nginx/nginx-0.7.62.tar.gz

$ tar xpf nginx-0.7.62.tar.gz

$ cd nginx-0.7.62

I Recommend using nginx 0.6 series because I had a lot of “502 Bad Gateway” with 0.7 series.

$ wget http://sysoev.ru/nginx/nginx-0.6.37.tar.gz

$ tar xpf nginx-0.6.37.tar.gz

$ cd nginx-0.6.37

$ sudo ./configure --add-module=/opt/local/lib/ruby/gems/1.8/gems/passenger-2.2.5/ext/nginx/ \
  --with-http_ssl_module --user=nobody --group=nobody --with-http_gzip_static_module \
  --with-poll_module --prefix=/opt/local --with-pcre

Configuration summary
+ using system PCRE library
+ using system OpenSSL library
+ md5: using OpenSSL library
+ sha1 library is not used
+ using system zlib library

nginx path prefix: "/opt/local"
nginx binary file: "/opt/local/sbin/nginx"
nginx configuration prefix: "/opt/local/conf"
nginx configuration file: "/opt/local/conf/nginx.conf"
nginx pid file: "/opt/local/logs/nginx.pid"
nginx error log file: "/opt/local/logs/error.log"
nginx http access log file: "/opt/local/logs/access.log"
nginx http client request body temporary files: "client_body_temp"
nginx http proxy temporary files: "proxy_temp"
nginx http fastcgi temporary files: "fastcgi_temp"

$ sudo make

$ sudo make install

$ cd /opt/local/conf

$ sudo cp mime.types.default mime.types

$ sudo cp nginx.conf.default nginx.conf

Edit nginx.conf

$ mate nginx.conf

or

$ sudo vi nginx.conf

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95

 
user nobody;
worker_processes  2;
 
#error_log  logs/error.log;
#error_log  logs/error.log  notice;
error_log  logs/error.log  info;
 
# Pid
pid  logs/nginx.pid;
 
events {
worker_connections 1024;
}
 
http {
include       mime.types;
default_type  application/octet-stream;
 
sendfile        on;
#tcp_nopush     on;
 
#keepalive_timeout  0;
keepalive_timeout  65;
 
gzip  on;
# if a precompiled gzip of the file exists, use it and force http proxies
# to use separate cache's based on User-Agent
gzip_static on;
gzip_min_length 2000;
gzip_buffers    16 8k;
gzip_types      text/plain text/html text/css image/x-icon application/xml application/xml+rss text/javascript;
gzip_disable    "MSIE [1-6] \.";
gzip_vary         on;
gzip_comp_level   2;
 
gzip_proxied any;
 
server {
listen       80;
server_name  localhost;
location / {
root   /Users/fred/Sites ;
autoindex on;
index  index.html index.htm;
}
}
 
passenger_root /opt/local/lib/ruby/gems/1.8/gems/passenger-2.2.2;
passenger_max_pool_size 8;
passenger_max_instances_per_app 1;
# The maximum number of seconds that an application instance may be idle.
# That is, if an application instance hasn’t received any traffic after the given number of seconds,
# then it will be shutdown in order to conserve memory.
passenger_pool_idle_time 3600;
 
# Project 1
server {
listen 80;
client_max_body_size 250M;
server_name project1.local;
root /Users/fred/rails/project1/public;
passenger_enabled on;
rails_env development;
access_log  /Users/fred/rails/project1/log/nginx.access.log;
error_log  /Users/fred/rails/project1/log/nginx.error.log info;
}
 
# Project 2
server {
listen 80;
client_max_body_size 250M;
server_name project2.local;
root /Users/fred/rails/project2/public;
passenger_enabled on;
rails_env development;
access_log  /Users/fred/rails/project2/log/nginx.access.log;
error_log  /Users/fred/rails/project2/log/nginx.error.log info;
}
 
# Project 3
server {
listen 80;
client_max_body_size 250M;
server_name project3.local;
root /Users/fred/rails/project3/public;
passenger_enabled on;
rails_env development;
access_log  /Users/fred/rails/project3/log/nginx.access.log;
error_log  /Users/fred/rails/project3/log/nginx.error.log info;
}
 
# And so on... as many projects as you want
 
}

Now edit your /etc/hosts and add the hosts for your local project

$ mate /etc/hosts

127.0.0.1   project1.local

127.0.0.1   project2.local

127.0.0.1   project3.local

3. Start nginx

sudo nginx

4. go to your browser and open project1.local

5. Easy start/restart/stop

add this to your ~/.bash_profile file

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

 
function nginx_reload() {
FILE="/opt/local/logs/nginx.pid"
if [ -e $FILE ]; then
echo "Reloading NGINX..."
PID=`cat /opt/local/logs/nginx.pid`
sudo kill -HUP $PID
else
echo "Nginx pid file not found"
return 0
fi
}
 
function nginx_stop() {
FILE="/opt/local/logs/nginx.pid"
if [ -e $FILE ]; then
echo "Stopping NGINX..."
PID=`cat /opt/local/logs/nginx.pid`
sudo kill -INT $PID
else
echo "Nginx pid file not found"
return 0
fi
}
 
function nginx_restart() {
FILE="/opt/local/logs/nginx.pid"
if [ -e $FILE ]; then
echo "Stopping NGINX..."
PID=`cat /opt/local/logs/nginx.pid`
sudo kill -INT $PID
sleep 1
echo "Starting NGINX..."
sudo nginx
else
echo "Nginx pid file not found"
return 0
fi
}

Troubleshooting

1. Nginx is not running

- check the logs

- check if it is really not running:

$ ps aux | grep nginx

2. you see the nginx error “502 Bad Gateway”

- may there is a problem with the /var/folders/ permissions on OSX:

2009/06/13 16:14:33 [crit] 1106#0: *1 connect() to unix:/var/folders/xl/xlSRYvzFHH8Fcehc51ciyE+++TI/-Tmp-//passenger.1091/master/helper_server.sock failed (13: Permission denied) while connecting to upstream, client: 127.0.0.1, server: hassan.local, request: "GET / HTTP/1.1", upstream: "unix:/var/folders/xl/xlSRYvzFHH8Fcehc51ciyE+++TI/-Tmp-//passenger.1091/master/helper_server.sock:", host: "hassan.local"

to fix it I did this:

$ sudo find /var/folders/xl/ -name “master” -exec chmod 755 {} \;

$ sudo find /var/folders/xl/ -name “-Tmp-” -exec chmod 755 {} \;

everytime I reboot my mac I had to do that… I still dont know how to fix it…

anybody knows?

That is it for now.

You can do it by typing about:config in your firefox url tab.

Most blogs and tutorial will tell you to set high values to improve the speed, such as this one: www.mydigitallife.info/2007/10/16/speed-up-your-firefox-by-adjusting-your-http-pipelining/

Many people will go crazy and make values even higher such as:

network.http.pipelining.maxrequests 32
network.http.max-persistent-connections-per-proxy 128
network.http.max-persistent-connections-per-server 128
network.http.max-connections-per-server 256

These settings are very high and will create at least 32 connections to your server.

if you have many images and SSI includes, it could overload your apache webserver.
If you use apache2.2 with worker_mpm it will create 1 thread for each connection, thus you will have 32 new threads forked within just a few seconds.

Of course if you have a quad-core server with lots of ram you should not bother to read this.
But for most cheap vps and single core servers, it can really help.

so how you do it? simple, just use iptables conn_limit module

iptables -I INPUT -p tcp --syn --dport 80 -m connlimit --connlimit-above  8 -j REJECT

you may need to adjust the order or to insert or append like

# to apped to the end of the INPUT chain:
iptables -A input …
or
# to insert at position 10 of the input chain:
iptables -I input 10 …

I have tested this schema and work very well with firefox pipeline freaks.

the server will only then take up to 8 simultaneously connections per IP

to test the established connections you can try with netstat from the server:

netstat -na | grep -i esta | grep -v 127.0.0.1 | sort -n -t. -k2

Any comments, suggestions are welcome …

Update:

This can be not so good to legit users behind a proxy or firewall, because the the IP will be unique to all users behind the proxy/firewall.

in this case, you would increase the limit value.

requirements:
- app-admin/sysstat
- net-mail/sendEmail
- app-admin/procinfo
- local postfix server able to deliver emails.
- perl

emerge -va app-admin/sysstat net-mail/sendEmail app-admin/procinfo

I put the script under /etc/cron.daily

 
#!/bin/bash
 
SERVER="mydomain.com"
EMAIL_TO="your_email@gmail.com"
EMAIL_FROM="amin@mydomain.com"
 
# logged in users and what are they running
WHO=`w`
 
# processor stats
MPSTAT=`mpstat`
 
# virtual memory stats
VMSTAT=`vmstat`
 
# Top 20 memory hog applications
PS_MEM=`ps -A -o pid,pcpu,pmem,start_time,state,time,comm | perl -e '($_ = join "",<>) =~ s/(\t)/     /g; print;' |sort -g -k 3 -r | head -20`
 
# Top 10 CPU usage applications
PS_CPU=`ps -A -o pid,pcpu,pmem,start_time,state,time,comm | perl -e '($_ = join "",<>) =~ s/(\t)/     /g; print;' | sort -g -k 2 -r | head -10`
 
#  memory usage in MB
FREE=`free -m`
 
PROCINFO=`procinfo`
 
# iptables status
IPTABLES=`iptables -nL`
 
# established connections
NETSTAT=`netstat -na |grep -i esta |grep -v 127.0.0.1 |sort -n -t. -k2`
 
# line divider
DL="=================================================================================="
 
FINAL="${DL} 
`date`
${DL}
${SERVER} 
${DL} 
${WHO} 
${DL}
${FREE}
${DL}
${MPSTAT}
${DL}
${VMSTAT} 
${DL}
${PROCINFO}
${DL} 
Top 10 CPU processes
${PS_CPU} 
${DL}
Top 20 Memory processes
${PS_MEM} 
${DL}
${IPTABLES}
${DL}
${NETSTAT}
${DL}
"
 
echo "${FINAL}" | 
  perl -e '($_ = join "",<>) =~ s/(\t)/     /g; print;' | 
  sendEmail -f "${EMAIL_FROM}" -u "${SERVER} comparator" -t ${EMAIL_TO}

Resources:

- http://www.cyberciti.biz/tips/how-do-i-find-out-linux-cpu-utilization.html
- http://pagesperso-orange.fr/sebastien.godard/documentation.html
- http://caspian.dotconf.net/menu/Software/SendEmail/

To make Firefox faster is to store the whole profile folder into a Ram Disk.
RamDisk in Linux are called TmpFS. You can also use shared memory folder /dev/shm if you have it in your fstab.

This post in the gentoo forums explains how to do it in Gentoo linux.

I made a similar script to make it work in OSX Leopard.

The Script have 2 parts, Start.sh and Stop.sh
Here are the Scripts:

Start.sh

#!/bin/bash
# Run this script to enable the Ramdisk for Firefox profiles
VolumeName="Mozilla"
 
# Size in MB, make sure is not too low or not too high
SizeInMB=220
 
NumSectors=$((2*1024*SizeInMB))
 
DeviceName=`hdid -nomount ram://$NumSectors`
 
echo $DeviceName
 
diskutil eraseVolume HFS+ RAMDisk $DeviceName
 
# move the current profiles folder
mv Profiles Profiles_ &&
 
# make a symlink to the ramdisk
ln -s /Volumes/RAMDisk ./Profiles &&
 
# then copy it to the ramdisk
/bin/cp -r Profiles_/* Profiles

Stop.sh

#!/bin/bash
cd ~/Library/Cache/Firefox/
 
# clean the cache
rm -rf  Profiles/*/Cache/* &&
 
# will save your modifications back to the DISK
/usr/bin/rsync -av --delete ./Profiles/ ./Profiles_/ &&
 
# sometimes during unmount it will say disk is in use.
# make sure you close firefox before.
umount /Volumes/RAMDisk &&
rm -rf Profiles &&
mv Profiles_ Profiles

You can also use ‘tar’ instead of ‘rsync’. I just love rsync more.

* Warning: The ramdisk contents will be erased after you umount the ramdisk.

Have fun.

Gentoo Linux ROCKS

Update:

To speed up firefox even more  run these commands:

cd ~/Library/Caches/Firefox/Profiles
 
for i in */*.sqlite; do sqlite3 $i VACUUM;done;
 
cd ~/Library/Application\ Support/Firefox/Profiles
 
for i in */*.sqlite; do sqlite3 $i VACUUM;done;

But there is a trick, when creating a “new” clean user it will allow you to enable filevault for that new user.

So, here is how you do it.

But before you try, make sure you have enough free space, try to get rid of huge files and folders, backup your data. (in this case, we will copy your data, so you will have 2 copies for safety)

Also clean your applications cache, such as firefox cache, camino, opera,
~/Library/caches/com.apple.Safari/Webpage\ Previews/ , etc…: and close most of your applications.

$ cd ~/Library/Caches

$ find ./ -name “Cache” -exec rm -rf {} \;

$ rm -rf ~/Library/Caches/com.apple.Safari/Webpage\ Previews/Incoming/*

Steps

1. Create another admin user, for example “admin”, with administrator privileges

2. login as that user.

3. move your old user folder.

$ cd /Users/

$ sudo mv myusername myusername.bak

3. delete your old user from the “Accounts” preferences pane.

4. then create it again and check the option to use filevault

5. Logout from “admin” and login again as newuser.

6. Now copy the old data to your home folder. ( will take a very long time for that)
update: You must use rsync instead of cp, so that you also copy your VERY important hidden files. such as .ssh .gnupg .vimrc .gem .gitconfig etc..

$ sudo /usr/bin/rsync -av /Users/myusername.bak/ /Users/myusername/

# watch out if you have files that should not change the ownership, such as server backups.
$ sudo chown -R myusername ~/

7. Logout and login again, for your preferences to take effect

8. If everything looks fine, you might just delete the backup folder.

$ sudo rm -rf /Users/myusername.bak

if you have any confusions let me know in comments.

Safe sleep is the way OSX sleeps to RAM and as well create a sleepimage (which is the size of your RAM). In case you run out of battery, so you can still resume from the image if the battery is dead, (and you have plugged it in   Provides very fast wake up, uses the battery while sleeping.

Fast Sleep is just sleep to RAM, same as safe sleep but no image creation, and if your battery is dead, the mac will cold boot. Provides very fast wake up same as safe sleep, uses the battery while sleeping.

Hibernate is when it uses the sleepimage all the time. Slower sleep and slow wake up, but it does not use battery at all…

So…

I found in this website http://alt.cc/jk/2007/08/07/safe-sleep-addendum/ this nice script that handles when to FastSleep or when to Hibernate.

you can get safe sleep with the command “$ sudo pmset hibernate 3″

I have modified the script it little :

#!/bin/sh
MODE=`/usr/bin/pmset -g | grep hibernatemode | awk '{ print $2 }'`
LEFT=`/usr/bin/pmset -g batt | grep Internal | awk '{ print $2 }' | awk -F % '{ print $1 }'`
HIBERNATE=20
FASTSLEEP=50
echo "Running safesleep.sh => MODE: ${MODE} LEFT: ${LEFT}" >> /var/log/system.log
if [ $LEFT -lt $HIBERNATE ] && [ $MODE != 3 ] ; then
	{
		echo "Less than ${HIBERNATE}% remains" >> /var/log/system.log
		echo "Setting Hibernate (hibernate mode 1)" >> /var/log/system.log
		`/usr/bin/pmset -a hibernatemode 1`
		LS=`ls -al /private/var/vm/sleepimage`
		echo "The sleepimage should be created:" >> /var/log/system.log
		echo "${LS}" >> /var/log/system.log
	}
elif  [ $LEFT -gt $FASTSLEEP ] && [ $MODE != 0 ]; then
	{
		echo "Greater than ${FASTSLEEP}% remains" >> /var/log/system.log
		echo "Setting FastSleep (hibernate mode 0)" >> /var/log/system.log
		`/usr/bin/pmset -a hibernatemode 0`
		`rm -rf /var/vm/sleepimage`
	}
fi

save it as /Users/your_user_name/.crons/safesleep.sh

now make it permanent in a crontab to run every 10 minutes, but wait, anacron (osx default cron) only supports daily/weekly/monthly jobs! (unless you patch it)

I guess we’ll have to install fcron:

$ sudo port -d install fcron

$ sudo vi /opt/local/etc/fcrontab


@,runas(root) 10 sh /Users/your_user_name/.crons/safesleep.sh


$ sudo launchctl load -w /Library/LaunchDaemons/org.macports.fcron.plist

$ sudo launchctl start org.macports.fcron

that’s it.

enjoy

http://virtualbox.org

VirtualBox runs on Windows, Linux, Macintosh and OpenSolaris hosts and supports a large number of Guest Operating systems.

Im running windows XP on my macbook pro and it seems ok, thou OSX is really slow, when I run the virtual machine. (and yes, I install virtual box drivers and tools on the guest)

here is some VirtualBox Goodnes that explains baout the command line tool:

http://dkprojects.wordpress.com/2008/10/29/more-virtualbox-goodness-tackling-the-cli/

You would like to use haml, textile or HTML to edit your Article from the admin interface.

  create_table "articles", :force => true do |t|
    t.string   "title"
    t.text     "body"
    t.string   "formatting_type", :limit => 20, :default => "HTML"
  end

app/models/article.rb

It’s quite simple. All you have to do is to add this helper in your application_helper.rb

  def print_formated(type,text)
    case type
    when "HTML"
      text
    when "Plain Text"
      h text
    when "HAML"
      Haml::Engine.new(text).render
    when "Syntaxy"
      Syntaxi.line_number_method = 'none'
      Syntaxi.new(text).process
    when "Textile"
      RedCloth.new(text).to_html
    end
  end

In your views/articles/_form.haml add the select field.

  = label :article, :formatting_type
  = select(:article, :formatting_type, Article::FORMATTING_TYPES.collect {|p| p }, { :include_blank => false })

Then in the Show view (articles/show.haml)

 
.article
  .title
    %h1
      = h @article.title
  .body
    = print_formated(@article.formatting_type, @article.body)

that’s pretty much it.

Since you are Gentoo user, i don’t need to go to details. You know what you doing.

Update: Mod-rails now works with apache mpm-worker

1. Recompile Apache non-threaded

add this to /etc/portage/package.use

www-servers/apache -threads

and this to /etc/make.conf

APACHE2_MPMS="prefork"

2. Re emerge apache

# emerge -va apache

3. Passenger is in gentoo portage, but its in testing. Currently Version 2.0.1

# echo "www-apache/passenger" >> /etc/portage/package.keywords

4. Install Passenger

# emerge -va passenger

If it tries to install rails 2.2.2, rake, and lots of other gems that you already have installed trough rubygems, then run emerge with –nodeps option

# emerge -va --nodeps passenger

5. Edit /etc/conf.d/apache and add “-D PASSENGER” to apache options

for example mine looks like this:

APACHE2_OPTS="-D DEFAULT_VHOST -D INFO -D LANGUAGE -D PROXY -D PASSENGER"

That’s it.

Now just drop a similar vhost config file inside /etc/apache/vhosts.d/

This is a sample vhost file for a rails app.

<VirtualHost *:80>
  ServerName mydomain.com
  DocumentRoot /myapp/public
  Include /etc/apache2/vhosts.d/deflate.conf
  RailsBaseURI /
  # The maximum number of Ruby on Rails application instances that may be simultaneously active.
  # A larger number results in higher memory usage, but improved ability to handle concurrent HTTP clients.
  # normally 1 to 10. (1 for each 50mb ram)
  RailsMaxPoolSize 1
  # The maximum number of seconds that a Ruby on Rails application instance may be idle.
  # That is, if an application instance hasn't done anything after the given number of seconds,
  # then it will be shutdown in order to conserve memory. ( 1 hour)
  RailsPoolIdleTime 3600
  RailsEnv 'production'
  <Directory /myapp/public>
    Options FollowSymLinks
    AllowOverride None
    Order allow,deny
    Allow from all
    </Directory>
</VirtualHost>

My sample deflate.conf,
used to gzip the content

<Location />
	SetOutputFilter DEFLATE
	#
	# Netscape 4.x has some problems...
	BrowserMatch ^Mozilla/4 gzip-only-text/html
	#
	# Netscape 4.06-4.08 have some more problems
	BrowserMatch ^Mozilla/4\.0[678] no-gzip
	#
	# MSIE masquerades as Netscape, but it is fine
	BrowserMatch \bMSIE !no-gzip !gzip-only-text/html
	# NOTE: Due to a bug in mod_setenvif up to Apache 2.0.48
	# the above regex won't work. You can use the following
	# workaround to get the desired effect:
	BrowserMatch \bMSI[E] !no-gzip !gzip-only-text/html
	# Don't compress images
	SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary
	SetEnvIfNoCase Request_URI \.(?:exe|t?gz|zip|bz2|sit|rar)$ no-gzip dont-vary
	SetEnvIfNoCase Request_URI \.pdf$ no-gzip dont-vary
	# Make sure proxies don't deliver the wrong content
	Header append Vary User-Agent env=!dont-vary
</Location>

DeflateFilterNote Input instream
DeflateFilterNote Output outstream
DeflateFilterNote Ratio ratio
LogFormat '"%r" %{output_info}n/%{input_info}n (%{ratio_info}n%%)' deflate
CustomLog /var/log/apache2/deflate_log deflate

* Update on July 10, 2008.

- Now using gentoo portage to install it. it’s more smooth.

Note:

Personally I found that Thin + nginx uses less memory(Nginx 4MB + each thin server) than
apache + passenger, which uses quite more. (Apache: 50MB + each rails spawner)