Nginx As imap4/pop3 Proxy Using Apache As Auth Server Backend

It’s been a long times since i wrote my last article, i’ve been bussy with real life things.
As usual, I’ll get right to the subject of how to configure nginx as POP3/IMAP proxy server.

Nginx IP                 = 192.168.1.1
Postfix User Database IP = 192.168.1.5 (postfix + courier server + apache backend)

nginx server configuration

mail {
    server_name mail-proxy.example.com;
    # apache external backend
    auth_http  192.168.1.5:8081/auth.php;
    proxy  on;
    proxy_pass_error_message on;

    imap_capabilities "IMAP4rev1" "UIDPLUS" "IDLE" "LITERAL +" "QUOTA";

    pop3_auth plain apop cram-md5;
    pop3_capabilities "LAST" "TOP" "USER" "PIPELINING" "UIDL";

    ssl_certificate /etc/nginx/ssl_keys/db.mail-proxy.crt;
    ssl_certificate_key /etc/nginx/ssl_keys/db.mail-proxy.key;
    ssl_session_timeout 5m;
    ssl_protocols SSLv2 SSLv3 TLSv1;
    ssl_ciphers ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP;
    ssl_prefer_server_ciphers on;

    server {
      listen      143;
      protocol    imap;
      starttls    on;
      auth_http_header X-Auth-Port 143;
      auth_http_header User-Agent "Nginx POP3/IMAP4 proxy";
    }

    server {
      protocol    pop3;
      listen      110;
      starttls    on;
      pop3_auth   plain;
      auth_http_header X-Auth-Port 110;
      auth_http_header User-Agent "Nginx POP3/IMAP4 proxy";
    }

    server {
      listen      993;
      ssl         on;
      protocol    imap;
      auth_http_header X-Auth-Port 993;
      auth_http_header User-Agent "Nginx POP3/IMAP4 proxy";
    }

    server {
      protocol    pop3;
      listen      995;
      ssl         on;
      pop3_auth   plain;
      auth_http_header X-Auth-Port 995;
      auth_http_header User-Agent "Nginx POP3/IMAP4 proxy";
    }
}

Monitoring And Debugging Memcached Server Using phpMemCachedAdmin

I have found a very cool program for monitoring and debugging Memcached server, it’s called  phpMemcachedAdmin. This program allows to see in real-time (top-like) or from the start of the server, stats for get, set, delete, increment, decrement, evictions, reclaimed, cas command, as well as server stats (network, items, server version) with googlecharts and server internal configuration You can go further to see each server slabs, occupation, memory wasted and items (key & value).

Another part can execute commands to any memcached server : get, set, delete, flush_all, as well as execute any commands (like stats) with telnet To extract these informations, phpMemCacheAdmin uses, as you wish, direct communication with server, PECL Memcache or PECL Memcached API.

Explanation above is an excerpt from the website. This is a simple guide how do I install phpMemCachedAdmin.

Download phpMemCachedAdmin tarball

# wget http://phpmemcacheadmin.googlecode.com/files/phpMemcachedAdmin-1.2.1-r233.tar.gz

Since we will install in /var/www/html/memcached/ directory, run this command:

# mkdir -p /var/www/html/memcached
# tar -xvzf phpMemcachedAdmin-1.2.1-r233.tar.gz -C /var/www/html/memcached/
# chmod 0777 /var/www/html/memcached/Config/Memcache.php

Apache Configuration

( i’m using apache as reverse proxy backend )

# cd /etc/httpd/conf.d
# vi memcached.conf

Sharing PHP Session In web Clusters Using Memcache

If you have multiple load balanced webservers all serving the same site, sessions should be shared among thosememcache logo servers, and not reside on each server individually. Because we don’t know where user gets load-balanced to which backend server. A common way solving this problems are by using custom class that overrides default behavior and stores sessions in a MySQL database. All webservers in clusters connect to the same database. However, the main problem that must be taken into consideration when using a database is the bottleneck.

Example loadbalancing backend using nginx

http {
  upstream backend {
    server 192.168.1.1:8000 weight=3;
    server 192.168.1.2:8001;
    server 192.168.1.3:8002;
    server 192.168.1.4:8003;
  }

  server {
    listen 80;
    server_name www.example.com;
    location / {
      proxy_pass http://backend;
    }
  }
}

Nginx, Simple Http Authentication Using ngx_http_auth_pam_module Module

I already have a list of users in a mysql database that I use for postfix smtp authentication/sasl.

I wonder, whether the list can be used for http authentication 😀 . Well, let’s find out.

First of all, of course we must compile nginx to support http_auth_pam_module module.
download ngx_http_auth_pam_module-1.2.tar.gz

When compiling from source build as usual adding the -add-module option:

./configure --add-module=$PATH_TO_MODULE

My pam_mysql for postfix smtp authentication /etc/pam.d/smtp

auth required pam_mysql.so user=user passwd=pass host=localhost db=db table=mailbox usercolumn=username passwdcolumn=password crypt=1 md5=1 sqlLog=0
account sufficient pam_mysql.so user=user passwd=pass host=localhost db=db table=mailbox usercolumn=username passwdcolumn=password crypt=1 md5=1 sqllog=0

Nginx, Strip All Newlines Using nginx-nonewlines Module

nginx logo
Nginx

Another Nginx module i’ve tested today, nginx-nonewlines , basically, this module same as Evan Miller’s nginx mod_strip module , Evan Miller’s mod_strip module removes unnecessary whitespace (spaces, tabs, and newlines) from HTML documents and automatically leaves comments as well as <pre></pre> tags untouched. while using using nginx-nonewlines we should define  <!–SC_OFF–> and  <!–SC_ON–> within the html code area that we’re intend to leaves them intact.

Here’s quick and dirty compiling nginx-nonewlines module

  • download the module
https://github.com/vedang/nginx-nonewlines/archives/master
  • add this line to ./configure step, compile and install
  # ./configure \
  ....
  ....
  --add-module=/path/to/vedang-nginx-nonewlines-ba02b59
  ....
  ....
  # make && make install

add folowwing directive to nginx’s config.

Nginx Blocking Spoofed Google Bot

nginx logo
Nginx

I’ve found something interesting in the nginx mailing list today http://forum.nginx.org/read.php?2,202715,202715#msg-202715 . someone asked whether it is possible to block fake user agent such as google bot. sometimes, a lot of bots flood our servers disguise as google bot or other legal bot. Most likely, to scrape our website’s contents. Original google bot always uses the ip address which is owned by Google Inc. Many website owners complained, bad bots only drain their bandwidth usage. 🙂

The first option to use “if” directive in nginx.

if ($http_user_agent ~* "Google Bot") {
	allow 66.x;
	allow 70.x;
	deny all;
}

However, “if” directive considered to be a bad practice when use for anything rather than “return” or “rewrite”. Here’s an example from Igor Sysoev:

Nginx And Simple Permalink

What is permalink?

permalink definition on a wiki website:

A permalink, or permanent link, is a URL that points to a specific blog or forum entry after it has passed from the front page to the archives. Because a permalink remains unchanged indefinitely, it is less susceptible to link rot. Most modern weblogging and content-syndication software systems support such links. Other types of websites use the term permanent links, but the term permalink is most common within the blogosphere. Permalink is a portmanteau word made from permanent link. Permalinks are often simply stated so as to be human-readable.

How’s the shape of permalink structure?

Generally, dynamic page’s url will look like this:

http://www.example.com/index.php?page=10

become:

http://www.example.com/what-is-permalink,10.html

Nginx Memcached Module, Caching Webiste Image Using Memcached

During my exploration of optimzing website using memcached, i’ve found this tutorial. It’s quite interesting using key value application (memcached) for serving website’s static content, Such as image, jss or js file. As we know, Reading/writing Hardrive is slower than accessing memory. although compared with SSD or 7,200 RPM hard drives. Even when an application is loaded, any files that are opened for use in that application are loaded into memory. As well as how web servers work.

First time following the tutorial, it isn’t work. image files not loaded into memcached, no warn, now error. after slightly modifying the script, image files can be stored into the memcached perfectly.

Here’s the php script after modification, i saved it as mem.php

<?php

function rscandir($base='', &$data=array()) {
        $array = array_diff(scandir($base), array('.', '..'));

        foreach($array as $value) {
                if (is_dir($base."/".$value)) {
                        $data = rscandir($base."/".$value,$data);

                }
                elseif (is_file($base."/".$value)) {
                $rest = substr($value, -4);
                        if ((!strcmp($rest,'.jpg')) || (!strcmp($rest,'.png')) || (!strcmp($rest,'.gif'))) {
                                $data[] = $base."/".$value;
                        }
                }

        }
return $data;

}

$mylist=rscandir("/path/to/public_html");

$srch = array('/path/to/public_html');
$newval = array('');

$memcache_obj = memcache_connect("127.0.0.1", 11211);

while (list($key, $val) = each($mylist)) {
        $url=str_replace($srch,$newval,$val);
        echo "$key => $val -> ".filesize($val)."\n";
        $value = file_get_contents($val);
        memcache_add($memcache_obj, $url, $value, false, 0);
}

?>

Nginx Memcached module And PHP

I’ve seen lots of tutorials on internet explaining how set nginx to use memcached module. But, i’ve rarely seen them explaining, even the easiest one on how to populate memcached with data generated by php. When i’m getting in touch with memcached for the first time, i thought default configuration like this will be working out of the box. without touching php script. 😀

server {

	listen 80;
	server_name  example.com www.example.com;

	# ...

	location ~ \.php$ {
		set $memcached_key "kutu:$request_uri";
		memcached_pass 127.0.0.1:11211;

		default_type       text/html;
		error_page 404 405 502 = @cache_miss;
	}

	location @cache_miss {
		fastcgi_pass 127.0.0.1:9000;
		fastcgi_index index.php;
		fastcgi_param SCRIPT_FILENAME /path/to/yoursite/htdocs$fastcgi_script_name;
		include /etc/nginx/fastcgi_params;
	}

	# ...
}

Well, i was totally confused :D. so, to prevent other people misinterpret on how nginx memcached module works, i wrote simple examples, and simple explainations on how to configure nginx and populate data generated from php.

Nginx And Unix User Directories

As mentioned in the nginx wiki webiste, Nginx does not natively support user dirs, but as of 0.7.42 it can be done by using regex captures. i've tested it using this simple configuration. [text highlight="6,7,8"] server { listen 192.168.200.18:80; server_name _; access_log /var/log/nginx/nginx-userdirs-access.log main; location ~ ^/~(.+?)(/.*)?$ { alias /home/$1/public_html$2; }…