Posted ininotify postfix rsync

Postfix, One Way Maildir Replication / Backup Using Inotify And Rsync

After I wrote about Maildir replication, using ChironFS and DRBD, this time I will write how to make maildir replication, using a very well known program utility called rsync. basically, rsync itself, does not do realtime replication process. rsync only perform the synchronization/copy process when needed or scheduled by using the crontab. like cp, rsync is used to copy files from one directory to another directory in one system, or to a directory on another system. and vice versa.

How do we make the process of replication/copy that is almost realtime by using rsync?

we will use the inotify-tools (inotifywait) to monitor changes to system files or directories, in this case is the postfix maildir. Inotify has been included in the mainline Linux kernel from release 2.6.13 (June 18, 2005), and could be compiled into 2.6.12 and possibly earlier releases by use of a patch.

What is inotify?

Inotify is a Linux kernel subsystem that acts to extend filesystems to notice changes to the filesystem, and report those changes to applications. It replaces an earlier facility, dnotify, which had similar goals.

OK, without further ado, let’s continue with the first step, install inotify-tools. on my centos machine, it can be done in the following way.

$ sudo yum -y install inotify-tools

Assume that we have two servers, first server contains a postfix + maildir. second servers is used to backup maildir from the first server. using inotifywait, any changes in the maildir on first server will trigger rsync to update the maildir on the backup server. However, first we will make rsync can do the login automatically to the backup server via ssh using Public Key Based Authentication.

On First server

[first_server] $ ssh-keygen -t dsa -f ~/.ssh/identity && cat ~/.ssh/identity.pub | ssh -l postfix second_server -p 12345 'sh -c "cat - >>~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"'
Tags: , , , 2
Posted inpostfix

postfix-2.8.4-RC1 SRC RPM

postfix-2.8.4-RC1 HISTORY 20110420 Performance: a high load of DSN success notification requests could slow down the queue manager. Solution: make the trace client asynchronous, just like the bounce and defer clients. Problem reported by Eduardo M. Stelmaszczyk of terra.com.br. Files: global/abounce.[hc], *qmgr/qmgr_active.c (the qmgr_active.c files are identical). 20110418 Bugfix (introduced…
Tags: , No Comments
Posted inpostfix

postfix-2.9.20110615 SRC RPM

postfix-2.9.20110615 HISTORY 20110614 Linux kernel version 3 support. Linus Torvalds has reset the counters for reasons not related to changes in code.  Files: makedefs, util/sys_defs.h. 20110615 Workaround: some Spamhaus RHSBL rejects lookups with "No IP queries" even if the name has an alphanumerical prefix. We play safe, and skip both…
Tags: , No Comments
Posted indrdb postfix

Postfix, Realtime Maildir Replication Using Dual-primary DRBD with OCFS2

As I’ve been written on my previous post about Maildir Replication using ChironFS in postfix, I will explain step by step it can also be done by using drbd and ocfs2.

Compiling & Installing

note: if your Linux distribution is shipped with a kernel older than 2.6.33 you have to install a kernel module package and packages for the user land code. If your distribution contains a Linux-2.6.33 kernel or newer you only need to install the user land code.

In my case i have my linux distribution older than 2.6.33, so i will also compile drbd kernel module.
Download drbd source and create Binary RPMS packages

$ wget http://oss.linbit.com/drbd/8.3/drbd-8.3.10.tar.gz
$ tar xvzf drbd-8.3.10.tar.gz
$ cd drbd-8.3.10
$ ./configure --enable-spec --with-km
$ cp ../drbd*.tar.gz `rpm -E %_sourcedir`
$ rpmbuild -bb drbd.spec
$ rpmbuild -bb drbd-km.spec
$ sudo rpm -ivh /path/to/RPMS/drbd-*

I’ll be using loop files for this setup since I don’t have access to raw partitions.

but if you have raw block device available you can subtitute this part:

resource r0 { meta-disk internal; device /dev/drbd0; disk /dev/loop0;

Into:

resource r0 {
	meta-disk internal;
	device /dev/drbd0;
	disk /dev/sdxx;

sdxx can be sda1, sdb1 sdb2 or what ever your raw disk device called

# dd if=/dev/zero of=/drbd-postfix.img bs=1M count=5000
# losetup /dev/loop0 /drbd-postfix.img

Place this DRBD resource file in /etc/drbd.d/r0.res. Be sure to adjust the server names and IP addresses for your servers.

resource r0 {
	meta-disk internal;
	device /dev/drbd0;
	disk /dev/loop0;

	syncer { rate 1000M; }
        net {
                allow-two-primaries;
                after-sb-0pri discard-zero-changes;
                after-sb-1pri discard-secondary;
                after-sb-2pri disconnect;
        }
	startup { become-primary-on both; }

	on postfix1 { address 192.168.200.18:7789; }
	on postfix2 { address 192.168.200.114:7789; }
}
Tags: , , No Comments
Posted injavascript php

Create Your Own Visitor’s IP Address Badge Using PHP And JavaScript

A web badge is a small image or text used on websites to promote web standards, products used in the creation of a web page or product, to indicate a specific content license that is applied to the content or design of a website, to comply with an application’s terms of service, to encourage your visitors to check your social network status, or even to display visitor’s informations. such as ip address, user agent, hostname etc.

Today’s story, we’ll make a badge for visitor’s ip address using php and javascript. This badge, for an example:
[cfields]v_ip_badge[/cfields]

create a php script called ipaddr.php as follows

<?php
if ( isset($_SERVER["REMOTE_ADDR"]) )    {
	$ip=$_SERVER["REMOTE_ADDR"] . ' ';
} else if ( isset($_SERVER["HTTP_X_FORWARDED_FOR"]) )    {
	$ip=$_SERVER["HTTP_X_FORWARDED_FOR"] . ' ';
} else if ( isset($_SERVER["HTTP_CLIENT_IP"]) )    {
	$ip=$_SERVER["HTTP_CLIENT_IP"] . ' ';
}
$IpAddr = $ip;
?>

if (typeof(v_ip_BackColor)=="undefined")
  v_ip_BackColor = "white";
if (typeof(v_ip_ForeColor)=="undefined")
  v_ip_ForeColor= "black";
if (typeof(v_ip_FontPix)=="undefined")
  v_ip_FontPix = "16";
if (typeof(v_ip_DisplayFormat)=="undefined")
  v_ip_DisplayFormat = "You are visiting from:<br>IP Address: %%IP%%";
if (typeof(v_ip_DisplayOnPage)=="undefined" || v_ip_DisplayOnPage.toString().toLowerCase()!="no")
  v_ip_DisplayOnPage = "yes";

v_ip_HostIP = "<?php echo $IpAddr ?>";

if (v_ip_DisplayOnPage=="yes") {
  v_ip_DisplayFormat = v_ip_DisplayFormat.replace(/%%IP%%/g, v_ip_HostIP);
  document.write("<table border='0' cellspacing='0' cellpadding='5' style='background-color:" + v_ip_BackColor + "; color:" + v_ip_ForeColor + "; font-size:" + v_ip_FontPix + "px'><tr><td>" + v_ip_DisplayFormat + "</td></tr></table>");
}
Tags: , , 2
Posted inmaildir postfix

Postfix, Maildir Replication In Real Time Using ChironFS

In this tutorial I will describe how to set up postfix and maildir which can automatically replicate email when email is delivered. We will not touch the postfix configuration at all. we will manipulate maildir directory behavior using ChironFS, instead.

what is ChironFS?

This the Chiron Filesystem. It is a Fuse based filesystem. It’s main purpose is to guarantee filesystem availability using replication. But it isn’t a RAID implementation. RAID replicates DEVICES not FILESYSTEMS.

Things we need before installing chironfs

  • fuse
  • fuse-devel
  • fuse-libs
  • ChironFS source/binary from here

Compile ChironFS source

RPM:

$ rpmbuild --rebuild chironfs-1.0.0-1.fc8.src.rpm
$ rpm -ivh /path/to/chironfs-1.0.0-1.fc12.rpm
Tags: , , 1
Posted inmemcache

Memcached Replication On Server Clusters

When I was looking for ways to replicate the contents of memcached for high-availability performance, I found this memcached-repcached application. that has the ability to replicate the contents of one memcached to another.

Repcached key features

  • Multi master replication.
  • Asynchronous data repliacation.
  • Support all memcached command (set, add, delete, incr/decr, flush_all, cas)

People probably already know about memcached . It’s a robust, high performance key-value based memory object cache interface. but unfortunately, lack the ability to create redundancy and replication in memcached server clusters. although replication could be done at the application level. However, it all depends on each individual’s taste.

This is a quick and dirty experiment I have tried using memcached-repcached application on 2 servers.

Download memcached-repcached from repcached.lab.klab.org
Or you can download source rpm version from here
[download#33]

On the first server and second server extract memcached-1.2.8-repcached-2.2.tar.gz, compile with –enable-replication option when configure.

$ tar xvzf memcached-1.2.8-repcached-2.2.tar.gz
$ cd memcached-1.2.8-repcached-2.2
$ ./configure --enable-replication
$ make
Tags: 6
Posted inmemcache nginx

Monitoring And Debugging Memcached Server Using phpMemCachedAdmin

I have found a very cool program for monitoring and debugging Memcached server, it’s called  phpMemcachedAdmin. This program allows to see in real-time (top-like) or from the start of the server, stats for get, set, delete, increment, decrement, evictions, reclaimed, cas command, as well as server stats (network, items, server version) with googlecharts and server internal configuration You can go further to see each server slabs, occupation, memory wasted and items (key & value).

Another part can execute commands to any memcached server : get, set, delete, flush_all, as well as execute any commands (like stats) with telnet To extract these informations, phpMemCacheAdmin uses, as you wish, direct communication with server, PECL Memcache or PECL Memcached API.

Explanation above is an excerpt from the website. This is a simple guide how do I install phpMemCachedAdmin.

Download phpMemCachedAdmin tarball

# wget http://phpmemcacheadmin.googlecode.com/files/phpMemcachedAdmin-1.2.1-r233.tar.gz

Since we will install in /var/www/html/memcached/ directory, run this command:

# mkdir -p /var/www/html/memcached
# tar -xvzf phpMemcachedAdmin-1.2.1-r233.tar.gz -C /var/www/html/memcached/
# chmod 0777 /var/www/html/memcached/Config/Memcache.php

Apache Configuration

( i’m using apache as reverse proxy backend )

# cd /etc/httpd/conf.d
# vi memcached.conf
Tags: , , 9
Posted iniptables memcache nginx

Sharing PHP Session In web Clusters Using Memcache

If you have multiple load balanced webservers all serving the same site, sessions should be shared among thosememcache logo servers, and not reside on each server individually. Because we don’t know where user gets load-balanced to which backend server. A common way solving this problems are by using custom class that overrides default behavior and stores sessions in a MySQL database. All webservers in clusters connect to the same database. However, the main problem that must be taken into consideration when using a database is the bottleneck.

Example loadbalancing backend using nginx

http {
  upstream backend {
    server 192.168.1.1:8000 weight=3;
    server 192.168.1.2:8001;
    server 192.168.1.3:8002;
    server 192.168.1.4:8003;
  }

  server {
    listen 80;
    server_name www.example.com;
    location / {
      proxy_pass http://backend;
    }
  }
}
Tags: , , , No Comments
Posted inperl smtp

Measure Response Time Of SMTP Connections Using Perl

Normally, to check if my server smtp connection alive, I just do telnet to port 25 from my workstation. if the smtp banner displayed, it means that the connection to the smtp server is good. I have done this for years.  😆

$ telnet smtp.example.com 25
Trying xxx.xxx.xx.xxx...
Connected to xxx.xxx.xx.x.
Escape character is '^]'.
220 smtp.example.com ESMTP Postfix
ehlo host.example.com
250-smtp.example.com
250-PIPELINING
250-SIZE 52428800
250-ETRN
250-STARTTLS
250-AUTH LOGIN PLAIN
250-AUTH=LOGIN PLAIN
250-ENHANCEDSTATUSCODES
250-8BITMIME
250 DSN
quit
221 2.0.0 Bye
Connection closed by foreign host.

Although many tools that are deliberately created for this purpose, still, I prefer just using telnet to port 25. I have made a simple perl script, with the intention that the things I do for years manually can be done automatically.

Modules required:
Tags: , , No Comments