postfix, auto postmap using inotify
postfix and dovecot 2.0.9 rejecting over quota user on smtp stage
Postfix Changing Outgoing IP By Time Interval Using TCP_TABLE And Perl
Someone asked me if i can make a perl scripts that can change the ip address based on time interval, say he want ip address 1.2.3.4 used within one hour, if done next ip address will be used within next one hour..and so on. when it came to highest number of ip address in array, they will be reset back to the start. first i suggest him to look at the articles i wrote. But then i decide to write Perl script which was made for the purposes mention above.
here we are..
Postfix section:
master.cf 127.0.0.1:2527 inet n n n - 0 spawn user=nobody argv=/etc/postfix/ip_by_time.pl ip1 unix - - n - - smtp -o syslog_name=postfix-ip1 -o smtp_helo_name=smtp1.example.com -o smtp_bind_address=1.2.3.1 ip2 unix - - n - - smtp -o syslog_name=postfix-ip2 -o smtp_helo_name=smtp2.example.com -o smtp_bind_address=1.2.3.2 ip3 unix - - n - - smtp -o syslog_name=postfix-ip3 -o smtp_helo_name=smtp3.example.com -o smtp_bind_address=1.2.3.3 ip4 unix - - n - - smtp -o syslog_name=postfix-ip4 -o smtp_helo_name=smtp4.example.com -o smtp_bind_address=1.2.3.4 .... ....
main.cf
transport_maps = tcp:[127.0.0.1]:2527 127.0.0.1:2527_time_limit = 3600s
Postfix, Rate Limiting Inbound Emails Using SenderScore And Memcache
I received email from someone fiew days ago, he directed me to an article about senderscore and and asked if I could make it usable. Actually, I’m not very familiar with how senderscore work. I’ve read the article and see the FAQ at https://senderscore.org/. I have found that senderscore can be queried with a format like this:
reversed.ip.address.score.senderscore.com
Ie, I want to know the score value of ip address 202.127.97.97, the format of the query would be like this:
$ dig a 97.97.127.202.score.senderscore.com +short 127.0.4.75
Look at the answers given by senderscore’s NS. last octet is the score of the ip address 202.127.97.97, which scored 75.
Excerpts from senderscore faq:
All scores are based on a scale of 0 to 100, where 0 is the worst, and 100 is the best possible score. A score represents that IP address’s rank as measured against other IP addresses, much like a percentile ranking.
Now back to the article, The authors make a perl module that can perform queries to senderscore ns, put a “reputation score” into memcache, at the same time, calculating how many times an ip address connected to our smtp.
Let’s begin, first of all download Policy::Memcache from this git repository
Create a working directory, and extract the tarball.
$ mkdir pol-mem && cd pol-mem $ tar --extract --file=petermblair-libemail-f73612c.tar.gz petermblair-libemail-f73612c/perl/senderscore/memcache/ $ mv petermblair-libemail-f73612c/perl/senderscore/memcache/* .
Postfix, Omar Kilani’s Memcache Patch Try-Out
I was rewrote Omar Kilani’s memcache patch couple of weeks ago. But that was not tested due to lack of time and unavailability of servers that can be used.
Now, i got chance to implement simple test. This is my configuration:
main.cf
smtpd_recipient_restrictions = ... ... check_recipient_access memcache:/etc/postfix/memcache.cf, ... ...
memcache.cf
servers = localhost:11211 key_format = %s
Entry on memcache
spam@example.com REJECT not allowed
Query using postmap
$ postmap -q "spam@example.com" memcache:/etc/postfix/memcache.cf postmap: dict_memcache_lookup: using key_format '%s' postmap: plmemcache_get: fetching key spam@example.com from memcache postmap: plmemcache_get: key spam@example.com =>; REJECT not allowed postmap: dict_memcache_lookup: spam@example.com returned REJECT not allowed REJECT not allowed
A little bit too verbose i guess, but it can be adjusted by modifying source code.
postfix, integrating memcache as a lookup table using tcp_table
I have not had time to do the test “postfix memcached patch” because there are no idle servers that can be used for the experiment. instead, I’ve made a tutorial how to integrate memcached as a “postfix lookup table” with the help of tcp_table and a simple perl script.
Indeed, tcp_table “table lookup protocol” is one of the most powerful tools as well as the regexp and pcre, in my opinion. although client-server connection is not protected and and the server is not authenticated.
yes, I did a lot of experiments using tcp_table and perl scripts. it made me realize that I can do almost everything I need and make postfix as my favorite MTA.
Things required:
- perl Cache::Memcached module
- memcached
- postfix with tcp_table support
OK, first we create a simple perl script that allows you to handle the protocols of tcp_table. let’s call it memc.pl
#!/usr/bin/perl use strict; use warnings; use Sys::Syslog qw(:DEFAULT setlogsock); use Cache::Memcached; # Configure the memcached server my $memd = new Cache::Memcached { 'servers' => [ '127.0.0.1:11211' ], }; # # Initalize and open syslog. # openlog('postfix/memcached','pid','mail'); sub qrymemc { return unless /^get\s+(.+)/i; my $kmemc = lc($1); chomp($kmemc); trim($kmemc); my $vmemc = $memd->get($kmemc); if (defined $vmemc) { return ($kmemc,$vmemc); } return; } sub trim{ $_[0]=~s/^\s+//; $_[0]=~s/\s+$//; return; } # # Autoflush standard output. # select STDOUT; $|++; while (<>) { chomp; if (/^get\s+(.+)/i) { my $data = lc($1); my @res = qrymemc($data); syslog("info","data: %s", $data); if (@res) { chomp(@res); print "200 $res[1]\n"; syslog("info","Found: key = %s, value = %s", $res[0], $res[1]); next; } } print "200 DUNNO\n"; }
Postfix, old memcache lookup table patch
Yesterday, I was idly fiddling with the old patch postfix “memcached lookup table” created by Omar Kilani . unfortunately, patches can only be used for old postfix distributions (2.1.x – Released 2005-04-01, 2.2.x – Released 2005-04-01).
I rewrote the patch (code was not modified) so it can be applied against last postfix-2.9-20110706 snapshot.
This patches required memcached and libmemcache .
I was successfully compiled it, but not test it yet whether it will work or not. so it’s not recommended for use on production servers.
$ wget ftp://ftp.porcupine.org/mirrors/postfix-release/experimental/postfix-2.9-20110706.tar.gz $ tar xzf postfix-2.9-20110706.tar.gz $ cd postfix-2.9-20110706
Postfix memcache patch can be download here:
[download#41]
Patch postfix source distribution
$ patch -p1 < ../postfix-2.9-20110706-memcache.patch patching file html/DATABASE_README.html patching file html/Makefile.in patching file html/MEMCACHE_README.html patching file html/memcache_table.5.html patching file man/Makefile.in patching file man/man5/memcache_table.5 patching file proto/DATABASE_README.html patching file proto/Makefile.in patching file proto/MEMCACHE_README.html patching file proto/memcache_table patching file README_FILES/AAAREADME patching file README_FILES/DATABASE_README patching file README_FILES/MEMCACHE_README patching file src/global/dict_memcache.c patching file src/global/dict_memcache.h patching file src/global/mail_dict.c patching file src/global/Makefile.in
Postfix, Dynamic OverQuota User Map Script Using Bash And Inotifywait
I recently experimented with a simple bash script, inotifywait and smtpd_recipient_restrictions (check_recipient_access) to map email users who have exceeded the quota.
Well, during testing, i’ve noticed when using hash/texthash lookup tables, it needed to be reloaded in order smtpd detect changes in table.so i’ve made quick test on mysql_tables it seem updating record on tables will immediately able to be queried
Mapping can be done as follows:
main.cf:
smtpd_recipient_restrictions = check_recipient_access mysql:/etc/postfix/mysql_quota_access.cf, ... ...
mysql_quota_access.cf
user = user password = password hosts = localhost dbname = postfixdb query = SELECT qaction FROM quota WHERE username='%s'
create mysql table called quota:
CREATE TABLE quota ( id INT NOT NULL AUTO_INCREMENT PRIMARY KEY, username VARCHAR(100), qaction VARCHAR(100) ) TYPE=innodb;
Here’s the idea, inotifywait will continuously monitor the maildir directory recursively, and updates “qaction” field on “quota” mysql table whenever new mail arrived or whenever there is email deleted from the maildir.
initial map, can be produced by retrieving user information from database.for example, username information in the database “postfixdb” with the table name “mailbox” and field “username”.
# for i in `mysql -u user -ppassword -D postfixdb -e 'SELECT username FROM mailbox' | grep -v username`;do mysql -u user -ppassword -D postfixdb -e "INSERT INTO quota (username, qaction) VALUES ('$i', 'DUNNO')";
With this script,value of qaction field on mysql quota table will change continuously as the user’s maildir contents that keeps changing.
Postfix, One Way Maildir Replication / Backup Using Inotify And Rsync
After I wrote about Maildir replication, using ChironFS and DRBD, this time I will write how to make maildir replication, using a very well known program utility called rsync. basically, rsync itself, does not do realtime replication process. rsync only perform the synchronization/copy process when needed or scheduled by using the crontab. like cp, rsync is used to copy files from one directory to another directory in one system, or to a directory on another system. and vice versa.
How do we make the process of replication/copy that is almost realtime by using rsync?
we will use the inotify-tools (inotifywait) to monitor changes to system files or directories, in this case is the postfix maildir. Inotify has been included in the mainline Linux kernel from release 2.6.13 (June 18, 2005), and could be compiled into 2.6.12 and possibly earlier releases by use of a patch.
What is inotify?
Inotify is a Linux kernel subsystem that acts to extend filesystems to notice changes to the filesystem, and report those changes to applications. It replaces an earlier facility, dnotify, which had similar goals.
OK, without further ado, let’s continue with the first step, install inotify-tools. on my centos machine, it can be done in the following way.
$ sudo yum -y install inotify-tools
Assume that we have two servers, first server contains a postfix + maildir. second servers is used to backup maildir from the first server. using inotifywait, any changes in the maildir on first server will trigger rsync to update the maildir on the backup server. However, first we will make rsync can do the login automatically to the backup server via ssh using Public Key Based Authentication.
On First server
[first_server] $ ssh-keygen -t dsa -f ~/.ssh/identity && cat ~/.ssh/identity.pub | ssh -l postfix second_server -p 12345 'sh -c "cat - >>~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"'