Geo Location DNSBL Using Perl, Memcached And GeoIP

In my last article about the DNSBL and memcached, I wrote how to use memcached to store the data for the DNSBL. It led me to other new ideas to make the Geo Location DNSBL. In previous article I put data into memcached with the following format:

_prefix_ip 127.0.0.2-10
_prefix_ = _a_ or _txt_
ip = ip address to blacklist
Example:
_a_192.168.1.1	127.0.0.3
_txt_192.168.1.1	Blacklisted

Each ip address has a pair of _a_ and _txt_ record.
For This experiment records will be store in following format:

_prefix_country_code 127.0.0.2-10
_prefix_ = _a_ or _txt_
country_code = country code to blacklist
Example:
_a_ID	127.0.0.3
_txt_ID	Blacklisted

dnsbl-geo.pl Perl script

#!/usr/bin/perl
use Net::DNS::Nameserver;
use Cache::Memcached;
use Geo::IP;
use strict;
use warnings;

our $our_dnsbl = ".dnsbl.example.com";

# Configure the memcached server
my $memd = new Cache::Memcached {
            'servers' => [ '127.0.0.1:11211' ],
};

my $gi = Geo::IP->new(GEOIP_STANDARD);

#sub reverse_ipv4 {
#        my $ip = $_[0];
#        my ($a1, $a2, $a3, $a4) = split /\./, $ip;
#        my $reversed_ipv4 = join('.', $a4,$a3,$a2,$a1);
#        return $reversed_ipv4;
#}

sub reverse_ipv4 {
        my @ips = split /\./, $_[0];
        my @r;
        push @r, pop @ips while @ips;
        return join('.', @r);
}

sub strip_domain_part {
        my $strip_domain = $_[0];
        $strip_domain =~ s/$our_dnsbl//ig;
        return $strip_domain;
}

sub reply_handler {
        my ($qname, $qclass, $qtype, $peerhost,$query,$conn) = @_;
        my ($rcode, @ans, @auth, @add);
        my ($memc_a_val, $memc_txt_val);

        #print "Received query from $peerhost to ". $conn->{"sockhost"}. "\n";
        #$query->print;

        my $striped_domain = strip_domain_part($qname);
        my $reverse_striped_domain = reverse_ipv4($striped_domain);

        my $country_code = $gi->country_code_by_addr($reverse_striped_domain);

        if ($qtype eq "A" && $qname eq $striped_domain . $our_dnsbl) {
                my $vmemc_a_val = sprintf("_a_%s", lc($country_code));
                $memc_a_val = $memd->get($vmemc_a_val);
                if (defined($memc_a_val)) {
                        my ($ttl, $rdata) = (86400, $memc_a_val);
                        push @ans, Net::DNS::RR->new("$qname $ttl $qclass $qtype $rdata");
                        $rcode = "NOERROR";
                }
        } elsif ( $qname eq "dnsbl.example.com" ) {
                $rcode = "NOERROR";

        } else {
                $rcode = "NXDOMAIN";
        }

        if ($qtype eq "TXT" && $qname eq $striped_domain . $our_dnsbl) {
                my $vmemc_txt_val = sprintf("_txt_%s", lc($country_code));
                $memc_txt_val = $memd->get($vmemc_txt_val);
                if (defined($memc_txt_val)) {
                        my ($ttl, $rdata) = (86400, $memc_txt_val);
                        push @ans, Net::DNS::RR->new("$qname $ttl $qclass $qtype $rdata");
                        $rcode = "NOERROR";
                }
        } elsif ( $qname eq "dnsbl.example.com" ) {
                $rcode = "NOERROR";

        } else {
                $rcode = "NXDOMAIN";
        }

        # mark the answer as authoritive (by setting the 'aa' flag
        return ($rcode, \@ans, \@auth, \@add, { aa => 1 });
        $memd->disconnect_all();
}

my $ns = Net::DNS::Nameserver->new(
     LocalAddr    => "192.168.200.18",
     LocalPort    => 5353,
     ReplyHandler => \&reply_handler,
     Verbose      => 0,
) || die "couldn't create nameserver object\n";

$ns->main_loop;

DNSBL Using Perl And Memcached

DNSBL is a DNS based blackhole list, which can be used as countermeasure against unsolicited mail spam. One of the most efficient ways to block mail spam is to do it on SMTP conversation stage by denying incoming connects from spam sources, where the source machine is identified by its IP address which is checked against one or more DNSBLs on the fly.

I wrote a very simple perl script, which mimicked the way the original DNSBL works. However, I’ve created this script just respond to A and TXT records. But that’s the basic principle of how the DNSBL works.

Note: This is not recommended for real use. it’s volatile, the records will be vaporized upon server reboot. Think of this experiment just for learning purposes and fun only :mrgreen: .

DNSBL server perl scripts we called it dnsbl.pl (This script is lousy, fix it if necessary):

#!/usr/bin/perl
use Net::DNS::Nameserver;
use Net::CIDR 'addr2cidr';
use Cache::Memcached;
use strict;
use warnings;

our $our_dnsbl = ".dnsbl.example.com";

# Configure the memcached server
my $memd = new Cache::Memcached {
            'servers' => [ '127.0.0.1:11211' ],
};

sub reverse_ipv4 {
        my $ip = $_[0];
        my ($a1, $a2, $a3, $a4) = split /\./, $ip;
        my $reversed_ipv4 = join('.', $a4,$a3,$a2,$a1);
        return $reversed_ipv4;
}

#sub reverse_ipv4 {
#        my @ips = split /\./, $_[0];
#        my @r;
#        push @r, pop @ips while @ips;
#        return join('.', @r);
#}

sub strip_domain_part {
        my $strip_domain = $_[0];
        $strip_domain =~ s/$our_dnsbl//ig;
        return $strip_domain;
}

sub truncating_ipv4 {
        my $ip_addr = $_[0];
        my ($a1, $a2, $a3, $a4) = split /\./, $ip_addr;

        my $net_work_addr_ess = $ip_addr;
        my $net_work_addr = join('.', $a1,$a2,$a3);
        my $net_work = join('.', $a1,$a2);
        my $net = $a1;

        my @truncated_ipv4_lists = ($net_work_addr_ess,$net_work_addr,$net_work,$net);
        return @truncated_ipv4_lists;
}

sub test_cidr {
        my @cidr_list = Net::CIDR::addr2cidr($_[0]);
        return @cidr_list;
}

sub reply_handler {
        my ($qname, $qclass, $qtype, $peerhost,$query,$conn) = @_;
        my ($rcode, @ans, @auth, @add);
        my ($memc_a_val, $memc_txt_val, $memc_cidr_val, $memc_match_val);

        #print "Received query from $peerhost to ". $conn->{"sockhost"}. "\n";
        #$query->print;

        my $striped_domain = strip_domain_part($qname);
        my $reverse_striped_domain = reverse_ipv4($striped_domain);
        my @truncated_ipv4_lists = truncating_ipv4($reverse_striped_domain);
        my @cidr_lists = test_cidr($reverse_striped_domain);

        if ($qtype eq "A" && $qname eq $striped_domain . $our_dnsbl) {
                foreach my $truncated_ipv4_list (@truncated_ipv4_lists) {
                        $memc_a_val = $memd->get("_a_" . $truncated_ipv4_list);
                        last if(defined($memc_a_val));
                }

                foreach my $cidr_list (@cidr_lists) {
                        $memc_cidr_val = $memd->get("_a_" . $cidr_list);
                        last if(defined($memc_cidr_val));
                }

                for(;;) {
                        if (defined($memc_a_val)) {
                                $memc_match_val = $memc_a_val;
                                last;
                        }
                        if (defined($memc_cidr_val)) {
                                $memc_match_val = $memc_cidr_val;
                                last;
                        }
                }

                my ($ttl, $rdata) = (86400, $memc_match_val);
                push @ans, Net::DNS::RR->new("$qname $ttl $qclass $qtype $rdata");
                $rcode = "NOERROR";
        } elsif ( $qname eq "dnsbl.example.com" ) {
                $rcode = "NOERROR";

        } else {
                $rcode = "NXDOMAIN";
        }

        if ($qtype eq "TXT" && $qname eq $striped_domain . $our_dnsbl) {
                foreach my $truncated_ipv4_list (@truncated_ipv4_lists) {
                        $memc_txt_val = $memd->get("_txt_" . $truncated_ipv4_list);
                        last if(defined($memc_txt_val));
                }

                foreach my $cidr_list (@cidr_lists) {
                        $memc_cidr_val = $memd->get("_txt_" . $cidr_list);
                        last if(defined($memc_cidr_val));
                }

                for(;;) {
                        if (defined($memc_txt_val)) {
                                $memc_match_val = $memc_txt_val;
                                last;
                        }
                        if (defined($memc_cidr_val)) {
                                $memc_match_val = $memc_cidr_val;
                                last;
                        }
                }

                my ($ttl, $rdata) = (86400, $memc_match_val);
                push @ans, Net::DNS::RR->new("$qname $ttl $qclass $qtype $rdata");
                $rcode = "NOERROR";
        } elsif ( $qname eq "dnsbl.example.com" ) {
                $rcode = "NOERROR";

        } else {
                $rcode = "NXDOMAIN";
        }
        # mark the answer as authoritive (by setting the 'aa' flag
        return ($rcode, \@ans, \@auth, \@add, { aa => 1 });
        $memd->disconnect_all();
}

my $ns = Net::DNS::Nameserver->new(
     LocalAddr    => "192.168.200.18",
     LocalPort    => 5353,
     ReplyHandler => \&reply_handler,
     Verbose      => 0,
) || die "couldn't create nameserver object\n";

$ns->main_loop;

postfix, integrating memcache as a lookup table using tcp_table

I have not had time to do the test “postfix memcached patch” because there are no idle servers that can be used for the experiment. instead, I’ve made a tutorial how to integrate memcached as a “postfix lookup table” with the help of tcp_table and a simple perl script.

Indeed, tcp_table “table lookup protocol” is one of the most powerful tools as well as the regexp and pcre, in my opinion. although client-server connection is not protected and and the server is not authenticated.

yes, I did a lot of experiments using tcp_table and perl scripts. it made me realize that I can do almost everything I need and make postfix as my favorite MTA.

Things required:

OK, first we create a simple perl script that allows you to handle the protocols of tcp_table. let’s call it memc.pl

#!/usr/bin/perl
use strict;
use warnings;
use Sys::Syslog qw(:DEFAULT setlogsock);
use Cache::Memcached;

# Configure the memcached server
my $memd = new Cache::Memcached {
            'servers' => [ '127.0.0.1:11211' ],
};

#
# Initalize and open syslog.
#
openlog('postfix/memcached','pid','mail');

sub qrymemc {
        return unless /^get\s+(.+)/i;
        my $kmemc = lc($1);
        chomp($kmemc);
        trim($kmemc);
        my $vmemc = $memd->get($kmemc);
        if (defined $vmemc) {
                return ($kmemc,$vmemc);
        }
        return;
}

sub trim{
        $_[0]=~s/^\s+//;
        $_[0]=~s/\s+$//;
        return;
}

#
# Autoflush standard output.
#
select STDOUT; $|++;

while (<>) {
        chomp;
        if (/^get\s+(.+)/i) {
                my $data = lc($1);
                my @res = qrymemc($data);
                syslog("info","data: %s", $data);
                if (@res) {
                        chomp(@res);
                        print "200 $res[1]\n";
                        syslog("info","Found: key = %s, value = %s", $res[0], $res[1]);
                        next;
                }
        }
        print "200 DUNNO\n";
}

Postfix, old memcache lookup table patch

Yesterday, I was idly fiddling with the old patch postfix “memcached lookup table” created by Omar Kilani . unfortunately, patches can only be used for old postfix distributions (2.1.x – Released 2005-04-01, 2.2.x – Released 2005-04-01).

I rewrote the patch (code was not modified) so it can be applied against last postfix-2.9-20110706 snapshot.
This patches required memcached and libmemcache .

I was successfully compiled it, but not test it yet whether it will work or not. so it’s not recommended for use on production servers.

$ wget ftp://ftp.porcupine.org/mirrors/postfix-release/experimental/postfix-2.9-20110706.tar.gz
$ tar xzf postfix-2.9-20110706.tar.gz
$ cd postfix-2.9-20110706

Postfix memcache patch can be download here:
[download#41]

Patch postfix source distribution

$ patch -p1 < ../postfix-2.9-20110706-memcache.patch
patching file html/DATABASE_README.html
patching file html/Makefile.in
patching file html/MEMCACHE_README.html
patching file html/memcache_table.5.html
patching file man/Makefile.in
patching file man/man5/memcache_table.5
patching file proto/DATABASE_README.html
patching file proto/Makefile.in
patching file proto/MEMCACHE_README.html
patching file proto/memcache_table
patching file README_FILES/AAAREADME
patching file README_FILES/DATABASE_README
patching file README_FILES/MEMCACHE_README
patching file src/global/dict_memcache.c
patching file src/global/dict_memcache.h
patching file src/global/mail_dict.c
patching file src/global/Makefile.in

Postfix, Dynamic OverQuota User Map Script Using Bash And Inotifywait

I recently experimented with a simple bash script, inotifywait and  smtpd_recipient_restrictions (check_recipient_access) to map email users who have exceeded the quota.

Well, during testing, i’ve noticed when using hash/texthash lookup tables, it needed to be reloaded in order smtpd detect changes in table.so i’ve made quick test on mysql_tables it seem updating record on tables will immediately able to be queried

Mapping can be done as follows:
main.cf:

smtpd_recipient_restrictions =
    check_recipient_access mysql:/etc/postfix/mysql_quota_access.cf,
	...
	...

mysql_quota_access.cf

user = user
password = password
hosts = localhost
dbname = postfixdb
query = SELECT qaction FROM quota WHERE username='%s'

create mysql table called quota:

 CREATE TABLE quota (
 id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
 username VARCHAR(100),
 qaction VARCHAR(100)
 ) TYPE=innodb;

Here’s the idea, inotifywait will continuously monitor the maildir directory recursively, and updates “qaction” field on “quota” mysql table whenever new mail arrived or whenever there is email deleted from the maildir.

initial map, can be produced by retrieving user information from database.for example, username information in the database “postfixdb” with the table name “mailbox” and field “username”.

# for i in `mysql -u user -ppassword -D postfixdb -e 'SELECT  username FROM mailbox' | grep -v username`;do mysql -u user -ppassword -D postfixdb -e "INSERT INTO quota (username, qaction) VALUES ('$i', 'DUNNO')";

With this script,value of qaction field on mysql quota table will change continuously as the user’s maildir contents that keeps changing.

Create GeoIP Badge Using PHP Image Processing And GD

When making the badges I always use javascript. This tutorial shows you how to use the GD library to dynamically create simple badges on your site.

Basically, image generation is done in three steps:

  • allocating an image
  • drawing into the allocated space
  • releasing the allocated data in picture format to the browser

[cfields]badgegd[/cfields]

As usual, obtaining client Ip Address simply by calling this php predefined variable, this time we will also trying to support for IPv6. One drawback, i still don’t know how to resize canvas to be fit with text if overflown, dynamically. Ie: the long IPv6 address.

<?php echo $_SERVER["REMOTE_ADDR"]; ?>