- This article provides basic background about location aware DNS and how to setup it to work with Bind.
- Article can be found at Geolocation-aware DNS with Bind.
- This can be referred to Bind-GeoIP and GeoDNS Bind Patch.
- Some bash script can be found at http://phix.me/geodns/
- Can also refer to the topic “How to build your own CDN using BIND, GeoIP, Nginx, and Varnish“
Overview
I was recently asked by my employer to bring our DNS in-house from UltraDNS where we originally hosted all our domain names. Due to various requirements within the company, they were utilising UltraDNS’s geo-targetting feature to enable internet users in different areas of the world to resolve hosts on their domains to varying IP addresses, depending on the geographical (country) location of these users.
Having already been exposed to BIND’s views feature some years ago, I googled on how it would be possible to make BIND geo-aware. There is not much documentation about this online but I found one such solution which involved patching the BIND source code. All well and good but, in all honesty, this seemed like using a sledge hammer to crack a nut. Besides, our company does not like patching (hacking) source code unless there is a real requirement to do so as it normally entails maintenance by having to refit changes into revisions of the BIND source code as and when the ISC release newer versions of BIND.
I analysed the patching BIND method further and the solution still uses two fundamental things to achieve a geo-aware DNS setup; BIND’s views feature and the freely downloadable GeoIP data available from MaxMind. It was then I realised that to make BIND geo-aware, all that is required is to reformat the data in the MaxMind GeoIP CSV file into something which BIND likes, and will accept in its configuration file. The easiest and most manageable way to achieve this is by using the BIND Access Control List clause, but here lies the problem. The MaxMind GeoIP CSV file operates in IP rangeswhereas BIND ACLs operate on IP networks, in classic net/mask notation. So, basically, I had to formulate a method to transform MaxMind IP ranges into BIND ACLs. This method is attainable by using the Linux BASH script(s) shown below.
The result is the automatic creation of a single and maintainable GeoIP.acl include file that can be instantly added into any already running BIND DNS server, without the requirement for source code patching and recompilation, producing a geo-aware production-ready DNS server in a matter of minutes.
Linux BASH script(s) to fetch, unzip, reformat and generate the GeoIP.acl include file for BIND
There are two different BASH scripts documented below which will generate the GeoIP.acl include file for BIND. The second is an improvement over the first but I’ve left it documented anyway as it was my original implementation. The first uses an iterative BASH loop (slower) whereas the second uses a recursive AWK function (much faster). Both achieve exactly the same thing by employing different programming constructs. For speed and efficiency, I recommend using the second recursive script.
NOTE: By default, some distributions of Linux use a non-GNU version of AWK which lacks the bitwise AND function. In this instance, GAWK must be installed (the GNU version of AWK) for the scripts below to function correctly (thanks to Ruben for pointing this out).
Each script will attempt to download the latest MaxMind GeoIP CSV file (which is actually a ZIP file). Once downloaded, it will use this file and reprocess it each time it is executed. Removing the ZIP file and then rerunning the script will force it to perform another fetch from MaxMind. Once the ZIP file has been fetched, each script will unzip it, reformat the enclosed GeoIP CSV file (taking several passes to do this if the iterative version is used) and then generate the file GeoIP.acl which is the include file that can be added into BIND’s configuration to make it geo-aware.
Iterative Version (slowest)
#!/bin/bash
[ -f GeoIPCountryCSV.zip ] || wget -T 5 -t 1 http://geolite.maxmind.com/download/geoip/database/GeoIPCountryCSV.zip
echo -n "Creating initial CBE (Country,Begin,End) CSV file..."
unzip -p GeoIPCountryCSV.zip GeoIPCountryWhois.csv | awk -F " '{print $10","$6","$8}' > cbe0.csv
echo -ne "DONEnSplitting CBE CSV file..."
lc0=0; lc1=$(wc -l cbe0.csv | awk '{print $1}')
while [ $lc0 -lt $lc1 ]
do
lc0=$lc1; echo -ne "n$lc0t"
awk -F , '{m = 2^32-2^int(log($3-$2+1)/log(2)); n = and(m,$3); if (n == and(m,$2)) print; else printf "%s,%u,%un%s,%u,%un",$1,$2,n-1,$1,n,$3}' cbe0.csv > cbe1.csv
mv -f cbe1.csv cbe0.csv; lc1=$(wc -l cbe0.csv | awk '{print $1}')
echo -ne "+$[$lc1-$lc0]t"; [ $lc0 -lt $lc1 ] && echo -n "OK"
done
echo -ne "DONEnGenerating BIND GeoIP.acl file..."
(for c in $(awk -F , '{print $1}' cbe0.csv | sort -u)
do
echo "acl "$c" {"
grep "^$c," cbe0.csv | awk -F , '{printf "t%u.%u.%u.%u/%u;n",$2/2^24%256,$2/2^16%256,$2/2^8%256,$2%256,32-int(log($3-$2+1)/log(2))}'
echo -e "};n"
done) > GeoIP.acl
rm -f cbe0.csv
echo "DONE"
exit 0Here’s this script in action!
$ ./GeoIP.sh
--00:00:00-- http://geolite.maxmind.com/download/geoip/database/GeoIPCountryCSV.zip
=> `GeoIPCountryCSV.zip'
Resolving geolite.maxmind.com... 64.246.48.99
Connecting to geolite.maxmind.com|64.246.48.99|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1,556,500 (1.5M) [application/zip]
100%[================================================================================>] 1,556,500 820.41K/s
00:00:02 (818.35 KB/s) - `GeoIPCountryCSV.zip' saved [1556500/1556500]
Creating initial CBE (Country,Begin,End) CSV file...DONE
Splitting CBE CSV file...
106184 +31276 OK
137460 +23038 OK
160498 +11755 OK
172253 +6413 OK
178666 +3544 OK
182210 +1905 OK
184115 +949 OK
185064 +463 OK
185527 +202 OK
185729 +94 OK
185823 +38 OK
185861 +19 OK
185880 +5 OK
185885 +2 OK
185887 +0 DONE
Generating BIND GeoIP.acl file...DONERecursive Version (fastest)
#!/bin/bash
[ -f GeoIPCountryCSV.zip ] || wget -T 5 -t 1 http://geolite.maxmind.com/download/geoip/database/GeoIPCountryCSV.zip
echo -n "Creating CBE (Country,Begin,End) CSV file..."
unzip -p GeoIPCountryCSV.zip GeoIPCountryWhois.csv | awk -F " '{print $10","$6","$8}' > cbe.csv
echo -ne "DONEnGenerating BIND GeoIP.acl file..."
(for c in $(awk -F , '{print $1}' cbe.csv | sort -u)
do
echo "acl "$c" {"
grep "^$c," cbe.csv | awk -F , 'function s(b,e,l,m,n) {l = int(log(e-b+1)/log(2)); m = 2^32-2^l; n = and(m,e); if (n == and(m,b)) printf "t%u.%u.%u.%u/%u;n",b/2^24%256,b/2^16%256,b/2^8%256,b%256,32-l; else {s(b,n-1); s(n,e)}} s($2,$3)'
echo -e "};n"
done) > GeoIP.acl
rm -f cbe.csv
echo "DONE"
exit 0Here’s this script in action!
$ ./GeoIP.sh
--00:00:00-- http://geolite.maxmind.com/download/geoip/database/GeoIPCountryCSV.zip
=> `GeoIPCountryCSV.zip'
Resolving geolite.maxmind.com... 64.246.48.99
Connecting to geolite.maxmind.com|64.246.48.99|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1,556,500 (1.5M) [application/zip]
100%[================================================================================>] 1,556,500 820.41K/s
00:00:02 (818.35 KB/s) - `GeoIPCountryCSV.zip' saved [1556500/1556500]
Creating CBE (Country,Begin,End) CSV file...DONE
Generating BIND GeoIP.acl file...DONEBoth of these scripts will generate the file GeoIP.acl in the current working directory which looks something like this:
acl "A1" {
64.46.32.0/23;
64.46.35.0/24;
64.46.40.64/26;
64.46.42.0/23;
64.46.47.0/24;
66.38.243.0/24;
67.15.183.0/25;
69.10.130.128/26;
69.10.139.0/25;
69.10.140.192/26;
...
acl "GB" {
2.6.190.56/29;
9.20.0.0/17;
12.129.72.32/29;
23.0.0.0/9;
25.0.0.0/8;
32.58.57.0/29;
32.58.58.0/28;
32.58.59.0/29;
32.60.34.96/27;
51.0.0.0/8;
...
217.204.159.96/29;
217.204.159.104/30;
217.204.159.112/28;
217.204.159.128/25;
217.204.160.0/19;
217.204.192.0/18;
217.205.0.0/16;
217.206.0.0/15;
217.237.189.240/29;
217.243.204.144/29;
};
...
217.194.132.0/24;
217.194.145.144/29;
217.194.146.192/26;
217.194.147.240/28;
217.194.149.32/28;
217.194.149.168/29;
217.194.156.0/26;
217.194.157.48/28;
217.194.157.144/29;
217.194.157.168/29;
};How do these scripts work?
I wont go into the technicalities of how these scripts work (this is left as an exercise for the reader) but the first iterative script creates a new CSV file containing 3 fields (Country,Begin,End) and then repeatedly searches for and splits these IP ranges on network boundaries so we are left with a CSV file that has exactly the same coverage of IPs as before but has been processed so that the IP ranges reside on values that allow for each range to be expressed concisely in net/mask notation. The final part of the script then uses this CSV file to generate the GeoIP.acl include file.
The second recursive script achieves the same result faster by creating a new CSV file as before, containing 3 fields (Country,Begin,End), and then performing recursive range splitting “on the fly” withinawk itself, for each country, to generate the GeoIP.acl include file.
Once either of these scripts have finished running, you can slot the newly created GeoIP.acl file straight into your existing BIND configuration file, by adding the line:
include "/path/to/GeoIP.acl";
to named.conf. It will then be possible to create custom geo-views within BIND, like this:
view "north_america" {
match-clients { US; CA; MX; };
recursion no;
zone "example555.com" {
type master;
file "pri/example555-north-america.db";
};
};
view "south_america" {
match-clients { AR; CL; BR; PY; PE; EC; CO; VE; BO; UY; };
recursion no;
zone "example555.com" {
type master;
file "pri/example555-south-america.db";
};
};
view "other" {
match-clients { any; };
recursion no;
zone "example555.com" {
type master;
file "pri/example555-other.db";
};
};If you decide to cron these scripts within your BIND name server(s), do remember to reload named (normally achieved by running the command service named reload on RedHat/CentOS) so the new ACL definitions within the GeoIP.acl file are loaded into BIND’s memory.
Summary
I hope this article proves useful for others (that’s why I have documented it). Interestingly, my original implementation of this was by using a PHP script coupled with MySQL, loading the MaxMind CSV file into a database table, and then running SELECT, UPDATE and INSERT queries to split up the IP ranges. Whilst this worked, it depended on having PHP and MySQL installed and configured. The above scripts achieve exactly the same thing but only using BASH commands and utilities, such as awk, grep and sort, which in my view, is far cleaner!
Incidently, it is actually possible to produce the GeoIP.acl file without using grep or any intermediate CSV file (shown below). These scripts may be used instead but with markedly longer execution times and, because of this, an echo statement, outputting the current country code to standard error, has been introduced into their main loops to give an indication of progress while the scripts are running.
Recursive Versions (smallest)
#!/bin/bash
[ -f GeoIPCountryCSV.zip ] || wget -T 5 -t 1 http://geolite.maxmind.com/download/geoip/database/GeoIPCountryCSV.zip
unzip GeoIPCountryCSV.zip || exit 1
(for c in $(awk -F " '{print $10}' GeoIPCountryWhois.csv | sort -u)
do
echo "$c" >&2
echo "acl "$c" {"
awk -F " 'function s(b,e,l,m,n) {l = int(log(e-b+1)/log(2)); m = 2^32-2^l; n = and(m,e); if (n == and(m,b)) printf "t%u.%u.%u.%u/%u;n",b/2^24%256,b/2^16%256,b/2^8%256,b%256,32-l; else {s(b,n-1); s(n,e)}} c == $10 {s($6,$8)}' c=$c GeoIPCountryWhois.csv
echo -e "};n"
done) > GeoIP.acl
rm -f GeoIPCountryWhois.csv
exit 0We can marginally reduce the execution time of the above script by adjusting its awk line to match the current country using a regular expression, as opposed to setting the awk variable c and then checking if c == $10, as follows:
#!/bin/bash
[ -f GeoIPCountryCSV.zip ] || wget -T 5 -t 1 http://geolite.maxmind.com/download/geoip/database/GeoIPCountryCSV.zip
unzip GeoIPCountryCSV.zip || exit 1
(for c in $(awk -F " '{print $10}' GeoIPCountryWhois.csv | sort -u)
do
echo "$c" >&2
echo "acl "$c" {"
awk -F " 'function s(b,e,l,m,n) {l = int(log(e-b+1)/log(2)); m = 2^32-2^l; n = and(m,e); if (n == and(m,b)) printf "t%u.%u.%u.%u/%u;n",b/2^24%256,b/2^16%256,b/2^8%256,b%256,32-l; else {s(b,n-1); s(n,e)}} '"/,"$c",/"' {s($6,$8)}' GeoIPCountryWhois.csv
echo -e "};n"
done) > GeoIP.acl
rm -f GeoIPCountryWhois.csv
exit 0Do note, however, that I personally prefer the previous grep method as it is much faster than these two scripts because it initially reformats the data within the CSV file into something that allows for fast regex pattern matching on the country field (by moving this field to the beginning of each line) allowing awk to take care of the more complicated task of IP range splitting that operates on the begin (2nd) and end (3rd) integer IP fields.
Geo-aware BIND for the IPv6 network? Without patching? Absolutely!
Over the last decade, IPv6 has become more and more mainstream. More recently, as of the 3rd of February 2011, IANA allocated the last remaining 5 IPv4 /8 blocks to each RIR, thus completely exhausting the IANA pool, meaning there is now no further free IPv4 address space available for allocation. Due to this, I predict demand for adoption of IPv6 is now likely to rise over the coming years.
As much as I have not yet seen any requirement for geo-aware DNS serving on the IPv6 network, I would imagine this will gradually become needed as services begin to migrate away from IPv4 to IPv6. BIND already handles IPv6 addresses within its ACLs so I have published further scripts below that allow the creation of a GeoIPv6.acl include file containing IPv6 net/mask entries, using the freely downloadable GeoIPv6 CSV file available from MaxMind.
It was a challenge to come up with a working solution using the same principles as in the above scripts, but across a much larger address space. This is because IPv6 uses a 128 bit address space, compared to IPv4 being only 32 bits. The scripts above get away with using simple BASH utilities such as awk for doing the necessary IP range splitting with 32 bits but, as I found out, awk is unable to handle numbers which are up in the realms of 64 bits and beyond. So I’ve had to pull various different Linux utilities into play here to achieve this.
In order to handle large numbers up to and beyond 64 bits in magnitude, one has to look at other programming languages and the libraries they offer. After evaluating today’s available languages like Python (which handles large numbers out the box) and PHP (which can only handle large numbers with an additional library installed), I decided to go with Perl. Perl has, on most standard installs, the bignumlibrary that is available and ready to go. This library is transparent and as soon as it is included into a script, all number processing will automatically use it. It has all the necessary operations like bitwise AND that the above scripts make use of. However, when writing the Perl script below, I ran into an inconsistency with the log function whilst using the bignum library and, for anything above 64 bits, bignumalso exhibits major rounding anomalies. To avoid this curveball, I decided to bring the common Linux arbitrary precision calculator bc into play to take over both of these roles. Together, Perl and bc offer the accuracy and speed required to split decimal IP ranges with magnitudes of 64 bits and beyond.
So, here are the scripts. The first script is, as before, a standard BASH script (called GeoIPv6.sh). It is much the same as before but rather than piping the filtered grep lines to awk, it pipes them to a newly created Perl script instead. It also contains some further adjustments at the top to download the latest GeoIPv6 CSV file from MaxMind’s servers, as well as an optional pipe of the Perl script output to sed to abbreviate IPv6 addresses to their “double-colon (::) notation” equivalent.
#!/bin/bash
d=http://geolite.maxmind.com/download/geoip/database/
f=$(wget -qT 5 -t 1 -O- $d | egrep -o 'GeoIPv6-[0-9]{8}.csv.gz' | sort -r | head -1)
[ -z "$f" ] && exit 1; [ -f $f ] || wget -T 5 -t 1 $d$f || exit 1
echo -n "Creating CBE (Country,Begin,End) CSV file..."
gunzip -c $f | awk -F " '{print $10","$6","$8}' > cbe.csv
echo -e "DONEnGenerating BIND GeoIPv6.acl file..."
(for c in $(awk -F , '{print $1}' cbe.csv | sort -u)
do
echo "$c" >&2
echo "acl "${c}v6" {"
grep "^$c," cbe.csv | ./GeoIPv6.pl | sed 's (:0)+/ ::/ '
echo -e "};n"
done) > GeoIPv6.acl
rm -f cbe.csv
echo "DONE"
exit 0The Perl script I have called GeoIPv6.pl, with the following contents:
#!/usr/bin/perl
use strict;
use bignum;
use IPC::Open2; open2(*BCOUT,*BCIN,'bc -l');
sub rs {
my ($b,$e) = @_;
print BCIN "scale=40; l($e-$b+1)/l(2)n";
my ($l) = split('.',<BCOUT>);
my $m = 2**128-2**$l;
my $n = $m & $e;
if ($n == ($m & $b)) {
my @x; for (my $p = 112; $p > 0; $p -= 16) {
print BCIN "scale=0; $b/2^$pn";
push(@x,<BCOUT>%65536);
}
printf "t%x:%x:%x:%x:%x:%x:%x:%x/%u;n",$x[0],$x[1],$x[2],$x[3],$x[4],$x[5],$x[6],$b%65536,128-$l;
} else {
rs($b,$n-1); rs($n,$e);
}
}
while (<STDIN>) {chomp($_); my ($c,$b,$e) = split(',',$_); rs($b,$e)}This Perl script effectively reads from standard input in precisely the same way as the original awk script does (expecting each line to be in the format of a CBE (Country,Begin,End) CSV file) but, unlikeawk, can perform IP range splitting on 128 bit decimal numbers, printing IPv6 net/mask entries to standard output. Note the use of a dual pipe to the Linux arbitrary precision calculator bc to manage the logarithmic division calculation and also to accurately truncate values before they are passed to the printf function (done by a small for loop that places these entries into an array). Most importantly, note that we must increase the default scale of 20 within bc to at least 40 to be able to accurately cope with the logarithmic division calculation. Observe:
$ echo 'l(2^128-1)/l(2)' | bc -l 128.00000000000000000132 $ echo 'scale=20; l(2^128-1)/l(2)' | bc -l 128.00000000000000000132 $ echo 'scale=39; l(2^128-1)/l(2)' | bc -l 128.000000000000000000000000000000000000088 $ echo 'scale=40; l(2^128-1)/l(2)' | bc -l 127.9999999999999999999999999999999999999956
The reason we also choose to open a dual pipe to bc within Perl is to stop the forking of a separate bc process each time we need to perform a division calculation (forking a new process is costly in terms of CPU time). By opening up a dual pipe to a single persistent bc process, we can simply throw and retrieve each calculation into and out off it quickly. The IPC::Open2 Perl module is required to do dual pipes and this may need to be installed on your system.
Once these two scripts have been created, it will be possible to run ./GeoIPv6.sh to generate the GeoIPv6.acl include file for BIND. Note that the execution time here will be far greater than before, since we are using Perl with bignum support, and passing division calculations to a separate persistent bc process. As such, the BASH script has been modified to output the current country code being processed to standard error to indicate progress. Once the script has completed execution, the GeoIPv6.acl include file will have been created in the current working directory, which looks something like this:
acl "ADv6" {
2001:4df8::/32;
};
acl "AEv6" {
2001:8f8::/32;
2a00:d30::/32;
2a00:f28::/32;
};
acl "AMv6" {
2001:1bb0::/32;
2001:4d00::/32;
2a00:f38::/32;
2a00:1290::/32;
2a00:1500::/32;
2a02:d18::/32;
...
acl "GBv6" {
2001:630::/32;
2001:678:4::/47;
2001:67c:18::/48;
2001:67c:90::/48;
2001:67c:b4::/48;
2001:67c:c0::/48;
2001:67c:d4::/48;
2001:6f8::/32;
2001:710::/32;
2001:768::/32;
...
2a02:ce8::/32;
2a02:da0::/32;
2a02:df8::/32;
2a02:e38::/32;
2a02:e68::/32;
2a02:eb0::/32;
2a02:ef8::/32;
2a02:f70::/32;
2a02:fb0::/32;
2a02:fb8::/32;
};
...
2001:43d8::/32;
2001:43f8:20::/48;
2001:43f8:30::/48;
2001:43f8:40::/48;
2001:43f8:50::/48;
2001:43f8:70::/45;
2001:43f8:90::/48;
2001:43f8:a0::/48;
2001:43f8:d0::/48;
};
acl "ZWv6" {
2001:42b0::/32;
};Faster Python Implementation of the above Perl Script
I have recently been learning Python at my current place of employment. After much procrastination, below is my Python implemention of my original Perl script above. When used within the BASH script, execution time to generate the GeoIPv6.acl include file is reduced to about one quarter that of when the Perl script is used. This is most notably because it is a self-contained script that does not depend on making external calls to the common Linux arbitrary precision calculator bc (an external process) as it utilises the mag function from the additional Python library mpmath to determine the magnitude of the IPv6 ranges it is potentially having to split.
#!/usr/bin/python
from sys import stdin
from mpmath import mag
def s(b,e):
l = mag(e-b+1)-1
m = 2**128-2**l
n = m & e
if n == m & b:
print 't%x:%x:%x:%x:%x:%x:%x:%x/%u;' % tuple([b/2**p%65536 for p in xrange(112,-1,-16)]+[128-l])
else:
s(b,n-1)
s(n,e)
for r in (map(int,l.split(',')[1:3]) for l in stdin): s(*r)To use this Python script instead, just place the above inside an executable file called GeoIPv6.py and then change the line:
grep "^$c," cbe.csv | ./GeoIPv6.pl | sed 's (:0)+/ ::/ '
in GeoIPv6.sh to:
grep "^$c," cbe.csv | ./GeoIPv6.py | sed 's (:0)+/ ::/ '
Performance versus Maintainability (pros/cons for/against this ACL method compared to BIND source code patching)
John ‘Warthog9’ Hawley, the chief administrator of www.kernel.org (a high-traffic site which implemented BIND GeoDNS on the 19th of September 2008 via patching), recently contacted me about this HOWTO with some interesting points concerning the implications of using this ACL method over BIND source code patching. I will briefly discuss this here, as it will affect which route you take when implementing GeoDNS within BIND.
In a nutshell, patching BIND for GeoDNS support results in a DNS server that can answer queries at an extremely rapid rate compared with this ACL method (I have confirmed this; it is quite easy to test; see below). This is because the MaxMind binary database is a binary search tree data structure, and so the worst case maximum number of lookups required to determine the country location of an IPv4 address will be 32 iterations (and most times, far less than this). Similary, for their IPv6 binary database, this number changes to 128 iterations. As you can imagine, patching the MaxMind GeoIP C library directly into BIND to achieve GeoDNS will result in a server which is able to process, lookup and answer DNS queries with very few CPU cycles. As such, if your DNS servers are high-traffic servers, responding to many DNS requests per second, it would be advisable to go with the source code patching route.
Alternatively, if maintainability is of more importance to you, the ACL method described in this HOWTO is still a viable option, but with the consequence of a substantial performance hit. According to John (who has been chatting with Paul Vixie, the primary author and architect of BIND until release 8), the ACL feature was never designed with the intention to store and hold the number of ACL entries that the above scripts generate, for GeoDNS purposes. This I can believe, as the scripts above (for IPv4) produce an ACL definition file containing over 200,000 ACL entries, which BIND has to load and subsequently store in its memory once launched. I am not fully aware of the data structures used within BIND to store ACLs, but they will be far less efficient than the simple binary search tree that MaxMind offer with their binary GeoIP databases. It is for this reason that the ACL method described in this HOWTO will result in a far slower DNS server, depending on how many views you create and the ACLs assigned to them.
To give you an idea of just how much of a performance hit this ACL method induces, I have a small low-power server on my network running a CentaurHauls VIA Nehemiah CPU @ 1 GHz (2000 BogoMips) with a 192.168.0.0/16 IP address (see RFC 1918; all other hosts on my LAN are in this network so none of them would be a match in any of the above ACLs). When loading BIND with the GeoIP.acl include file, and creating a catch-all view that matches any client (not using any of the ACLs in the GeoIP.acl include file), the DNS response time tends to be about 2 ms. If, however, another view is created before this catch-all one in named.conf, and the clause:
match-clients { A1; A2; AD; AE; AF; AG; AI; AL; AM; AN; ... VI; VN; VU; WF; WS; YE; YT; ZA; ZM; ZW; };is added to this view (forcing it to attempt a match across every single ACL definition inside the GeoIP.acl file), the response time sores to around 85 ms. In other words, the amount of work that we have now asked BIND to do, in order for it to verify if any of the ACLs are a match for a client with IP address in 192.168.0.0/16, has resulted in it slowing down by a factor of 40 (a rough guestimate figure only) which is a substantial performance hit that needs to be considered. For this reason, if using the ACL method described in this HOWTO, try and limit the number of views you create and the number of ACLs assigned to them as this will lower the amount of work BIND has to do when answering DNS queries made to it.
In short, you should determine if speed (source code patching) or maintainability (ACL include file) is of more importance to you and be fully aware of the pros and cons of each method of GeoDNS implementation within BIND. As a systems administrator, use your head to decide which method to go with. As www.kernel.org is a global site, ranked around 10,000 across all sites on the internet (according to Alexa), John has done the right thing and gone with the patching method when deploying BIND GeoDNS servers for Kernel.org.
——– ANOTHER ARTICLE ———
So it might happen that you are offering a service on a number of identically configured servers that are geographically distributed (for example, VPN servers, streaming media servers,mirrors for software downloads), and you want it to be reachable using a single URL, but at the same time you want to send users to the server that is “closer” to them (“closer” according to geolocation information).
Probably the cleanest, most scalable and most resilient way to implement that is by using anycast routing, associating the URL with a single IP address and announcing that same IP address from all the locations into the Internet, thus letting users go to the location that’s closest to them in terms of routing. For example, some root DNS servers use this technique. This also works to transparently redirect users to another server when one location is down and the advertisement is withdrawn from there. However, this requires that you run dynamic routing (BGP) at all the locations, or you need your ISP’s cooperation to announce the addresses on your behalf. Furthermore, you’ll probably need a large enough block if IP addresses to be announced, which might not be the case if you don’t own many public IPs. Finally, the BGP method is not problem-free anyway, since routing convergence in the Internet at large could take time, and there might be some devices that filter advertisements of the same addresses coming from different locations (hopefully those should be very few or none though).
So another, less optimal but probably easier, way to get a similar result, assuming you control the authoritative DNS server(s) for the name in the URL, is to have the DNS server look at the source IP address of the queries it gets, and serve different replies based on that. For this example, we’re going to assume that there are three servers, one in Europe (IP 192.168.0.1), one in Asia (IP 10.0.0.1), and one in the US (IP 172.16.0.1), and they should all be accessible using the namemirror.example.com. So ideally our goal here is to have the DNS server resolve the name to 192.168.0.1 if the query comes from a european IP, 10.0.0.1 if the query comes from an asian IP, and 172.16.0.1 if it’s from America.
Note that our DNS server will almost never be queried directly by the end clients that need to access the service; rather, it will be queried by other DNS servers that want to resolve the names on behalf of their clients. However, it’s reasonable to assume that end clients will generally be using DNS servers that are geographically close to them (for example, their company’s or their ISP’s DNS). Sure, there will be exceptions, but the worst that can happen is that, say, an asian client is sent to the european server, for example, so it’s not really something to worry about, and geoIP information cannot be 100% accurate anyway, so those things would probably happen in any case.
Bind views
Bind has just the perfect feature to implement the strategy described above, and it’s called views. Views allow to define different “virtual” configurations within the same server, and also to specify who should see which configuration. A typical use of views is to provide the so-called split (or split-horizon) DNS service. For example in an enterprise when you want internal clients to be able to use and resolve internal names that should not be visible outside, you define a view for the internal clients that serves a zone containing the internal names, and another view for external clients that serves a zone with only the official names that should be visible from the Internet. The term split DNS seems to be used for any scenario where the server can give different answers for the same queries depending on some characteristics of the query, so our geographic DNS project can indeed be classified as an example of split DNS.
As said, the key is that the server uses the source IP address of the query to select which one of the defined views should be consulted to answer. This means that there should be a way to associate a view with one or more source IP addresses, and that is indeed the case. When a client sends a query, the view that matches its IP address is used. (There are other selectors that can be used, but they are not relevant here.)
In practice, this means doing something like this in named.conf:
view "internal" {
match-clients { 10.0.0.0/8; 192.168.33.0/24; };
// some configuration fragment we want the internal clients to see
// ...
};
view "internet" {
match-clients { any; };
// some (possibly different) configuration fragment we want Internet clients to see
// ...
};The good news is that lists of IP addresses can be given names (ACL in Bind terms), and the names rather than the addresses can be used in the match-clientsdirective. Also note that the simpler allow-query directive could be used in place of match-clients.
So back to our scenario, the idea is to define three views, one per server, each of which will serve a different IP address for mirror.example.com. To do this, we’ll create three zone files, each resolving the name to the IP address of the server in the particular region (Europe, Asia, America).
Sources of geolocation information
So what needs to be done is to generate suitable ACLs for each view. Roughly speaking, we should be able to associate all the european IPs to the european view and so on. That is quite a lot of data (we will limit the example to IPv4, but it shouldn’t be too hard to extend it to IPv6). What we need is some big database that associates an IPv4 address or (much better) address block to a specific country. With that and some text processing, it should be possible to automatically create a number of (more or less huge) ACLs, one per country. Those ACLs could then be aggregated by region (Europe, Asia, America) and associated with the relevant view. Queries coming from countries that are not exactly located in one of the three regions (eg African countries, Greenland and the like) can either be sent to a default catch-all view, or those countries can be listed in one of the three main views, perhaps based on distance.
That said, there are a few sources of geolocation information; see for example MaxMind or WIPmania. They provide geolocation files in various formats; for our purposes, we will use WIPmania’s textual format, downloadable from the “CIDR” link in the above page, which is probably the easiest to parse. Essentially, the file contains a lot of lines in the format
IP address/mask country;
for example
139.92.66.0/24 EU; 139.92.67.0/24 SE; 139.92.68.0/22 FR; 139.92.72.0/21 FR; 139.92.80.0/22 FR; ... 158.193.0.0/16 SK; 158.194.0.0/16 CZ; 158.195.0.0/16 SK; 158.196.0.0/16 CZ; 158.197.0.0/16 SK; 158.198.0.0/16 JP; 158.201.0.0/16 JP; ...
ACL generation
The information in the above format can very easily be turned into a list of Bind ACLs with this Perl code using hashes (note I’m not a Perl programmer):
#!/usr/bin/perl
# do_acl.pl
# Outputs Bind ACLs from geoIP information
use warnings;
use strict;
my $h;
my %ip; # each hash element is an array reference
my $country;
open $h,$ARGV[0] or die "Error opening input file";
while(<$h>) {
/^(S+) (S+);$/;
push @{$ip{$2}}, $1;
}
close $h or die "Error closing file";
# Traverse the hash and print the ACLs
for $country (keys %ip) {
print "acl "$country" {n";
print " $_;n" for (@{$ip{$country}});
print "};nn";
}The script can be run like this:
$ do_acl.pl worldip.conf > geo_acl.conf
If everything went fine, geo_acl.conf should contains something like this:
acl "GL" {
88.83.0.0/19;
194.177.224.0/19;
};
acl "DJ" {
41.189.225.0/24;
41.189.232.0/23;
193.251.143.0/24;
193.251.167.0/26;
193.251.167.64/27;
193.251.167.96/28;
193.251.224.0/25;
193.251.224.128/26;
193.251.224.192/28;
193.251.224.208/29;
196.201.192.0/20;
213.187.131.168/29;
};
acl "JM" {
63.75.234.0/23;
65.183.0.0/20;
66.36.201.0/24;
...The order of the ACL might not be the same as shown as they come from a hash, but that’s not really important.
Putting all together
So now we come to the interesting bits. Essentially, we should decide which countries are closer to which server, and put the ACLs for those countries in the relevant server’s view. That might sound like a lot of work, but it has to be done only once. Here’s a minimal example of a named.conf:
...
include "geo_acl.conf";
view "europe" {
match-clients { FR; DE; IT; AT; ...other european ACLs here... };
zone "example.com" {
type master;
file "/path/to/db-europe.example.com";
};
};
view "america" {
match-clients { US; CA; MX; ...other american ACLs here... };
zone "example.com" {
type master;
file "/path/to/db-america.example.com";
};
};
view "asia" {
match-clients { JP; IN; BD; ...other asian ACLs here... };
zone "example.com" {
type master;
file "/path/to/db-asia.example.com";
};
};And, as one might expect, the zone files are something like
# cat db-asia.example.com ... mirror IN A 10.0.0.1 ... # cat db-america.example.com ... mirror IN A 172.16.0.1 ... # cat db-europe.example.com ... mirror IN A 192.168.0.1 ...
Even if you decided to list all the countries in some view, it’s probably a good safety belt to define another view after those above as a catch-all default in case some query is coming from an IP that is not in any ACL:
// "unknown" clients are sent to the asian server...
view "default" {
match-clients { any; };
zone "example.com" {
type master;
file "/path/to/db-asia.example.com";
};
};In this example these clients are sent to the asian server, but any other server can of course be used. You can also omit the default view, and just put the view you want to use as default last (eg “asia” here), and use match-clients { any; }; there. I prefer to be explicit and have a “default” view, also because it makes it slightly easier to automate the generation of the configuration in case you want to do it with a script starting with some mapping between countries and areas.
Testing
Probably the perfect site to test this kind of setup is http://just-ping.com. Just enter the name you want to test in the textbox (ie mirror.example.com for this example), and click on “ping!”. The website tries to ping (and thus resolve) the name you supply from several locations around the world, so you can check that the same name resolves to different IPs in different locations. If you see that happening, then it’s working!
You can also use this website to find out which other sites use geolocation DNS techniques, for example if you try pinging www.yahoo.com or www.google.com you’ll see that those names are resolved to different IPs based on the location. (And, btw, it’s nice to see how www.google.com actually resolves:
www.google.com. 16258 IN CNAME www.l.google.com. www.l.google.com. 60 IN CNAME www-tmmdi.l.google.com. www-tmmdi.l.google.com. 59 IN A 216.239.59.104 www-tmmdi.l.google.com. 59 IN A 216.239.59.147 www-tmmdi.l.google.com. 59 IN A 216.239.59.99 www-tmmdi.l.google.com. 59 IN A 216.239.59.103
note the CNAME chain)
Caveats
Provide explicit names
It’s useful to include specific DNS names to explicitly use a given server, like for example mirror-europe.example.com, mirror-asia.example.com, mirror-america.example.com, and have them resolve to the IP of the specific server, regardless of the source of the query. That can be useful for testing, and also the alternate names could be provided to users to be used as last resort when the generic name isn’t working (for example, because the server in their area to which they would be sent is down). So one could do
# cat db-asia.example.com ... mirror-america IN A 172.16.0.1 mirror-asia IN A 10.0.0.1 mirror-europe IN A 192.168.0.1 mirror IN A 10.0.0.1 ...
or even
# cat db-asia.example.com ... mirror-america IN A 172.16.0.1 mirror-asia IN A 10.0.0.1 mirror-europe IN A 192.168.0.1 mirror IN CNAME mirror-asia.example.com. ...
and equivalent things for the other zone files.
Zones not in any view
It’s important to note that when views are used, there cannot be zones “outside” of any view, or Bind will complain. You can check that this is the case if you see in the log messages like these:
Dec 10 20:41:17 kermit named[1126]: loading configuration from '/etc/bind/named.conf' Dec 10 20:41:18 kermit named[1126]: /etc/bind/named.conf:7: when using 'view' statements, all zones must be in views
The solution is to put all the zones in each view, including the root hint “.” zone, the “localhost” zone, the “127.in-addr.arpa” zone and friends.
Slave servers
What we have so far is just a single DNS server, and good practices dictate that at least two (or even three) should be available (see RFC 2182, section 5, “How many secondaries?”). One could either use multiple master servers, or set up slaves. Setting up a slave server when views are involved isn’t too difficult, but there are some rules to follow if you want all the views replicated in the slave.
The problem with a “normal” slave is that it’s seen by the master as just another client, and when a zone transfer is requested the master will send the zone corresponding to the view that matches the slave’s IP address, and only that zone. With old versions of Bind, working around the problem required to assign multiple IPs to the slave, and set up things such that it issued multiple zone transfer requests to the master, using a different source IP for each request. For this to work, the master needed to explicitly associate the slave’s various source IPs to the various views.
With reasonably recent versions of Bind, there is a much cleaner mechanism available, and it’s based on TSIG (Transaction SIGnature) keys. Altough TSIG keys were introduced mainly to be used for dynamic DNS, they work just fine for the purposes of triggering the transfer of zones in different views.
TSIG is based on the communicating parties sharing a secret. What it does is to “sign” DNS transactions by calculating an HMAC over the messages and appending it, so the receiver can authenticate them by recalculating it and see it if matches the transmitted one. While that is certainly good, for our goal here the important thing is thata TSIG-signed message includes the name of the key used to sign it, and that name can be used as a special ACL and thus associated to a specific view.
Essentially, in our example three keys can be defined, one per view, and the servers are configured to use those keys when communicating with their peer. This way, the master will send different zones depending on the key with which the slave signs the *XFR request. Conversely, the slave will accept NOTIFY messages from the master and assign them to the view corresponding to the TSIG key used by the master. The RFC suggests to name a key after the servers that will use it; here, also considering that all the keys will be used between the same pair of servers, for simplicity the convention is relaxed and the name just identifies the view.
Here is an example of how to do that in our scenario, assuming the master and the slave DNS have addresses 10.10.10.10 and 10.20.20.20 respectively:
; On the master
key "key_europe" { algorithm hmac-md5; secret "YWJjZGVmZw=="; };
key "key_asia" { algorithm hmac-md5; secret "aGlqa2xtbg=="; };
key "key_america" { algorithm hmac-md5; secret "b3BxcnN0dQ=="; };
acl all_keys { key key_america; key key_europe; key key_asia; };
view "europe" {
// this view matches either european IPs, or whoever uses the key key_europe
match-clients { key key_europe; !all_keys FR; DE; IT; AT; ...other european ACLs here... };
// use this key to NOTIFY the right view on the slave
server 10.20.20.20 { keys key_europe; };
zone "example.com" {
type master;
file "/path/to/db-europe.example.com";
};
};
view "asia" {
// this view matches either asian IPs, or whoever uses the key key_asia
match-clients { key key_asia; !all_keys; JP; IN; BD; ...other asian ACLs here... };
// use this key to NOTIFY the right view on the slave
server 10.20.20.20 { keys key_asia; };
zone "example.com" {
type master;
file "/path/to/db-asia.example.com";
};
};
view "america" {
// this view matches either american IPs, or whoever uses the key key_america
match-clients { key key_america; !all_keys; US; CA; MX; ...other american ACLs here... };
// use this key to NOTIFY the right view on the slave
server 10.20.20.20 { keys key_america; };
zone "example.com" {
type master;
file "/path/to/db-america.example.com";
};
};
; On the slave
key "key_europe" { algorithm hmac-md5; secret "YWJjZGVmZw=="; };
key "key_asia" { algorithm hmac-md5; secret "aGlqa2xtbg=="; };
key "key_america" { algorithm hmac-md5; secret "b3BxcnN0dQ=="; };
acl all_keys { key key_america; key key_europe; key key_asia; };
view "europe" {
match-clients { key key_europe; !all_keys; FR; DE; IT; AT; ...other european ACLs here... };
// use this key to get the right view from the master when requesting *XFR
server 10.10.10.10 { keys key_europe; };
zone "example.com" {
type slave;
masters { 10.10.10.10; };
file "/path/to/db-europe.example.com";
};
};
view "asia" {
match-clients { key key_asia; !all_keys; JP; IN; BD; ...other asian ACLs here... };
// use this key to get the right view from the master when requesting *XFR
server 10.10.10.10 { keys key_asia; };
zone "example.com" {
type slave;
masters { 10.10.10.10; };
file "/path/to/db-asia.example.com";
};
};
view "america" {
match-clients { key key_america; !all_keys; US; CA; MX; ...other american ACLs here... };
// use this key to get the right view from the master when requesting *XFR
server 10.10.10.10 { keys key_america; };
zone "example.com" {
type slave;
masters { 10.10.10.10; };
file "/path/to/db-america.example.com";
};
};So with the above config, the slave will use the key “key_europe” when talking to the master; the master will recognize it as being a client for view “europe”, and will transfer the zone in that view. In the same way, the other keys will trigger the transfer of the zones in the respective views. Similarly, the master will use the key “key_europe” when sending NOTIFY messages to the slave, and that will allow the salve to match the received NOTIFY message with the view “europe” (and again, the same for the other keys and views). Note the special ACL all_keys. This is absolutely essential for having zone transfers with views working correctly. If you omit the!all_keys clause in the match-clients directive, depending on the actual location of the servers, if a server presents a TSIG key that doesn’t match the one for that view, but its IP address matches a geographic ACL for that view, then the wrong view will be selected and problems will most certainly occur (very hard to troubleshoot, btw). See this thread on bind-users for more information.
Remember that the “secret” string must be a valid base64-encoded string (here they are just “abcdefg”, “hijklmn” and “opqrstu”, you might want to set them to something a bit stronger, perhaps using dnssec-keygen as described for example here).
GSLB?
The setup described here could be described as a (very) poor man’s version of DNS GSLB (Global Server Load Balancing). However, it lacks most of the more sophisticated features of GSLB, in particular the fact that a GSLB solution actively monitors the services and dynamically adjusts its configuration based on load, latency, availablility to name some.
However, even if we are poor, our solution could be partially automated, perhaps with scripts that monitor the services and in case of problems or failures could dynamically trigger a reconfiguration of the DNS server(s) and a zone reload. The new information would suffer from the TTL propagation delay associated to DNS records, but in most cases would still be better that having manual intervention only.
Complete the configuration
The above configuration fragments are for demonstration purposes only, and lack many parts that you want to have in a real configuration, such as the definition of logging, whether zone transfers, dynamic updates, recursive queries are allowed and from whom, etc.
