I.T. Security and Linux Administration

Mar 25 2012   2:42PM GMT

IP Banlist with Automagic Updating

Eric Hansen Eric Hansen Profile: Eric Hansen


First let me start off by saying that this can be used for iptables with some minor tweaking, but I chose to implement this using tcp_wrappers instead (/etc/hosts.allow; hosts.deny).  Main reason being is I wrote this for Rob to make his task of updating a list of banned IPs that much easier.

Requirements

Before getting started, there are some requirements.  One of them being that this was written with Bash in mind, so if you use Zsh or some other shell, there’s no guarantees it will work right out of the box.  Secondly, cURL [I]OR[/I] wget must be installed.  Originally I wrote this with only cURL support, but also decided to work in wget since it was trivial.

Notes

There is not a whole lot of error checking, and there are areas that can be improved.  I’ve never claimed, nor will I ever, to be a script-fu master.  If what I write works, then I’m proud of myself.  If not, I see how I can make it work.  So, basically consider it your homework to add the line into tcp_wrappers after the list is generated.  The script will be provided at the end of this post, like usual.  Also, be aware that this is generated in CIDR format (which both tcp_wrapper & iptables support).  The format can be changed, but that will be extra credit.

Purpose

Maintaining an ACL (access control list) can be very daunting, for one.  Especially when you have to maintain the same ACL on multiple servers.  You can push out a customized file for the servers and be done with it.  But, how do you even populate that list?  While the source for these IPs is not the de-facto, it provides a good amount to use.

Why, then, write this script?  Easy.  For those who want to keep an up-to-date bad IP (reported for whatever reason).  Maybe you’re running a network of cloud servers and want to reduce the threat of attacks.

How To Use

Throughout the rest of this article I’ll assume you’re using the default name of the script, fetch.sh.

So how is this script used exactly?  Simple, it takes up to 2 arguments:

  1. Country list
  2. Bad IP filename

The country list argument has to be enclosed in quotes if you’re wanting more than 1 originating-country’s IP list.  Also, the list must be in ISO format.  So, for example if you want all reported IPs from Afghanistan and USA, it would be “AF US”.  By default all countries are fetched (see the CL variable for more).

The bad IP filename is the absolute path to the file where the IPs will be stored.  By default the path (and file) is /tmp/bad_ips.  If you want to specify it as something else then just do this:

./fetch.sh /path/to/file/IPs.list

If you run the script without any arguments, it’ll use the defaults for both options.  If you run it with just one argument then it assumes the argument is the bad IP filename.  However, the defaults can be changed by editing the Bash script.

Examples of Use

Fetching IPs For First Time

These examples of use assume using defaults.  Running the script for the first time typically doesn’t take as long as there’s no IPs to check.

ehansen@hometest:~/CodeDump$ ./fetch.sh

— Bad IP Fetcher v1.0 —

Script Usage: ./fetch.sh [country list] [IP list filename]

– Country list: List of 2-character code countries to get reportedly bad IPs from.

– IP list filename: This is the output file where the IPs will be stored.

If only 1 argument is given, it is assumed to be the output filename.

A word of warning, this script may take a while as it checks every IP address to see if it is already in the ban list.

Output file: /tmp/bad_ips

Fetching 253 countries.

Use cURL: 1

[06:09:52 PM] Fetching IP list from http://www.countryipblocks.net/country-blocks/select-formats/

Starting cURL…done.

!! Elapsed time to HTML for IPs: 0:00:03

[06:09:55 PM] Fetching IPs and storing them in /tmp/bad_ips…done.

!! Elapsed time for storing IP list: 0:00:05

[06:10:00 PM] A total of 143798 blacklisted IPs were fetched, ./fetch.sh stored 143798 of them in /tmp/bad_ips, while 0 were already present.

!! Script execution time: 0:00:08

ehansen@hometest:~/CodeDump$

Getting Updates

The script checks to see if the current IP from the fetched list exists in the file already, and if not adds it.  This takes longer to run due to the constant checking (mind you, the server I ran this on is a REALLY, REALLY old desktop [reaching the 10 years mark], with extremely low-end specs).  Here is the output from running it a second time:

ehansen@hometest:~/CodeDump$ ./fetch.sh

— Bad IP Fetcher v1.0 —

Script Usage: ./fetch.sh [country list] [IP list filename]

– Country list: List of 2-character code countries to get reportedly bad IPs from.

– IP list filename: This is the output file where the IPs will be stored.

If only 1 argument is given, it is assumed to be the output filename.

A word of warning, this script may take a while as it checks every IP address to see if it is already in the ban list.

Output file: /tmp/bad_ips

Fetching 253 countries.

Use cURL: 1

[05:24:13 PM] Fetching IP list from http://www.countryipblocks.net/country-blocks/select-formats/

Starting cURL…done.

!! Elapsed time to HTML for IPs: 0:00:03

[05:24:16 PM] Fetching IPs and storing them in /tmp/bad_ips…done.

!! Elapsed time for storing IP list: 0:35:36

[05:59:52 PM] A total of 143798 blacklisted IPs were fetched, ./fetch.sh stored 0 of them in /tmp/bad_ips, while 143798 were already present.

!! Script execution time: 0:35:39

ehansen@hometest:~/CodeDump$

How To Use In tcp_wrappers

Now the fun part, after all of this talking with the script.  Now we get to use it.

Well, assuming the IP list is stored in /tmp/bad_ips, we will set this up to block these IPs from using SSH.  All you need to do is open up /etc/hosts.deny and add this line:

sshd: /tmp/bad_ips

If the ACL starts with a / then tcp_wrappers treats it as a file and will read the IP list (each IP separated with a newline) in.  Best of all there is no need to restart the system or any services, changes take effect immediately.

Note that hosts.allow takes precedence over hosts.deny.  This is why I prefer iptables over tcp_wrappers, but its still not a bad firewall.

Any service that can be configured with tcp_wrappers can use this script and file.

Script / Downloads

fetch.sh – https://github.com/SecurityForUs/CodeDump/blob/master/fetch.sh

 Comment on this Post

 
There was an error processing your information. Please try again later.
Thanks. We'll let you know when a new response is added.
Send me notifications when other members comment.

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy

Forgot Password

No problem! Submit your e-mail address below. We'll send you an e-mail containing your password.

Your password has been sent to: