How to install a Squid & Dansguardian content filter on Ubuntu Server

Being a family man and a geek, our household has both children and lots of tech; there are 6 or so computers, various tablets, smartphones and other devices capable of connecting to, and displaying content from, the Internet.

For a while now I’ve wanted to provide a degree of content filtering on our network to prevent accidental, or deliberate, access to some of the worst things the Internet has to offer. What I didn’t want to do however was blindly hand control of this very important job to my ISP (as our beloved leader would like us all to do). Also, I absolutely believe this is one of my responsibilities as a parent; it is not anyone else’s. In addition, there are several problems I have with our government’s chosen approach:

  • Filtering at the ISP network-side means the ISP must try and inspect all my internet traffic all of the time (what else could they potentially do with this information I wonder?)
  • If the ISP’s filter prevents access to content which we feel that our kids should be able access, how can I change that? Essentially I can’t.
  • I reckon that most kids of mid-teenage years will have worked out ways to bypass these filters anyway (see footnote) leaving more naive parents in blissful ignorance; thinking their kids are protected when in fact they are not.

With the above in mind I set about thinking how I could provide a degree of security on our home network using tried and trusted Open Source tools…

Firstly this is how our networked looked before.
Home network (before filtering)

The BT Router is providing the DHCP service in the above diagram.

The Ubuntu 12.04 Server is called vimes (after Commander Vimes in the Discworld novels by Terry Pratchett) and is still running the same hardware that I described way back in 2007! It’s a low power VIA C7 processor, 1G of RAM and it now has a couple of Terabytes of disk. It’s mainly used as a central backup controller and dlna media store/server for the house.

I never did get Untangle working on it, but now it seemed like a good device to use to do some filtering… There are loads of instructions on the Internet about using Squid & Dansguardian but none covered quite what I wanted to achieve: A dhcp serving, bridging, transparent proxy content filter.

Architecturally, my network needed to look like this:

Home network (after filtering)

As you can see above, the physical change is rather negligible. The Ubuntu server now sits between the home LAN and the broadband router rather than as just another network node on the LAN as it was before.

The configuration of the server to provide what I required can be broken down into several steps.

1. Get the Ubuntu server acting as a transparent bridge

This is relatively straightforward. First install the bridge-utils package: sudo apt-get install bridge-utils

Then I made a backup of my /etc/network/interfaces file and replaced it with this one:

# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).
  
# The loopback network interface
auto lo
iface lo inet loopback

# Set up interfaces
iface eth0 inet manual
iface eth1 inet manual

# Bridge setup
auto br0
iface br0 inet static
  bridge_ports eth0 eth1
  address 192.168.1.2
  broadcast 192.168.1.255
  netmask 255.255.255.0
  gateway 192.168.1.1

Probably the most interesting part of this file is where we assign a static IP address to the bridge itself. Without this I would not be able to connect to this server as both ethernet ports are now just transparent bridge ports so not actually listening for IP traffic at all.

(Obviously you will need to determine the correct IP address scheme for your own network)

2. Disable DHCP on the router and let Ubuntu do it instead

The reason for this is mostly down to the BT Home Hub… For some bizarre reason, BT determined that they should control what DNS servers you can use. Although I’m not using it right now, I might choose to use OpenDNS for example, but I can’t change the DNS addresses served by the BT Home Hub router so the only way I can control this is to turn off DHCP on the router altogether and do it myself.

Install the dhcp server: sudo apt-get install dhcp3-server

Tell the dhcp server to listen for requests on the bridge port we created before by editing the file /etc/default/isc-dchp-server so that the INTERFACES line reads: INTERFACES="br0".

Then edit the dhcp configuration file /etc/dhcp/dhcpd.conf so we allocate the IP addresses we want to our network devices. This is how mine looks:

ddns-update-style none;

default-lease-time 600;
max-lease-time 7200;

# If this DHCP server is the official DHCP server for the local
# network, the authoritative directive should be uncommented.
authoritative;

# Use this to send dhcp log messages to a different log file (you also
# have to hack syslog.conf to complete the redirection).
log-facility local7;

subnet 192.168.1.0 netmask 255.255.255.0 {
        range 192.168.1.16 192.168.1.254;

        option subnet-mask 255.255.255.0;
        option routers 192.168.1.1;
        
        #Google DNS
        option domain-name-servers 8.8.8.8, 8.8.4.4;
        #OpenDNS
        #option domain-name-servers 208.67.222.222, 208.67.220.220;

        option broadcast-address 192.168.1.255;
}

There are many options and choices to make regarding setting up your DHCP server. It is extremely flexible; you will probably need to consult the man pages and other on-line resources to determine what is best for you. Mine is very simple. It serves one block of IP addresses within the range 192.168.1.16 to 192.168.1.254 to all devices. Currently I’m using Google’s DNS servers but as you can see I’ve also added OpenDNS as a comment so I can try it later if I want to.

3. Install Squid and get it working as a transparent proxy using IPTables

This bit took a while to get right but, as with most things it seems to me, in the end the actual configuration is fairly straightforward.

Install Squid: sudo apt-get install squid.

Edit the Squid configuration file /etc/squid3/squid.conf… By default this file contains a lot of settings. I made a backup and then reduced it to just those lines that needed changing so it looked like this:

http_port 3128 transparent

acl localnet src 192.168.1.0/24
acl localhost src 127.0.0.1/255.255.255.255
acl CONNECT method CONNECT

http_access allow localnet
http_access allow localhost
always_direct allow all

cache_dir aufs /var/spool/squid3 50000 16 256

Probably the most interesting part in the above is the word “transparent” after the proxy port. Essentially this means we do not have to configure every browser on our network: http://en.wikipedia.org/wiki/Proxy_server#Transparent_proxy. The final line of the file is just some instructions to configure where the cache is stored and how big it is. Again, there are tons of options available which the reader will need to find out for themselves…

To actually cause all the traffic on our LAN to go through the proxy rather than just passing through the bridge transparently requires a bit of configuration on the server using ebtables to allow easier configuration of the Linux kernel’s bridge & iptables to redirect particular TCP/IP ports to the proxy.

First I installed ebtables: sudo apt-get install ebtables

My very simplistic understanding of the following command is that it essentially tells the bridge to identify IP traffic for port 80 (http) and pass this up to the kernel’s IP stack for further processing (routing) which we then use iptables to handle.

sudo ebtables -t broute -A BROUTING -p IPv4 --ip-protocol 6 --ip-destination-port 80 -j redirect --redirect-target ACCEPT

Then we tell iptables to forward all port 80 traffic from the bridge to our proxy:

sudo iptables -t nat -A PREROUTING -i br0 -p tcp --dport 80 -j REDIRECT --to-port 3128

Restart Squid: sudo service squid3 restart

At this point http browser traffic should now be passing through your bridge and squid proxy before going on to the router and Internet. You can test to see if it is working by tailing the squid access.log file.

I found that squid seemed to be very slow at this juncture. So I resorted to some google fu and looked for some help on tuning the performance of the system. I came across this post and decided to try the configuration suggestions by adding the following lines to my squid.conf file:

#Performance Tuning Options
hosts_file /etc/hosts
dns_nameservers 8.8.8.8 8.8.4.4
cache_replacement_policy heap LFUDA
cache_swap_low 90
cache_swap_high 95
cache_mem 200MB
logfile_rotate 10
memory_pools off
maximum_object_size 50 MB
maximum_object_size_in_memory 50 KB
quick_abort_min 0 KB
quick_abort_max 0 KB
log_icp_queries off
client_db off
buffered_logs on
half_closed_clients off
log_fqdn off

This made an immediate and noticeable difference to the performance; enough so in fact that I haven’t yet bothered to go any further with tuning investigations. Thanks to the author Tony at last.fm for the suggestions.

4. Install Dansguardian and get it filtering content

sudo apt-get install dansguardian is all you need to install the application.

To get it to work with our proxy I needed to make a couple of changes to the configuration file /etc/dansguardian/dansguardian.conf.

First, remove or comment out the line at the top that reads UNCONFIGURED - Please remove this line after configuration I just prefixed it with a #.

Next we need to configure the ports by changing two lines so they look like this:

filterport = 8080
proxyport = 3128

Finally, and I think this is right, we need to set it so that Dansguardian and squid are both running as the same user so edit these two lines:

daemonuser = ‘proxy’
daemongroup = ‘proxy’

As you will see in that file, there are loads of other configuration options for Dansguardian and I will leave it up to the reader to investigate these at their leisure.

One suggestion I came across on my wanderings around the Interwebs was to grab a copy of one of the large collections of blacklisted sites records and install these into /etc/dansguardian/blacklists/. I used the one linked to from the Dansguardian website here http://urlblacklist.com/ which says it is OK to download once for free. As I understand it, having a list of blacklist sites will reduce the need for Dansguardian to parse every url or all content but this shouldn’t be relied on as the only mechanism as obviously the blacklist will get out-of-date pretty quickly.

Dansguardian has configurable lists of “phrases” and “weights” that you can tailor to suit your needs.

Now that’s installed we need to go back and reconfigure one of the iptables rules so that traffic is routed to Dansguardian rather than straight to Squid first and also enable communication between Squid and Dansguardian. You can flush (empty) the existing iptables rules by running iptables -F.

Now re-enter the rules as follows:

sudo iptables -t nat -A PREROUTING -i br0 -p tcp –dport 80 -j REDIRECT –to-port 8080
sudo iptables -t nat -A OUTPUT -p tcp –dport 80 -m owner –uid-owner proxy -j ACCEPT
sudo iptables -t nat -A OUTPUT -p tcp –dport 3128 -m owner –uid-owner proxy -j ACCEPT
sudo iptables -t nat -A OUTPUT -p tcp –dport 3128 -j REDIRECT –to-ports 8080

Restart Squid and Dansguardian: sudo service squid3 restart & sudo service dansguardian restart.

Now if you try to connect to the internet from behind the server your requests should be passed through Dansguardian and Squid automatically. If you try and visit something that is inappropriate your request should be blocked.

If it all seems to be working OK then I suggest making your ebtables and iptables rules permanent so they are restored after a reboot.

This can be achieved easily for iptables by simply running sudo iptables-save.

I followed these very helpful instructions to achieve a similar thing for the ebtables rule.

And that’s it. Try rebooting the server to make sure that it all still works without you having to re-configure everything. Then ask your kids and wife to let you know if things that they want to get to are being blocked. YOU now have the ability to control this – not your ISP… 😀

Footnotes

Be aware that on the network diagrams above the Wifi service provided by the BT Homehub router, and the LAN on the router side of the server, are not protected by these instructions. For me this is fine as the coverage of that Wifi network only makes it as far as the Kitchen anyway. And if it was more visible I could always change the key and only let my wife and I have access.

Also, I should make it clear that I know what I have above is not foolproof. I am completely aware that filtering/monitoring encrypted traffic is virtually impossible and there are plenty of services available that provide ways to circumvent what I have here. But I am also not naive and I reckon that if my kids have understood enough about networking and protocols etc. to be able to use tunnelling proxies or VPN services then they are probably mature enough to decide for themselves what they want to look at.

Of course there are plenty of additional mechanisms one can put in place if desired.

  • Time-based filters preventing any Internet access at all at certain times
  • Confiscation of Internet connected devices at bedtime
  • Placing computers and gaming consoles in public rooms of the house and not in bedrooms
  • And many more I’m sure you can think of yourself

As I see it, the point is simply this: As a parent, this is your responsibility…

GeoTools: Geolocation services for vtiger CRM

As many of you know already, our company Libertus Solutions does quite a lot of work with the open source CRM called vtiger. It’s a very competent and accomplished product made even more so by its well thought out extension capabilities.

In this post I’m really pleased to announce our first open source vtlib module for vtiger called GeoTools.

It was derived from another project on the vtiger forge called Maps, which we have taken and extended in true open source style. Standing on the shoulders of giants, and all that…

GeoTools introduces Geolocation features to vtiger in a standard vtlib module package. It adds the ability to perform distance-based searches on your data.

GeoTools uses the Google Maps API to gather positional data, that’s latitude and longitude coordinates, for the entity records that have been configured in the GeoTools Settings area. Once we have acquired this positional data we can then perform location-based calculations to display the results on an embedded Google Map, and as a list view of entity records.

Anyway enough of the words already! Here’s a video:

As soon as the forge site is up I’ll update this and provide links to the code.

Update: Here we are: This will be a moving target for some time yet – it’s still rather “beta” grade code…

Using Facebook XMPP chat on Ubuntu

The first step to recovery is to admit you have a problem. Hi everyone, my name is Alan and I do have a Facebook account. There, done it. Feels better already.
I don’t use it that much, and frankly I find it a little disturbing the way it mixes up all your friends, family and work contacts so they all talk to each other. But this isn’t a post about my insecurities and paranoid delusions. No, it is a post about Ubuntu and XMPP. Facebook now does XMPP, which is an instant messaging protocol also known as Jabber. It is the same thing Google talk uses and the same thing that the most awesome OLPC XO uses for communication.

  • First up you need to set a facebook username up. Log on to facebook and go to your account settings page. Set your username if you haven’t already. I chose alanbelltolc, to match my twitter and identi.ca names. Now I think you have to log out of facebook and back in – this might not be appears to be a necessary step.
  • Now run Empathy, Applications-Internet-Empathy Instant Messenger.
  • Press F4 or go to Edit-Accounts in the Empathy menu.
  • Press the Add button and choose Jabber from the dropdown list of account types.
  • Press the Create button.
  • Your login ID is username@chat.facebook.com – we think it prefers all lower case
  • Your password is your facebook password
  • Now make sure the account is enabled (checkbox next to the account name on the left)
  • It may ask you if you want to let it save your password in the gnome keyring at this point.
  • Make yourself available and the names and pictures of both of your friends should appear!

The account setting dialog as you go through the setup:

You can chat with your friends

You even get lovely libnotify popups like this one >>

Update
If you want to try it out on someone please feel free to find me on Facebook and . . . um what is the verb? XMPP me? Jab me? Next up I will have to take Debian off my OLPC, put Sugar back on and try and get Sugar to use Facebook as a back end.

The Google Chrome Key

Back in 1995 something very odd happened. Microsoft released a new version of their operating system, jumping from version 3.x to version 95 which, as well as being a pretty innovative bit of version numbering, brought in a few other new things including the start menu. Now to get the start menu to work they pulled off an astonishing move and added a new key to computer keyboards, not just keyboards made by Microsoft (I am not even sure they did make them at the time) but keyboards made by all manufacturers. This key had a little windows advert on it and was called the Windows key. Quite how the competition regulators let this pass at the time and ever since is a mystery to me, but to this day if you go and buy a Logitech keyboard for your Ubuntu Linux desktop you will have a windows logo staring at you as you type like the eye of Sauron.

Chrome Key

Keyboards without a windows key are few and far between. Here is a picture of one from ZaReason.

Todays announcement from Google that they are going to release an operating system may cause keyboard manufacturers to think about their little windows adverts a bit more carefully. Initially Google is targeting netbooks with integrated keyboards, presumably they won’t tolerate a windows logo on the keyboard. If and when the Chrome OS moves to a more desktop like platform will Google start pressing OEMs to add a Chrome key? Or will the key be neutralised to a more generic symbol that doesn’t advertise an operating system at all?
Chrome Key

Google Chrome OS

This could be pretty big.

Google announced, in their own rather subtle way – via a blog post – their new Google Chrome OS. It’s quite exciting simply because it is from Google and what the objective of the OS is:

Speed, simplicity and security are the key aspects of Google Chrome OS. We’re designing the OS to be fast and lightweight, to start up and get you onto the web in a few seconds. The user interface is minimal to stay out of your way, and most of the user experience takes place on the web. And as we did for the Google Chrome browser, we are going back to the basics and completely redesigning the underlying security architecture of the OS so that users don’t have to deal with viruses, malware and security updates. It should just work.

For us Freedom lovers there’s good new too:

Google Chrome OS is an open source, lightweight operating system that will initially be targeted at netbooks. Later this year we will open-source its code, and netbooks running Google Chrome OS will be available for consumers in the second half of 2010…

… We have a lot of work to do, and we’re definitely going to need a lot of help from the open source community to accomplish this vision. We’re excited for what’s to come and we hope you are too. Stay tuned for more updates in the fall and have a great summer.

Techcrunch had a great headline on this announcement: Google Drops A Nuclear Bomb On Microsoft. And It’s Made of Chrome.

Here’s a quick thought that that headline inspired…

If Google are ultimately as successful in the OS space as they have been in the on-line space, then I can see some major investigations and calls from such places as Redmond for Google to be split into smaller entities. In a somewhat ironic repeat of what happened to AT&T back in the 70s.

Which isn’t necessarily a bad thing in my mind. Even though Google claim to “do no evil” a monopoly is a monopoly is a monopoly. This is early days and clearly we aren’t anywhere near there yet but they are growing fast and becoming even more ubiquitous than AT&T; which was essentially only a USA monopoly.

I’d love to hear other thoughts on this:

Do you care what happens?
Do you think it is too early in the morning and I should stay in bed rather than talk drivel?
Do you think Google Chrome OS is a non-entity?

Is Microsoft ‘Buying-Off’ Linux Netbook Vendors? [Updated]

There seems to be something of a military campaign afoot. One that surely shows signs of desperation and anxiety on behalf of the instigator.

First we had Asus telling it’s potential customers that “It’s Better With Windows” using phraseology such as:

“Windows helps you easily get online and connect to your devices and services – without dealing with an unfamiliar environment or major compatibility issues.”

And now we hear that PC World (IMHO probably the worst place to buy a PC or get technical advice anywhere in the UK) are dropping Linux netbooks too. And why?

… because Windows makes it easier to share content, and provides customers with a simpler, more familiar computing experience on the move,

Share content huh? Does Ubuntu (or any other Linux distro of choice) work with Facebook, Twitter, Flickr, Google Apps, or anywhere else where we “share” content? Does email work? Does OpenOffice.org allow me to share stuff with those unfortunate enough to have spent hundreds of pounds on another Office suite? (Hint: The answer is Yes.)

Simpler… Hmmm, I wonder how much truth there is in this? How hard is it to install software on [say] Ubuntu vs Windows. Wander over here and take a look. (It’s pretty funny really).

familiar – Well, OK I’ll give them that. But I could make Ubuntu blue and have big child-like buttons if I really wanted to. Other than that though – what’s the big deal? I have a mouse, keyboard, screen. Yep, check. I move the mouse and click (or double click) on things and I type stuff. Yep. Check. Ahh no I’ve got it. The familiarity is with the dear old BSOD. Now I don’t get that. In fact my PC doesn’t really crash at all. It doesn’t need defragging (whatever that is), it doesn’t need disinfecting, it doesn’t require frequent re-builds because it gets so slooooooooow after a few months of use. Now I see. That’s what users want and are familiar with.

And how about all those Drivers we have to install and update and search the ‘net for? Hey? Drivers? What are they?

And how often do you update your Anti-virus software? You do have AV software don’t you? Oh of course, sorry I forgot. Nope. Viruses are not really much of an issue with a proper operating system. Take a look at the Wildlist.org (the list of “Active” computer viruses) All 451 from March this year target the Win32 API.

And how about all those strange licenses that many people don’t read, but that “protect” you from doing something illegal – like helping your neighbour perhaps?

This whole thing reminds me of the stupid and cringe-worthy adverts you see in the press where Vendor X says that they “Recommend Proprietary OS Home Premium”. Do they hell. Do you think they really believe that? Or do you think they have been given incentives to say so? I often wonder if there could be a case here with the Trades Description Act (if that still exists). IANAL so don’t really know but it seems as though many vendors’ adverts are not being totally honest when they make these recommendations. It’s bloody obvious they are being paid in some form or another to say that.

So, it seems to me that Microsoft are getting a bit fed up with all these really cool little netbooks running Ubuntu or Android or something else that isn’t Windows. They have very deep pockets and can afford to buy off some of the people some of the time. But I do not think they can afford to buy everyone all of the time and some vendors probably have a bit more integrity in the first place.

I suggest that you shop around and – even if you buy a PC with Windows for some weird reason – you go to a vendor or supplier who has not been bought off. At least you can have some level of trust in them.

Of course, you could always visit Naked Computers.org and find a vendor that will sell you a computer without an operating system at all. You can then install Ubuntu or something else on it. You almost certainly have a legal CD of XP lying around somewhere. I have several and don’t even use it anymore, so why should I be forced to buy more of their crap, unsafe, bug-ridden software?

Phew. Well. I’m glad I got that off my chest.

[UPDATE] How coincidental is this? Asus, suddenly pull their Android netbook and can’t really say anything about it…

A day after an Asus Eee PC running Google’s Android operating system was shown at Computex Taipei, top executives from the company said the project will be put on the backburner. …

… “Frankly speaking, the first question, I would like to apologise that, if you look at Asus booth we’ve decided not to display this product,” he said. “I think you may have seen the devices on Qualcomm’s booth but actually, I think this is a company decision so far we would not like to show this device. That’s what I can tell you so far. I would like to apologise for that.”

He declined further comment on the subject.

Yeah right. Of course he did.

Next Page »