Squid with the delay pools feature

Bandwith limiting HOWTO

• We have 115,2 kbits/s ppp (modem) internet link (115,2/10 = 11,5 kbytes/s).
  Note: with eth connections (network card) we would divide 115,2 by 8; with ppp we divide by 10, because of start/stop bits (8 + 1 + 1 = 10).
• We have some LAN stations and their users are doing bulk downloads all the time.
• We want web pages to open fast, no matter how many dowloads are happening.
• Our internet interface is ppp0.
• Our LAN interface is eth0.
• Our network is 192.168.1.0/24

---

2.1. What do we need

Believe it or not, shaping the incoming traffic is an easy task and you don't have to read tons of books about routing or queuing algorithms.
To make it work, we need at least Squid proxy; if we want to fine tune it, we will have to get familiar with ipchains or iptables and CBQ.
To test our efforts, we can install IPTraf.

---

2.2. How does it work?

Squid is probably the most advanced HTTP proxy server available for Linux. It can help us save bandwidth in two ways:

• The first is a main characteristic of proxy servers -- they keep downloaded web pages, pictures, and other objects in memory or on a disk. So, if two people are requesting the same web page, it isn't downloaded from the internet, but from the local proxy.
  
• Apart from normal caching, Squid has a special feature called delay pools. Thanks to delay pools, it is possible to limit internet traffic in a reasonable way, depending on so-called 'magic words', existing in any given URL. For example, a magic word could be '.mp3', '.exe' or '.avi', etc. Any distinct part of a URL (such as .avi) can be defined as a magic word.
  

With that, we can tell the Squid to download these kinds of files at a specified speed (in our example, it will be about 5 kbytes/s). If our LAN users download files at the same time, they will be downloaded at about 5 kbytes/s altogether, leaving remaining bandwidth for web pages, e-mail, news, irc, etc.
Of course, the Internet is not only used for downloading files via web pages (http or ftp). Later on, we will deal with limiting bandwidth for Napster, Realaudio, and other possibilities.

---

3. Installing and Configuring Necessary Software

---

3.1. Installing Squid with the delay pools feature

As I mentioned before, Squid has a feature called delay pools, which allows us to control download bandwidth. Unfortunately, in most distributions, Squid is shipped without that feature.
So if you have Squid already installed, I must disappoint you -- you need to uninstall it and do it once again with delay pools enabled in the way I explain below.
...
...

3.3. Solving remaining problems

OK, we have installed Squid and configured it to use delay pools. I bet nobody wants to be restricted, especially our clever LAN users. They will likely try to avoid our limitations, just to download their favourite mp3s a little faster (and thus causing your headache).
I assume that you use IP-masquerade on your LAN so that your users could use IRC, ICQ, e-mail, etc. That's OK, but we must make sure that our LAN users will use our delay pooled Squid to access web pages and use ftp.
We can solve most of these problems by using ipchains (Linux 2.2.x kernels) or iptables (Linux 2.4.x kernels).

---

3.3.1. Linux 2.2.x kernels (ipchains)

We must make sure that nobody will try to cheat and use a proxy server other than ours. Public proxies usually run on 3128 and 8080 ports:
/sbin/ipchains -A input -s 192.168.1.1/24 -d ! 192.168.1.1 3128 -p TCP -j REJECT
/sbin/ipchains -A input -s 192.168.1.1/24 -d ! 192.168.1.1 8080 -p TCP -j REJECT
We must also make sure that nobody will try to cheat and connect to the internet directly (IP-masquerade) to download web pages:
/sbin/ipchains -A input -s 192.168.1.1/24 -d ! 192.168.1.1 80 -p TCP -j REDIRECT 8080
If everything is working, we add these lines to the end of our initializing scripts. Usually, it can be /etc/rc.d/rc.local.
We might think to block ftp traffic (ports 20 and 21) to force our LAN users to use Squid, but it's not a good idea for at least two reasons:

• Squid is a http proxy with ftp support, not a real ftp proxy. It can download from ftp, it can also upload to some ftp, but it can't delete/change name of files on remote ftp servers.
  When we block ports 20 and 21, we won't be able to delete/change name of files on remote ftp servers.
  
• IE5.5 has a bug -- it doesn't use a proxy to retrieve the ftp directory. Instead it connects directly via IP-masquerade.
  When we block ports 20 and 21, we won't be able to browse through ftp directories, using IE5.5.
  

So, we will block excessive ftp downloads using other methods. We will deal with it in chapter 4.

---

3.3.2. Linux 2.4.x kernels (iptables)

We must make sure that nobody will try to cheat and use a proxy server other than ours. Public proxies usually run on 3128 and 8080 ports:
/sbin/iptables -A FORWARD -s 192.168.1.1/24 -d ! 192.168.1.1 --dport 3128 -p TCP -j DROP
/sbin/iptables -A FORWARD -s 192.168.1.1/24 -d ! 192.168.1.1 --dport 8080 -p TCP -j DROP
We must also make sure that nobody will try to cheat and connect to the internet directly (IP-masquerade) to download web pages:

/sbin/iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8080

If everything is working, we add these lines to the end of our initializing scripts. Usually, it can be /etc/rc.d/rc.local.
We might think to block ftp traffic (ports 20 and 21) to force our LAN users to use Squid, but it's not a good idea for at least two reasons:

• Squid is a http proxy with ftp support, not a real ftp proxy. It can download from ftp, it can also upload to some ftp, but it can't delete/change name of files on remote ftp servers.
  When we block ports 20 and 21, we won't be able to delete/change name of files on remote ftp servers.
  
• IE5.5 has a bug -- it doesn't use a proxy to retrieve the ftp directory. Instead it connects directly via IP-masquerade.
  When we block ports 20 and 21, our LAN users won't be able to browse through ftp directories, using IE5.5.
  

So, we will block excessive ftp downloads using other methods. We will deal with it in chapter 4.