TorrentFlux is a great program/interface to download your torrents remotely on a linux machine. It is based on php and it uses a modified bittornado client to download the torrents.
The problem: The bittornado client is able to put specific limits on a per torrent basis. That means that you can put a 100kb/sec download and 50kb/sec upload limit per torrent through torrentflux’s web interface. If you have 20 torrents though, this easily becomes 20*50=1Mb/sec upload “limit”. There are cases that you don’t want this to happen and you want both a per torrent limit (eg 50kb/sec) and a global limit (eg 300kb/sec).
The solution: My solution is based on iptables, layer7 filter and tc (iproute2). I am using layer7 filter to pick out the bittorrent packets, iptables to mark those packets with specific values and tc to shape those marked packets into categories. Beware that the method I am using works mostly on the “uploading” part (outgoing traffic). It is not that hard to make it work for the incoming traffic as well, but it is my personal view that downloading with a few Mb/sec is not as harmfull as uploading with a few Mb/sec. I usually have my downloads seeded over many weeks…so it’s good for my ratio to have the torrent downloaded as fast as possible and then seed it endlessly. I usually like to seed until i get a ratio over 1000% per torrent (that means 10 times as much uploaded traffic than downloaded). The following example configs are created for use on a 100mbit line and keeping in mind that outgoing torrent traffic should not exceed 2-2.5Mbits (~250-300kb/sec).
#emerge -avt net-misc/l7-filter net-misc/l7-protocols
Here’s how my netfilter configuration looks like:
CONFIG_NETFILTER=y# CONFIG_NETFILTER_DEBUG is not set# CONFIG_BRIDGE_NETFILTER is not set## Core Netfilter Configuration## CONFIG_NETFILTER_NETLINK is not setCONFIG_NETFILTER_XTABLES=yCONFIG_NETFILTER_XT_TARGET_CLASSIFY=m# CONFIG_NETFILTER_XT_TARGET_CONNMARK is not setCONFIG_NETFILTER_XT_TARGET_MARK=mCONFIG_NETFILTER_XT_TARGET_NFQUEUE=mCONFIG_NETFILTER_XT_MATCH_COMMENT=mCONFIG_NETFILTER_XT_MATCH_CONNBYTES=mCONFIG_NETFILTER_XT_MATCH_CONNMARK=mCONFIG_NETFILTER_XT_MATCH_CONNTRACK=mCONFIG_NETFILTER_XT_MATCH_DCCP=mCONFIG_NETFILTER_XT_MATCH_ESP=mCONFIG_NETFILTER_XT_MATCH_HELPER=mCONFIG_NETFILTER_XT_MATCH_LENGTH=mCONFIG_NETFILTER_XT_MATCH_LIMIT=mCONFIG_NETFILTER_XT_MATCH_MAC=mCONFIG_NETFILTER_XT_MATCH_MARK=m# CONFIG_NETFILTER_XT_MATCH_POLICY is not setCONFIG_NETFILTER_XT_MATCH_MULTIPORT=mCONFIG_NETFILTER_XT_MATCH_PKTTYPE=m# CONFIG_NETFILTER_XT_MATCH_QUOTA is not setCONFIG_NETFILTER_XT_MATCH_REALM=mCONFIG_NETFILTER_XT_MATCH_SCTP=mCONFIG_NETFILTER_XT_MATCH_STATE=m# CONFIG_NETFILTER_XT_MATCH_STATISTIC is not setCONFIG_NETFILTER_XT_MATCH_STRING=mCONFIG_NETFILTER_XT_MATCH_TCPMSS=m## IP: Netfilter Configuration#CONFIG_IP_NF_CONNTRACK=mCONFIG_IP_NF_CT_ACCT=yCONFIG_IP_NF_CONNTRACK_MARK=y# CONFIG_IP_NF_CONNTRACK_EVENTS is not setCONFIG_IP_NF_CT_PROTO_SCTP=mCONFIG_IP_NF_FTP=mCONFIG_IP_NF_IRC=mCONFIG_IP_NF_NETBIOS_NS=mCONFIG_IP_NF_TFTP=mCONFIG_IP_NF_AMANDA=mCONFIG_IP_NF_PPTP=mCONFIG_IP_NF_H323=mCONFIG_IP_NF_SIP=mCONFIG_IP_NF_QUEUE=mCONFIG_IP_NF_IPTABLES=yCONFIG_IP_NF_MATCH_IPRANGE=yCONFIG_IP_NF_MATCH_LAYER7=m# CONFIG_IP_NF_MATCH_LAYER7_DEBUG is not setCONFIG_IP_NF_MATCH_TOS=yCONFIG_IP_NF_MATCH_RECENT=mCONFIG_IP_NF_MATCH_ECN=mCONFIG_IP_NF_MATCH_DSCP=mCONFIG_IP_NF_MATCH_AH=mCONFIG_IP_NF_MATCH_TTL=mCONFIG_IP_NF_MATCH_OWNER=mCONFIG_IP_NF_MATCH_ADDRTYPE=mCONFIG_IP_NF_MATCH_HASHLIMIT=mCONFIG_IP_NF_FILTER=yCONFIG_IP_NF_TARGET_REJECT=yCONFIG_IP_NF_TARGET_LOG=yCONFIG_IP_NF_TARGET_ULOG=mCONFIG_IP_NF_TARGET_TCPMSS=yCONFIG_IP_NF_NAT=mCONFIG_IP_NF_NAT_NEEDED=yCONFIG_IP_NF_TARGET_MASQUERADE=mCONFIG_IP_NF_TARGET_REDIRECT=mCONFIG_IP_NF_TARGET_NETMAP=mCONFIG_IP_NF_TARGET_SAME=mCONFIG_IP_NF_NAT_SNMP_BASIC=mCONFIG_IP_NF_NAT_IRC=mCONFIG_IP_NF_NAT_FTP=mCONFIG_IP_NF_NAT_TFTP=mCONFIG_IP_NF_NAT_AMANDA=mCONFIG_IP_NF_NAT_PPTP=mCONFIG_IP_NF_NAT_H323=mCONFIG_IP_NF_NAT_SIP=mCONFIG_IP_NF_MANGLE=mCONFIG_IP_NF_TARGET_TOS=mCONFIG_IP_NF_TARGET_ECN=mCONFIG_IP_NF_TARGET_DSCP=mCONFIG_IP_NF_TARGET_TTL=mCONFIG_IP_NF_TARGET_CLUSTERIP=m# CONFIG_IP_NF_RAW is not setCONFIG_IP_NF_ARPTABLES=mCONFIG_IP_NF_ARPFILTER=mCONFIG_IP_NF_ARP_MANGLE=m
#echo "net-firewall/iptables extensions l7filter" >> /etc/portage/package.use
#emerge -avt net-firewall/iptables sys-apps/iproute2
# Generated by iptables-save v1.3.5 on Fri Jan 12 20:50:52 2007
*mangle
:PREROUTING ACCEPT [1102387:193393325]
:INPUT ACCEPT [1102372:193390208]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [2100485:2922693566]
:POSTROUTING ACCEPT [2100483:2922690566]
-A PREROUTING -s IP.OF.MACHINE -p tcp -m multiport --sports 22,80 -j MARK --set-mark 1001
-A PREROUTING -d IP.OF.MACHINE -p tcp -m multiport --dports 22,80 -j MARK --set-mark 1001
-A PREROUTING -m layer7 --l7proto ssh -j MARK --set-mark 1001
#-A PREROUTING -m layer7 --l7proto bittorrent -j MARK --set-mark 11090
-A PREROUTING -m mark --mark 1001 -j RETURN
-A POSTROUTING -s IP.OF.MACHINE -p tcp -m multiport --sports 22,80 -j MARK --set-mark 1001
-A POSTROUTING -d IP.OF.MACHINE -p tcp -m multiport --dports 22,80 -j MARK --set-mark 1001
-A POSTROUTING -m mark --mark 1001 -j RETURN
-A POSTROUTING -m connmark --mark 0x0 -j MARK --set-mark 11030
-A POSTROUTING -m layer7 --l7proto dns -j MARK --set-mark 11010
-A POSTROUTING -m layer7 --l7proto ssh -j MARK --set-mark 11010
-A POSTROUTING -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j MARK --set-mark 11010
-A POSTROUTING -p icmp -j MARK --set-mark 11010
-A POSTROUTING -m layer7 --l7proto bittorrent -j MARK --set-mark 11090
COMMIT
# Completed on Fri Jan 12 20:50:52 2007
# Generated by iptables-save v1.3.5 on Fri Jan 12 20:50:52 2007
*nat
:PREROUTING ACCEPT [407:30699]
:POSTROUTING ACCEPT [111:6662]
:OUTPUT ACCEPT [111:6662]
COMMIT
# Completed on Fri Jan 12 20:50:52 2007
# Generated by iptables-save v1.3.5 on Fri Jan 12 20:50:52 2007
*filter
:INPUT ACCEPT [266369:32040284]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [479227:676859047]
COMMIT
# Completed on Fri Jan 12 20:50:52 2007
You need to change IP.OF.MACHINE with the IP of your linux box.
# Main Link
LINK=100000
SHAPEDLINK=50000
# High Priority
HIGHPRIO=10000
HIGHPRIO_MAX=$SHAPEDLINK
# Normal
NORMAL=512
NORMAL_MAX=$SHAPEDLINK
# Downloads
TOR=512
TOR_MAX=2048
# del old
tc qdisc del dev $DEV root 2> /dev/null > /dev/null
# add root
tc qdisc add dev $DEV root handle 100: htb default 1
tc class add dev $DEV parent 100: classid 100:1 htb rate ${LINK}kbit
tc qdisc add dev $DEV parent 100:1 handle 1: htb
tc class add dev $DEV parent 1: classid 1:1 htb rate ${SHAPEDLINK}kbit
# some more rules
tc class add dev $DEV parent 100: classid 100:1 htb rate ${LINK}kbit
tc qdisc add dev $DEV parent 100:1 sfq perturb 10
tc filter add dev $DEV parent 100:0 protocol ip prio 1 handle 1001 fw flowid 100:1
tc class add dev $DEV parent 1:1 classid 1:10 htb rate ${SHAPEDLINK}kbit ceil ${SHAPEDLINK}kbit prio 5
tc qdisc add dev $DEV parent 1:10 sfq perturb 10
# High priority
tc class add dev $DEV parent 1:10 classid 1:1010 htb rate ${HIGHPRIO}kbit ceil ${HIGHPRIO_MAX}kbit prio 0
tc qdisc add dev $DEV parent 1:1010 sfq perturb 10
tc filter add dev $DEV parent 1:0 protocol ip prio 0 handle 11010 fw flowid 1:1010
# normal
tc class add dev $DEV parent 1:10 classid 1:1030 htb rate ${NORMAL}kbit ceil ${NORMAL_MAX}kbit prio 5
tc qdisc add dev $DEV parent 1:1030 sfq perturb 10
tc filter add dev $DEV parent 1:0 protocol ip prio 5 handle 11030 fw flowid 1:1030
# bittorent
tc class add dev $DEV parent 1:10 classid 1:1090 htb rate ${TOR}kbit ceil ${TOR_MAX}kbit prio 10
tc qdisc add dev $DEV parent 1:1090 sfq perturb 10
tc filter add dev $DEV parent 1:0 protocol ip prio 10 handle 11090 fw flowid 1:1090
The rules are pretty straightforward…so I am not going to fully explain them. The basic concept is that you create a “shaped” partition of your bandwith and you add classes (high priority, normal , bittorrent) there. The trick is that you can skip anything you don’t want shaped by marking it with iptables 1001 mark.
In my iptables example above, I mark as 1001 the outgoing ssh and http traffic. This way I can shape the seeding of my torrents using TorrentFlux but I can download via http without any traffic shaping the torrents to my PC at home. I can also ssh to the machine without any latency caused by the shaping because the sshd port (22) is marked with 1001.
The only problem I faced with those scripts was that sometimes the layer7 filter for bittorrent let’s some torrent traffic pass by. My solution to that was to change NORMAL_MAX=$SHAPEDLINK to NORMAL_MAX=2048 for example. Then, even “normal traffic” was shaped. Remember that anything I didn’t want shaped, was marked as 1001 on the iptables script…so the machine was still very responsive even after shaping the “normal traffic”.
To check how your scripts are doing in terms of shaping you can download this excellent perl script: tc-viewer. Click here for a screenshot: tc-viewer htb screenshot
The above example configs are very very generic. If you have a server that serves many other duties apart from ssh, http and bittorrent, then this script might not work out of the box for you.
-A POSTROUTING -p tcp --sport 61000:63000 -j MARK --set-mark 11090
-A POSTROUTING -m layer7 --l7proto bittorrent -j MARK --set-mark 11090
Enjoy shaped encrypted bittorent uploads! Keep seeding…
The testing platform we are using today is an HP Vectra slimline PC. Considering the computer was FREE (as in beer) after a company upgraded their workstations, the specifications are nothing to scoff at.
is a BEAST - in a good way. It’s really hungry for a faster processor than I can throw at it. The list of features really blow everything out of the water. It’s not just a router or firewall platform, it’s like someone asked themselves a question: “What is EVERYTHING a small office could EVER need in a networking server?” ClarkConnect provides three different robust VPN connectivity solutions using IPSec, PPTP, OpenVPN, along with web proxy and web filtering. Additionally, it provides an SSH server, Quality of Service (QoS) filtering for common P2P applications, Intrusion Detection, and much, much more including email server, file, print, database and web serving. Not to mention a fairly comprehensive group ware suite, which has calendar, contact, tasks lists, and provides a paid option for using Microsoft Outlook Connector to allow everything to go right into Microsoft Office Outlook.
to be the baseline for features, usability and extensibility. The installation CD is simple, but employs a non-linear configuration that some may have difficulty using the first time around. A nice touch is including MemTest86 on the CD and including that as an option on the initial bootup. The program will systematically test your RAM and determine if there is a fault, and as a computer gets older, the likelihood of that happening becomes more of a reality.
is by far, the smallest of the bunch. The entire thing is contained in a measly 8 MB CD image! monowall is first and foremost, a routing platform. Nothing more, nothing less. The distribution comes in two flavors, either for embedded systems or for regular PCs. Installation the first time around may be difficult for a beginner, since it refers to network cards by their FreeBSD driver name, instead of something a human can easily interpret. Which is easier to understand: “fxp0″ or “Intel Pro 10/100+”? Why not provide both peices of information to the user?
is a hybrid of sorts, that has multiple sources for it’s major components. It was originally derived from monowall, but uses OpenBSD’s ported Packet Filter, a package management system to provide an integrated extensibility to the platform and Alternate Queuing (ALTQ) from FreeBSD. This Frankenstein is no slouch when it comes to performance, features and usability.
installation is simplistic, and the GREEN/RED interface descriptions are an easy idea to grasp. One of the best features is a Java SSH client that runs right in the web interface - slick. Smoothwall’s VPN is designed to connect multiple Smoothwalls to each other, but IPSec is supported fully, and addons can be found for other VPN implementations.
“is very easy to install, use and manage, without losing its flexibility.” I had a completely different experience. Although Endian is only 106 MB and would easily fit within the 1GB limitation of our testbed, installation failed at 96% - reporting that there was not enough space on the drive.
is a close match to every other distribution we’ve looked at so far, with a few nice touches. Their website says that they have the following feature at first look, seems pretty kickass: “Anonymisation Gateway: The Gibraltar Anonymisation Gateway makes your overall network traffic anonymous and it makes sure you can surf in the internet anonymously.”

