[LARTC] Shaping traffic over a linux bridge

[Hugh Buchanan]

I have since fixed my problem.  I am not sure if it's useful to anyone, 
but I'll briefly describe what I did to get it working.

In the end, I am not using the 'iptables' program at all.  I am using 
ebtables to mangle the packets on eth1 as they come in.  What I was 
doing before with ebtables was mostly right.. I have pretty simple 
ebtable lines that mark the packets.  Packet marks can be 32 bits.

And my shaping is very different now too.. I shape on the outside (eth0) 
interface.. I can shape incoming traffic going over the eth1 interface 
as well, although I don't need it.

I also find that cbq doesn't work too happily.. HTB works like a charm, 
and really is more useful for my application.  I ended up needing to use 
the 'tc' binary supplied by the HTB people.

I am not completely done yet.. but at the very least it works.. it is 
now just a matter of setting up all of the classes/rules for the 200+ 
MAC addresses I'll be shaping.

Hugh

Hugh Buchanan wrote:

> I should add some additional comments.
>
> I have gone through most of the LARTC archives dealing with tc.. it 
> seems a lot of people have attempted this, but no one ever posts 
> solutions to these things.
>
> There are a bunch of archive posts I found somewhat helpful.
> http://lists.netfilter.org/pipermail/netfilter/2002-May/034041.html
> http://mailman.ds9a.nl/pipermail/lartc/2003q1/007571.html
>
> Using those two posts, I decided to give u32 filtering a try to avoid 
> the mangling confusion I have.. and it doesn't seem to change much.
>
> Shaping works perfectly using NAT/MASQUERADE and iptables mangling.  
> When I throw bridging into the mix it stops working.
>
> I have yet to try a 2.5.x kernel.. I would prefer not to, but I 
> suppose since I have no ideas right now on how to proceed, I better 
> try 2.5.x.
>
> If anyone figures this out, send me your address.. i'll send you a box 
> of cookies or something.
>
> Hugh
>
> Hugh Buchanan wrote:
>
>> Greetings,
>>
>> I am using tc/cbq to do some traffic shaping over a linux bridge.  My
>> system is running the 2.4.20 kernel with the latest bridge-nf and
>> ebtables patches.  I am also running ebtables 2.0.3 and iptables 1.2.8.
>>
>> Last week, since I had yet to discover ebtables/bridging I was testing
>> the shaping with a basic NAT setup using totally seperate networks on
>> each ethernet interface.
>>
>> I now have a fully functional bridge.  I was amazed to see how easy it
>> is!  I love it!
>>
>> But now I am trying to add tc back into the mix.  I have looked at the
>> simple and real life examples, and a typical mark line would be
>> something like
>> ebtables -A FORWARD -p ipv4 -i eth0 -j mark --set-mark 2 --mark-target
>> CONTINUE
>>
>> So now I have two questions.
>>
>> How many bits are available for marking?  On docum.org, I saw examples
>> for mark values up to '5', which leads me to assume that there are at
>> least 3 bits, meaning I can mark up to '7' (or is it '8'?).  What is the
>> numeric range here?
>>
>> And what I am wanting to do is possible, right?  I haven't found any
>> real life examples for traffic shaping over a bridge yet (links
>> appreciated).  I have found this though:
>> "When you create a bridge with the bridge-utils, you get a new device :
>> br0. You can shape traffic on this device, but you can not use iptables
>> to mark packets and the fw filter to use that mark. But you can use the
>> u32 filter."
>>
>> Does this mean I need to use some other process besides normal mangling?
>>
>> What it comes down to are two issues (that are almost the same).  I
>> don't know how to mark packets coming in from the LAN, and I don't know
>> which interface to bind tc to.
>>
>> Here is a diagram of my setup:
>>
>> 64.119.201.0/24 ---- eth1 [bridge, br0] eth0 ---- 64.119.201.1 (router)
>>
>> And here is the script I am playing with.  I have tried binding tc and
>> ebtables and iptables to all three interfaces (eth0,eth1,br0) and it
>> doesn't seem to change anything.. but then again I don't know if my
>> packets are even being marked.  Should I be using a 2.5.x kernel instead
>> of the patched 2.4.20?  I need this for production use.. stability is
>> important.
>>
>> #!/bin/sh
>> OUTSIDE="eth0"
>> INSIDE="eth1"
>> BRIDGE="br0"
>> LAN="64.119.201.0/24"
>> OPTION="allot 1514 maxburst 20 avpkt 1000 prio 3"
>>
>> ebtables -P INPUT ACCEPT
>> ebtables -P OUTPUT ACCEPT
>> ebtables -P FORWARD ACCEPT
>> ebtables -F
>> ebtables -t nat -F
>>
>> iptables -P INPUT ACCEPT
>> iptables -P OUTPUT ACCEPT
>> iptables -P FORWARD ACCEPT
>> iptables -F
>> iptables -t nat -F
>>
>> echo "Loading ebtables rules"
>> ebtables -A FORWARD -p ipv4 -i eth1 --ip-source 64.119.201.114 -j mark
>> --set-mark 2 --mark-target CONTINUE
>> ebtables -A FORWARD -p ipv4 -j mark --set-mark 1 --mark-target CONTINUE
>>
>> echo "Loading iptables rules"
>> iptables -A PREROUTING -i eth1 -s $LAN -t mangle -j MARK --set-mark 1
>> iptables -A PREROUTING -i eth1 -t mangle -j MARK --set-mark 2
>>
>> ###############################################################################
>> # i have tried $INSIDE, $OUTSIDE, and $BRIDGE here
>> DEV="dev $INSIDE"
>> RATE_TOT=10kbit
>> SERVERS=500kbit
>> SERVERS_WEIGHT=50kbit
>> GEN=50kbit
>> GEN_WEIGHT=5kbit
>>
>> echo "Clearing qdiscs"
>> tc qdisc del dev br0 root
>> tc qdisc del dev eth0 root
>> tc qdisc del dev eth1 root
>>
>> echo "Inserting qdiscs"
>> tc qdisc add $DEV root handle 10: cbq bandwidth $RATE_TOT avpkt 1000
>> tc class add $DEV parent 10:0 classid 10:2 cbq bandwidth $RATE_TOT rate
>> $RATE_TOT $OPTION bounded
>>
>> tc qdisc add $DEV parent 10:2 handle 20: cbq bandwidth $RATE_TOT allot
>> 1514 avpkt 1000
>> tc class add $DEV parent 20: classid 20:2 cbq bandwidth $RATE_TOT rate
>> $RATE_TOT $OPTION prio 3
>> tc class add $DEV parent 20:2 classid 20:10 cbq bandwidth $RATE_TOT rate
>> $SERVERS $OPTION weight $SERVERS_WEIGHT bounded
>> tc class add $DEV parent 20:2 classid 20:20 cbq bandwidth $RATE_TOT rate
>> $GEN     $OPTION weight $GEN_WEIGHT bounded
>>
>> echo "Adding tc filters"
>> tc filter add $DEV parent 10: protocol ip prio 3 handle 1 fw classid 
>> 10:2
>> tc filter add $DEV parent 10: protocol ip prio 3 handle 2 fw classid 
>> 10:2
>>
>> tc filter add $DEV parent 20: protocol ip prio 3 handle 1 fw classid 
>> 20:20
>> tc filter add $DEV parent 20: protocol ip prio 3 handle 2 fw classid 
>> 20:10
>>
>> echo "eth0:"
>> tc -s qdisc ls dev eth0
>> echo "eth1:"
>> tc -s qdisc ls dev eth1
>> echo "br0:"
>> tc -s qdisc ls dev br0
>>
>> Hugh Buchanan
>> Userfriendly.com
>