<html><div style='background-color:'><DIV class=RTE>
<P>Hi all.</P>
<P>I'm going on my 3rd week trying to get a simple traffic shapping to work the right way :( !!</P>
<P>My goal it to shape the traffic coming from one machine (pc1) to another machine (pc2) throught the "eth0" interface. My test configuration is as follows:</P>
<P>PC1</P>
<P> IP: 192.168.105.237<BR> Mask: 255.255.255.0<BR> OS: Red Hat Linux Kernel 2.4.20-8<BR> Rules:<BR>=========================================================================<BR> interface="eth0"</P>
<P> #Delete previous stored configuration<BR> tc qdisc del dev $interface root</P>
<P> #Creating the root Qdisc (Queueing Disk)<BR> tc qdisc add dev $interface root handle 1: htb default 12</P>
<P> #Definition of classes<BR> tc class add dev $interface parent 1: classid 1:1 htb rate 100kbit ceil 100kbit <BR> tc class add dev $interface parent 1:1 classid 1:10 htb rate 30kbit ceil 100kbit <BR> tc class add dev $interface parent 1:1 classid 1:11 htb rate 10kbit ceil 100kbit <BR> tc class add dev $interface parent 1:1 classid 1:12 htb rate 60kbit ceil 100kbit </P>
<P> tc qdisc add dev $interface parent 1:10 handle 20: sfq perturb 10<BR> tc qdisc add dev $interface parent 1:11 handle 30: sfq perturb 10<BR> tc qdisc add dev $interface parent 1:12 handle 40: sfq perturb 10</P>
<P> #Definition of the filters<BR> tc filter add dev $interface protocol ip parent 1:0 prio 1 u32 match ip src 192.168.105.237<BR> match ip dport 20000 0xffff flowid 1:10<BR> tc filter add dev $interface protocol ip parent 1:0 prio 1 u32 match ip src 192.168.105.237<BR> match ip dport 20001 0xffff flowid 1:11<BR>=========================================================================</P>
<P><BR>PC2:</P>
<P> IP: 192.168.105.211<BR> Mask: 255.255.255.0<BR> OS: Red Hat Linux Kernel 2.4.20-8</P>
<P><BR>I'm using the TG (<A href="http://www.postel.org/tg">www.postel.org/tg</A>) as a TCP traffic generator, to establish three 90kbits/s TCP flows from PC1(any port) to PC2(port 20000, 20001 and 20002), with different durations and pause times, which as can be shown in the next files:</P>
<P>=========================================================================<BR> on 0:2 tcp 192.168.105.211.20000<BR> at 3 setup<BR> at 5 arrival constant 0.0888889 length constant 1000<BR> time 3<BR> wait 3<BR> at 11 arrival constant 0.0888889 length constant 1000<BR> time 15<BR>=========================================================================</P>
<P>=========================================================================<BR> on 0:2 tcp 192.168.105.211.20001<BR> at 3 setup<BR> at 5 arrival constant 0.0888889 length constant 1000<BR> time 9<BR> wait 3<BR> at 17 arrival constant 0.0888889 length constant 1000<BR> time 6<BR> wait 2<BR>=========================================================================</P>
<P>=========================================================================<BR> on 0:2 tcp 192.168.105.211.20002<BR> at 3 setup<BR> at 5 arrival constant 0.0888889 length constant 1000<BR> time 15<BR> wait 10<BR>=========================================================================</P>
<P>I'm also using the TCPTRACE program, so that I can get some statistics from a captured file made with the help of ETHEREAL (capture filter="ip src 192.168.105.237 && port 20000 || port 20001 || port 20002"), installed on PC2.</P>
<P>Before adding the TC rules to the system, I've runned my test system and got the following output from the ethereal captured file using the command: "tcptrace -l <ethereal.file>":</P>
<P>=========================================================================<BR>=========================================================================<BR> 886 packets seen, 886 TCP packets traced<BR> elapsed wallclock time: 0:00:00.024537, 36108 pkts/sec analyzed<BR> trace file elapsed time: 0:00:23.034820<BR> TCP connection info:<BR> 3 TCP connections traced:<BR> TCP connection 1:<BR> host a: 192.168.105.237:36069<BR> host b: 192.168.105.211:20000<BR> complete conn: no (SYNs: 1) (FINs: 1)<BR> first packet: Sun Oct 16 16:21:21.332143 2005<BR> last packet: Sun Oct 16 16:21:44.366963 2005<BR> elapsed time: 0:00:23.034820<BR> total packets: 205<BR> filename:
linksharing.ethereal<BR> a->b: b->a:<BR> total packets: 205 total packets: 0 <BR> ack pkts sent: 204 ack pkts sent: 0 <BR> pure acks sent: 2 pure acks
sent: 0 <BR> sack pkts sent: 0 sack pkts sent: 0 <BR> dsack pkts sent: 0 dsack pkts sent: 0 <BR> max sack blks/ack: 0 max sack blks/ack:
0 <BR> unique bytes sent: 201000 unique bytes sent: 0 <BR> actual data pkts: 201 actual data pkts: 0 <BR> actual data bytes: 201000 actual data bytes: 0 <BR> rexmt data pkts:
0 rexmt data pkts: 0 <BR> rexmt data bytes: 0 rexmt data bytes: 0 <BR> zwnd probe pkts: 0 zwnd probe pkts: 0 <BR> zwnd probe bytes: 0 zwnd probe
bytes: 0 <BR> outoforder pkts: 0 outoforder pkts: 0 <BR> pushed data pkts: 200 pushed data pkts: 0 <BR> SYN/FIN pkts sent: 1/1 SYN/FIN pkts sent: 0/0 <BR> req
1323 ws/ts: Y/Y req 1323 ws/ts: N/N <BR> adv wind scale: 0 adv wind scale: 0 <BR> req sack: Y req sack: N <BR>
sacks sent: 0 sacks sent: 0 <BR> urgent data pkts: 0 pkts urgent data pkts: 0 pkts <BR> urgent data bytes: 0 bytes urgent data bytes: 0 bytes<BR> mss requested: 1460 bytes mss
requested: 0 bytes<BR> max segm size: 1448 bytes max segm size: 0 bytes<BR> min segm size: 552 bytes min segm size: 0 bytes<BR> avg segm size: 999 bytes avg segm size: 0 bytes<BR> max win adv: 5840
bytes max win adv: 0 bytes<BR> min win adv: 5840 bytes min win adv: 0 bytes<BR> zero win adv: 0 times zero win adv: 0 times<BR> avg win adv: 5840 bytes avg win adv: 0
bytes<BR> initial window: 201000 bytes initial window: 0 bytes<BR> initial window: 201 pkts initial window: 0 pkts <BR> ttl stream length: 201000 bytes ttl stream length: NA <BR> missed data: 0 bytes missed data:
NA <BR> truncated data: 0 bytes truncated data: 0 bytes<BR> truncated packets: 0 pkts truncated packets: 0 pkts <BR> data xmit time: 20.909 secs data xmit time: 0.000 secs <BR> idletime max: 3109.6 ms idletime max: NA
ms <BR> throughput: 8726 Bps throughput: 0 Bps <BR> ================================<BR> TCP connection 2:<BR> host c: 192.168.105.237:36070<BR> host d: 192.168.105.211:20001<BR> complete conn: yes<BR> first packet: Sun Oct 16 16:21:21.332520 2005<BR> last packet: Sun Oct 16 16:21:41.332416 2005<BR> elapsed time: 0:00:19.999896<BR> total packets: 340<BR> filename: linksharing.ethereal<BR> c->d:
d->c:<BR> total packets: 172 total packets: 168 <BR> ack pkts sent: 171 ack pkts sent: 168 <BR> pure acks sent: 2 pure acks sent: 166 <BR> sack pkts
sent: 0 sack pkts sent: 0 <BR> dsack pkts sent: 0 dsack pkts sent: 0 <BR> max sack blks/ack: 0 max sack blks/ack: 0 <BR> unique bytes sent:
168000 unique bytes sent: 0 <BR> actual data pkts: 168 actual data pkts: 0 <BR> actual data bytes: 168000 actual data bytes: 0 <BR> rexmt data pkts: 0 rexmt data pkts:
0 <BR> rexmt data bytes: 0 rexmt data bytes: 0 <BR> zwnd probe pkts: 0 zwnd probe pkts: 0 <BR> zwnd probe bytes: 0 zwnd probe bytes: 0 <BR> outoforder
pkts: 0 outoforder pkts: 0 <BR> pushed data pkts: 166 pushed data pkts: 0 <BR> SYN/FIN pkts sent: 1/1 SYN/FIN pkts sent: 1/1 <BR> req 1323 ws/ts: Y/Y req
1323 ws/ts: Y/Y <BR> adv wind scale: 0 adv wind scale: 0 <BR> req sack: Y req sack: Y <BR> sacks sent:
0 sacks sent: 0 <BR> urgent data pkts: 0 pkts urgent data pkts: 0 pkts <BR> urgent data bytes: 0 bytes urgent data bytes: 0 bytes<BR> mss requested: 1460 bytes mss requested: 1460 bytes<BR> max segm
size: 1448 bytes max segm size: 0 bytes<BR> min segm size: 552 bytes min segm size: 0 bytes<BR> avg segm size: 999 bytes avg segm size: 0 bytes<BR> max win adv: 5840 bytes max win adv: 64000
bytes<BR> min win adv: 5840 bytes min win adv: 8000 bytes<BR> zero win adv: 0 times zero win adv: 0 times<BR> avg win adv: 5840 bytes avg win adv: 60610 bytes<BR> initial window: 1000 bytes initial
window: 0 bytes<BR> initial window: 1 pkts initial window: 0 pkts <BR> ttl stream length: 168000 bytes ttl stream length: 0 bytes<BR> missed data: 0 bytes missed data: 0 bytes<BR> truncated data: 0 bytes
truncated data: 0 bytes<BR> truncated packets: 0 pkts truncated packets: 0 pkts <BR> data xmit time: 17.847 secs data xmit time: 0.000 secs <BR> idletime max: 3104.7 ms idletime max: 3019.2 ms <BR> throughput: 8400 Bps
throughput: 0 Bps <BR> ================================<BR> TCP connection 3:<BR> host e: 192.168.105.237:36071<BR> host f: 192.168.105.211:20002<BR> complete conn: yes<BR> first packet: Sun Oct 16 16:21:21.332771 2005<BR> last packet: Sun Oct 16 16:21:38.378520 2005<BR> elapsed time: 0:00:17.045748<BR> total packets: 341<BR> filename: linksharing.ethereal<BR> e->f: f->e:<BR> total packets: 172 total
packets: 169 <BR> ack pkts sent: 171 ack pkts sent: 169 <BR> pure acks sent: 2 pure acks sent: 167 <BR> sack pkts sent: 0 sack pkts
sent: 0 <BR> dsack pkts sent: 0 dsack pkts sent: 0 <BR> max sack blks/ack: 0 max sack blks/ack: 0 <BR> unique bytes sent: 168000 unique bytes sent: 0 <BR>
actual data pkts: 168 actual data pkts: 0 <BR> actual data bytes: 168000 actual data bytes: 0 <BR> rexmt data pkts: 0 rexmt data pkts: 0 <BR> rexmt data bytes: 0
rexmt data bytes: 0 <BR> zwnd probe pkts: 0 zwnd probe pkts: 0 <BR> zwnd probe bytes: 0 zwnd probe bytes: 0 <BR> outoforder pkts: 0 outoforder pkts:
0 <BR> pushed data pkts: 167 pushed data pkts: 0 <BR> SYN/FIN pkts sent: 1/1 SYN/FIN pkts sent: 1/1 <BR> req 1323 ws/ts: Y/Y req 1323 ws/ts: Y/Y <BR> adv wind
scale: 0 adv wind scale: 0 <BR> req sack: Y req sack: Y <BR> sacks sent: 0 sacks sent:
0 <BR> urgent data pkts: 0 pkts urgent data pkts: 0 pkts <BR> urgent data bytes: 0 bytes urgent data bytes: 0 bytes<BR> mss requested: 1460 bytes mss requested: 1460 bytes<BR> max segm size: 1448 bytes max segm size: 0
bytes<BR> min segm size: 552 bytes min segm size: 0 bytes<BR> avg segm size: 999 bytes avg segm size: 0 bytes<BR> max win adv: 5840 bytes max win adv: 64000 bytes<BR> min win adv: 5840 bytes min win
adv: 8000 bytes<BR> zero win adv: 0 times zero win adv: 0 times<BR> avg win adv: 5840 bytes avg win adv: 60630 bytes<BR> initial window: 1000 bytes initial window: 0 bytes<BR> initial window: 1
pkts initial window: 0 pkts <BR> ttl stream length: 168000 bytes ttl stream length: 0 bytes<BR> missed data: 0 bytes missed data: 0 bytes<BR> truncated data: 0 bytes truncated data: 0 bytes<BR> truncated packets: 0
pkts truncated packets: 0 pkts <BR> data xmit time: 14.919 secs data xmit time: 0.000 secs <BR> idletime max: 2112.8 ms idletime max: 2113.0 ms <BR> throughput: 9856 Bps throughput: 0 Bps </P>
<P>=========================================================================<BR>=========================================================================</P>
<P><BR> After adding the refered TC rules I've runned the configuration again, I can see that the number of captured packets has decreased a lot (almost half) !! So I've type the following TC command to find out what was effectively sent from PC1 to PC2, "tc -d -s class show dev eth0" and got the following output:</P>
<P>=========================================================================<BR> class htb 1:11 parent 1:1 leaf 30: prio 0 quantum 1000 rate 10Kbit ceil 100Kbit burst 1611b/8 mpu 0b cburst 1727b/8 mpu 0b level 0 <BR> Sent 42108 bytes 34 pkts (dropped 0, overlimits 0) <BR> lended: 30 borrowed: 4 giants: 0<BR> tokens: -758272 ctokens: 106496</P>
<P> class htb 1:1 root rate 100Kbit ceil 100Kbit burst 1727b/8 mpu 0b cburst 1727b/8 mpu 0b level 7 <BR> Sent 370941 bytes 295 pkts (dropped 0, overlimits 0) <BR> lended: 12 borrowed: 0 giants: 0<BR> tokens: 108032 ctokens: 108032</P>
<P> class htb 1:10 parent 1:1 leaf 20: prio 0 quantum 1000 rate 30Kbit ceil 100Kbit burst 1637b/8 mpu 0b cburst 1727b/8 mpu 0b level 0 <BR> Sent 122770 bytes 89 pkts (dropped 0, overlimits 0) <BR> lended: 81 borrowed: 8 giants: 0<BR> tokens: -415481 ctokens: 100352</P>
<P> class htb 1:12 parent 1:1 leaf 40: prio 0 quantum 1000 rate 60Kbit ceil 100Kbit burst 1675b/8 mpu 0b cburst 1727b/8 mpu 0b level 0 <BR> Sent 206063 bytes 172 pkts (dropped 0, overlimits 0) <BR> lended: 172 borrowed: 0 giants: 0<BR> tokens: 174507 ctokens: 108032<BR>=========================================================================</P>
<P>So we can see that before the TC rules, I Had about 537kbytes transfered (201k+168k+168k), after having applied the TC rules I've only got a total of 370.941 bytes !!!!!!!! Where have they gone ?</P>
<P>I've also plotted a graph with gnuplot, showing me that (somehow) my rules were "correct", I've got an average value of 30kbits/s, a 10kbits/s and a 60kbits/s.</P>
<P>Is it normal that some packets get dropped by the rules or not (taking into account my test configuration) ?</P>
<P>This is my last researched subject in my master thesis (<A href="http://www.fe.up.pt/si/teses_posgrad.tese?p_sigla=MEEC&P_ALU_NUMERO=030553029">www.fe.up.pt/si/teses_posgrad.tese?p_sigla=MEEC&P_ALU_NUMERO=030553029</A>), and this is the only thing that has kept me from finishing it !!!</P>
<P>Thanks lot for your answers !!</P>
<P>Best regards.</P>
<P><BR>Paulo Augusto<BR></P>
<P> </P></DIV></div></html>