[LARTC] HTB doesn't respect rate values

Mon, 7 Jul 2003 20:38:04 +0200 (CEST)

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.
  Send mail to mime@docserver.cac.washington.edu for more info.

--8323328-908300613-1057603084=:18108
Content-Type: TEXT/PLAIN; charset=ISO-8859-2
Content-Transfer-Encoding: QUOTED-PRINTABLE

Well. The right way for your case would be to limit single
subqueue in SFQ. See line 24 of attached patch - and try patch
itself.
devik

On Mon, 7 Jul 2003, Sergiusz Brzezi=F1ski wrote:

> U=BFytkownik devik napisa=B3:
> >>I did pfifo with limit 10 and HTB started to work. I noticed drops and
> >>rate was OK. Sometimes (for 10-40 seconds) but seldom it worked bad (1:=
2
> >>got less, than it should) but there wasn't drops during this time. I
> >>tried this also with 12kbit at it was similar.
> >>
> >>If I good understand, there is something (TCP stack or what) whitch
> >>works BEVORE htb and this makes some connections slower or faster, so
> >>HTB has later nothing to do.
> >>
> >>The question for me is: how can I set it (the mechanism bevore HTB) to
> >>give HTB full control over the bandwitch? I don't wont to use pfifo (no=
t
> >>everywhere). I would like to use sfq for some classes.
> >
> >
> > you can try to play with /proc/sys/net/ipv4/tcp_{w,r,}mem. If wmem is
> > smaller than space in qdisc (15kB for 10 pfifo, approx 200kB for SFQ)
> > then TCP will back off before it fill the qdisc...
> > Also with sfq it should work - it is like pfifo with limit 128 - you
> > might need to increase wmem.
> > Note that the problem occur only when you have qdisc on the same machin=
e
> > as sending app.
>
> - I did this:
> # echo "4096 2048000 5120000" > /proc/sys/net/ipv4/tcp_wmem
> than I tried if in /proc/sys/net/ipv4/tcp_wmem there are really the new
> values (i set so high values because i wanted really be sure that the
> amount of memory will be OK :) - defaults where: 4096 16384 131072)
>
> Well, it helped in 80%. Why only in 80? I repeated my test with SFQ and:
>
> - it worked better than bevore, there where long time periods
> (15-20sec.) with right rate-values
> - but class whitch should become 98kbit still got sometimes only 38kbit;
> it happen seldom and short but it was a fact!
> - what was very strange: there were still no drops and no overlimits
> (!!!) in stats ("tc -s class show dev eth0"); in a test with "pfifo
> limit 10" I could see: when there were drops - the rate was OK, when
> there were no drops - the rate got lower than it should; with SFQ there
> were NO DROPS AT ALL so the question is: who (or what) really makes the
> whole work? It doesn't look like HTB-work.(have I right?)
>
> Should I also do something with "/proc/sys/net/ipv4/tcp_mem"?
> Is the min value in tcp_wmem (4096) OK?
> Do you have some more ideas?
>
> I would make some experiments but I'm really not familiar with this
> theme. So the only thing I can do is to ask YOU or someone else from the
> group.
>
> Sergiusz
>
> >>U=BFytkownik devik napisa=B3:
> >>
> >>>Interestingly from what I see HTB didn't come into play.
> >>>All drop and overlimits counters are zero. It seems that
> >>>www server haven't managed to send more.
> >>>Please try to add pfifo with limit 10 under both classes.
> >>>Because you are sending from the same computer, your
> >>>TCP stack uses send queue management which counts packets
> >>>in qdisc and backs off. It MIGHT cause problems ...
> >>>
> >>>-------------------------------
> >>>    Martin Devera aka devik
> >>>Linux kernel QoS/HTB maintainer
> >>>  http://luxik.cdi.cz/~devik/
> >>>
> >>>On Sat, 5 Jul 2003, Sergiusz Brzezi=F1ski wrote:
> >>>
> >>>
> >>>
> >>>>Thak you for your hints!
> >>>>
> >>>>
> >>>>>1) 6kbit is really too small it should be at least 10 ..
> >>>>
> >>>>I tried with 12, 20 and even with 30kbit for 1:3
> >>>>
> >>>>I noticed, that it work for some seconds (or 1-2 minutes) but than th=
e
> >>>>1:3 class gets more then it should get :(.
> >>>>
> >>>>
> >>>>
> >>>>>2) it should workeven with 6k:
> >>>>> - look at stats (tc -s class show dev eth0) before and
> >>>>>   after the test - you are interested in drops. Also try
> >>>>>   it during the test to look whether queues are build up.
> >>>>>
> >>>>
> >>>>I made a test with settings:
> >>>>---------------------------------
> >>>>
> >>>>tc qdisc del root dev eth0
> >>>>tc qdisc add dev eth0 root handle 1:0 htb default 3
> >>>>
> >>>>tc class add dev eth0 parent 1:0 classid 1:1 htb rate 128kbit ceil
> >>>>128kbit burst 20kbit
> >>>>
> >>>>tc class add dev eth0 parent 1:1 classid 1:2 htb rate 98kbit ceil
> >>>>128kbit quantum 4900 burst 20kbit
> >>>>
> >>>>tc class add dev eth0 parent 1:1 classid 1:3 htb rate 30kbit ceil
> >>>>128kbit quantum 1500
> >>>>
> >>>>tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip spo=
rt
> >>>>80 0xffff flowid 1:2
> >>>>
> >>>>Bevore test: (reseted htb)
> >>>>--------------------------------
> >>>># tc -s class show dev eth0
> >>>>
> >>>>class htb 1:1 root rate 128Kbit ceil 128Kbit burst 2559b cburst 1762b
> >>>> Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
> >>>> lended: 0 borrowed: 0 giants: 0
> >>>> tokens: 244140 ctokens: 168131
> >>>>
> >>>>class htb 1:2 parent 1:1 prio 0 rate 98Kbit ceil 128Kbit burst 2559b
> >>>>cburst 1762b
> >>>> Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
> >>>> lended: 0 borrowed: 0 giants: 0
> >>>> tokens: 318876 ctokens: 168131
> >>>>
> >>>>class htb 1:3 parent 1:1 prio 0 rate 30Kbit ceil 128Kbit burst 1637b
> >>>>cburst 1762b
> >>>> Sent 0 bytes 0 pkts (dropped 0, overlimits 0)
> >>>> lended: 0 borrowed: 0 giants: 0
> >>>> tokens: 666503 ctokens: 168131
> >>>>
> >>>>After test:
> >>>>------------
> >>>>class htb 1:1 root rate 128Kbit ceil 128Kbit burst 2559b cburst 1762b
> >>>> Sent 5843869 bytes 4715 pkts (dropped 0, overlimits 0)
> >>>> rate 15427bps 12pps
> >>>> lended: 1461 borrowed: 0 giants: 0
> >>>> tokens: -21142 ctokens: -97151
> >>>>
> >>>>class htb 1:2 parent 1:1 prio 0 rate 98Kbit ceil 128Kbit burst 2559b
> >>>>cburst 1762b
> >>>> Sent 2735702 bytes 1811 pkts (dropped 0, overlimits 0)
> >>>> rate 6397bps 4pps
> >>>> lended: 1802 borrowed: 9 giants: 0
> >>>> tokens: 312898 ctokens: 163555
> >>>>
> >>>>class htb 1:3 parent 1:1 prio 0 rate 30Kbit ceil 128Kbit burst 1637b
> >>>>cburst 1762b
> >>>> Sent 3108167 bytes 2904 pkts (dropped 0, overlimits 0)
> >>>> rate 9488bps 8pps
> >>>> lended: 1452 borrowed: 1452 giants: 0
> >>>> tokens: -561135 ctokens: -97151
> >>>>
> >>>>Description of the test:
> >>>>------------------------
> >>>>On the beginning it was everything OK, after 1 min, 1:2 lost his 98kb=
it.
> >>>>Than he got sometimes his 98kbit again and sometimes he got even 30kb=
it.
> >>>>
> >>>>
> >>>>1. Can I do something more to find out what happen?
> >>>>2. What does mean: "queues are build up" ?
> >>>>
> >>>>Sergiusz
> >>>>
> >>
> >>
> >>
> >
> >
>
>
>
>

--8323328-908300613-1057603084=:18108
Content-Type: TEXT/PLAIN; charset=US-ASCII; name="sfq.diff"
Content-Transfer-Encoding: BASE64
Content-ID: <Pine.LNX.4.33.0307072038040.18108@devix>
Content-Description: 
Content-Disposition: attachment; filename="sfq.diff"

ZGlmZiAtcnViQiAvdXNyL3NyYy9saW51eC0yLjIuMTYvbmV0L3NjaGVkL3Nj
aF9zZnEuYyBnYXRlay9uZXQvc2NoZWQvc2NoX3NmcS5jDQotLS0gL3Vzci9z
cmMvbGludXgtMi4yLjE2L25ldC9zY2hlZC9zY2hfc2ZxLmMJU3VuIEFwciAy
NSAwMjo1MTo0OCAxOTk5DQorKysgZ2F0ZWsvbmV0L3NjaGVkL3NjaF9zZnEu
YwlTdW4gU2VwIDI0IDE1OjQ1OjUwIDIwMDANCkBAIC0xMTEsNiArMTExLDgg
QEANCiAJaW50CQlwZXJ0dXJiYXRpb247DQogCXNmcV9pbmRleAl0YWlsOwkJ
LyogSW5kZXggb2YgY3VycmVudCBzbG90IGluIHJvdW5kICovDQogCXNmcV9p
bmRleAltYXhfZGVwdGg7CS8qIE1heGltYWwgZGVwdGggKi8NCisJc2hvcnQJ
CWZsb3dzOwkJLyogTnVtIG9mIGZsb3dzICovDQorCXNob3J0CQltYXhfZmxv
d3M7CS8qIE1heCBudW0gb2YgZmxvd3MgKi8NCiANCiAJc2ZxX2luZGV4CWh0
W1NGUV9IQVNIX0RJVklTT1JdOwkvKiBIYXNoIHRhYmxlICovDQogCXNmcV9p
bmRleAluZXh0W1NGUV9ERVBUSF07CS8qIEFjdGl2ZSBzbG90cyBsaW5rICov
DQpAQCAtMjY1LDYgKzI2Nyw3IEBADQogCV9fc2tiX3F1ZXVlX3RhaWwoJnEt
PnFzW3hdLCBza2IpOw0KIAlzZnFfaW5jKHEsIHgpOw0KIAlpZiAocS0+cXNb
eF0ucWxlbiA9PSAxKSB7CQkvKiBUaGUgZmxvdyBpcyBuZXcgKi8NCisJICAg
IAlpZiAoKytxLT5mbG93cyA+IHEtPm1heF9mbG93cykgcS0+bWF4X2Zsb3dz
ID0gcS0+Zmxvd3M7DQogCQlpZiAocS0+dGFpbCA9PSBTRlFfREVQVEgpIHsJ
LyogSXQgaXMgdGhlIGZpcnN0IGZsb3cgKi8NCiAJCQlxLT50YWlsID0geDsN
CiAJCQlxLT5uZXh0W3hdID0geDsNCkBAIC0yNzYsMTEgKzI3OSwxMyBAQA0K
IAkJfQ0KIAl9DQogCWlmICgrK3NjaC0+cS5xbGVuIDwgU0ZRX0RFUFRILTEp
IHsNCisJCS8vIGFyYml0cmFyeSBoYWNrIHRvIGxpbWl0IG1heGltYWwgZmxv
dydzIGJhY2tsb2cNCisJCWlmIChxLT5xc1t4XS5xbGVuID4gNSkgZ290byBk
cm9wOw0KIAkJc2NoLT5zdGF0cy5ieXRlcyArPSBza2ItPmxlbjsNCiAJCXNj
aC0+c3RhdHMucGFja2V0cysrOw0KIAkJcmV0dXJuIDE7DQogCX0NCi0NCitk
cm9wOg0KIAlzZnFfZHJvcChzY2gpOw0KIAlyZXR1cm4gMDsNCiB9DQpAQCAt
MzAwLDYgKzMwNSw3IEBADQogCV9fc2tiX3F1ZXVlX2hlYWQoJnEtPnFzW3hd
LCBza2IpOw0KIAlzZnFfaW5jKHEsIHgpOw0KIAlpZiAocS0+cXNbeF0ucWxl
biA9PSAxKSB7CQkvKiBUaGUgZmxvdyBpcyBuZXcgKi8NCisJICAgIAlpZiAo
KytxLT5mbG93cyA+IHEtPm1heF9mbG93cykgcS0+bWF4X2Zsb3dzID0gcS0+
Zmxvd3M7DQogCQlpZiAocS0+dGFpbCA9PSBTRlFfREVQVEgpIHsJLyogSXQg
aXMgdGhlIGZpcnN0IGZsb3cgKi8NCiAJCQlxLT50YWlsID0geDsNCiAJCQlx
LT5uZXh0W3hdID0geDsNCkBAIC0zNDEsNiArMzQ3LDcgQEANCiANCiAJLyog
SXMgdGhlIHNsb3QgZW1wdHk/ICovDQogCWlmIChxLT5xc1thXS5xbGVuID09
IDApIHsNCisJICAgIAlxLT5mbG93cy0tOw0KIAkJYSA9IHEtPm5leHRbYV07
DQogCQlpZiAoYSA9PSBvbGRfYSkgew0KIAkJCXEtPnRhaWwgPSBTRlFfREVQ
VEg7DQpAQCAtNDE3LDYgKzQyNCw4IEBADQogCQlxLT5kZXBbaStTRlFfREVQ
VEhdLnByZXYgPSBpK1NGUV9ERVBUSDsNCiAJfQ0KIAlxLT5tYXhfZGVwdGgg
PSAwOw0KKwlxLT5tYXhfZmxvd3MgPSAwOw0KKwlxLT5mbG93cyA9IDA7DQog
CXEtPnRhaWwgPSBTRlFfREVQVEg7DQogCWlmIChvcHQgPT0gTlVMTCkgew0K
IAkJcS0+cXVhbnR1bSA9IHBzY2hlZF9tdHUoc2NoLT5kZXYpOw0KQEAgLTQ1
MCw4ICs0NTksOCBAQA0KIAlvcHQucGVydHVyYl9wZXJpb2QgPSBxLT5wZXJ0
dXJiX3BlcmlvZC9IWjsNCiANCiAJb3B0LmxpbWl0ID0gU0ZRX0RFUFRIOw0K
LQlvcHQuZGl2aXNvciA9IFNGUV9IQVNIX0RJVklTT1I7DQotCW9wdC5mbG93
cyA9IFNGUV9ERVBUSDsNCisJb3B0LmRpdmlzb3IgPSBxLT5tYXhfZmxvd3M7
DQorCW9wdC5mbG93cyA9IHEtPmZsb3dzOw0KIA0KIAlSVEFfUFVUKHNrYiwg
VENBX09QVElPTlMsIHNpemVvZihvcHQpLCAmb3B0KTsNCiANCg==
--8323328-908300613-1057603084=:18108--