[LARTC] Request for guidance

[Jim Lloyd]

Hello all,

I have leased 1/3rd of a rack (14U space) in a top notch data center.  
I'll be racking a layer 2 managed switch (a Dell PowerConnect 5224),  
four of my own servers (1U Opteron servers, single socket, dual core,  
dual NIC), as well 2 servers for 2 friends who will be subleasing  
from me. The package includes 6Mbps of bandwidth, burstable to  
100Mbps. Bandwidth is tracked with 5-min samples, and as long as my  
95th percentile is less than 6Mbps each month, there is no extra  
charge for bandwidth.

I would like to use tc bandwidth shaping so that I can
1) ensure that I never have to pay for extra bandwidth in any month
2) be able to guarantee all servers a predefined minimum slice of  
bandwidth

I am a software engineer and have only in the last couple years  
acquired some spotty knowledge of advanced networking concepts. I  
have been pouring over available documentation the last several days  
and it is very clear that I can satisfy my minimum requirements quite  
easily. However, its also clear that there is the potential for me to  
do some very fancy things that might be too fancy for my own good.  
So, I am looking for a little guidance from some experts willing to  
steer me in the right direction.

For example, I have a choice between setting up one server as either  
a router or a bridge. The bridge approach seems quite interesting/ 
powerful, but I wonder if it would introduce unnecessary complexity  
that I would later regret. So far, it seems like the main advantage  
of a bridge is that if it has problems, I can easily bypass it.  
Otherwise, there is just the coolness factor of having a transparent  
firewall.

I may want to carve up the /25 network assigned to me by the data  
center into some smaller networks (a /28 network for each of my  
friends, a /26 network for me), each with their own VLAN, so that  
with one firewall I can protect all servers from external attacks,  
but also protect my subleasers from each other. I can probably get my  
host to carve up the /25 network for me. If not, then I am forced to  
be a router. At first I thought this precluded configuring as a  
bridge, but now I see that I can configure a server as both a router  
and a bridge. I have a strong suspicion that is too fancy for my own  
good.

One question I have is not so much about linux routing & traffic  
control, but instead a question about VLANs. If I configure server as  
a bridge, it needs to be logically between the data center's upstream  
router, and my layer 2 switch. I can of course do that by instructing  
the datacenter to do the physical cabling that way. However, if I  
understand VLANs correctly, I can also just instruct the datacenter  
to cable everything to my switch. I would then make a two-port VLAN  
between the upstream router and the external interface of my bridge,  
and should get the same effect. Is that correct?

The following are two things I am interested in trying to do in the  
future (if possible), but should probably wait to do until I have had  
some experience with a simpler configuration, but I would like  
mention now anyway. One reason is that if I don't do them now, I  
can't test them while I still have the servers in my possession,  
where I can most easily recover from mistakes. How risky will it be  
to make changes like the following to the setup remotely, if I want  
to minimize the chance of paying a sysadmin at the datacenter $100/ 
hour to help me recover from a mistake?

It is possible that I will run some p2p service from one or more  
servers. If so, there may be as much of a need to control inbound  
bandwidth as there is to control outbound bandwidth. I understand  
that one can't do shaping on ingress. From the documentation I have  
seen so far, I haven't seen a clear example of controlling inbound  
bandwidth to a bridge via an  egress qdisc on the internal interface.  
If I do that, should I use RED for that purpose?

Finally, there is one thing that it would be nice to be able to do in  
the future, which is to try to do my bandwidth shaping based on the 5- 
min samples and 95th percentile measurements, and ideally understand  
the monthly billing cycle. Suppose that without bandwidth shaping my  
95%-ile for one month would be 10Mbps or more. Is there a way to do  
bandwidth shaping so that I can allow 4.5% of my traffic in a month  
to be unmodified, but still have my 95%-ile be just under the 6Mbps  
limit?

FYI, I'll most likely be running Fedora Core 5, x86_64 on my servers,  
including the one that serves as the firewall/(router|bridge), unless  
someone here has good reason to steer me to a different configuration.

Thanks in advance for any guidance. I plan to write up my  
configuration and lessons learned and will of course give credit in  
that write-up to all that contribute.

Jim

p.s. I have room for one more server in the rack in case anyone is  
interested in subleasing. I'm not looking to profit from subleasing,  
so your share of the cost would be a just prorated share of the total  
cost. Contact me privately at this email address if interested.