Why can't you queue download traffic on an interface? The reason openbsd.org's FAQ gives is:
"Note that queueing is only useful for packets in the outbound direction. Once a packet arrives on an interface in the inbound direction it's already too late to queue it -- it's already consumed network bandwidth to get to the interface that just received it. The only solution is to enable queueing on the adjacent router or, if the host that received the packet is acting as a router, to enable queueing on the internal interface where packets exit the router."
But this is wrong. It's not too late to queue it; by queueing it and dropping some packets of inbound traffic the sending host slows down the speed at which it sends.
I'm using pf to do NAT on my box, and I can shape download traffic using the 'queueing on the internal interface' hack; so why can't I do it elegantly on one interface? Shaping NAT traffic downloads works fine with this hack, but I also run some services on the external interface. With downloads queued on the internal interface there's no way to queue the services' download traffic, which means an external service can hog up all the bandwidth and I can't do anything.
I know this is possible because IPFW with dummynet doesn't have any problems. If everyone loves PF because of its elegance why can't it do something as simple as queue download traffic?
> Why can't you queue download traffic on an interface? The reason > openbsd.org's FAQ gives is:
> "Note that queueing is only useful for packets in the outbound > direction. Once a packet arrives on an interface in the inbound > direction it's already too late to queue it -- it's already consumed > network bandwidth to get to the interface that just received it. The > only solution is to enable queueing on the adjacent router or, if the > host that received the packet is acting as a router, to enable queueing > on the internal interface where packets exit the router."
> But this is wrong. It's not too late to queue it; by queueing it and > dropping some packets of inbound traffic the sending host slows down > the speed at which it sends.
Are you sure about that? Perhaps a well behaved sending host would, but what if it isn't? Also, if you're being DDOSd, will this even matter?
> Shaping NAT traffic downloads works fine with this hack, but I also run > some services on the external interface. With downloads queued on the > internal interface there's no way to queue the services' download > traffic, which means an external service can hog up all the bandwidth > and I can't do anything.
I haven't heard of any firewall that successfully could. If you're being DDOSd, you're being DDOSd. No firewall with any special set of rules can improve your bandwidth in that case. If the pipe is filled, it's filled.
> I know this is possible because IPFW with dummynet doesn't have any > problems.
I don't use IPFW, so I can't speak on its specific technical merits. Maybe you should ask the IPFW devs how they're able to perform this magic. Last I heard, the PF devs were technicians, engineers, and scientists, not a single magic-user among them. (IRL anyhow. :-))
If everyone loves PF because of its elegance why can't it do
> something as simple as queue download traffic?
I think that you _can_ configure PF to do this, but I believe that what the developers are trying to say is, `What's the point'? We're talking about trying to control traffic _before_ it hits your interface. Even if the remote sending host is well-behaved enough to slow down its sending rate, it still has to interact with PF before PF can decide whether to pass the packets, drop the packets, or tell the sending host to `bugger off'.
So, you can queue the download traffic, but that really has a minor to no effect on traffic outside of your firewall. The queuing actually occurs on _your_ side of the external interface.
My $0.02 CDN. Take it for what it's worth, or exchange it for better currency. :-)
-- W. Steven Schneider <steven_schnei...@telus.net>
> I haven't heard of any firewall that successfully could. If you're > being DDOSd, you're being DDOSd. No firewall with any special set > of rules can improve your bandwidth in that case. If the pipe is > filled, it's filled.
Yes, if you're being DDoSed then incoming traffic shaping won't do anything, but if you're using TCP streams from cooperative hosts you can shape incoming traffic very effectively; you drop packets, sender realises packets are getting lost, sender slows down sending packets. It works when you use the hack of queueing on the internal interface when you're using NAT, it clearly works, so why can't you do it on a single interface?
> I don't use IPFW, so I can't speak on its specific technical merits. > Maybe you should ask the IPFW devs how they're able to perform this > magic. Last I heard, the PF devs were technicians, engineers, and > scientists, not a single magic-user among them. (IRL anyhow. :-))
There's no magic about it :-P
> I think that you _can_ configure PF to do this, but I believe that > what the developers are trying to say is, `What's the point'? We're > talking about trying to control traffic _before_ it hits your > interface. Even if the remote sending host is well-behaved enough > to slow down its sending rate, it still has to interact with PF > before PF can decide whether to pass the packets, drop the packets, > or tell the sending host to `bugger off'. > So, you can queue the download traffic, but that really has a minor > to no effect on traffic outside of your firewall. The queuing > actually occurs on _your_ side of the external interface.
As I explained above shaping download traffic works great, but only using the two interface hack. People on other boards are telling me I should put another box beetween the LAN and the internet so I can use the two interface hack for all traffic, but this seems stupid; if it works over two interfaces so it can certainly work on one interface, so why doesn't it?
>> I haven't heard of any firewall that successfully could. If you're >> being DDOSd, you're being DDOSd. No firewall with any special set >> of rules can improve your bandwidth in that case. If the pipe is >> filled, it's filled. > Yes, if you're being DDoSed then incoming traffic shaping won't do > anything, but if you're using TCP streams from cooperative hosts you > can shape incoming traffic very effectively; you drop packets, sender > realises packets are getting lost, sender slows down sending packets. > It works when you use the hack of queueing on the internal interface > when you're using NAT, it clearly works, so why can't you do it on a > single interface?
Obviously, I misunderstood. It's still true that the queuing actually occurs on your side of the firewall, and that it has to be on the outbound traffic.
Now I might still be wrong, but I think that you want to control your machine's ack rate. Have you looked at http://www.benzedrine.cx/ackpri.html? This guy has some ideas that might be applicable to your situation.
Thanks for the help, but I still don't think you understand. This isn't a limitation in my understanding of PF and ALTQ, this is a limitation of PF and ALTQ itself; you cannot shape traffic coming in on an interface. There's no good reason why you shouldn't be able to do this. I want to know how I can get this added to a todo list, everyone who has a firm grasp of PF and ALTQ, and isn't a zealot, agrees that this is an important feature which PF lacks, and should be implemented asap.
kestas....@gmail.com writes: > Thanks for the help, but I still don't think you understand. This isn't > a limitation in my understanding of PF and ALTQ, this is a limitation > of PF and ALTQ itself; you cannot shape traffic coming in on an > interface. There's no good reason why you shouldn't be able to do this.
No, there is plenty of good reason to not do so. Spending extra processor cycles queuing (i.e. delaying) packets that have already been received, e.g. have already used network bandwidth, is not a good thing. What you are asking is "please make my network slower so well-behaved peers will throttle their long lived TCP connections" The cost of doing that will slow *everything else* down, too. It buys you nothing with UDP, ICMP, and mis-behaved TCP hosts (of which there are all too many). It exacerbates DOS scenarios. It makes things like popular web servers slower -- http requests are too short lived to get much benefit from TCP congestion control.
> I want to know how I can get this added to a todo list, everyone who > has a firm grasp of PF and ALTQ, and isn't a zealot, agrees that this > is an important feature which PF lacks, and should be implemented asap.
Since you appear to be the only person who is eager to slow down your network processing for minimal benefit (if any at all) you will probably have to do the work yourself.
Most hosts are well behaved, and UDP also throttles if the protocol is intelligent. The fact that this is useful is obvious because so many people do it on their gateway using two interfaces. I know it's useful because before implementing it one person downloading would hog up the whole network; with incoming traffic shaping the person hogging up the network immidiately gets throttled down and other queues can get a chance.
Extra processor cycles? Downloading at full pace my Pentium II 350MHz isn't using 5% of it processor.
Finally I'm not the only person who wants this. From the thread in bit.listserv.openbsd-pf:
> It's a question of what's "useful". Not useful against > a malicious agent, but useful for traffic shaping regardless. > It surely has its caveats, but has its use, too. I'm stuck with ipfw on > one last machine, because I can't limit bulk TCP traffic with pf. > I can speak for myself - I can't afford both the hardware and the > electricity bill for a separate machine. Maybe downstream limiting isn't > very robust, but IMO is the biggest thing pf/altq lacks.
I think most people who use pf for anything more than the most trivial traffic shaping applications realise that this is a major feature which is lacking.
> Thanks for the help, but I still don't think you understand.
I spent a fair bit of today researching the issue to make sure that I understand you better. I believe I do now.
> This isn't > a limitation in my understanding of PF and ALTQ, this is a limitation > of PF and ALTQ itself; you cannot shape traffic coming in on an > interface.
Yes, I was aware that it can't and that this is a limitation of PF/ALTQ. I'm not entirely convinced that it's serious one, however.
Do you have access to any documents that show measurable advantages that queuing incoming traffic would have over not queuing it? Personally, I've been finding mostly "ad hoc" accounts being cited as evidence for shaping incoming traffic.
> There's no good reason why you shouldn't be able to do this. > I want to know how I can get this added to a todo list, everyone who > has a firm grasp of PF and ALTQ, and isn't a zealot, agrees that this > is an important feature which PF lacks, and should be implemented asap.
Your statement assumes that anyone who disagrees with you has a poor grasp of PF/ALTQ, and is a zealot. That polarizes things just a bit, don't you think?
Keeping in mind that I'm not a developer myself, the following points might be helpful.
If you could demonstrate the importance of shaping incoming traffic to Theo and the developers. Better yet, submit a patch, that might go further than simply telling them that it, "should be implemented asap."
Generous donations may also help. I hear that they're rather fond of a good beer. :-)
-- W. Steven Schneider <steven_schnei...@telus.net>
> I spent a fair bit of today researching the issue to make sure that > I understand you better. I believe I do now.
Thanks, I appreciate that you'd spend some time trying to understand this.
> Do you have access to any documents that show measurable advantages > that queuing incoming traffic would have over not queuing it? > Personally, I've been finding mostly "ad hoc" accounts being cited > as evidence for shaping incoming traffic.
Hmm, okay. These images show me downloading two files on two different hosts. I want one host to recieve 30KB/s, and the other to recieve the rest. But most importantly I want it to happen immidiately (so the priority queue can be used to quickly get web pages). I run a download on the low priority host, and start a download on the high priority host, and run date before and after, and see the time it takes to stabilise at a fair speed for each. Here it is without download queueing: http://img151.imagevenue.com/img.php?loc=loc30&image=82445_asdf2.JPG It takes over 20 seconds for the speeds to stabilise at around 25KB/s each. This means if someone's downloading it'll take over 20 seconds to download a fairly lightweight webpage.
Remember people do use download queueing using the two interface hack I use, but this functionality is limited only to NAT or dedicated traffic shaping boxes; I've shown you why it's useful, and lots of people use it with NAT, so why can't you do it on one interface?
> Your statement assumes that anyone who disagrees with you has a poor > grasp of PF/ALTQ, and is a zealot. That polarizes things just a > bit, don't you think?
I don't know how a statement can assume anything, or what is polarized, but yes that's true. It's not a personal attack to say that I doubt you're familiar with PF's traffic shaping capabilities, I'm not familiar with lots of things. I think anyone who understands PF's traffic shaping capabilities must understand its limitations, and this is a severe limitation.
> If you could demonstrate the importance of shaping incoming traffic > to Theo and the developers. Better yet, submit a patch, that might > go further than simply telling them that it, "should be implemented > asap."
I've done the former, not that it needs to be done as I'm sure the developers understand this download shaping limitation intimately, and unfortunately I can't do the latter as I don't know C very well, and I think working on something as complex and mission critical as PF+ALTQ would be daunting to someone who knew C well.
> Generous donations may also help. I hear that they're rather fond > of a good beer. :-)
I said I'd be happy to donate in another thread, to help get this moved along faster. Maybe a couple of days wages, but the only person who responded suggested a fee of $20,000. I'm a student earning minimum wage, I'm the last person who could splash out /$20,000/ to fund a hackathon to get this feature re-implemented.
|
