Why can't you queue download traffic on an interface? The reason
openbsd.org's FAQ gives is:

"Note that queueing is only useful for packets in the outbound
direction. Once a packet arrives on an interface in the inbound
direction it's already too late to queue it -- it's already consumed
network bandwidth to get to the interface that just received it. The
only solution is to enable queueing on the adjacent router or, if the
host that received the packet is acting as a router, to enable queueing
on the internal interface where packets exit the router."

But this is wrong. It's not too late to queue it; by queueing it and
dropping some packets of inbound traffic the sending host slows down
the speed at which it sends.

I'm using pf to do NAT on my box, and I can shape download traffic
using the 'queueing on the internal interface' hack; so why can't I do
it elegantly on one interface?
Shaping NAT traffic downloads works fine with this hack, but I also run
some services on the external interface. With downloads queued on the
internal interface there's no way to queue the services' download
traffic, which means an external service can hog up all the bandwidth
and I can't do anything.

I know this is possible because IPFW with dummynet doesn't have any
problems. If everyone loves PF because of its elegance why can't it do
something as simple as queue download traffic?

Regards,
Kestas

http://lists.freebsd.org/mailman/htdig/freebsd-pf/2005-November/00165...

Daniel

On 04/29/2006 08:05:47 AM, kestas....@gmail.com wrote:

It's a question of what's "useful".  Not useful against
a malicious agent, but useful for traffic shaping regardless.

I'd promised some benchmarks to demonstrate this quite some time
ago, but just have not been able to get to it.  In the meantime
I've had to set up an entirely separate box, a bridge
with only 2 interfaces, just to get inbound queueing.  :-(
I'm doing VOIP, and in practice inbound queueing definately works.
However, I've still flakey voip issues that I haven't
worked out and so have not been willing to report anything.
To support inbound queueing for voip in a really
elegent fashion you probably want additional semantics
in the queue so you can always have enough free bandwidth
to handle another call and don't have to just
throttle data way back just in case there's lots of
voip traffic.  But that's yet another issue....

Karl <k...@meme.com>
Free Software:  "You don't pay back, you pay forward."
                  -- Robert A. Heinlein

What I tried to express in the last paragraph of the referenced mail was
that it's not pf that's lacking anything, but altq. There simply are no
input queues with altq. If you added those queues to altq, you might not
even have to change a single line in pf to get them used.

While there are now ties between pf and altq (pf classifying packets for
altq, and pfctl setting up queues), that doesn't mean the core of altq
is maintained by the people who maintain pf. It's in separate source
files, and if you compare the respective CVS histories, you'll see the
difference.

Question's of the form "why hasn't this been done" sound kind of weird
to me, almost as if they were rhetorical. Assuming you haven't donated
blood in the last month, "why haven't you"? Are you really interested in
the petty excuses humans have for not doing things? :)

Daniel

| But it wouldn't change anything, because the congestion is upstream of
| your ALTQ box. You can drop as many packets as you like after you
| received them, that doesn't free up any bandwidth on your downlink.

It surely has its caveats, but has its use, too. I'm stuck with ipfw on
one last machine, because I can't limit bulk TCP traffic with pf. Of
course, downstream limiting will never throttle DoS attacks, ICMP or
'dumb' UDP traffic with no acknowledgements, but works just fine for
everything else. Even on ipfw's contemptible 'dummynet'.

| If you want to do this with ALTQ, you can do so by limiting outgoing
| packets on the "other" interface, assuming the box is forwarding all
| packets between two interfaces.

I can speak for myself - I can't afford both the hardware and the
electricity bill for a separate machine. Maybe downstream limiting isn't
very robust, but IMO is the biggest thing pf/altq lacks.

-- sh

application_pgp-signature_part < 1K Download

I was hoping you'd tell me something along the lines of "Because we
already have, you just need to do XYZ".

Unfortunately I also can't afford an extra machine to get the basic
functionality of download queueing.

Well you can only specifiy the total bandwidth for a certain interface,
and all queues which belong to an interface are one-way. You'd have to
have a way to specify download bandwidth on an interface and have a
seperate set of child queues for download. Then you'd have to have a
way of sending incoming and outgoing packets belonging to a certain
state to two queues.

It's doable, but it certainly isn't an internal ALTQ problem.
I read ALTQ and pf were now 'married' (in Theo's words), so I'd think
the problem would require changes to both pf and ALTQ.

I'd have a look at this problem myself, but I'm not good with C. I was
hoping there was some sort of todo list I could petition this to be
added too, because lots of people here seem to agree this is pf's (and
ALTQ's) worst problem.
If there's some sort of bounty system with OpenBSD I'd be happy to set
a bounty so get this done faster.
Kestas

On 04/29/2006 10:58:39 AM, Daniel Hartmeier wrote:

Where would be the proper forum to discuss input queueing then?

Karl <k...@meme.com>
Free Software:  "You don't pay back, you pay forward."
                  -- Robert A. Heinlein

Stock altq could put token buckets on input interfaces for rate
limiting purposes, referred to as the "traffic conditioner" (CDNR).  
That capability was removed when the classifier was merged with pf.

Old messages that might be relevant:
http://www.benzedrine.cx/pf/msg02871.html
http://www.benzedrine.cx/pf/msg07159.html (bottom)

I'm not sure if this thread is a statistically relevant measurement.

If the main reason why the workaround with the additional machine is
problematic is hardware and electricity cost, and there are 'lots of
people' that care deeply about this, and each of them would save, say,
$500 if the feature were implemented, it shouldn't be hard to collect
$20k (two man-months, a reasonable estimate, I'd say) and send that to
Theo with the request for implementation during a hackathon.

Or, to turn the argument around, if the entire support for the feature
is five people offering lollipops and eternal gratitute, developer time
is probably better spent elsewhere.

By all means, try it. There are more readers (and developers) on the
misc@ and tech@ lists, so I'd start there.

Daniel

I don't think time spent developing PF or ALTQ could be better spent
developing something other than download queueing. Everyone here seems
to agree it's PF's worst deficiency.

I'm thinking perhaps there's some messy hack I can come up with using
virtual interfaces, does anyone have any ideas?

I said lots of people *here*, and there aren't close to 40 here, and
$500 is very steep for an extra box which has to do so little, and this
wouldn't require a whole hackathon to code. I'm not sure why you're
being so resistant to a feature request.

I posted in misc, but no-one was biting, and I've just sent one to
tech.

Interesting, I wonder how difficult it would be to get the
functionality back. Any ideas on why it was dropped in the first place?

Kestas

  i queue the incoming downstream on the outbound-towards-my-LAN side.

  works just as good as it possibly could if pf had a "download" queue
  mechanism, if not better.

  if you are in a colo situation and you can't afford a little soekris
  ( or somethin' ) to drop in between and do this -- well... kudos
  on finding so inexpensive a colo that you can afford that but not
  a little 1 time ~250 USD or so for the investment on soekris/whatever

--

  jared

[ openbsd 3.9-current GENERIC ( mar 15 ) // i386 ]

This works adequetly (How could it be "better"? Sounds like zealot
speak to me.) if the boxes only function is NAT, but if it also runs
external services queueing inbound traffic on the internal interface
doesn't work, because external services get priority.

Kestas