Why can't you queue download traffic on an interface? The reason
openbsd.org's FAQ gives is:

"Note that queueing is only useful for packets in the outbound
direction. Once a packet arrives on an interface in the inbound
direction it's already too late to queue it -- it's already consumed
network bandwidth to get to the interface that just received it. The
only solution is to enable queueing on the adjacent router or, if the
host that received the packet is acting as a router, to enable queueing
on the internal interface where packets exit the router."

But this is wrong. It's not too late to queue it; by queueing it and
dropping some packets of inbound traffic the sending host slows down
the speed at which it sends.

I'm using pf to do NAT on my box, and I can shape download traffic
using the 'queueing on the internal interface' hack; so why can't I do
it elegantly on one interface?
Shaping NAT traffic downloads works fine with this hack, but I also run
some services on the external interface. With downloads queued on the
internal interface there's no way to queue the services' download
traffic, which means an external service can hog up all the bandwidth
and I can't do anything.

I know this is possible because IPFW with dummynet doesn't have any
problems. If everyone loves PF because of its elegance why can't it do
something as simple as queue download traffic?

Regards,
Kestas

http://lists.freebsd.org/mailman/htdig/freebsd-pf/2005-November/00165...

Daniel

On 04/29/2006 08:05:47 AM, kestas....@gmail.com wrote:

It's a question of what's "useful".  Not useful against
a malicious agent, but useful for traffic shaping regardless.

I'd promised some benchmarks to demonstrate this quite some time
ago, but just have not been able to get to it.  In the meantime
I've had to set up an entirely separate box, a bridge
with only 2 interfaces, just to get inbound queueing.  :-(
I'm doing VOIP, and in practice inbound queueing definately works.
However, I've still flakey voip issues that I haven't
worked out and so have not been willing to report anything.
To support inbound queueing for voip in a really
elegent fashion you probably want additional semantics
in the queue so you can always have enough free bandwidth
to handle another call and don't have to just
throttle data way back just in case there's lots of
voip traffic.  But that's yet another issue....

Karl <k...@meme.com>
Free Software:  "You don't pay back, you pay forward."
                  -- Robert A. Heinlein

What I tried to express in the last paragraph of the referenced mail was
that it's not pf that's lacking anything, but altq. There simply are no
input queues with altq. If you added those queues to altq, you might not
even have to change a single line in pf to get them used.

While there are now ties between pf and altq (pf classifying packets for
altq, and pfctl setting up queues), that doesn't mean the core of altq
is maintained by the people who maintain pf. It's in separate source
files, and if you compare the respective CVS histories, you'll see the
difference.

Question's of the form "why hasn't this been done" sound kind of weird
to me, almost as if they were rhetorical. Assuming you haven't donated
blood in the last month, "why haven't you"? Are you really interested in
the petty excuses humans have for not doing things? :)

Daniel

| But it wouldn't change anything, because the congestion is upstream of
| your ALTQ box. You can drop as many packets as you like after you
| received them, that doesn't free up any bandwidth on your downlink.

It surely has its caveats, but has its use, too. I'm stuck with ipfw on
one last machine, because I can't limit bulk TCP traffic with pf. Of
course, downstream limiting will never throttle DoS attacks, ICMP or
'dumb' UDP traffic with no acknowledgements, but works just fine for
everything else. Even on ipfw's contemptible 'dummynet'.

| If you want to do this with ALTQ, you can do so by limiting outgoing
| packets on the "other" interface, assuming the box is forwarding all
| packets between two interfaces.

I can speak for myself - I can't afford both the hardware and the
electricity bill for a separate machine. Maybe downstream limiting isn't
very robust, but IMO is the biggest thing pf/altq lacks.

-- sh

application_pgp-signature_part < 1K Download

I was hoping you'd tell me something along the lines of "Because we
already have, you just need to do XYZ".

Unfortunately I also can't afford an extra machine to get the basic
functionality of download queueing.

Well you can only specifiy the total bandwidth for a certain interface,
and all queues which belong to an interface are one-way. You'd have to
have a way to specify download bandwidth on an interface and have a
seperate set of child queues for download. Then you'd have to have a
way of sending incoming and outgoing packets belonging to a certain
state to two queues.

It's doable, but it certainly isn't an internal ALTQ problem.
I read ALTQ and pf were now 'married' (in Theo's words), so I'd think
the problem would require changes to both pf and ALTQ.

I'd have a look at this problem myself, but I'm not good with C. I was
hoping there was some sort of todo list I could petition this to be
added too, because lots of people here seem to agree this is pf's (and
ALTQ's) worst problem.
If there's some sort of bounty system with OpenBSD I'd be happy to set
a bounty so get this done faster.
Kestas

On 04/29/2006 10:58:39 AM, Daniel Hartmeier wrote:

Where would be the proper forum to discuss input queueing then?

Karl <k...@meme.com>
Free Software:  "You don't pay back, you pay forward."
                  -- Robert A. Heinlein

Stock altq could put token buckets on input interfaces for rate
limiting purposes, referred to as the "traffic conditioner" (CDNR).  
That capability was removed when the classifier was merged with pf.

Old messages that might be relevant:
http://www.benzedrine.cx/pf/msg02871.html
http://www.benzedrine.cx/pf/msg07159.html (bottom)

I'm not sure if this thread is a statistically relevant measurement.

If the main reason why the workaround with the additional machine is
problematic is hardware and electricity cost, and there are 'lots of
people' that care deeply about this, and each of them would save, say,
$500 if the feature were implemented, it shouldn't be hard to collect
$20k (two man-months, a reasonable estimate, I'd say) and send that to
Theo with the request for implementation during a hackathon.

Or, to turn the argument around, if the entire support for the feature
is five people offering lollipops and eternal gratitute, developer time
is probably better spent elsewhere.

By all means, try it. There are more readers (and developers) on the
misc@ and tech@ lists, so I'd start there.

Daniel

I don't think time spent developing PF or ALTQ could be better spent
developing something other than download queueing. Everyone here seems
to agree it's PF's worst deficiency.

I'm thinking perhaps there's some messy hack I can come up with using
virtual interfaces, does anyone have any ideas?

I said lots of people *here*, and there aren't close to 40 here, and
$500 is very steep for an extra box which has to do so little, and this
wouldn't require a whole hackathon to code. I'm not sure why you're
being so resistant to a feature request.

I posted in misc, but no-one was biting, and I've just sent one to
tech.

Interesting, I wonder how difficult it would be to get the
functionality back. Any ideas on why it was dropped in the first place?

Kestas

  i queue the incoming downstream on the outbound-towards-my-LAN side.

  works just as good as it possibly could if pf had a "download" queue
  mechanism, if not better.

  if you are in a colo situation and you can't afford a little soekris
  ( or somethin' ) to drop in between and do this -- well... kudos
  on finding so inexpensive a colo that you can afford that but not
  a little 1 time ~250 USD or so for the investment on soekris/whatever

--

  jared

[ openbsd 3.9-current GENERIC ( mar 15 ) // i386 ]

This works adequetly (How could it be "better"? Sounds like zealot
speak to me.) if the boxes only function is NAT, but if it also runs
external services queueing inbound traffic on the internal interface
doesn't work, because external services get priority.

Kestas

Intresting definition for "Everyone". It seems the definition does not
include the developers.

If you can not live without "download queueing" you can always:

1. Write and submit a patch
2. Fund a developer to do the work for you.
3. Go run something else.

I guess you would get bored after a while and choose #3.
Feel free to surprise me though.

Note that "unending trolling on the mailing lists"  

Not that I know of.

This might surprise you but OpenBSD does not run on requests, or polls
or democracy. If a developer feels that such a feature is intresting/important
and have resources to spare, than the feature will be implemented.

The best way to get results is to 'shut up and code'.

Input condititioners are different from queues. You would *not* have the
same amount of control over the traffic as you would have when using a separate box.

Can

On 5/1/06, Can Erkin Acar <cana...@eee.metu.edu.tr> wrote:

I don't know about "worst deficiency", but it sounds like it could be useful.

Well that's a way of looking at it.  Alternately, some coders may wake
up one day and wonder what they should code.  Maybe all their itches
are being scratched, but they still want to code something.  Maybe
they don't know which direction will be of the most benefit to the
community at large.  I know I personally have gotten (and implemented)
ideas from others.

For example, just today I pushed out a new distribution of
pf_dns_lookup.  This program will resolve a set of IPs or hosts, and
either print them out (to be redirected to a file) or stuff them into
a table.  This is to follow the IPs of DDNS clients.  I got the idea
from a discussion about how to handle dynamic DNS on this very list.
--
"Curiousity killed the cat, but for a while I was a suspect" -- Steven Wright
Security Guru for Hire http://www.lightconsulting.com/~travis/ -><-
GPG fingerprint: 9D3F 395A DAC5 5CCC 9066  151D 0A6B 4098 0C55 1484

  to answer that, i believe there's no room for discussion there, then.

  if the machine's only function is NAT it does not
  run external services.

  if i have an external service which i want to have
  jurisdiction over the input traffic on, i run it behind the LAN side.

  if i have an external service that i don't care about
  getting download traffic control over, i run it where it
  makes the most sense to me.

  perhaps you don't have the same liberty of convenience.

--

  jared

[ openbsd 3.9-current GENERIC ( mar 15 ) // i386 ]

[In a message on 01 May 2006 01:51:35 PDT,

Is it a firewall or a dessert topping?  Firewalls should firewall, not
serve services.  If you can't afford one box to firewall, and another
to provide services, well, you're in a fix.  I got a Sun IPX I'll give
you if you pay shipping (anything to end this topic) -- Hardly sucks
any juice, too.

I'm sure I'm part of a large majority of list members who would be
thrilled to see this topic end.  While there's no harm in asking for a
feature expansion, and a discussion about the technical feasibility of
it, when it devolves into accusations of "zealot speak", it's time to
move on.

Sean

PS You're all buying your CDs, right?  Once I'm employed again, I will be.

Why not? This isn't a corporate HQ where the box comes under heavy
load, it's my home firewall/gateway/file server/development box;
there's no reason it can't perform all those roles (other than pf being
unable to shape download traffic).

There was one accusation of zealot speak, because the comment made no
sense whatsoever, and he has still not justified it. For some reason
most people who are replying to this thread seem to be looking for any
reason why this feature request should be ignored; it's not necessary,
it's not needed, pay $20000 if you want it done, someone mentioned
"zealot speak", download shaping isn't needed on firewalls, etc.. Why
the resistance? The other two major firewalls iptables and IPFW can do
it, why can't PF?

Thanks Travis, it's good to hear from someone who isn't pretending to
be a/speak for the developers.

Kestas

On Tuesday 02 May 2006 09:29, kestas....@gmail.com wrote:

Because it's not deemed a really urgent, or even wanted, feature, obviously.
The majority of users/developers has a separate firewall and then "download
queing" is just a matter of doing it on the inside interface. Most of the
previous questions about "download queuing" has been from people who didnt
know that you could achieve the this by queuing on the inside interface, not
by people who wanted to shape traffic to services the firewall box itself.

---
Lars Hansson

Kestas, several core PF developers have responded to your original
message and various follow-ups, essentially trying to elicit some sort
of fact-based reasoning why this feature should be developed by them.

Your response to them has been essentially "Some other product has this"
mixed in with some repetions of "I want this", combined with a liberal
dose of name-calling.  Neither of these approaches are terribly useful
as tools of persuasion.

Discussions of this general mold seem to pop up with alarming frequency,
so I will outline some of the smarter ways to get developers' or more
experienced users' attention -

* "Some other product has this, why can't PF do this"

  Well, in most cases it is likely that PF can indeed do what you want,
  but possibly in a slightly different way than what you would do with
  the other product.  If you are able to explain to other list readers
  what it is you want to achieve, some helpful user is likely to point
  you in the right direction.

  Offering explanations of what you want to achieve is always a lot more
  useful than quoting a chunk of some other product's configuration file
  claiming that PF is deficient if its users can't parse product foo's
  configuration file.

* "I want this! I'm sure PF does not have this"

  It is possible that you have indeed found a useful feature which could
  usefully be implemented.  If you have identified a missing feature
  which you feel is essential to your network, then clearly something
  needs to be done.  

  Making the case for a new feature to be implemented takes a bit of
  work (offering up some code for review usually helps), most important
  is making a well reasoned case that this is a) actually a useful
  feature and b) the feature belongs in the base system.

  It is quite possible that your project satisfies condition a, but not
  condition b, which means that it is better off as a separate program
  to be maintained outside the base system.  Examples of "a, but not b"
  features which became widely used programs maintained outside the base
  system are pftop and expiretable.

* "You're a bunch of mindless moron zealots! I want to talk to the real
  developers!"

  The likelihood that the stranger on the PF mailing list telling you
  that your favorite missing feature is not worth implementing is indeed
  a real developer with commit access to the source tree is far from
  negligible.  

  If your explanation of the obvious goodness of your wished-for feature
  does not convince, you should consider the possibility that a) your
  idea isn't actually that obviously good or b) you need to work a bit
  more on that explanation.  Abuse and name-calling never helps your
  case, ever.

--
Peter N. M. Hansteen, member of the first RFC 1149 implementation team
http://www.blug.linux.no/rfc1149/ http://www.datadok.no/ http://www.nuug.no/
"First, we kill all the spammers" The Usenet Bard, "Twice-forwarded tales"
20:11:56 delilah spamd[26905]: 146.151.48.74: disconnected after 36099 seconds.

What if your firewall box has ssh access on the external interface and
you want to make sure no-one accessing sshd can hog up the bandwidth;
you can't do this with pf.
What if you're using OpenBSD as a desktop computer, you might want to
allow certain applications different bandwidth allowances; you can't do
this with pf (without an extra box).
What if you've got an OpenBSD multi user sshd machine: you want to
allow all users internet access, but you want to make sure they can all
have an equal share of the download bandwidth; you can't do this with
pf (without an extra box, and some sort of very messy identd+tables
hack).

As far as firewalls go I'm a big fan of pf, actually I just finished
writing a fairly detailed walkthrough of my pf.conf (
http://kuliukas.com/guide.html ), which is why I don't want to have to
use/recommend something else if I need this functionality; I think
download shaping would be the icing on the cake.
I'm not demanding anyone do anything, I'm not trolling, I just want to
get this acknowledged as an area for potential development. Why
everyone's so resistant to this is beyond me. That this is the only
extra feature I'd like to have in pf I think reflects pretty well on
pf.

Kestas

* kestas....@gmail.com <kestas....@gmail.com> [2006-05-01 02:50]:

it's nice that you think so.
now, let me tell you some news: it does not matter what you think.
what matters is what we think, the ones that write the code.

when we see a clean, well written diff that implements this and makes
sense, we might incorporate that.
maybe you could get one of us to code that if you fund him to do that
(let me tell you beforehands that you're looking at a 4 digit number
for sure).
endless whining here will make sure we'll never implement it unless we
have a reallyt urgent need. since we hadn't had that in the past, what,
4 years, it's kinda unlikely that changes.

I'll summarize again for you. pick one:

1) submit a diff
2) pay a developer to do it
3) get over it

note that "continue whining on the mailing lists and annoy developers
enough so that they eventually might unsubscribe" is not on the list.

--
Henning Brauer, h...@bsws.de, henn...@openbsd.org
BS Web Services, http://bsws.de
OpenBSD-based Webhosting, Mail Services, Managed Servers, ...

No problem. I, dhartmei@, in my role as OpenBSD and pf developer[1], hereby
acknowledge inbound queueing in pf/altq as an area of potential development.

Several users, on several occasions, have shown interest in this feature,
and so far no strong technical reason has been named why it should not be
added.

I vouch that, if and when development in that area has occurred, I'll do
my best to review, test, and reach a consensus to commit that work.

I'm not resistant to this feature, I just don't want to implement it
myself. I'd rather do laundry and watch a movie.

That doesn't mean I discourage anyone else from doing it. Go ahead, knock
yourself out. It's not a trivial task, you'll earn respect.

Is that official enough? Can we move on now?

Daniel

[1] http://www.openbsd.org/cgi-bin/cvsweb/src/sys/net/pf.c?rev=1.1&conten...

Get over what? This is a suggestion, a feature request.

As Travis H. said:

If you classify a feature request, and justifications and defences for
that feature request, as whining, then just ignore this thread. Why
threaten to unsubscribe? After a post like that I couldn't care less.

Kestas

On 05/02/2006 02:22:33 AM, Lars Hansson wrote:

To be fair, this only works if you've a single "inside interface".

Karl <k...@meme.com>
Free Software:  "You don't pay back, you pay forward."
                  -- Robert A. Heinlein