High Road (Re: OpenBSD Benchmarked... results: poor!)

[Chuck Yerkes <chuck+obsd@2003.snew.com>]

Quoting James Herbert (jamesherbert@gmx.net):
> Found this today and was intrigued.
> http://bulk.fefe.de/scalability/
> Wonder what you guys think about this?

Read the thread to now - lots of reactions, but many are
along the lines of "it works for me".

I gotta say, I love it when tech support or developeres take this
attitude - its really non-helpful and counterproductive.

It was a paper and the author's comments in his paper and
/. indicate that
1) He's a Linux guy first, not a BSD guy.
    this isn't a character flaw - at least he's not an MS contractor.
2) it was done under time constraints.
   This seems to have kept him from doing things "right".
   I'd do it on a box with removable disks (I have a VALinux
   box that has 6 removable drives).  I'd put each OS on its
   own disk (try to use the pretty RH gui to do a 400MB Linux
   install.  Try to find man pages for every file installed
   on linux).
   These contraints made it "less thorough" science than
   it should be.


His tests run on his setup caused his kernel to stop.
I won't dispute that.  I *will* view a kernel crash as
a problem.
I will wonder what his code was that caused this crash
(running as root allow opportunities denied to general
users).
The code, (his or the OSs) fixed should cause errors to
be returned and perhaps more allocation to be denied.

The next step would be to remove that bottleneck.

OpenBSD seems to be (wisely) moving many of the variables we used
to need customer kernels for into sysctl variables.
That's good.
The only change I do to GENERIC is to lock some SCSI disk
targets to specfic SCSI IDs (I turn some things off on occasion
and and having SD2 become SD1 is a PITA.  That can all be done
via config to a compiled kernel.  No worries there.
I also remote random PIDs on devel machines for ingrained habits
of "ps ax|sort -n".  That's mine.


The tests run stress specific parts of the kernel for a high
end network server.  Web and mail can beat up a stack pretty
well (web worse because connections last 0.25 seconds and come
back for more).

There are tunings to handle that and he could learn.

Failing: Used OpenBSD 3.4-current, rather than -stable.

I will take issue with his subjective remarks about
- the BSD IPv6 stack (which comes from KAME via Itojun-san).
- Booting after cyl 1024.  Not a problem on my real machines
  (read "machines without 1982 BIOS" like Sparc, PPC, etc).
  And I find that folks using the mono-partition are usually
  less experienced.  My / partition is usually < 100MB.
- and Dude, the OpenBSD/NetBSD fork happened 7 years ago.  Linux
  has diverged from its parent, SCO, a great deal in 7 years ;)

I will also take issue with other things missing:
- PreFAB Linux:
  While it's easy to rebuild a Linux kernel, he can't go in and
rebuild, say glibc easily.  Or hop over to /usr/src/usr.sbin/netstat
and tweak that to show him things it doesn't by default.

  I wanted/needed "."  in usernames (www.example.com) and changed
chown to not use "." as a separator and started offering patches
to the OS that used "." (rc files, etc) and watched others
whack through the remaining ones in a week.

This gets to security and maintainability.
  + I don't install packages built on some untrusted stranger's machine.
  + I *never* play "find the right RPM" game.
  + Oh, this needs those other RPMs?
  + Heavens!  You replaced the sendmail package with your own build?
  + Now MIMEDefang won't install.
  + Got all the gnome 3 packages installed to find that you have
    the wrong subversion of libc but need that for another package?

built from source on your own machine means it works. Ports means
that dependancies get built.  Problems are rare. Far more rare
than the linux games I've played on RH, Suse and debian.

And unrelated to about anything: man pages.  Linux sometimes has
them.  Often not.  Inexcusable.  If there's an info page, then
a man page stub can point you do it.

And I'll skip over add on packages going into /usr/bin/ and such.


The bottom line, however, are that he had problems that crashed
the machine.  If this is not a result of -current instability,
then he's got a legit issue.

I'd rather see the BSD's show their HUGE advantage of a centralized
code collective and react by addressing it quickly.   I'm overjoyed
that when I report a bug it's often addressed within a week.
When I've mailed the odd hardware to a developer, I can see
support appearing in CVS in the weeks after it arrives.  Hell,
I can see the CVS logs fro the whole damn OS!

BSD excels at identifying problems quickly and fixing them.
Reacting by dismissing him with "I don't see this problem on my
machine" ignores the strength of OpenBSD's developers working
together on a whole system with fixes distributed rapidly.  pf
came from nowhere to very fast and with a very advanced config
file parsing in a matter of months.  (triggered by petty anger,
but  perhaps that's just a useful thing to harness here  :)


Spinning off forks should not cause a crash.  His nmap study
indicates that something is amiss.

That's the symptom of the problem?  What's the problem? What's the answer?