Re: Network performance openbsd 3.6

[Claudio Jeker <cjeker_(_at_)_diehard_(_dot_)_n-r-g_(_dot_)_com>]

On Tue, Apr 12, 2005 at 02:08:03PM +0200, Matthew Whittaker-Williams wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi there,
> 
> I`ve been using openbsd for quite some time now.
> I used to tweak my machines alot to get as much out of it as i could.
> But since the 3.6 release some options has been removed from the
> kernel and been replaced by sysctl settings.
> There for i was trying to figure out what option would actually raise
> the nmbclusters.
> Unfortunally i wasn`t able to find any documentation on this and from
> reading the mailing list alot of people say you shouldn`t tweak
> anything anymore and just be happy with the defaults.
> After talking to some people they pointed out to tune kern.maxclusters
> what would be in effect nmbclusters.
> 
> My main concern at the moment is this weird report netstat gives me.
> 
> - - ( Test machine ) Bridged setup between em interfaces and 1 fxp
> management interface
> 
> With a newly build kernel without unnessecary hardware as well as no
> options tweaked.
> 

<SNIP>

> netstat -m :
> 
> 602 mbufs in use:
> ~        576 mbufs allocated to data
> ~        1 mbuf allocated to packet headers
> ~        25 mbufs allocated to socket names and addresses
> 576/592/6144 mbuf clusters in use (current/peak/max)
> 1344 Kbytes allocated to network (96% in use)
> 0 requests for memory denied
> 0 requests for memory delayed
> 0 calls to protocol drain routines
> 

As you can see here you have 576 clusters in use with a peak of 592 and a
absolute max of 6144. So you are using <10% of the possible mclusters.
The (96% in use) refers to the useage of the 1344 Kbytes that are
allocated for network stuff. This is no hard limit, it tells you that of
the 1344KB memory that are acctually allocated only 4% are wasted.
Acctually the 4% are not wasted they are just not used.

> vmstat -m:
> 

<SNIP>

> In use 3718K, total allocated 4056K; utilization 91.7%
> 
> I am totaly confused about this utilization, because how can it be 91%
> in use without anything running.

Same again here. Your system uses 4056K for various pools of those 4M
3718K are acctually in use (91.7%). The 8.3% or 338K that are unused are
objects that are on the free list of some pools.

> This even when pf is disabled and the only connection to the machine
> is my ssh connection.
> 

<SNIP>

> In use 2694K, total allocated 2896K; utilization 93.0%
> 
> What i don`t seem to understand is how openbsd can use so much memory
> with default settings.

Do I understand you right. You think that 2896K is much memory?

> And when tweaking it somewhat it will still report > 90 % usage.
> It might be i am interperting this wrong but from what i know is when
> it reaches 100 % usage it will deny connections that are being made.
> The weird thing is all my freebsd systems only report 3% usage and a
> different openbsd box on a via C3 proccessor only reports:
> 

FreeBSD report a different thing than OpenBSD.

<SNIP>

> netstat -m:
> 
> 27 mbufs in use:
> ~        1 mbuf allocated to data
> ~        1 mbuf allocated to packet headers
> ~        25 mbufs allocated to socket names and addresses
> 0/82/6144 mbuf clusters in use (current/peak/max)
> 208 Kbytes allocated to network (3% in use)
> 0 requests for memory denied
> 0 requests for memory delayed
> 0 calls to protocol drain routines
> 
> 
> On my freebsd systems it only reports 3%.
> So i am a bit dazed on how openbsd deals with this.
> 

In other words your freebsd box needs currently 6k of RAM for the network
but wastes another 202k just to make people feel save.

> There for i am wondering if any of you fokes can give me any advice on
> where to look at next on how to get this usage down some what so i can
> change the machine to production.
> 

Unless you realy trigger a case where no mbufs or cluster can be allocated
you don't need to play with the buttons. At least I don't have to push
buttons for my routers and firewalls.

> Could it perhaps have to do with SMP being enabled?
> I am having my second thoughts about this em driver aswell, because
> since i`ve been using em cards under openbsd it seems to take alot
> more memory then the normal fxp card do.
> 

em cards are gigabit and need therefor more memory reserved for the recv
dma to get good preformance.

> Well let me know
> 
> In hope to figure this thing out...
> 


-- 
:wq Claudio