Many interrupts on a sparc64 with em cards

[Niclas Sodergard <nickus@nickus.net>]

Dear all,

I'm testing to see if a Sun Ultra 10 could replace our firewall at work. 
It's running OpenBSD 3.6 and I've added four Intel gigE cards (Intel 
PRO/1000MT (82541EI)). I've tried to push some traffic through it and I'm 
getting a very high interrupt load or at least I believe so.

I've used ftp to copy from em0 to em1 and I get about 8.5MB/s and the 
system is spending 80% of the cpu time doing interrupts. I then hooked up 
two other computers to em2 and em3 and copied data there as well. When I 
do ftp transfers over all four interfaces I get a total bandwidth of 
9-11MB/s (8000-10000 packets/second) and the interrupt load is at 85-90% 
all the time.

Is this normal for this kind of system? Are there any other brand of 
network card that would be preferable? Or is it just that the Ultra 10 is 
too slow.

I've attached a dmesg output from the machine

cheers,
Nickus

console is keyboard/display
Copyright (c) 1982, 1986, 1989, 1991, 1993
 	The Regents of the University of California.  All rights reserved.
Copyright (c) 1995-2004 OpenBSD. All rights reserved.  http://www.OpenBSD.org

OpenBSD 3.6 (GENERIC) #304: Fri Sep 17 13:01:33 MDT 2004
     deraadt@sparc64.openbsd.org:/usr/src/sys/arch/sparc64/compile/GENERIC
total memory = 268435456
avail memory = 235307008
using 1638 buffers containing 13418496 bytes of memory
bootpath: /pci@1f,0/pci@1,1/ide@3,0/disk@0,0
mainbus0 (root): Sun Ultra 5/10 UPA/PCI (UltraSPARC-IIi 440MHz)
cpu0 at mainbus0: SUNW,UltraSPARC-IIi @ 440 MHz, version 0 FPU
cpu0: physical 32K instruction (32 b/l), 16K data (32 b/l), 2048K external (64 b/l)
psycho0 at mainbus0 addr 0xfffc4000
SUNW,sabre: impl 0, version 0: ign 7c0 bus range 0 to 2; PCI bus 0
DVMA map: c0000000 to e0000000
IOTDB: 13a8000 to 1428000
pci0 at psycho0
ppb0 at pci0 dev 1 function 1 "Sun Simba PCI-PCI" rev 0x13
pci1 at ppb0 bus 1
ebus0 at pci1 dev 1 function 0 "Sun PCIO Ebus2" rev 0x01
auxio0 at ebus0 addr 726000-726003, 728000-728003, 72a000-72a003, 72c000-72c003, 72f000-72f003
power at ebus0 addr 724000-724003 ipl 37 not configured
SUNW,pll at ebus0 addr 504000-504002 not configured
sab0 at ebus0 addr 400000-40007f ipl 43: rev 3.2
sabtty0 at sab0 port 0
sabtty1 at sab0 port 1
comkbd0 at ebus0 addr 3083f8-3083ff ipl 41: layout 46
wskbd0 at comkbd0: console keyboard
com0 at ebus0 addr 3062f8-3062ff ipl 42, mouse: ns16550a, 16 byte fifo
lpt0 at ebus0 addr 3043bc-3043cb, 30015c-30015d, 700000-70000f ipl 34: polled
fdthree at ebus0 addr 3023f0-3023f7, 706000-70600f, 720000-720003 ipl 39 not configured
clock0 at ebus0 addr 0-1fff: mk48t59: hostid 80f0e1d7
flashprom at ebus0 addr 0-fffff not configured
audioce0 at ebus0 addr 200000-2000ff, 702000-70200f, 704000-70400f, 722000-722003 ipl 35 ipl 36: nvaddrs 0
audio0 at audioce0
hme0 at pci1 dev 1 function 1 "Sun HME" rev 0x01: address 08:00:20:f0:e1:d7
nsphy0 at hme0 phy 1: DP83840 10/100 media interface, rev. 1
hme0: using ivec 3021 for interrupt
vgafb0 at pci1 dev 2 function 0 "ATI Mach64 GP" rev 0x5c
wsdisplay0 at vgafb0: console (std, sun emulation), using wskbd0
pciide0 at pci1 dev 3 function 0 "CMD Technology PCI0646" rev 0x03: DMA, channel 0 configured to native-PCI, channel 1 configured to native-PCI
pciide0: using ivec 1820 for native-PCI interrupt
wd0 at pciide0 channel 0 drive 0: <ST39120A>
wd0: 16-sector PIO, LBA, 8693MB, 17803297 sectors
wd0(pciide0:0:0): using PIO mode 4, DMA mode 2
pciide0: channel 1 disabled (no drives)
ppb1 at pci0 dev 1 function 0 "Sun Simba PCI-PCI" rev 0x13
pci2 at ppb1 bus 2
em0 at pci2 dev 1 function 0 "Intel PRO/1000MT (82541EI)" rev 0x00: ivec 10, address: 00:0e:0c:22:a6:ee
em1 at pci2 dev 2 function 0 "Intel PRO/1000MT (82541EI)" rev 0x00: ivec 14, address: 00:0e:0c:22:ae:98
em2 at pci2 dev 3 function 0 "Intel PRO/1000MT (82541EI)" rev 0x00: ivec 18, address: 00:0e:0c:3a:da:68
em3 at pci2 dev 4 function 0 "Intel PRO/1000MT (82541EI)" rev 0x00: ivec 1c, address: 00:0e:0c:22:a3:30
pcons at mainbus0 not configured
No counter-timer -- using %tick at 440MHz as system clock.
root on wd0a
rootdev=0xc00 rrootdev=0x1a00 rawdev=0x1a02
WARNING: clock gained 4 days -- CHECK AND RESET THE DATE!