[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

pf: A success story

I finally did it.
I replaced our main firewall, a 2.9/ipf box, with a 3.0/pf box.

There was one thing that prevented me from doing this until recently, a big
bummer: the box crashed after some hours of operation. Two weeks ago Daniel
and I found the error (related to ICMP error messages and some not very 
sensefull rules for that, it's fixed in -current). I backported -current pf 
to my frankenstein'd tree.

Old and new box are identical hardware-wise: Duron 700, 128 MB RAM, 3x
21143-based NICs using dc(4). With 2.9 and ipf, it ran at over 90% CPU usage 
at prime time and delays began to be noticeable. The rule file was already
fairly high optimized.

Short story: I haven't seen the 3.0/pf box less than 89% idle CPU-wise. We
are having about 10000 packets per seconds each on the main external and the
main internal interface; I have about 1000 rules. One should also note that
the new box does even more than the old one, I added two more /24s which
also leads to a lot of additional rules. When I tried to add just a _few_
rules for this new space to ipf the load instantly was at 100% even outside
the prime time.
There wasn't a _single_ block I couldn't explain,
there is not the slightest evidence of a problem (opposed to out-of-window
error occurring regularily with ipf). I never had such a fine-grained
control on packet filtering, and colleagues here without a clue about pf
(nor ipf) understand the rule file without further explanations (ok, small
exceptions, though that's not due to pf's syntax but due to complicated
filter rules based on tcp flags and stuff they neither have a clue about). 
Thanks to the rule label addition, I have a more detailed and nontheless 
easier to implement accounting than ever. nmap syn scans now take about 45
minutes per host and report zero open ports, nmap's OS detection fails.
That's great.


pfctl -si output as of now:

Status: Enabled  Time: 1017952733  Since: 1017671773  Debug: Urgent
Bytes In IPv4: 0           Bytes Out: 0         
         IPv6: 0           Bytes Out: 0         
Inbound Packets IPv4:  Passed: 0           Dropped: 0         
                IPv6:  Passed: 0           Dropped: 0         
Outbound Packets IPv4: Passed: 0           Dropped: 0         
                 IPv6: Passed: 0           Dropped: 0         
States: 20241
pf Counters
state searches            1459984318
state inserts             9641366 
state removals            9621125 
Counters
match                     706343270
bad-offset                0       
fragment                  287     
short                     20      
normalize                 9928    
memory                    0       

I've seen 25000 concurrent states during normal operations. This is with 
aggressive timeouts, with normal timeouts I've seen over 40000 states.
btw, that's (that beeing the counters) after 3 days 6h.

I'm heavily impressed.

I have to publically express a hughe "Thank you" to Daniel. The amount of
help he gave, the incredible speed in implementing suggestions, the analysis
of the crashes due to the
icmp-error-messages-with-statefull-filtering-memory-leak and the pool_get
issue we did together, the uncounted discussions about modifications, 
changes, new features, improvements; the analysis of some blocks we both 
did not understand initially (but that were sooooooooo logical afterwards), 
that's just incredible.
Daniel, I owe you more beers than we can drink ;-). Not to forget that he is
a really nice guy and all this conversations didn't just had these great
results, it was (and is) also a pleasure and funny. And now that after roughly
half a year of intensive conversation we noticed we both have german as
native language... ;-))  

For completeness, dmesg below.

Now further testing 3.1-beta, the changes art did in the vm area are very
promising...

Greetz

Henning


OpenBSD 3.0-henning (cr2x) #0: Sun Mar 31 16:32:40 CEST 2002
    root@bss004:/usr/src/sys/arch/i386/compile/cr2x
cpu0: AMD Duron ("AuthenticAMD" 686-class) 702 MHz
cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,SYS,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR
real mem  = 133804032 (130668K)
avail mem = 121856000 (119000K)
using 1658 buffers containing 6791168 bytes (6632K) of memory
mainbus0 (root)
bios0 at mainbus0: AT/286+(35) BIOS, date 03/16/01, BIOS32 rev. 0 @ 0xfb3c0
apm0 at bios0: Power Management spec V1.2
apm0: AC on, battery charge unknown
pcibios0 at bios0: rev. 2.1 @ 0xf0000/0xb848
pcibios0: PCI IRQ Routing Table rev. 1.0 @ 0xfdd80/160 (8 entries)
pcibios0: PCI Exclusive IRQs: 10 11 15
pcibios0: PCI Interrupt Router at 000:07:0 ("VIA VT82C596A PCI-ISA" rev 0x00)
pcibios0: PCI bus #1 is the last bus
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 "VIA VT8363 Host" rev 0x03
ppb0 at pci0 dev 1 function 0 "VIA VT8363 PCI-AGP" rev 0x00
pci1 at ppb0 bus 1
pcib0 at pci0 dev 7 function 0 "VIA VT82C686 PCI-ISA" rev 0x40
pciide0 at pci0 dev 7 function 1 "VIA VT82C571 IDE" rev 0x06: ATA100, channel 0 configured to compatibility, channel 1 configured to compatibility
wd0 at pciide0 channel 0 drive 0: <ST31720A>
wd0: 16-sector PIO, LBA, 1625MB, 3303 cyl, 16 head, 63 sec, 3329424 sectors
wd0(pciide0:0:0): using PIO mode 4, DMA mode 2
pciide0: channel 1 ignored (disabled)
"VIA VT82C686 SMBus" rev 0x40 at pci0 dev 7 function 4 not configured
dc0 at pci0 dev 8 function 0 "DEC 21142/3" rev 0x41: irq 10 address 00:40:f4:0a:d1:f5
sqphy0 at dc0 phy 17: Seeq 84220 10/100 media interface, rev. 0
dc1 at pci0 dev 9 function 0 "DEC 21142/3" rev 0x41: irq 11 address 00:00:cb:53:5a:fc
sqphy1 at dc1 phy 17: Seeq 84220 10/100 media interface, rev. 0
dc2 at pci0 dev 10 function 0 "DEC 21142/3" rev 0x41: irq 15 address 00:00:cb:53:5f:e2
sqphy2 at dc2 phy 17: Seeq 84220 10/100 media interface, rev. 0
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard
pcppi0 at isa0 port 0x61
sysbeep0 at pcppi0
npx0 at isa0 port 0xf0/16: using exception 16
pccom0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pccom0: console
biomask 4000 netmask cc00 ttymask cc02
pctr: user-level cycle counter enabled
mtrr: Pentium Pro MTRR support
dkcsum: wd0 matched BIOS disk 80
root on wd0a
rootdev=0x0 rrootdev=0x300 rawdev=0x302


-- 
http://2suck.net/hhwl.html
Unix is very simple, but it takes a genius to understand the simplicity.
(Dennis Ritchie)