[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
kernel/page fault in ipip_input function
First, I'm sorry for:
1. Sending directly to bugs@openbsd.org -- sending mail is not possible
from system where this happens.
2. No ddb output (probably 'show registers' is the most important) --
-- it's remote system and I need it running as fast as possible (so
ddb.panic=0). If it is necessary I can turn it on and drive there
when this happens, but as the last resort :)
3. Probably screwed cut & paste from messages / dmesg log.
SYSTEM
------
Pre 3.6 -- It is one of the last available snapshots before 3.6 release.
I'm tracking cvs-changes and I think there was no relevant
commit to fix this. If I'm wrong I'm sorry for this.
And 15th Sep was the last cvs update, which this system is
running.
PROBLEM
-------
problematic point is (probably):
ipip_input+0x295
Oct 31 17:35:14 router-milan /bsd: uvm_fault(0xd05d63e0, 0x450000, 0, 1) -> e
Oct 31 17:35:14 router-milan /bsd: fatal page fault in supervisor mode
Oct 31 17:35:14 router-milan /bsd: trap type 6 code 0 eip d026f4b9 cs 8
eflags 10286 cr2 450008 cpl 30
Oct 31 17:35:14 router-milan /bsd: panic: trap type 6, code=0, pc=d026f4b9
Oct 31 17:35:14 router-milan /bsd: Starting stack trace...
Oct 31 17:35:14 router-milan /bsd:
panic(0,d3bec7b0,6,d026f4b9,8,10286,daf5fc6c,
d0317433,d04dc39b,6,0,d026f4b9,30,db816802,db88da00,3c807ea7,daf5fda4,dbb0d010,1
4,92c18a57,0,d05d63e0,d05d63e0,450000,450008,1,1,daf5e000,0,0,daf5fdec,d0100eb6,
58,10,10,10,db88da00,db816802,daf5fdec,450008,d0a05128,dbb0d024,c2836d0a,6,0,d02
6f4b9,8,10286,db88da00,dbb0d010,0,d01d1953,0,d0b9510c,d0b95108,0,d05dd2d0,0,dbb0
d024,d05d79a0,45004be2) at panic+0x8e
Oct 31 17:35:14 router-milan /bsd: panic(d04dc39b,6,0,d026f4b9,30) at
panic+0x8e
Oct 31 17:35:14 router-milan /bsd: trap() at trap+0x243
Oct 31 17:35:14 router-milan /bsd: --- trap (number 6) ---
Oct 31 17:35:14 router-milan /bsd: ipip_input(db88da00,14,0,0,0) at
ipip_input+0x295
Oct 31 17:35:14 router-milan /bsd:
ip4_input(db88da00,14,dbb0d01c,dbb0d020,db88da00) at ip4_input+0x37
Oct 31 17:35:14 router-milan /bsd: in_gif_input(db88da00,14,0,0,db88da00) at
in_gif_input+0xaf
Oct 31 17:35:14 router-milan /bsd: ipv4_input(db88da00,1,db88da00,d01febd9) at
ipv4_input+0x63b
Oct 31 17:35:14 router-milan /bsd: ipintr(d059ef5c,18,0,d0310d69,db88da00) at
ipintr+0x77
Oct 31 17:35:14 router-milan /bsd: Xsoftnet() at Xsoftnet+0x2d
Oct 31 17:35:15 router-milan /bsd: --- interrupt ---
Oct 31 17:35:15 router-milan /bsd: 0:
Oct 31 17:35:15 router-milan /bsd: End of stack trace.
Oct 31 17:35:15 router-milan /bsd: uvm_fault(0xd05d63e0, 0x33d2b000, 0, 1) -> e
Oct 31 17:35:15 router-milan /bsd: fatal page fault in supervisor mode
Oct 31 17:35:15 router-milan /bsd: trap type 6 code 0 eip 33d2bb94 cs 8
eflags 10287 cr2 33d2bb94 cpl 80
Oct 31 17:35:15 router-milan /bsd: panic: trap type 6, code=0, pc=33d2bb94
Oct 31 17:35:15 router-milan /bsd: Starting stack trace...
Oct 31 17:35:15 router-milan /bsd:
panic(0,d3bec7b0,6,33d2bb94,8,10287,daf5fa98,
d0317433,d04dc39b,6,0,33d2bb94,80,3,20,210,1de6d0a,0,0,0,0,d05d63e0,d05d63e0,33d
2b000,33d2bb94,1,1,daf5e000,0,0,daf5fb0c,d0100eb6,58,10,10,10,20,3,daf5fb0c,db81
6802,80,d059b948,33d2bb94,6,0,33d2bb94,8,10287,d01b502a,db816802,d029ce20,4,0,0,
d01b4fe0,4,d05b7348,41851345,a9330,daf5fb4c,d01e9a59) at panic+0x8e
Oct 31 17:35:15 router-milan /bsd: panic(d04dc39b,6,0,33d2bb94,80) at
panic+0x8e
Oct 31 17:35:15 router-milan /bsd: trap() at trap+0x243
Oct 31 17:35:15 router-milan /bsd: --- trap (number 6) ---
Oct 31 17:35:15 router-milan /bsd: curpcb(0,20,daf5fb4c,d01e9a53,d059b948,daf5fb
30,d010147e,50) at 0x33d2bb94
Oct 31 17:35:15 router-milan /bsd: softclock(d04dc39b,daf5fc18,30,d0310d69,3) at
softclock+0x1d1
Oct 31 17:35:15 router-milan /bsd: Xsoftclock() at Xsoftclock+0x10
Oct 31 17:35:15 router-milan /bsd: --- interrupt ---
Oct 31 17:35:15 router-milan /bsd: 0x30:
Oct 31 17:35:15 router-milan /bsd: End of stack trace.
DMESG
-----
OpenBSD 3.6 (GENERIC) #3: Wed Sep 15 19:48:21 CEST 2004
root@router-daedroth.pilsfree.czf:/usr/src/sys/arch/i386/compile/GENERIC
cpu0: AMD Athlon(TM) MP ("AuthenticAMD" 686-class) 1.50 GHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE
real mem = 133783552 (130648K)
avail mem = 115326976 (112624K)
using 1658 buffers containing 6791168 bytes (6632K) of memory
mainbus0 (root)
bios0 at mainbus0: AT/286+(5a) BIOS, date 08/01/03, BIOS32 rev. 0 @
0xf1940
apm0 at bios0: Power Management spec V1.2 (BIOS mgmt disabled)
apm0: APM power management enable: unrecognized device ID (9)
apm0: APM engage (device 1): power management disabled (1)
apm0: AC on, battery charge unknown
pcibios0 at bios0: rev 2.1 @ 0xf0000/0x1ff2
pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xf1f20/208 (11 entries)
pcibios0: PCI Interrupt Router at 000:17:0 ("VIA VT82C586 ISA" rev 0x00)
pcibios0: PCI bus #1 is the last bus
bios0: ROM list: 0xc0000/0x8000 0xc8000/0x1000
cpu0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 "VIA VT8377 PCI" rev 0x80
ppb0 at pci0 dev 1 function 0 "VIA VT8377 AGP" rev 0x00
pci1 at ppb0 bus 1
vga1 at pci1 dev 0 function 0 "Matrox MGA G400/G450 AGP" rev 0x04
wsdisplay0 at vga1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
skc0 at pci0 dev 9 function 0 "3Com 3c940" rev 0x12: irq 12
skc0: 3Com Gigabit LOM (3C940)
sk0 at skc0 port A: address 00:0e:a6:59:9e:bf
eephy0 at sk0 phy 0: Marvell 88E1000* Gigabit PHY
em0 at pci0 dev 10 function 0 "Intel PRO/1000MT (82541EI)" rev 0x00: irq
11, address: 00:0e:0c:5e:92:4c
wi0 at pci0 dev 11 function 0 "Intersil PRISM2.5" rev 0x01: irq 5
wi0: PRISM2.5 ISL3874A(Mini-PCI), Firmware 1.1.1 (primary), 1.8.0
(station), address 00:60:b3:6d:91:1d
wi1 at pci0 dev 14 function 0 "Intersil PRISM2.5" rev 0x01: irq 10
wi1: PRISM2.5 ISL3874A(Mini-PCI), Firmware 1.1.1 (primary), 1.8.0
(station), address 00:60:b3:6d:79:36
"VIA VT8237 SATA" rev 0x80 at pci0 dev 15 function 0 not configured
"VIA VT82C571 IDE" rev 0x06 at pci0 dev 15 function 1 not configured
pcib0 at pci0 dev 17 function 0 "VIA VT8237 ISA" rev 0x00
xl0 at pci0 dev 19 function 0 "3Com 3c905B 100Base-TX" rev 0x30: irq 12,
address 00:10:5a:53:37:2f
exphy0 at xl0 phy 24: 3Com internal media interface
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
wdc0 at isa0 port 0x1f0/8 irq 14
wd0 at wdc0 channel 0 drive 0: <ST340014A>
wd0: 16-sector PIO, LBA48, 38166MB, 78165360 sectors
wd0(wdc0:0:0): using BIOS timings
pcppi0 at isa0 port 0x61
midi0 at pcppi0: <PC speaker>
sysbeep0 at pcppi0
lpt0 at isa0 port 0x378/4 irq 7
it0 at isa0 port 0x290/8: IT87
npx0 at isa0 port 0xf0/16: using exception 16
pccom1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
pccom2: irq 5 already in use
biomask e355 netmask ff75 ttymask fff7
pctr: user-level cycle counter enabled
mtrr: Pentium Pro MTRR support
dkcsum: wd0 matched BIOS disk 80
root on wd0a
rootdev=0x0 rrootdev=0x300 rawdev=0x302
WARNING: / was not properly unmounted
OTHER INFORMATIONS
------------------
- IPsec is used on wi0 and wi3
- IPv6 is not configured here, but some IPv6 traffic from others, IPv6
configured systems, flows on xl0 interface. Don't know if it helps
them, but I've added 'pass quick inet6' in pf.conf.
ifconfig -A
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33224
inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x9
sk0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
address: 00:0e:a6:59:9e:bf
media: Ethernet autoselect (100baseTX full-duplex,flag0,flag1)
status: active
inet 10.109.131.30 netmask 0xffffffe0 broadcast 10.109.131.31
inet6 fe80::20e:a6ff:fe59:9ebf%sk0 prefixlen 64 scopeid 0x1
inet 10.109.131.254 netmask 0xffffffff broadcast 10.109.131.255
em0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
address: 00:0e:0c:5e:92:4c
media: Ethernet autoselect (1000baseT full-duplex)
status: active
inet 10.109.255.2 netmask 0xffffffe0 broadcast 10.109.255.31
inet6 fe80::20e:cff:fe5e:924c%em0 prefixlen 64 scopeid 0x2
wi0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
address: 00:60:b3:6d:91:1d
nwid: pf-milan-daedroth
powersave: off
media: IEEE802.11 autoselect (DS11)
status: active
inet 10.109.131.193 netmask 0xfffffffc broadcast 10.109.131.195
inet6 fe80::260:b3ff:fe6d:911d%wi0 prefixlen 64 scopeid 0x3
wi1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
address: 00:60:b3:6d:79:36
nwid: pf-milan
powersave: off
media: IEEE802.11 autoselect hostap (DS2)
status: active
inet 10.109.131.190 netmask 0xffffffc0 broadcast 10.109.131.191
inet6 fe80::260:b3ff:fe6d:7936%wi1 prefixlen 64 scopeid 0x4
xl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
address: 00:10:5a:53:37:2f
media: Ethernet autoselect (100baseTX full-duplex)
status: active
inet 10.109.255.129 netmask 0xfffffffc broadcast 10.109.255.131
inet6 fe80::210:5aff:fe53:372f%xl0 prefixlen 64 scopeid 0x5
inet 10.1.2.234 netmask 0xffffff00 broadcast 10.1.2.255
pflog0: flags=141<UP,RUNNING,PROMISC> mtu 33224
pfsync0: flags=0<> mtu 2020
enc0: flags=0<> mtu 1536
objdump -S ip_ipip.o | less
#ifdef INET6
case 6:
ip6 = (struct ip6_hdr *) ipo;
260: 8b 8d f0 fe ff ff mov 0xfffffef0(%ebp),%ecx
266: 89 8d ec fe ff ff mov %ecx,0xfffffeec(%ebp)
nxt = ip6->ip6_nxt;
itos = (ntohl(ip6->ip6_flow) >> 20) & 0xff;
26c: 8b 01 mov (%ecx),%eax
26e: 66 c1 c8 08 ror $0x8,%ax
272: c1 c8 10 ror $0x10,%eax
275: 66 c1 c8 08 ror $0x8,%ax
279: c1 e8 14 shr $0x14,%eax
27c: 88 85 f9 fe ff ff mov %al,0xfffffef9(%ebp)
if (!ip_ecn_egress(ECN_ALLOWED, &otos, &itos)) {
282: 83 c4 fc add $0xfffffffc,%esp
285: 8d 85 f9 fe ff ff lea 0xfffffef9(%ebp),%eax
28b: 50 push %eax
28c: 8d 85 fa fe ff ff lea 0xfffffefa(%ebp),%eax
292: 50 push %eax
293: 6a 01 push $0x1
295: e8 fc ff ff ff call 296 <ipip_input+0x212>
29a: 83 c4 10 add $0x10,%esp
29d: 85 c0 test %eax,%eax
29f: 0f 84 03 02 00 00 je 4a8 <ipip_input+0x424>
m_freem(m);
return;
}
ip6->ip6_flow &= ~htonl(0xff << 20);
2a5: 8b 85 ec fe ff ff mov 0xfffffeec(%ebp),%eax
2ab: 8b 10 mov (%eax),%edx
2ad: 81 e2 f0 0f ff ff and $0xffff0ff0,%edx
2b3: 89 10 mov %edx,(%eax)
ip6->ip6_flow |= htonl((u_int32_t) itos << 20);
2b5: 0f b6 85 f9 fe ff ff movzbl 0xfffffef9(%ebp),%eax
2bc: c1 e0 14 shl $0x14,%eax
2bf: 66 c1 c8 08 ror $0x8,%ax
2c3: c1 c8 10 ror $0x10,%eax
2c6: 66 c1 c8 08 ror $0x8,%ax
2ca: 09 c2 or %eax,%edx
2cc: 8b 8d ec fe ff ff mov 0xfffffeec(%ebp),%ecx
2d2: 89 11 mov %edx,(%ecx)
break;
2d4: eb 0d jmp 2e3 <ipip_input+0x25f>
#endif
relevant C code (from ip_ipip.c):
#ifdef INET6
case 6:
ip6 = (struct ip6_hdr *) ipo;
nxt = ip6->ip6_nxt;
itos = (ntohl(ip6->ip6_flow) >> 20) & 0xff;
-->0x295--> if (!ip_ecn_egress(ECN_ALLOWED, &otos, &itos)) {
m_freem(m);
return;
}
ip6->ip6_flow &= ~htonl(0xff << 20);
ip6->ip6_flow |= htonl((u_int32_t) itos << 20);
break;
#endif
I don't know how to move forward. If somebody give me some help it would
be great.
I think it's quite repeatable as this system is crashing ~ once a week.
Thank you,
Martin.