[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Help request to debug a crash
- To: tech_(_at_)_openbsd_(_dot_)_org
- Subject: Help request to debug a crash
- From: Pailloncy Jean-Gérard <jg_(_at_)_rilk_(_dot_)_com>
- Date: Thu, 2 Dec 2004 13:32:03 +0100
Hi,
I have a Tyan Trinity GC-SL motherboard.
I have rarely some random crash, sometime during boot.
I had already (few months ago) send a mail to the mailig list, but with
less information.
I did not have at this time a debugging kernel.
So, I compile a debug kernel.
I did not activate the support for the WDT card, I add just some debug
option.
And I wait few weeks to get a crash
Then I do "shudown -r now" and at reboot a crash...
Note 1: I do not understand the msgs "timeout delayed -1" and "init:
copying out path `/sbin/init' 11" I got.
Note 2: Sometime the system crash not at boot but under heavy load.
Any help will be welcomed.
Cordialement,
Jean-Gérard Pailloncy
# dmesg
OpenBSD 3.6 (POOLDBG) #3: Wed Nov 10 13:39:33 CET 2004
admin_(_at_)_mail_(_dot_)_rilk_(_dot_)_com:/usr/src/sys/arch/i386/compile/POOLDBG
cpu0: Intel(R) Pentium(R) 4 CPU 2.66GHz ("GenuineIntel" 686-class) 2.66
GHz
cpu0:
FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,
CFLUSH,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,CNXT-ID
real mem = 1073324032 (1048168K)
avail mem = 972730368 (949932K)
using 4278 buffers containing 53768192 bytes (52508K) of memory
mainbus0 (root)
bios0 at mainbus0: AT/286+(00) BIOS, date 10/31/03, BIOS32 rev. 0 @
0xfdb80
apminfo: 30102, code f0000[ffff]/f0000[ffff], data 400[bff], ept ef50
apm0 at bios0: Power Management spec V1.2
apm0: AC on, battery charge unknown
pcibios0 at bios0: rev 2.1 @ 0xf0000/0x10000
pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xf4fb0/272 (15 entries)
pcibios0: PCI Interrupt Router at 000:01:7 ("ServerWorks CSB5
SouthBridge" rev 0x00)
pcibios0: PCI bus #0 is the last bus
bios0: ROM list: 0xc0000/0x8000 0xc8000/0x2200 0xca800/0x1000
0xcb800/0x1800
cpu0 at mainbus0
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 "ServerWorks CMIC_SL Host" rev 0x32
pchb1 at pci0 dev 0 function 1 "ServerWorks CMIC_SL Host" rev 0x00
pci1 at pchb1 bus 1
"Industrial Computer Source WDT 50x Watchdog Timer" rev 0x02 at pci0
dev 3 function 0 not configured
vga1 at pci0 dev 7 function 0 "ATI Rage XL" rev 0x27
wsdisplay0 at vga1: console (80x25, vt100 emulation)
wsdisplay0: screen 1-5 added (80x25, vt100 emulation)
fxp0 at pci0 dev 8 function 0 "Intel 82557" rev 0x10: irq 9, address
00:e0:81:27:eb:5c
inphy0 at fxp0 phy 1: i82555 10/100 media interface, rev. 4
pcib0 at pci0 dev 15 function 0 "ServerWorks CSB5 SouthBridge" rev 0x93
pciide0 at pci0 dev 15 function 1 "ServerWorks CSB5 IDE" rev 0x93: DMA
atapiscsi0 at pciide0 channel 0 drive 0
scsibus0 at atapiscsi0: 2 targets
cd0 at scsibus0 targ 0 lun 0: <SAMSUNG, CD-ROM SC-152A, CA08> SCSI0
5/cdrom removable
cd0(pciide0:0:0): using PIO mode 4, DMA mode 2
pchb2 at pci0 dev 15 function 3 "ServerWorks CSB5 PCI" rev 0x00
pchb3 at pci0 dev 16 function 0 "ServerWorks CIOBX2" rev 0x05
pchb4 at pci0 dev 16 function 2 "ServerWorks CIOBX2" rev 0x05
pci2 at pchb4 bus 2
ami0 at pci2 dev 3 function 0 "Symbios Logic MegaRAID" rev
0x01pci_mapreg_map: limited PCI mapping from 10000 to 1000
: irq 10 LSI 523/64b/lhc
ami0: FW 712T, BIOS vG116, 64MB RAM
ami0: 1 channels, 0 FC loops, 1 logical drives
scsibus1 at ami0: 40 targets
sd0 at scsibus1 targ 0 lun 0: <AMI, Host drive #00, > SCSI2 0/direct
fixed
sd0: 715410MB, 91201 cyl, 255 head, 63 sec, 512 bytes/sec, 1465159680
sec total
em0 at pci2 dev 7 function 0 "Intel PRO/1000MT (82545EM)" rev 0x01: irq
11, address: 00:e0:81:27:eb:5d
isa0 at pcib0
isadma0 at isa0
pckbc0 at isa0 port 0x60/5
pckbdprobe: reset error 5
pckbd0 at pckbc0 (kbd slot)
pckbc0: using irq 1 for kbd slot
wskbd0 at pckbd0: console keyboard, using wsdisplay0
pmsprobe: reset error 5
pmsiprobe: reset error 5
pcppi0 at isa0 port 0x61
midi0 at pcppi0: <PC speaker>
sysbeep0 at pcppi0
lpt0 at isa0 port 0x378/4 irq 7
lpt_isa_probe: mask ff data 55 failed
lpt_isa_probe: mask ff data 55 failed
npx0 at isa0 port 0xf0/16: using exception 16
pccom0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pccom0: console
pccom1 at isa0 port 0x2f8/8 irq 3: ns16550a, 16 byte fifo
fdc0 at isa0 port 0x3f0/6 irq 6 drq 2
fd0 at fdc0 drive 0: 1.44MB 80 cyl, 2 head, 18 sec
biomask f565 netmask ff65 ttymask ffe7
pctr: user-level cycle counter enabled
timeout delayed -1
timeout delayed -3
dkcsum: checksum of sd0 is d7c99676
timeout delayed -1
timeout delayed -3
timeout delayed -5
dkcsum: attempting to match with BIOS drive 80 csum d7c99676
timeout delayed -1
dkcsum: sd0 matched BIOS disk 80
timeout delayed -1
root on sd0a
rootdev=0x400 rrootdev=0xd00 rawdev=0xd02
timeout delayed -1
WARNING: / was not properly unmounted
timeout delayed -1
timeout delayed -2
timeout delayed -1
init: copying out path `/sbin/init' 11
timeout delayed -1
timeout delayed -25
timeout delayed -10
timeout delayed -1
timeout delayed -25
timeout delayed -14
timeout delayed -12
timeout delayed -3
# diff POOLDBG GENERIC
10,14d9
< makeoptions DEBUG="-g"
< option DEBUG
< option POOL_DIAGNOSTIC
< option MALLOC_DEBUG
<
# gdb.4
GNU gdb 4.16.1
Copyright 1996 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and
you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for
details.
This GDB was configured as "i386-unknown-openbsd3.5".
(gdb) file bsd.gdb
Reading symbols from bsd.gdb...done.
(gdb) target kvm /var/crash/bsd.12.core
Undefined target command: "kvm /var/crash/bsd.12.core". Try "help
target".
(gdb) target kcore /var/crash/bsd.12.core
panic: trap type 6, code=2, pc=d01efb5a
#0 0x1000 in ?? ()
(gdb) bt
#0 0x1000 in ?? ()
#1 0xd03139c1 in boot (howto=256)
at
/admin/src/sys/arch/i386/compile/POOLDBG/../../../../arch/i386/i386/
machdep.c:2241
#2 0xd01f1443 in panic ()
at
/admin/src/sys/arch/i386/compile/POOLDBG/../../../../kern/subr_prf.c:
224
#3 0xd031c1a3 in trap (frame={tf_fs = 88, tf_gs = -798031856, tf_es =
-780468208,
tf_ds = 274858000, tf_edi = -674088960, tf_esi = -799111648,
tf_ebp = -797970988, tf_ebx = -674086872, tf_edx = -1608372224,
tf_ecx = 1514,
tf_eax = -674086864, tf_trapno = 6, tf_err = 2, tf_eip =
-803275942,
tf_cs = 80, tf_eflags = 66182, tf_esp = -674082816, tf_ss = 80,
tf_vm86_es = -780566528, tf_vm86_ds = 941161504, tf_vm86_fs =
256576,
tf_vm86_gs = -797970976})
at
/admin/src/sys/arch/i386/compile/POOLDBG/../../../../arch/i386/i386/
trap.c:268
#4 0xd0100eb6 in calltrap ()
#5 0xd0169123 in fxp_add_rfabuf (sc=0xd1798000, oldm=0xd7d25200)
at
/admin/src/sys/arch/i386/compile/POOLDBG/../../../../dev/ic/fxp.c:1514
#6 0xd016850f in fxp_intr (arg=0xd1798000)
at
/admin/src/sys/arch/i386/compile/POOLDBG/../../../../dev/ic/fxp.c:965
#7 0xd01017f6 in Xresume_legacy9 ()
# gdb
GNU gdb 6.1
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and
you are
welcome to change it and/or distribute copies of it under certain
conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for
details.
This GDB was configured as "i386-unknown-openbsd3.6".
(gdb) file bsd.gdb
Reading symbols from bsd.gdb...done.
(gdb) target kvm /var/crash/bsd.12.core
0x00001000 in ?? ()
(gdb) bt
#0 0x00001000 in ?? ()
#1 0x00000100 in ?? ()
#2 0xd06fed20 in ?? ()
#3 0xd04e69bb in i386_features ()
#4 0x00000000 in ?? ()
#5 0x00000011 in ?? ()
#6 0x00000000 in ?? ()
#7 0x00000008 in ?? ()
#8 0xd032b1f4 in sdsize ()
at
/admin/src/sys/arch/i386/compile/POOLDBG/../../../../scsi/sd.c:1183
#9 0xd03139c1 in boot (howto=256)
at
/admin/src/sys/arch/i386/compile/POOLDBG/../../../../arch/i386/i386/
machdep.c:2241
#10 0xd01f1443 in panic (fmt=0xd04e69bb "trap type %d, code=%x, pc=%x")
at
/admin/src/sys/arch/i386/compile/POOLDBG/../../../../kern/subr_prf.c:
224
#11 0xd031c1a3 in trap (frame=
{tf_fs = 88, tf_gs = -798031856, tf_es = -780468208, tf_ds =
274858000, tf_edi = -674088960, tf_esi = -799111648, tf_ebp =
-797970988, tf_ebx = -674086872, tf_edx = -1608372224, tf_ecx = 1514,
tf_eax = -674086864, tf_trapno = 6, tf_err = 2, tf_eip = -803275942,
tf_cs = 80, tf_eflags = 66182, tf_esp = -674082816, tf_ss = 80,
tf_vm86_es = -780566528, tf_vm86_ds = 941161504, tf_vm86_fs = 256576,
tf_vm86_gs = -797970976})
at
/admin/src/sys/arch/i386/compile/POOLDBG/../../../../arch/i386/i386/
trap.c:268
#12 0xd0100eb6 in calltrap ()
---Type <return> to continue, or q <return> to quit---
#13 0x00000058 in ?? ()
#14 0xd06f0010 in ?? ()
#15 0xd17b0010 in ?? ()
#16 0x10620010 in ?? ()
#17 0xd7d23800 in ?? ()
#18 0xd05e8620 in max_linkhdr ()
#19 0xd06fedd4 in ?? ()
#20 0xd7d24028 in ?? ()
#21 0xa0223000 in ?? ()
#22 0x000005ea in ?? ()
#23 0xd7d24030 in ?? ()
#24 0x00000006 in ?? ()
#25 0x00000002 in ?? ()
#26 0xd01efb5a in _pool_get (pp=0xd05e8620, flags=0,
file=0xd04abb40
"/admin/src/sys/arch/i386/compile/POOLDBG/../../../../dev/ic/fxp.c",
line=1514)
at
/admin/src/sys/arch/i386/compile/POOLDBG/../../../../kern/subr_pool.c:
821
#27 0xd0169123 in fxp_add_rfabuf (sc=0xd1798000, oldm=0xd7d25200)
at
/admin/src/sys/arch/i386/compile/POOLDBG/../../../../dev/ic/fxp.c:1514
#28 0xd016850f in fxp_intr (arg=0xd1798000)
at
/admin/src/sys/arch/i386/compile/POOLDBG/../../../../dev/ic/fxp.c:965
#29 0xd01017f6 in Xresume_legacy9 ()
#30 0xd1798000 in ?? ()
---Type <return> to continue, or q <return> to quit---
#31 0x00000000 in ?? ()
#32 0xd06f0058 in ?? ()
#33 0x00000010 in ?? ()
#34 0xd01d0010 in systrace_policy (fst=0xd06f0058, pol=0x10)
at
/admin/src/sys/arch/i386/compile/POOLDBG/../../../../dev/systrace.c:
1047
Previous frame inner to this frame (corrupt stack?)
# ps -N bsd.12 -M bsd.12.core -ax -O paddr
PID PADDR TT STAT TIME COMMAND
1 d7d15000 ?? Is 0:00.00 (init)
26807 d7d15b88 00 Is+ 0:04.00 (sh)
24951 d7d15cd0 00 R+ 0:04.00 (dmesg)
# vmstat -N bsd.12 -M bsd.12.core -m
Memory statistics by bucket size
Size In Use Free Requests HighWater Couldfree
16 201 55 3195 1280 0
32 213 43 579 640 0
64 448 64 2121 320 0
128 140 20 711 160 0
256 308 12 595 80 0
512 201 7 381 40 0
1024 85 3 106 20 0
2048 20 0 28 10 0
4096 24 1 218 5 0
8192 8 0 8 5 0
16384 2 0 2 5 0
32768 3 0 3 5 0
Memory usage type by bucket size
Size Type(s)
16 devbuf, pcb, routetbl, ifaddr, sysctl, namecache, UFS mount,
dirhash,
proc, in_multi, VM swap, UVM amap, UVM aobj, temp
32 devbuf, pcb, routetbl, ifaddr, vnodes, sem, dirhash, VFS
cluster,
ether_multi, VM swap, UVM amap, temp
64 devbuf, routetbl, UFS mount, sem, dirhash, in_multi, pfkey
data,
UVM amap, NDP
128 devbuf, routetbl, ifaddr, vnodes, ttys, exec, UVM amap, NDP
256 devbuf, routetbl, ifaddr, sysctl, vnodes, UFS mount, shm, VM
map,
proc, NFS srvsock, NFS daemon, ttys, newblk, temp
512 devbuf, pcb, ifaddr, ioctlops, mount, UFS mount, shm,
dirhash, exec,
temp
1024 devbuf, ioctlops, UFS mount, proc, ttys, UVM aobj, crypto
data, temp
2048 devbuf, ifaddr, UFS mount, pagedep, VM swap
4096 devbuf, UFS mount, MSDOSFS mount, temp
8192 devbuf, NFS node, namecache, UFS quota, UFS mount, ISOFS
mount,
inodedep
16384 devbuf, UFS mount
32768 namecache, UFS mount
Memory statistics by type Type Kern
Type InUse MemUse HighUse Limit Requests Limit Limit Size(s)
devbuf 895 242K 244K 39322K 1253 0 0
16,32,64,128,256,512,1024,2048,4096,8192,16384
pcb 9 3K 3K 39322K 52 0 0
16,32,512
routetbl 90 6K 6K 39322K 142 0 0
16,32,64,128,256
ifaddr 123 20K 20K 39322K 124 0 0
16,32,128,256,512,2048
sysctl 2 1K 1K 39322K 2 0 0 16,256
ioctlops 0 0K 1K 39322K 33 0 0 512,1024
mount 12 6K 7K 39322K 14 0 0 512
NFS node 1 8K 8K 39322K 1 0 0 8192
vnodes 92 9K 9K 39322K 92 0 0
32,128,256
namecache 3 41K 41K 39322K 3 0 0
16,8192,32768
UFS quota 1 8K 8K 39322K 1 0 0 8192
UFS mount 48 150K 150K 39322K 48 0 0
16,64,256,512,1024,2048,4096,8192,16384,32768
shm 2 1K 1K 39322K 2 0 0 256,512
VM map 3 1K 1K 39322K 3 0 0 256
sem 2 1K 1K 39322K 2 0 0 32,64
dirhash 15 3K 3K 39322K 15 0 0
16,32,64,512
proc 4 3K 3K 39322K 4 0 0
16,256,1024
VFS cluster 0 0K 1K 39322K 10 0 0 32
NFS srvsock 2 1K 1K 39322K 2 0 0 256
NFS daemon 1 1K 1K 39322K 1 0 0 256
in_multi 32 2K 2K 39322K 32 0 0 16,64
ether_multi 8 1K 1K 39322K 8 0 0 32
ISOFS mount 1 8K 8K 39322K 1 0 0 8192
MSDOSFS mount 1 4K 4K 39322K 1 0 0 4096
ttys 144 84K 84K 39322K 144 0 0
128,256,1024
exec 0 0K 1K 39322K 271 0 0 128,512
pfkey data 1 1K 1K 39322K 2 0 0 64
pagedep 1 2K 2K 39322K 1 0 0 2048
inodedep 1 8K 8K 39322K 1 0 0 8192
newblk 1 1K 1K 39322K 1 0 0 256
VM swap 5 3K 3K 39322K 5 0 0
16,32,2048
UVM amap 53 2K 4K 39322K 4967 0 0
16,32,64,128
UVM aobj 2 2K 2K 39322K 2 0 0 16,1024
crypto data 1 1K 1K 39322K 1 0 0 1024
NDP 10 1K 1K 39322K 12 0 0 64,128
temp 87 23K 27K 39322K 694 0 0
16,32,256,512,1024,4096
Memory Totals: In Use Free Requests
634K 23K 7947
Memory resource pool statistics
Name Size Requests Fail Releases Pgreq Pgrel Npage Hiwat Minpg
Maxpg Idle
phpool 40 216 0 8 3 0 3 3 0
8 0
extentpl 20 220 0 194 1 0 1 1 0
8 0
pmappl 76 140 0 137 1 0 1 1 0
8 0
vmsppl 216 140 0 137 1 0 1 1 0
8 0
vmmpepl 88 3964 0 3929 2 0 2 2 0
8 1
vmmpekpl 88 283 0 282 1 0 1 1 0
8 0
aobjpl 52 1 0 0 1 0 1 1 0
8 0
amappl 40 1411 0 1395 1 0 1 1 0
8 0
bufpl 116 24 0 24 1 0 1 1 0
8 1
mbpl 256 646 0 325 30 8 22 22 1
8 1
mclpl 2048 578 0 258 165 0 165 165 4
3072 5
sockpl 200 77 0 77 1 0 1 1 0
8 1
procpl 328 148 0 137 1 0 1 1 0
8 0
zombiepl 72 137 0 137 1 0 1 1 0
8 1
ucredpl 80 1 0 0 1 0 1 1 0
8 0
pgrppl 24 2 0 0 1 0 1 1 0
8 0
sessionpl 48 2 0 0 1 0 1 1 0
8 0
pcredpl 24 148 0 137 1 0 1 1 0
8 0
filepl 48 547 0 544 1 0 1 1 0
8 0
fdescpl 292 149 0 137 1 0 1 1 0
8 0
pipepl 72 8 0 8 1 0 1 1 0
8 1
sigapl 316 140 0 137 1 0 1 1 0
8 0
wdcspl 96 7 0 7 1 0 1 1 0
8 1
scxspl 128 492 0 492 1 0 1 1 0
8 1
namei 1024 1310 0 1310 1 0 1 1 0
8 1
vnodes 196 164 0 0 9 0 9 9 0
8 0
nchpl 64 448 0 0 8 0 8 8 0
8 0
ffsino 284 149 0 0 11 0 11 11 0
8 0
pagedeppl 68 4 0 0 1 0 1 1 0
8 0
inodedeppl 84 17 0 0 1 0 1 1 0
8 0
newblkpl 32 35 0 35 1 0 1 1 0
8 1
bmsafemappl 32 2 0 0 1 0 1 1 0
8 0
allocdirectpl 68 35 0 31 1 0 1 1 0
8 0
freefragpl 32 5 0 5 1 0 1 1 0
8 1
freeblkspl 108 1 0 0 1 0 1 1 0
8 0
diraddpl 32 2 0 0 1 0 1 1 0
8 0
dirrempl 32 13 0 0 1 0 1 1 0
8 0
dirhash 1024 22 0 0 6 0 6 6 0
8 0
rtentpl 108 58 0 2 2 0 2 2 0
8 0
plimitpl 152 2 0 1 1 0 1 1 0
8 0
inpcbpl 216 34 0 34 1 0 1 1 0
8 1
In use 873K, total allocated 1040K; utilization 83.9%
Visit your host, monkey.org