Re: IPSec Error PAYLOAD_MALFORMED (OpenBSD-3.0 + Cisco 7200)

[Hakan Olsson <ho@crt.se>]

First, my guess is a mistyped shared secret. (A typo will cause bad
decryption on the reciever side --> "malformed payload".)

...

At a glance, your isakmpd.conf, pf.conf, policy etc look ok. (Do you need
to run DES + MD5? Most systems today support stronger variants...)

> ########################################################################
> 5) ipsecadm flush (just in case...)
>    ps axu       -> no isaknmp daemon running
>    netstat -nr  -> nothing under "Encap"
>    netstat -na  -> nothing listerning on port 500
> ########################################################################
> 6) Now execute isakmpd -c /etc/isakmpd.conf -d -DA=9 -D1=70 (snipped)

Why -DA=9 ? I assume you meant to type -DA=90, which give more (lots...),
and more detailing debug output. Still, what I saw in the log file mostly
looks like a misspelled shared secret. As '9' is not a high enough log
level to show if it actually was (most debug info start at level '10'), I
can't be more specific than this.

If you turn on more debugging and see a message such as "unknown payload
type NN", with NN larger than 14, it's almost certainly a bad passphrase.
(Why 14? If you have the source, see the isakmp_num.cst file :)

For general IPsec troubleshooting, look at the OpenBSD FAQ, ch 13.

/H

--
Håkan Olsson <ho@crt.se>        (+46) 708 437 337     Carlstedt Research
Unix, Networking, Security      (+46) 31 701 4264        & Technology AB