Re: "Out of memory!" errors running sa-learn on 3.6-STABLE AMD64

[Kevin <kkadow@gmail.com>]

On Mon, 31 Jan 2005 22:46:07 -0600 (CST), C. Bensend
<benny@bennyvision.com> wrote:
>   I just cut over my personal web/mailserver to a 3.6-STABLE system,
> running an AMD64 3000+ with 2GB of DDR400 RAM.  For the most part,
> things are running smoothly, but I'm seeing some strange memory
> issues every now and again.
. . .
>   The one I'm fighting with now is training my SpamAssassin Bayes
> database using 'sa-learn'.  I'm attempting to train a directory of
> 5000 corpus spam emails (something I did often on the old PIII-800
> with 512MB of RAM).

The sa-learn process can consume a significant amount of RAM,
and depending on the OS version you were running on the P3/800,
memory consumption of perl (sa-learn is a perl script)  may have
increased significantly with the move to ELF, etc.


>   What I'm getting is this (wrapped for your viewing comfort):
> 
> [benny@fusion ~]$ time sa-learn --showdots --spam --dir
> /home/benny/Maildir/.SPAM.corpus.2004.archive10/cur/
> ..................................................................
> ..................................................................
> ..................................................................
> ..................................................................
> ..............................................Out of memory!
> 
>   Out of memory?  I still have over a gig of RAM free, no swap is
> being used, and my user is in the 'staff' login group.  OK, so I'm
> obviously hitting some sort of limit - I changed my user to the
> 'daemon' login group (and yes, logged out and back in).  Same thing.
> Hrm.

OpenBSD's  default limits are set in /etc/login.conf based on the 'class',
you will want to adjust datasize-cur.  If you use  the 'ksh' shell, the 
built-in command 'ulimit -a'  will show the current limits.

The whole 'class' thing took me a while to get used to, being completely
distinct from the account's primary group and all.


>   This machine is running SpamAssassin 3.02 built from source, as I
> opted for a newer version than is in the ports tree.

I've encountered the exact same issue with sparc64, perhaps the issue
is in part related to 64-bit mode?  Or more likely, sa-learn is simply a
memory hog :)


Kevin