[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: Parsing tcpdump files
- To: tech_(_at_)_openbsd_(_dot_)_org
- Subject: Re: Parsing tcpdump files
- From: "Steve Bernard" <sbernard_(_at_)_gmu_(_dot_)_edu>
- Date: Sat, 01 Mar 2003 21:56:26 -0500
- Organization: George Mason University
Thanks for the suggestions. Up until now I've been doing basically what
you have suggested, but with small files (300MB), and using
BASH/grep/etc. for the parsing. Importing everything into a RDBMS was my
first strategy but that isn't feasible right now. I intend to store the
reporting data in MySQL but I don't have the disk space to duplicate
everything in the RDBMS. I was hoping that there were some tools for
parsing the data in binary format because converting to ASCII adds about
35% to the file sizes, and my customer prefers the data to be in
tcpdump's binary format. I still need to compare the file sizes using
hex format instead of ASCII. Unless I hear other suggestions I plan on
going forward using Perl or Python.
Joseph C. Bender wrote:
On Saturday 01 March 2003 08:30 pm, Steve Bernard wrote:
I'm supporting several granted network engineering and security analysis
research projects. Each project has specific data requirements and
capabilities. To facilitate this I need to perform string parsing, data
aggregation/sub-setting, statistical analysis, and reporting. They will
each do much more on their own but, this is what is required at my end. I
anticipate the capture files being around 1GB each.
Well, I just had to do some looking at a 12 hour capture of some database
I did the capture with Tcpdump, then did my initial sorting with tcpdump
reading the file with filtering for each major grouping of info I needed.
Because I was looking for something, I then pulled each file into Ethereal to
better see the captures, and display-filter the data in question.
However, it would seem that you'd want to write the ascii output of tcpdump to
files, then using $PARSING_LANGUAGE_OF_CHOICE to generate your numbers, or
pull the info into a RDBMS for using some major tool.
Visit your host, monkey.org