ANNOUNCE: CTM for OpenBSD - get the current OpenBSD source- or cvs-tree daily via email (public beta test)

[Thomas Graichen <graichen_(_at_)_axp5_(_dot_)_physik_(_dot_)_fu-berlin_(_dot_)_de>]

                              A N N O U N C E

 CTM for OpenBSD - get the current OpenBSD source- or cvs-tree daily via email

--------------------------------------------------------------------------------

WHAT IS CTM ?

ctm is a system which was designed by paul-henning kamp for makeing all the
changes to the FreeBSD source- and cvs-tree (and some other trees - like the
FreeBSD ports-tree) available on a daily (or multiple time per day) basis by
email.

this is a very good way to stay up to date with the current source- or
cvs-tree if you have a bad internet connection - for instance via modem. ctm
uses only very low bandwith for distributing the changes to the tree - and it
compresses these differences using gzip -9.

HOW DOES IT WORK ?

the basic idea of ctm is - that you subscribe to a special mailinglist to get
the daily changes of the OpenBSD source- or cvs-tree and then you'll get a
mail each day containing all the changes to the tree (maybe we increase the
number of ctm files sent per day in the future) - one such set of differences
is called a "delta". 

if you received your first delta - you have to download all the delta's until
the first delta you received by email - this always starts with a big - so
called "base delta" - which contains the full source tree you start
with. starting from this big one all other deltas only contain the changes to
the tree of the day of the delta generation. 

to avoid that you have to get the base delta and then some hundred relative
deltas a new base delta is generated as a startpoint each 100 relative
delta's - so that you always have to get less than hundred relative deltas for
starting ctm :-)

HOW MUCH BANDWIDTH DOES CTM USE ?

as said above the base delta contains the full source- or cvs-tree so it is
really big - you'll have to get it via ftp from the site below. here are the
sizes of the current base deltas:

  ~ 45 mbyte for OpenBSD-src.0001.gz - the base delta for the source-tree
  ~ 55 mbyte for OpenBSD-cvs.0001.gz - the base delta for the cvs-tree

and the size of the relative deltas vary from ~ 10 kbyte to ~ 100 kbyte
usually and sometimes they are some (1,2 or 3) mbytes - but they are always
split into 100 kbyte mails which will automatically assembled from the
ctm-programs - the bigger sizes come from bigger changes in the source- or
cvs-tree (update of the binutils, import of perl etc.)

HOW DO I USE CTM ?

ctm is very simple to use:

* subscribe to the ctm mailinglist you want

    echo subscribe OpenBSD-src | mail majordomo_(_at_)_OpenBSD_(_dot_)_org

  for the source-tree or

    echo subscribe OpenBSD-cvs | mail majordomo_(_at_)_OpenBSD_(_dot_)_org

  for the cvs-tree - you should also subscribe to

    echo subscribe ctm-announce | mail majordomo_(_at_)_OpenBSD_(_dot_)_org

  on this list i'll post announcements for users of ctm - and at the moment i
  also post there somekind of a log of the ctm delta generation - so you can
  see what is all in the ctm delta and if the assembling of it succeeded - you
  should really subscribe to this list if you use ctm

* wait until you get your first delta

* unpack and assemble the ctm delta using ctm_rmail (for the sources to all
  the ctm-programs see below)

    ctm_rmail  -p. -d. -b. the_mail_folder_containing_the_ctm_delta_mail

  after that you should have the bare ctm delta there (something like
  OpenBSD-cvs.0015.gz or so)

* if you have this - get all the deltas beginning with 0001 up to your first
  mail-received one via ftp from

    ftp.cs.tu-berlin.de:/pub/bsd/CTM/OpenBSD

  also known as

    ftp6.de.FreeBSD.org:/pub/bsd/CTM/OpenBSD

  for the tree you subscribed too - i'm very interested in sites all over the
  world who wan't to mirror this or want to put up the ctm deltas there too

* then run ctm (you can get the sources of ctm which should compile at least
  on OpenBSD just fine from the same ftp adress - maybe you have to tweak it a
  bit for other systems - you may need the libmd from the FreeBSD sources
  there eventually)

    cd where_you_want_your_source_tree_to_be
    ctm -v -v /where_your_ctm_deltas_are_stored/OpenBSD-src.*

  or

    cd where_you_want_your_cvs_tree_to_be
    ctm -v -v /where_your_ctm_deltas_are_stored/OpenBSD-cvs.*

  this will take some time for the base delta - and less time for the relative
  ones

* now you can run ctm_rmail and ctm on every new ctm delta to get up to date

WHAT SHOULD I CHOOSE - SOURCE-TREE OR CVS-TREE VIA CTM ?

this simply depends on your diskspace - the better is the cvs-tree - because
you can make modifications to the local checked out copy (ctm can't yet handle
modifications in files it want to update) - but the drawback is a _lot_ of
diskspace - you need currently 

  ~ 240 mbyte for the full cvs-tree and
  ~ 200 mbyte for the checked out source tree plus
  ~ 100 mbyte for the obj files while "make build"

if you have enough space then go for the cvs-tree - then you simply have to
check out a local copy of the source tree after the first ctm -v -v command
using

  cd /usr
  cvs -q checkout -PA src

and after each new applied ctm relative delta you have to run

  cd /usr
  cvs -q update -PAd src

to get your local copy up to date

if you don't have enough space so you can use ctm for getting the checked out
copy of the source tree - but then you have to think about the handling of
local modifications - here something someone from the NetBSD people is using
a mixture of null and union mounts and some scripts for doing this - if i
remember right it was something like this

  /usr/src-local is null mounted to /usr/src
  /usr/src-ctm is union mounted to /usr/src

so you have all local modifications in /usr/src-local and you have to run a
script which does something like

  for all files in /usr/src-local diff them against the /usr/src-ctm versions
  and apply this patch to a fresh ctm applied /usr/src-ctm tree mounted in the
  above way against a fresh /usr/src-local directory

the union mount of /usr/src-ctm guaranties that the /usr/src-ctm tree is not
modified which is very importand for the operation of ctm

all this is just a rough idea - all in all - it is doable because someone from
the NetBSD people is definitely doing it in a way close to this one

HOW STABLE IS CTM FOR OpenBSD ?

i will call this now a "public beta" state - i'm using ctm now for more than a
month to stay up to date with the OpenBSD cvs-tree without any problems - and
alss the FreeBSD people use ctm very extensively for years know without any
major problems i think (i also use ctm for FreeBSD now for about half a year
without any problems) 

so i think you really can trust ctm and the name "beta" is only because until
now ctm for OpenBSD was only tested by me.

WHERE CAN I FIND MORE INFORMATION ABOUT CTM FOR OpenBSD ?

nowhere :-) - but you can take all the information from FreeBSD and you have
the man-pages for the ctm-programs in the ctm source-package

there are some informations about the use and ideas of ctm in the FreeBSD
handbook - which you will find somethere under 

  http://www.FreeBSD.org/handbook/ctm.html

sorry - i don't know the exact adress for the ctm info - but look for the
handbook and there for the information about ctm - you'll shurely find it :-)

if you have problems with ctm you may also ask me - i'll try to help you as
far as i can and as far asi have enough time (which is a big problem in the
moment) 

OK - IF YOU GOT HERE ...

... and decided to use ctm from now on - here's the summary of adresses:

mailinglists:

  echo subscribe OpenBSD-src | majordomo_(_at_)_OpenBSD_(_dot_)_org
  echo subscribe OpenBSD-cvs | majordomo_(_at_)_OpenBSD_(_dot_)_org
  echo subscribe ctm-announce | majordomo_(_at_)_OpenBSD_(_dot_)_org

ftp server:

  ftp.cs.tu-berlin.de:/pub/bsd/CTM/OpenBSD
  ftp6.de.FreeBSD.org:/pub/bsd/CTM/OpenBSD

there you'll find the deltas and the sources to ctm (i'll let wolfram
schneider upload them there in the next hours/days - so you'll have to look
there again if they are'nt there)

i'll also make any importnat notes and announcements about ctm in
ctm-announce_(_at_)_OpenBSD_(_dot_)_org, misc_(_at_)_OpenBSD_(_dot_)_org and the big ones in
announce_(_at_)_OpenBSD_(_dot_)_org 

and pleeese - send me a mail if you use ctm for OpenBSD so that i can see how
many people are using it !

THANKS ...

... go to:

poul-henning kamp (phk_(_at_)_FreeBSD_(_dot_)_org)
  for making ctm possible and helping me getting the delta generation running

theo de raadt (deraadt_(_at_)_theos_(_dot_)_com)
  for making OpenBSD possible :-) and giving me all the ressources i needed
  for creating the ctm deltas on cvs.OpenBSD.org

wolfram schneider (wosch_(_at_)_FreeBSD_(_dot_)_org)
  for setting up the ftp space for the deltas on ftp.cs.tu-berlin.de

... and all the others who contributed indirectly

SO - GOOD LUCK WITH CTM FOR OpenBSD wishes

t 

 (graichen_(_at_)_physik_(_dot_)_fu-berlin_(_dot_)_de or graichen_(_at_)_FreeBSD_(_dot_)_org or graichen_(_at_)_OpenBSD_(_dot_)_org)

-- 
  thomas graichen    graichen_(_at_)_mail_(_dot_)_physik_(_dot_)_fu-berlin_(_dot_)_de    graichen_(_at_)_FreeBSD_(_dot_)_org

  perfection is reached, not when there is no longer anything to add, but when
      there is no longer anything to take away    antoine de saint-exupery

-- 
  thomas graichen    graichen_(_at_)_mail_(_dot_)_physik_(_dot_)_fu-berlin_(_dot_)_de    graichen_(_at_)_FreeBSD_(_dot_)_org

  perfection is reached, not when there is no longer anything to add, but when
      there is no longer anything to take away    antoine de saint-exupery