Re: PLEASE TEST snapshots (long)

[Rick Robino <rrobino@wavedivision.com>]

Steve,

Nice site, and I agree with your aims. However, my proposal is
underneath version tracking as it stops at rounding up a group of
test boxes running various versions on different arch's. Self serve
chaos, use at own risk, no hassle variety, better than not testing
that sparc port at all thing.

Hopefully people would hasten the update of their ports with such
a facility and then check them in somehwere - your site or the one
at infomatrix (http://infomatrix.ca/obsdproj.php).

I was thinking more along the lines of maintaining a list of
platforms, versions and users in one place; the machines being very
lightly maintained so as to not be a burden. Probably a limit on
outbound bandwidth and a fresh install of current or stable for
given architectures nightly and thats about it.

All the same, please do drop me a line when testing is over.
If this area needs help I'd be happy to contribute and if
there is something else like it already going on, I have no
desire to duplicate efforts.

--Rick

> On Sat, Apr 06, 2002 at 07:15:55PM -0500, STeve Andre' wrote:
> Coordinated testing is always a good idea.  Given the number of ports,
> about 6000 for all the platforms, there more than one year of full time
> work to test everything, and that doesn't include the ports.
> 
> Thats why I created the stupid little system at 35.8.65.46, so it could
> be the start of something that people might work together on.  Static
> web pages to hold info on testing is desperataly bad, but it gets
> something up for the moment.  A better database system could be
> built later.
> 
> So yes, there is interest in such a system from here.  I'm not sure
> that a different mailing list would be needed for it, but perhaps.  My
> ideas on this have evolved into a system where packages are 
> checked into the database, with falvours for the different platforms,
> and comment areas for each along with the name of the tester.
> 
> 6000+ packages is an incredible amount to test, and there will
> only be more in the future.
> 
> I don't think this idea will encounter resistence from the developers,
> but will be met with some skepticism in that people often talk but 
> don't do anything.  Thats why I came up with the idiot system I 
> did--do, and improve on it, but actually do something.  This is NOT
> to disparage what you are saying--I like it.  We should talk after
> the current testing phase is done.
> 
> --STeve Andre'
> 
> On Saturday 06 April 2002 04:23 pm, Rick Robino wrote:
> > > On Thu, Apr 04, 2002 at 09:46:33PM +0200, Marc Espie wrote:
> > > 3.1-beta is really, really close.
> >
> > ...
> >
> > > Please test the damn packages !
> > > Please test the damn packages !
> >
> > ...
> >
> >     Request for Comments Regarding User-community Test Bed:
> >
> > Would anyone be interested in an _informal_ but somewhat organized
> > coalition of advanced users establishng a test farm for ports and
> > general snapshots?
> >
> > I know that there are alot of people who thoughtfully keep themselves
> > out of the main kitchen, but might be able to offer different arch's
> > and versions without too much oversight on their part. Despite the
> > potential for abuse, an array of test systems loosely open to a wide
> > audience could be valuable for the individual who has something to
> > contribute and test, but doesn't have the time or money to keep a
> > properly varied test setup on hand.
> >
> > Most people on these particular lists can likely handle the operational
> > issues such as quarantine for one box at a time, updating from cvs,
> > dealing with out of space issues, reinstalling and banning after a
> > bad apple, their own self-defense and good behavior on other people's
> > systems, and so on. Many also are likely to have an extra non-i386
> > machine looking for a purpose.
> >
> > If approved, such an effort would need to set up some coordination
> > facilities, such as a website, mailing list - agreements on how and
> > what to use for regression tests. If some very loose and easy to
> > abide by standards could be agreed on, the effort could produce a
> > more accurate report from the field on what is working, and other
> > benefits - maybe fewer surprise bugs in less popular ports.
> >
> > This is just an idea. Seems to me that this kind of thing could fly
> > if very few regulations or other barriers to experimentation are
> > imposed. Just a bare minimum to get a better read, even if on an
> > inscrutable personal level for the experimenter. It would be outside
> > of and support in a (small?) way higher food-chain projects like
> > ports and obsd-only original development.
> >
> > Sorry for the length - I am not proposing more complication for the
> > folks who are already very busy - not even to worry about this
> > becoming an orphan they might have to deal with in the future.  Just
> > a little bit of herding and it could be destroyed if it fails to
> > be useful enough or outlives its purpose. I think anyone of like
> > mind would agree that such a burden doesn't deserve to live.
> >
> > Feedback? It is probably not a good idea to clutter the lists I
> > posted to (tech and ports) with the responses. Send them to:
> >
> > obsdtest@wavedivision.com
> >
> > I am willing to contribute some planning, various support services,
> > some hosting, a box, and to take responsibility for housekeeping
> > like status reports.
> >
> > Of course, if any of the full-timers (and you know who you are)
> > think this is a bad idea, I'll respect that. Just thought I'd
> > dip my toe in before I went to alot of effort. Whoever weighs
> > in, I will post a much shorter note in a week to update folks
> > on what the dialogue results were.
> >
> > p.s. I'd rather keep this out of misc right now; no offense to
> > them, just probably not the pertinent audience.

-- 

Rick Robino                                          v. (503) 332-4452
Internet Systems Architect                           f. (503) 439-8946
Wave Division Consulting                             @. wavedivision.com