Saga of bugs/wish lists OBSD 3.1 install w/ USB mass storage & keyboard

[bsd user <fbibsd@yahoo.com>]

Hi all,

This is long,  but I am just typing in real time as I go through
the installation of a new box it so that it might be  of use to
anyone  else out  there with  similar problems,  or that  it  might be
useful to point out possible  improvements to those who are developing
OpenBSD.

I apologize  for the tedious details  / length; it isn't  a flame, and
I'm wonderfully happy  that OpenBSD is being developed  and is as good
as it is (I've been using it for a couple years).

I just am typing this step  by step as I actually encounter or remeber
various  issues  so that  I  don't forget  or  find  myself unable  to
properly detail the conditions / circumstances involved.

OK

I just got the OpenBSD 3.1 release CD set (yea!), and a new box to run
it on.   I've been successfully  used/installed OpenBSD 2.8,  2.9, 3.0
before, though  never on  this particular system  nor from and  to USB
mass storage CD/HD.

The motherboard is all in one ser/lan/agp/par/usb/ide/fdd/ac97/game, a
VIA VPSD  EPIA with their EDEN  x86 compatible CPU  installed.  It's a
very neat, modern, tiny sized motherboard that features very low power
consumption, and  is designed to  run with NO  FANS (not even  for the
CPU)!

It is very compelling for "always on" internet appliance / server type
uses since it should be very quiet and consume well under ten watts of
power  for  the whole  system.   (CPU  2.5  watts actively  processing
normally, less in various sleep / idle / power management states).

Anyway,  needless  to  say,  I'd  really  like to  see  this  type  of
configuration cleanly supported  since it makes a lot  of sense (and I
got this box just for an OpenBSD 'always on' server!).

The EDEN processor should be as compatible as their (VIA's) C3; I know
of  no problems  with either;  it seems  to work  in the  boot process
anyway.
http://www.viavpsd.com/product/epia_mini_itx_spec.jsp?motherboardId=21

Hardware configuration:  
No FDD, No EIDE devices  
Motherboard VIA VPSD EPIA Mini ITX with VIA EDEN CPU, 
Memory: 2*512MB for 1GBy PC-133 SDRAM
Motherboard's PCI EIDE controller: Disabled  in CMOS setup (or not; no matter)
Motherboard's FDD controller: Disabled in CMOS setup 
Boot dev: USB CD (IDE  DVD/CD-ROM in ME-320U IDE to  USB 2.0 external enclosure)
HDD dev:  USB HD (ACOM  DATA Model: External  USB 2.0 Hard  Drive P/N: HD040U2E Capacity: 40GB

USB:  PC->Keyspan USB  1.1 powered  4  port hub  -> USB  CD, HD,  KBD,
trackball

----------------------------------------------------------------------
** APPARENT BUG / ANNOYING CIRCUMSTANCE #1:

Anyway the box boots off of  a USB 2.0 DVD-ROM, starts loading OpenBSD
3.1's boot image from release CD #1, and displays:

Boot  from CD  :  1. FD  2.88  MB System  Type  (0F) reading  boot....
probing: pc0 com0 apm mem[639K 1014M apm=on] 
disk: fd0 hd0*

and then it just sat there  for long periods of time (several minutes)
before proceeding in  the boot sequence.  Actually I  was going to say
it locked up, but as I typed this message it finally proceeded!

This seems a  bit excessive for a delay timeout for  what I presume is
non-configured  intentionally  absent hardware  for  which a  suitable
alternative is active, sensed, and connected. (i.e. hd0 probing is too
long if that's what it's waiting for!)

N.B. I  have *no* actual floppy  drive hardware or IDE  devices in the
system; the storage devices are the USB DVD ROM and the USB HDD.  This
probably is  not an untypical configuration, looking  forward into the
future of how preople will put things together.

N.B.  I tried this  both with  and without  the PCI  EIDE and  PCI FDD
controller  chips totally disabled  in the  system's "CMOS  setup".  I
presume it's got a built in LONG timeout probing for IDE HDD devices.

I'd suggest  that if the system  reports no IDE HDD,  and doesn't even
seem  to have  an enabled  HDD controller  channel, maybe  the timeout
could be less (especially  if there IS a USB / 1394  / SCSI / whatever
other type  of storage device).   Since there are so  many inexpensive
USB  / 1394  /  USB-FLASH type  storage  devices that  are simple  for
non-technical end users to mate to their systems, people will probably
use non IDE non SCSI configurations increasingly.

For my  next box I may  well just net  boot it diskless /  CD-less and
have NO local  mass storage devices of any sort  at all. i.e. configre
RAM memory disks / filesystems a custom kernel boot image and have the
LAN be the only peripheral.

After I get  the system installed I'll just unplug  the USB CD/DVD and
can reuse it on another system.  If I want to replace a failed HDD, or
upgrade  it, I  can just  plug in  another USB  unit in  thity seconds
without having to go into the PC's case at all.

So anyway  it'd be nice if this  was not excessively slow  to boot the
installation image on, and also if the problems below was corrected.

----------------------------------------------------------------------
** APPARENT  BUG /  ANNOYING CIRCUMSTANCE  #2  (would be  killer if  I
didn't have a workaround PS/2 keyboard to use for installation):

N.B. CMOS setup settings enabled use of integrated USB controller, and
ENABLEd the use of a USB Keyboard.

N.B. USB keyboard functioned for BIOS CMOS setup.

N.B. OpenBSD successfully booted from, saw, and 'almost' worked with a
USB HD on the same USB HUB as the USB Keyboard / Trackball.

History: Me:  takes coffee  break waiting for  the boot hdd  probes to
timeout...  ..ok after several minutes system has finished probing and
has gone further in the boot process.  The screen shows lots of kernel
diagnostic "dmesg" type messages, and kind of mixed in the *middle* of
the kernel diags is a  recognizable part of the (I)nstall (U)pgrade or
(S)hell or whatever (I didn't write it down) prompt.

NOTABLY *after* (in the VGA console text stream) the appearance of the
"(I)nstall  (U)pgrade  or  (S)hell"  prompt is  various  other  output
INCLUDING the detection / probing  / whatever (I didn't write it down)
of the  Microsoft Internet Keyboard  as ukbd!  So  it did see  the USB
keyboard and apparently plumb it as  a usb keyboard.  I can repeat the
boot process  to see exactly what  it said; I saw  nothing that looked
like an error message concerning this.

Personal note: I assume maybe the  kernel diag output is mixed up with
the  installation  script's  prompt   message  due  to  some  kind  of
independent  output  stream  buffering  /  printf  /  syslog  kind  of
thing.. not necessarily BAD, but possibly very confusing, and it looks
bad too. :)
 
So then...   
Me: "Oh cool!  It  might just work after  all!"  
Me: Types "I" on USB keyboard to  begin installation, hmm, nothing.
Me: Shift-Lock,  Scroll-Lock,   Num-Lock.   Nope.  USB keyboard isn't
reacting.   
Me: Plugs in PS/2 keyboard to box. Ah! PS/2 Keyboard works.  Doh!

N.B.  I've noticed  OpenBSD in  the past  to be  a little  'weak' with
respect  to keyboard  robustness for  PS/2 keyboards  e.g.  failure to
automatically recover  / initialize  if the keyboard  is disconnected,
hot plugged, whatever.   I don't seem to rememeber  whether I had much
luck with USB  keyboards in the past; my  general rememberance is that
I've never had a lot of luck  using *only* a USB keyboard on any of my
boxes  for everything including  CMOS setup  and O/S  installation for
various OS's.

This new box  seems to handle the CMOS setup  with USB keyboards fine,
but OpenBSD doesn't seem to plumb it during installation.

N.B: Oh this Microsoft Internet Keyboard does integrate a two external
port *non powered* USB hub into it (I don't use it, it's just there).

* I don't remember if it was  FreeBSD or OpenBSD that does this little
bit in  this paragraph  (memories of a  past release), but  it (*BSD?)
just WOULD NOT let me use this keyboard if I plugged the keyboard into
my four external port POWERED USB 1.1 HUB, but the keyboard WOULD work
(well sometimes anyway...  :) if I plugged it right  into the PC's USB
ports.   The  O/S  *saw*  the  plug in,  emitted  messages  about  its
detection, and just  refused to *use* / *plumb*  it that way something
about hub  to hub chaining or  whatever.  I only mention  this in case
that  might  have something  to  do  with  this particular  keyboard's
failure  in OPENBSD  3.1 installation.   I think  other  USB keyboards
(lacking hubs) did work whether they were connected to the powered hub
or  PC directly.   I never  did 'get  that' apparently  OS intentional
arbitrary limitation; if it works  plugged into the PC, it should work
when plugged into a powered USB hub just as well.  Anyway, I know this
is slightly vague documentation, but I doubt the problem(s) are at all
inscrutable with respect  to this.  I'll provide more  details / tests
if needed / when I have time in a few days.

USB sucks,  and is prone  to lots of  support flakiness still,  but it
beats the usual alternatives if it'd just work.

It'd be nice to  be able to use an all USB  system since at least it's
nice to be  able to have hubs, extension cords,  hot plugging, and all
that good stuff.   Especially with six or so CPUs in  the room and not
*really* wanting  to have six distinct  mice, keyboards, et.  al. or a
$400 intelligent KVM switch vs.  being able to just dynamically switch
/ plug  one USB hub into  the box I want  to talk to and  be done with
it...

OpenBSD3.1 install disc 1 specific behaviour clearly documented:

Well I feel  bad about being vague, so I just  tried this and recorded
the exact reactions of OpenBSD 3.1 disc 1 installation:

OK sitting at the 
"Configure  the nwtwork? [y]" prompt in install (3.1 rel CD1)
I  unplugged the MS Internet USB  kbd (non usefully working),
unplugged the  PS/2 keyboard (working  alternative), and plugged  in a
QTRONIX USB keyboard  (keyboard with a PS/2 mouse port  in it that you
can plug a PS/2 mouse into and  use them both over the USB channel) in
its place.

* The keyboard is plugged into a 4 port Keyspan powered USB 1.1 hub.

OpenBSD  3.1  VGA  console  screen shows:  ...various  messages  about
disconnect /  reconnect et.  al. as I  fiddled around until  I finally
plugged the  new keyboard in  for the last time...   
"wskbd1 detached"
"ukbd0 detached" 
"ukbd0  at uhub0 port 4 configuration  1 interface 0"
"ukbd0: QTRONIX USB Keyboard and  Mouse, rev 1.00/1.10, addr 6, iclass 3/1" 
"wskbd1 at ukbd0 mux 1"

OK so it sees the new USB keyboard (one I've had less "issues" with in
the past),  plumbs it, but apparently  not for console  input since it
isn't  able to type  into the  prompt or  change its  LEDs when  I hit
shift-lock.  The LEDs did cycle while  I was plugging it in and the OS
was plumbing it.  So it isn't  a problem with the MS Internet Keyboard
and  that  HUB,  and  OpenBSD  3.1  likes the  hub  and  the  keyboard
apparently.  I'd guess it's  not being intelligent about assigning the
console to the USB keyboard in these situations.

IMHO  *any* keyboard /  mouse that  is plugged  in dynamically  at any
point should get 'bound to' the main console input / main pointer uses
if there are no other such  devices still functional.  At least in the
'default' kernel / configuration.

In the past  (OpenBSD 2.9ish timeframe) I've had to  ssh into my other
box *many  times* over the  LAN to issue  a 'shutdown' /  'reboot' for
absolutely  no  other reason  than  that  OpenBSD  had simply  stopped
talking to its keyboard and  would not (not even with coaxing) recover
/ reinitialize  the console/keyboard even  when a new one  was plugged
in.  This may be true for PS/2 and or USB; I forget the details except
that it was very frustrating.

Use of whatever the xrefresh  / kbdcontrol / whatever commands I could
find kbdmap / that claimed to be able to diddle the console / keyboard
never were able to stimulate  recovery once into this state, nor would
any small number of "unplug" "replug" cycles.

Now the  REASON that it stopped  talking to a  given keyboard probably
did involve me manually  disconnecting that keyboard while rearranging
cables or borrowing the keyboard for another CPU or whatever (I have a
4 port non-intelligent mechanically  switched KVM PS/2 switch box, for
instance).

However excepting the case  where the computer hardware actually locks
up totally, it ought to be able to reinitialize the keyboard interface
if it needs to (maybe  even periodically; once a minute?) to guarantee
that  a switched  out,  failed, or  substituted  console input  device
doesn't effectively  incapacitate an 'always on' internet  server or a
workstation  that may  have lots  of  open programs  and unsaved  data
needing manual saving before the system shuts down.


----------------------------------------------------------------------
** APPARENT BUG / ANNOYING CIRCUMSTANCE #3:

At  various points  prior to  and after  the  installation's partition
definition / editing  / creation process on what I  am presuming to be
the USB  HDD: (ACOM DATA USB  2.0 Hard Drive) Model:  External USB 2.0
Hard Drive P/N: HD040U2E Capacity: 40GB

...the following kernel error message is printed:

sd0:   could  not   mode  sense   (4/5);  using   fictitions  geometry
sd0(umass0:1:0):  Check condition  on  opcode 0x8  
SENSE KEY:  Illegal Request 
ASC/ASCQ: ASC 0x21 ASCQ 0x00

Personal note:  I have no idea  whether the USB  enclosure chassis for
the HDD is really fully implementing all the proper USB diagnostics or
not, but I believe it must  be supporting "enough" of them since it is
certainly automatically  recognized as  being present, and  having the
right ~ 40 GB total capacity.

** I   would  suggest  that   "geometry"  features   are  increasingly
meaningless as  disk drive interfaces get more  and more "virtualized"
relative to  the physical  hardware.  e.g. a  compact flash  IDE solid
state drive /  PCMCIA ATA flash drive is an  IDE HDD, but "cylinders",
"interleave", "RPM",  "sectors / cylinder" et. al.  are pretty virtual
and really only  the total capacity matters much.   For a USB external
mass storage  device which may  be flash or  a real hard disc  unit, I
really wonder  if the OS's  attempt to optimize the  disk partitioning
based upon  what might be a  doubly or triply  virtualized geometry is
sane.   e.g. translations at  points :  
(1: inside  the device  at its controller level)  
(2: at  the device's external  controller interface e.g. USB or 1394)
(3: whatever the BIOS or OS does in terms of translations / encodings)

** Therefore it  might be nice if just  a "raw" mode or  flag could be
set  to cause  things like  interleaving, RPM  based  optimizations, #
heads based optimizations, maybe even spare sector/cylinder inclusions
(since often this is handled in external hardware/firmware/RAID/controller)
to 'go  away' (or  be set  to sane parameters for a purely logical block
virtualized type device).

Anyway this all seems to  be creating a more concrete serious problem,
e.g.:

----------------------------------------------------------------------
** APPARENT  BUG /  ANNOYING  CIRCUMSTANCE #4:  The partition  editing
program would not calculate incremental partition offsets! e.g.

...after (as above) such messages as: 
"sd0: could not mode sense (4/5); using fictitions  geometry"
"sd0(umass0:1:0): Check  condition on opcode 0x8"
"SENSE KEY: Illegal Request"
"ASC/ASCQ: ASC 0x21 ASCQ 0x00"

...with sd0 (as  above) being a USB mass  storage enclosure holding an
IDE disc.

"Do you want  to use the *entire* disk for OpenBSD  [no] yes"
...  the automatic process  created partition "c"  of what seemed to be
around the full size of the disk  (fine).  

It also auto created partition "i" (not "a"!) as something a little less
than  the disc capacity (weird, but ok).

** Personal note  -- maybe it assumed  to create 'i' not  'a' since it
never assumed  one would be intending  to put the system  root on sd0?
Bad assumption!

Then  I manually  deleted "i"  partition, entered  "a a",  entered the
size, and mount point (/), however  the offset for "a" prompted as [0]
not the usual [63].  So I entered 63 ofset for "a".

Then when  I went to  create b  (swap) with "a  b".  It asked  for the
size,  and again  prompted me  with a  default offset  of [0]  not the
expected (63 + partition a size), so I had to manually calculate it...

...then  again for  my  adding partitions  d,e,g,h,i  it would  always
prompt with  offset [0]  never automatically calculating  / accounting
for the size of the previously allocated partitions / areas.

Painful; I've never had that fail in past OpenBSD installations.

* Though I  do remember  a past OpenBSD  (2.9?) being kind  of finicky
when the geometry of a HDD that (80GB) was bigger than the BIOS really
could deal with.   It would only want to 'see' 40GB  of the 80GB total
because 40 was all  the BIOS could see, and I had  to go in and define
manual  geometry  parameters  during  the  OpenBSD  partition  editing
process  which was  a  little  weird /  complicated  since the  'right
numbers'  weren't really  well specified  by the  HDD's OEM  since the
geometry was virtualized anyway so they were suggesting numbers that'd
work  into people's  BIOS  limitations, not  numbers  that would  best
reflect the actual device characteristics.  Since *NIX doesn't use the
BIOS it doesn't really matter what the BIOS wants, but it'd be nice if
it'd just get the "total capacity" from the device itself and work off
of that alone  or in conjunction with what geometry  it can figure out
from the device itself.

* Oh and  added to the  'wish list' for  partition editing --  why not
display  things in quantities  of K,  MB, GB,  or so  on vs.  512 byte
blocks?  I  know it's traditional  and reflects various  internal data
structures  relating to  sectors, cylinders,  heads, tracks,  etc. but
that's  kind  of  all  obsolete.    And  seeing  the  size  listed  as
"89424293170" or whatever for a  40GBy partition isn't really the most
readable thing.  Especially  if one enters the data  as "512m" or "2g"
it'd be  nice to see it  like that (e.g. du  -h, du -k,  and a verbose
display down to the sector only as an alternate non-default listing).

So in summary:

* believe  'device' geometry  over 'bios'  geometry especially  if the
whole  disk is being  given to  OpenBSD and  one isn't  working around
existing "BIOS geometry" partitions.

* Deprecate use of (or allow  simple 'work around' "virtual" flag) for
sectors / interleave  / RPM / heads / tracks  / alternate blocking and
anything else that's probably  nonexistent, virtualized, or handled in
firmware/hardware in many modern storage devices.

* always calculate default offsets when interactively adding partitions.

* assume  that "sd0"  (or anything  other than hd0) may well be the
intended root drive.

But it gets worse...

----------------------------------------------------------------------
** APPARENT BUG / ANNOYING CIRCUMSTANCE #5 (killer; I'm screwed):

After finishing the  partition definition / editing with  "w" and "q",
and "done"  since sd0  (USB HD)  was the only  drive to  partition the
following problems happened:

"You have configured the following devices and mount points:" 
"sd0a /"
"============================================================"     
"The next step will overwrite any existing data on:" "sd0a" 
"Are you really sure that  you'd like to proceed? [no]  y" 
"Creating filesystems....."
"sd0:  could not mode  sense (4/5);  using fictitions  geometry" 
"sd0:could  not  mode  sense   (4/5);  using  fictitions  geometry"  
"newfs: /dev/rsd0a Device not configured"

"You will now be given a chance to configure the network......"

** Personal note: I  assume I'm hosed here, that the  disk has none of
the  configured filesystems  on it,  that installation  to a  USB mass
storage root drive cannot proceed,  do not pass go, collect $0.02 from
mailing list, don't get OpenBSD  installed on this box without install
script fixes.

It's nice  that this system  configuration *ALMOST* worked in  that it
did  boot OpenBSD  3.1  off the  USB CD,  and  that it  did "see"  was
*almost* willing to partition the USB  HD for a root install disc, but
it looks  like a few  things "aren't quite  there yet".  There  may be
some  unknown   level  of  cheesiness   in  the  external   USB  drive
controller's firmware, I can't say,  but it'd be nice if OpenBSD could
have been more robust to "do it anyway". (it does work under winblows)

----------------------------------------------------------------------
Further wish lists to think about  for the future, just off the top of
my head, and in no particular order:

* Enable installation of some or all filesystems to a filesystem image
(e.g. vnode'd or whatever) instead of  a device like hd0, sd0, et. al.
One example  if wanted to have  the system create  a bootable ISO-9660
image the  process of  creating the ISO  file containing  the selected
filesystems and  the "virtual"  2.88 MB floppy  for the  ISO-9660 boot
could  be  integrated  as  'target'  options in  the  general  install
script.  One  could perform  the  install to  a  file  in an  existing
development system / server and then just burn the image to CD to have
a customized  bootable CD based OpenBSD  system to run  on a different
box which could be diskless / headless (high security, low cost).

* Enable specification / configuration of ram disk (MFS?) file systems
during normal  installation script "partition editing".   So maybe I'd
like  a 1GBy  /tmp as  a dedicated  ram disk  for speed  and security,
perhaps.   Or maybe  I'd like  "/, /usr,  /var, /home"  as initialized
filesystems that'd be RAM resident after booting the image from LAN,
CD, HDD, whatever.   That would be highly fast,  secure (if the system
had  read  only boot  media  or LAN  booted,  even  if the  "writable"
filesystems were  damaged / compromised  one could just reboot  and be
back up perfectly cleanly).

* Enable specification of read  only vs. read-write filesystems during
partitioning / installation.  For security, integrity, and simple back
up it'd be  nice to know that e.g. /home /etc  /var were maybe 'dirty'
relative to  the installed origin,  but that / /usr  /usr/X11R6 hadn't
been  changed.    Obviously  one  could  edit   fstab  manually  after
installation, but the point is rather more towards evolving to make it
easier  on the  user  to generate  and  maintain a  division of  local
data/changes and "system" files that should really never change unless
due to an upgrade.

* Enable "client/server"  LAN installs so  that one box could  boot up
and maybe  share its drive  or whatever for  a "server" box to  run an
installation but with  the target system of that  installation being a
resource on  some client on  the LAN.  So  the console and  boot media
would be on  a server PC and  the client PC could just  boot floppy or
boot  LAN  and  get  its  installation  from  elsewhere.   This  being
different  than a 'FTP'  install where  the console  interaction would
still be  on the console of  the client box  (necessitating a monitor,
keyboard, or at least a serial console).

* Enable specification of mount_union 'translucent' filesystems during
install e.g.  /usr_base being  the "install time"  files for  /usr and
somewhere else like "/usr_overlay"  would be translucently upper layer
mounted over  /usr_base as /usr  so that writes /  modifications would
only  go to  /usr_overlay but  reads of  the unmodified  /  original /
undeleted  base system  files would  always occur  from  the /usr_base
which would  presumably be read-only.   And of course the  stack would
just be mounted as "/usr" just like normal.  Again, obviously possible
to do manually, but a lot  of work that could pretty easilly be rolled
into the  general installation  options.  This could  benefit security
audits as well as data back up and so on.

* Enable revision  control system  during system installation  so that
for all  files "installed"  entries could be  made for those  files in
some  kind  of revision  control  system.   e.g.  /etc/hosts would  be
created with the uninteresting default content, but one would have the
possibility of having "fresh out  of the box" SCCS, CVS, RCS, whatever
trees created so that if one  then wanted to modify a system file like
/etc/hosts, /etc/fstab, /etc/passwd  one would have the infrastructure
to recover the previous revision, audit the deltas, et. al.

* Enable SHA /  MD5 generation / tripwire hash /  database log for all
installed files so that upon installation there'd be a file (or files,
maybe one for each FS?) listing  the correct signature of all files in
that filesystem right from  the first installation.  Anything that was
compatible  with tripwire  or  other free  source  commonly used  file
integrity/security  auditing tools that  basically tracks  signature /
date  / time  / permission  changes  to system  files would  certainly
enhance convenience and security.  I'd think it'd be trivially easy to
implement right at the point  of unpackaging / installing / generating
the specific files.

----------------------------------------------------------------------
----------------------------------------------------------------------

Ok, that's today's saga!

Thanks in advance for any pointers / suggestions / thoughtfulness
of such matters.

Maybe I'll go get a IBM Travelstar 2.5" "laptop" EIDE hdd for internal
use in this box since they only take about 2.5 watts to run and go to
sleep nicely, which would be good for a low power consumption 
'always on' box.

I had really wanted to get it tested out / developed / installed via
USB, though, since in the end I may just want this box diskless. :(

Since the 3.1 default install kernel does explicitly support USB
mass storage devices, though, clearly it's not too unreasonable to
hope that such configurations as the one I tried above might eventually
be super cleanly operable in the future.

Thanks; keep up the good work!
Yahoo! - Official partner of 2002 FIFA World Cup
http://fifaworldcup.yahoo.com