At triumf, we are developing a system to use removable hard drives to store data collected by midas
daq stations. The basic idea is to replace storage on 300 GB DLT tapes with storage on removable
esata, usb2 or firewire 750 GB hard drives.
To minimize culture shock, we stay as close as possible to the "tape" paradigm. Two removable disks
are used in tandem. Data is written to the first removable disk until it is full. Then midas automatically
switches to the second disk and asks the operator to replace the full disk with a blank disk. Similar to
handling tapes, the operator takes the full disk and stores it on the shelf (offline); takes a blank disk
and connects it to the computer. To read data from one of the disks, the operator takes the disk from
the shelf and connects it to the daq computer or to some other computer equipped with a compatible
removable storage bay. The full data disks are mounted read-only to prevent accidental data
modifications.
Two pieces of software are needed to implement this system:
1) midas support for switching to alternate output disks as they become full. Data could be written to
the removable disk directly by the mlogger (no extra data copy on local disks) or by the lazylogger
(mlogger writes the data to the local disk, then the lazylogger copies it to the removable disk). Writing
directly to the removable disk is more efficient as it avoids the one extra data copy operation by the
lazylogger.
2) a user interface utility for mounting and dismounting removable disks. Handling of removable disks
cannot be fully automatic: before unplugging a removable disk, the user has to inform the system; after
connecting a removable disk, the user has to tell the system to mount it read-only (for existing data),
read-write (to add more data) or to initialize a blank disk (fdisk+mkfs). (Also, some SATA interfaces do
not implement automatic hot-plug: they have to be manually told "please look for new disks").
We are presently evaluating various internal SATA hot-plug enclosures. We evaluated external eSATA
and USB2 enclosures and decided not to use them: while the performance is adequate, presence of
extra bulky components (eSATA and USB cables, non-standardized power bricks) and the extra cost of
eSATA and USB hard drive enclosures makes them unattractive.
I am open to suggestions and comments. I am most interested in hearing which data path (mlogger or
the lazylogger) would be most useful for other users.
K.O. |