Adding pool
Following are the necessary steps to add a new pool group (ie add a new experiment) to trdata. Following example is for smd experiment.
Set up pool on data machine
Assume we set data pool on data machine trdata05 (ssh root@trdata05).
Create the pool (with 600GB of space)
/opt/d-cache/bin/dcache pool create --size=600G /data/dcache/pool/smd_05_00 smd_05_00 trdata05_00PoolDomain
in smd_05_00 05 indicates this is trdata05 and 00 means the first data device on this machine.
the restart dcache
service dcache restart
Note: nothing in dcache prevents you from oversubscribing the space on a particular machine. Ie, you could allocate two pools with 15TB of space on a device with just 20TB of space overall. You need to check that yourself.
Configuration on trdata00
Next steps are on trdata00 (ssh root@trdata00)
First, if dcache PNFS is not mounted, mount it
mount -o intr,rw,noac,hard,nfsvers=3 trdata00:/pnfs /pnfs
Now create a local user, with the same UID as the experiments user account.
groupadd -g 6016 smd adduser -u 6016 -g smd smd
Now create the top level directory in dcache for smd
/opt/d-cache/bin/chimera-cli.sh Mkdir /pnfs/triumf.ca/data/smd echo "data" | /opt/d-cache/bin/chimera-cli.sh Writetag /pnfs/triumf.ca/data/smd sGroup echo "StoreName smd" | /opt/d-cache/bin/chimera-cli.sh Writetag /pnfs/triumf.ca/data/smd OSMTemplate
Check that this worked
# cd /pnfs/triumf.ca/data/smd/ root@trdata00:/pnfs/triumf.ca/data/smd # cat '.(tag)(OSMTemplate)' StoreName smd root@trdata00:/pnfs/triumf.ca/data/smd # cat '.(tag)(sGroup)' data
Make the directory belong to the right user
/opt/d-cache/bin/chimera-cli.sh Chown /pnfs/triumf.ca/data/smd 6016 /opt/d-cache/bin/chimera-cli.sh Chgrp /pnfs/triumf.ca/data/smd 6016
Admin Configuration on trdata00
Now we do the last setup with the obscure dcache interface.
ssh -c blowfish -p 22223 -l admin trdata00
(there is a password...)
cd PoolManager psu create pgroup smd-pgroup psu create pool smd_05_00 psu addto pgroup smd-pgroup smd_05_00 psu removefrom pgroup default smd_05_00 psu create unit -store smd:data@osm psu create ugroup smd-cond psu addto ugroup smd-cond smd:data@osm psu create link smd-link smd-cond psu set link smd-link -readpref=10 -writepref=10 -cachepref=10 -p2ppread=-1 psu add link smd-link smd-pgroup save .. logoff
Test
Finally test if it worked from some random DAQ machine
mkdir /daq/pnfs/triumf.ca/data/smd/test dccp /etc/exports /daq/pnfs/triumf.ca/data/smd/test/.
Success!