Adding pool

Following are the necessary steps to add a new pool group (ie add a new experiment) to trdata. Following example is for smd experiment.

Set up pool on data machine

Assume we set data pool on data machine trdata05 (ssh root@trdata05).

Create the pool (with 600GB of space)

/opt/d-cache/bin/dcache pool create --size=600G /data/dcache/pool/smd_05_00 smd_05_00 trdata05_00PoolDomain

in smd_05_00 05 indicates this is trdata05 and 00 means the first data device on this machine.

the restart dcache

service dcache restart

Note: nothing in dcache prevents you from oversubscribing the space on a particular machine. Ie, you could allocate two pools with 15TB of space on a device with just 20TB of space overall. You need to check that yourself.

Configuration on trdata00

Next steps are on trdata00 (ssh root@trdata00)

First, if dcache PNFS is not mounted, mount it

mount -o intr,rw,noac,hard,nfsvers=3 trdata00:/pnfs /pnfs

Now create a local user, with the same UID as the experiments user account.

groupadd -g 6016 smd
adduser -u 6016 -g smd smd

Now create the top level directory in dcache for smd

/opt/d-cache/bin/chimera-cli.sh Mkdir /pnfs/triumf.ca/data/smd 
echo "data" | /opt/d-cache/bin/chimera-cli.sh Writetag /pnfs/triumf.ca/data/smd sGroup
echo "StoreName trekdata" | /opt/d-cache/bin/chimera-cli.sh Writetag /pnfs/triumf.ca/data/smd OSMTemplate

Check that this worked

# cd /pnfs/triumf.ca/data/smd/
root@trdata00:/pnfs/triumf.ca/data/smd
# cat '.(tag)(OSMTemplate)'
StoreName smd
root@trdata00:/pnfs/triumf.ca/data/smd
# cat '.(tag)(sGroup)'
data

Make the directory belong to the right user

/opt/d-cache/bin/chimera-cli.sh Chown /pnfs/triumf.ca/data/smd 6016
/opt/d-cache/bin/chimera-cli.sh Chgrp /pnfs/triumf.ca/data/smd 6016

Admin Configuration on trdata00

Now we do the last setup with the obscure dcache interface.

ssh -c blowfish -p 22223 -l admin trdata00

(there is a password...)

[trdata00.triumf.ca] (local) admin > cd PoolManager
[trdata00.triumf.ca] (PoolManager) admin > psu create pgroup smd-pgroup
[trdata00.triumf.ca] (PoolManager) admin > psu create pool smd_05_00
[trdata00.triumf.ca] (PoolManager) admin > psu addto pgroup smd-pgroup smd_05_00
[trdata00.triumf.ca] (PoolManager) admin > psu removefrom pgroup default smd_05_00
[trdata00.triumf.ca] (PoolManager) admin > psu create unit -store smd:data@osm
[trdata00.triumf.ca] (PoolManager) admin > psu create ugroup smd-cond
[trdata00.triumf.ca] (PoolManager) admin > psu addto ugroup smd-cond smd:data@osm
[trdata00.triumf.ca] (PoolManager) admin > psu create link smd-link smd-cond
[trdata00.triumf.ca] (PoolManager) admin > psu set link smd-link -readpref=10 -writepref=10 -cachepref=10 -p2ppread=-1
[trdata00.triumf.ca] (PoolManager) admin > psu add link smd-link smd-pgroup
[trdata00.triumf.ca] (PoolManager) admin > save
[trdata00.triumf.ca] (PoolManager) admin > ..

mkdir /daq/pnfs/triumf.ca/data/trek/test [trek@ladd00 ~]$ dccp tdc_CP_run1263.pdf/daq/pnfs/triumf.ca/data/trek/test/.