Adding pool: Difference between revisions

From DaqWiki
Jump to navigation Jump to search
No edit summary
Line 2: Line 2:
Following example is for smd experiment.
Following example is for smd experiment.


== Set up pool on data machine ==
= Set up pool on data machine =


Assume we set data pool on data machine trdata05.
Assume we set data pool on data machine trdata05.


Create the pool
Create the pool (with 600GB of space)


<pre>
<pre>
Line 20: Line 20:
</pre>
</pre>


groupadd -g 4226 trek
Note: nothing in dcache prevents you from oversubscribing the space on a particular machine.  Ie, you could allocate two pools with 15TB of space on a device with just 20TB of space overall.  You need to check that yourself.
adduser -u 4226 -g trek trek


mount -o intr,rw,noac,hard,nfsvers=3 trdata00:/pnfs /pnfs
= Configuration on trdata00 =


# mkdir /pnfs/triumf.ca/data/trek
Next steps are on trdata00.
mkdir: cannot create directory `/pnfs/triumf.ca/data/trek': Permission
denied


OK... follow my instructions from t2ksrm ops
First, if dcache PNFS is not mounted, mount it


/opt/d-cache/bin/chimera-cli.sh Mkdir /pnfs/triumf.ca/data/trek
<pre>
echo "data" | /opt/d-cache/bin/chimera-cli.sh Writetag
mount -o intr,rw,noac,hard,nfsvers=3 trdata00:/pnfs /pnfs
/pnfs/triumf.ca/data/trek sGroup
</pre>
echo "StoreName trekdata" | /opt/d-cache/bin/chimera-cli.sh Writetag
/pnfs/triumf.ca/data/trek OSMTemplate


# cd /pnfs/triumf.ca/data/trek/
Now create a local user, with the same UID as the experiments user account.
root@trdata00:/pnfs/triumf.ca/data/trek
# cat '.(tag)(OSMTemplate)'
StoreName trekdata
root@trdata00:/pnfs/triumf.ca/data/trek
# cat '.(tag)(sGroup)'


<pre>
groupadd -g 6016 smd
adduser -u 6016 -g smd smd
</pre>


/opt/d-cache/bin/chimera-cli.sh Chown /pnfs/triumf.ca/data/trek 4226
Now create the top level directory in dcache for smd
/opt/d-cache/bin/chimera-cli.sh Chgrp /pnfs/triumf.ca/data/trek 4226




trdata05
<pre>
/opt/d-cache/bin/chimera-cli.sh Mkdir /pnfs/triumf.ca/data/smd
echo "data" | /opt/d-cache/bin/chimera-cli.sh Writetag /pnfs/triumf.ca/data/smd sGroup
echo "StoreName trekdata" | /opt/d-cache/bin/chimera-cli.sh Writetag /pnfs/triumf.ca/data/smd OSMTemplate
</pre>


/opt/d-cache/bin/dcache pool create --size=600G
Check that this worked
/data/dcache/pool/trek_05_00 trek_05_00 trdata05_00PoolDomain
# cd /pnfs/triumf.ca/data/smd/
Created a pool in /data/dcache/pool/trek_05_00. The pool was added to
root@trdata00:/pnfs/triumf.ca/data/smd
trdata05_00PoolDomain in file:/opt/d-cache//etc/layouts/trdata05.conf.
# cat '.(tag)(OSMTemplate)'
 
StoreName smd
service dcache restart
root@trdata00:/pnfs/triumf.ca/data/smd
 
# cat '.(tag)(sGroup)'
Damn, data is going to wrong pool... why?
data
 
 
OK, maybe these are why all the new pools have been added to default
pool group
 
psu addto pgroup default lixe_03_01
psu addto pgroup default pienu_06
psu addto pgroup default lixe_03_03
psu addto pgroup default iris_05_00
psu addto pgroup default tactic_05_00
psu addto pgroup default tigress_05_00
psu addto pgroup default t2km11_05_00
psu addto pgroup default tmpPool
psu addto pgroup default t2km11_04
psu addto pgroup default trek_05_00
psu addto pgroup default titan_05_00
 
So try editting this and then do this:
 
"It can also be loaded into a running system with the *reload* command.
In this chapter we will describe the commands allowed in this file. "
 
so modify this file and then do this:
 
[trdata00.triumf.ca] (local) admin > cd PoolManager
[trdata00.triumf.ca] (PoolManager) admin > reload
  This command destroys the current setup
  and replaces it by the setup on disk
  Please use 'reload -yes' if you really want
  to do that.
[trdata00.triumf.ca] (PoolManager) admin > reload -yes
 
 
t2km11 copies work
 
[t2km11@ladd00] dccp snapshot.png /daq/pnfs/triumf.ca/data/t2km11/test/.
122078 bytes (119 kiB) in 1 seconds (119 kiB/s)
 
pienu copy works
 
[pienu@ladd00 ~]$ dccp vf48.c /daq/pnfs/triumf.ca/data/pienu/test/.
39609 bytes (38.7 kiB) in 0 seconds
 
iris copies work...
 
but trek doesn't !  Fix!!!
[trek@ladd00 ~]$ dccp tdc1.pdf /daq/pnfs/triumf.ca/data/trek/test/.
Failed open file in the dCache.
Can't open destination file : "No write pool available for
<size=0;new=true;stored=false;sClass=trekdata:data;cClass=-;hsm=osm;accessLatency=ONLINE;retentionPolicy=REPLICA;uid=4226;onerror=default;StoreName=trekdata;alloc-size=16608;gid=100;path=/daq/pnfs/triumf.ca/data/trek/test/./tdc1.pdf;timeout=-1;store=trekdata;group=data;bfid=<Unknown>;>


in the linkGroup [none]"
Make the directory belong to the right user
System error: Input/output error


Ah ha!  The command was supposed to be
/opt/d-cache/bin/chimera-cli.sh Chown /pnfs/triumf.ca/data/smd 6016
/opt/d-cache/bin/chimera-cli.sh Chgrp /pnfs/triumf.ca/data/smd 6016


echo "StoreName trekdata" | /opt/d-cache/bin/chimera-cli.sh Writetag
= Admin Configuration on trdata00 =
/pnfs/triumf.ca/data/trek OSMTemplate


right!  Right and the copy goes into the right pool, on trdata05.
Now we do the last setup with the obscure dcache interface.


So we have now fixed the default pools being used and got the trek pool
working!






OK, here's the other steps




Revision as of 19:59, 20 May 2015

Following are the necessary steps to add a new pool group (ie add a new experiment) to trdata. Following example is for smd experiment.

Set up pool on data machine

Assume we set data pool on data machine trdata05.

Create the pool (with 600GB of space) 

/opt/d-cache/bin/dcache pool create --size=600G /data/dcache/pool/smd_05_00 smd_05_00 trdata05_00PoolDomain

in smd_05_00 05 indicates this is trdata05 and 00 means the first data device on this machine.

the restart dcache 

service dcache restart

Note: nothing in dcache prevents you from oversubscribing the space on a particular machine. Ie, you could allocate two pools with 15TB of space on a device with just 20TB of space overall. You need to check that yourself.

Configuration on trdata00

Next steps are on trdata00.

First, if dcache PNFS is not mounted, mount it 

mount -o intr,rw,noac,hard,nfsvers=3 trdata00:/pnfs /pnfs

Now create a local user, with the same UID as the experiments user account. 

groupadd -g 6016 smd
adduser -u 6016 -g smd smd

Now create the top level directory in dcache for smd 


/opt/d-cache/bin/chimera-cli.sh Mkdir /pnfs/triumf.ca/data/smd 
echo "data" | /opt/d-cache/bin/chimera-cli.sh Writetag /pnfs/triumf.ca/data/smd sGroup
echo "StoreName trekdata" | /opt/d-cache/bin/chimera-cli.sh Writetag /pnfs/triumf.ca/data/smd OSMTemplate

Check that this worked

  1. cd /pnfs/triumf.ca/data/smd/

root@trdata00:/pnfs/triumf.ca/data/smd

  1. cat '.(tag)(OSMTemplate)'

StoreName smd root@trdata00:/pnfs/triumf.ca/data/smd

  1. cat '.(tag)(sGroup)'

data

Make the directory belong to the right user

/opt/d-cache/bin/chimera-cli.sh Chown /pnfs/triumf.ca/data/smd 6016 /opt/d-cache/bin/chimera-cli.sh Chgrp /pnfs/triumf.ca/data/smd 6016

Admin Configuration on trdata00

Now we do the last setup with the obscure dcache interface.




[trdata00.triumf.ca] (local) admin > cd PoolManager [trdata00.triumf.ca] (PoolManager) admin > [trdata00.triumf.ca] (PoolManager) admin > [trdata00.triumf.ca] (PoolManager) admin > psu create pgroup trek-pgroup [trdata00.triumf.ca] (PoolManager) admin > psu create pool trek_05_00 java.lang.IllegalArgumentException: Duplicated entry : trek_05_00 [trdata00.triumf.ca] (PoolManager) admin > psu addto pgroup trek-pgroup trek_05_00 [trdata00.triumf.ca] (PoolManager) admin > psu create unit -store trek:data@osm [trdata00.triumf.ca] (PoolManager) admin > psu create ugroup trek-cond [trdata00.triumf.ca] (PoolManager) admin > psu addto ugroup trek-cond Syntax error: (2) Invalid number of arguments Help : psu addto ugroup <uGroup> <unit> [trdsm [trdata00.triumf.ca] (PoolManager) admin > [trdata00.triumf.ca] (PoolManager) admin > psu addto ugroup trek-cond trek:data@osm java.lang.IllegalArgumentException: trek:data@osm already member of trek-cond [trdata00.triumf.ca] (PoolManager) admin > psu create link trek-link trek-cond [trdata00.triumf.ca] (PoolManager) admin > psu set link trek-link -readpref=10 -writepref=10 -cachepref=10 -p2ppref=-1 [trdata00.triumf.ca] (PoolManager) admin > psu add link trek-link trek-pgroup [trdata00.triumf.ca] (PoolManager) admin > save [trdata00.triumf.ca] (PoolManager) admin >


mkdir /daq/pnfs/triumf.ca/data/trek/test [trek@ladd00 ~]$ dccp tdc_CP_run1263.pdf/daq/pnfs/triumf.ca/data/trek/test/.

Retrieved from "https://daq00.triumf.ca/DaqWiki/index.php?title=Adding_pool&oldid=346"