HowTo: CMS XRootD configuration - Revision history

Mcaubet: /* storage-disk.xml file */

2017-06-30T07:54:10Z

storage-disk.xml file

Mcaubet: /* storage-disk.xml file */

2017-06-30T07:54:00Z

storage-disk.xml file

Mcaubet: /* Requirements */

2017-06-30T07:53:07Z

Requirements

Mcaubet: /* XRootD Local Redirector */

2017-06-30T07:49:31Z

XRootD Local Redirector

Mcaubet: Created page with "= XRootD CMS = == Requirements == === XRootD Local Redirector === * '''xrootd-cmst1.pic.es''' is an example of local CMS XRootD redirector at PIC. * Relevant services are '''x..."

2017-06-30T07:47:34Z

Created page with "= XRootD CMS = == Requirements == === XRootD Local Redirector === * '''xrootd-cmst1.pic.es''' is an example of local CMS XRootD redirector at PIC. * Relevant services are '''x..."

New page

= XRootD CMS =
== Requirements ==
=== XRootD Local Redirector ===
* '''xrootd-cmst1.pic.es''' is an example of local CMS XRootD redirector at PIC.
* Relevant services are '''xrootd''' and '''cmsd''' which should be up and running:
[root@xrootd01-cmst1 ~]# service xrootd status
[default] xrootd (pid 26682) is running...

[root@xrootd01-cmst1 ~]# service cmsd status
[default] cmsd (pid 26583) is running...
* XRootD configuration files is locate at: '''/etc/xrootd/xrootd-clustered.cfg''' and should be defined as follows:
[root@xrootd01-cmst1 ~]# cat /etc/xrootd/xrootd-clustered.cfg | grep -v '#' | grep .
xrd.port 1095
all.role server
all.sitename T1_ES_PIC
all.manager any xrootd-cms.infn.it+ 1213
xrootd.redirect xrootd-cmst1-door.pic.es:1094 /
all.export / nostage
cms.allow host *
xrootd.trace emsg login stall redirect
ofs.trace none
xrd.trace conn
cms.trace all
oss.namelib /usr/lib64/libXrdCmsTfc.so file:/etc/xrootd/storage-disk.xml?protocol=direct
xrootd.seclib /usr/lib64/libXrdSec.so
xrootd.fslib /usr/lib64/libXrdOfs.so
all.adminpath /var/run/xrootd
all.pidpath /var/run/xrootd
cms.delay startup 10
cms.fxhold 60s
xrd.report xrootd.t2.ucsd.edu:9931 every 60s all sync
xrootd.monitor all fstat 60s lfn ops ssq xfr 5 ident 5m dest fstat info user CMS-AAA-EU-COLLECTOR.cern.ch:9330
* Relevant packages that need to be installed are the following:
[root@xrootd01-cmst1 ~]# rpm -qa | grep xrootd | sort
cms-xrootd-dcache-1.2-7.osg.el6.noarch
xrootd-4.5.0-1.el6.x86_64
xrootd-client-4.5.0-1.el6.x86_64
xrootd-client-devel-4.5.0-1.el6.x86_64
xrootd-client-libs-4.5.0-1.el6.x86_64
xrootd-cmstfc-1.5.1-6.osg.el6.x86_64
xrootd-compat-3.3.6-1.el6.x86_64
xrootd-compat-client-libs-3.3.6-1.el6.x86_64
xrootd-compat-libs-3.3.6-1.el6.x86_64
xrootd-compat-server-libs-3.3.6-1.el6.x86_64
xrootd-devel-4.5.0-1.el6.x86_64
xrootd-libs-4.5.0-1.el6.x86_64
xrootd-selinux-4.5.0-1.el6.noarch
xrootd-server-4.5.0-1.el6.x86_64
xrootd-server-devel-4.5.0-1.el6.x86_64
xrootd-server-libs-4.5.0-1.el6.x86_64
* The above packages can be installed from the official XRootD repositories:
[root@xrootd02-cmst1 yum.repos.d]# cat /etc/yum.repos.d/xrootd-stable.repo
[xrootd-stable]
name=XrootD official repository
baseurl=http://xrootd.org/binaries/stable/slc/6/$basearch http://xrootd.cern.ch/sw/repos/stable/slc/6/$basearch
enabled=1
gpgcheck=0
enablegroups=1
metadata_expire=43200
* PNFS mount is a required mountpoint. Read-only mode should be enough, and the ''sync'' option is recommended:
[root@xrootd01-cmst1 ~]# mount | grep pnfs
nfsv41.pic.es:/pnfs/pic.es/data/cms on /pnfs/pic.es/data/cms type nfs4
(ro,sync,noatime,nodiratime,minorversion=1,nolock,noac,intr,bg,hard,addr=2001:67c:1148:201::11,clientaddr=2001:67c:1148:201::17)

== dCache door configuration ==
* '''xrootd-cmst1-door.pic.es''' (usually ''door06.pic.es'') is the alias for the CMS XRootD door at PIC.
* dCache CMS XRootD layout file example, where:
:* ''xrootd.loginbroker.tags=srm'' to disable glue publishing
:* ''xrootd.mover.queue=xrootd-wan'' to dedicated a specific mover queue (we want split WAN vs LAN movers)

############################################
# Domain: xrootd-cmst1-door06Domain
[xrootd-cmst1-${host.name}Domain]
dcache.java.memory.heap=2048m
'''xrootd.loginbroker.tags=srm'''
'''xrootd.mover.queue=xrootd-wan'''
'''# Cell: xrootd'''
'''[xrootd-cmst1-${host.name}Domain/xrootd]'''
'''xrootd.cms.tfc.path=/etc/dcache/storage-disk.xml'''
'''xrootd.cms.tfc.protocol=direct'''
'''xrootd.net.port=1094'''
'''xrootd.plugins=gplazma:gsi,authz:cms-tfc'''
## End xrootd-cmst1-door06Domain
############################################
* Relevant packages that need to be installed are the following:
[root@door06 ~]# rpm -qa | grep ^dcache- | sort
'''dcache-2.16.38-1.noarch'''
dcache-srmclient-3.0.9-1.noarch
* TFC plugin must be installed.
:* Can be downloaded from https://www.dcache.org/downloads/xrootd4j/index.shtml (we use the ''.tar.gz'' file)
[root@door06 ~]# ls -ltrha /usr/local/share/dcache/plugins/
total 12K
drwxr-xr-x. 3 root root 4.0K Jun 20 10:09 ..
drwxr-xr-x. 3 root root 4.0K Jun 20 10:16 .
drwxr-xr-x. 2 root root 4.0K Jun 20 10:16 '''xrootd4j-cms-plugin-1.3.7'''

== dCache pool configuration ==
* Pool configuration is needed for the XRootD monitoring service.
* An example of a dCache CMS pool layout file for ''dc040.pic.es'' is shown below:

############################################
# Domain: dc040_1Domain
[${host.name}_1Domain]
dcache.java.options.extra=-Djava.net.preferIPv6Addresses=true -Dcom.sun.management.jmxremote.port=7001 -Dcom.sun.management.jmxremote.ssl=false -
Dcom.sun.management.jmxremote.authenticate=false
dcache.java.memory.heap=6000m
'''pool.mover.xrootd.port.max=${dcache.net.wan.port.max}'''
'''pool.mover.xrootd.port.min=${dcache.net.wan.port.min}'''
pool.plugins.meta=org.dcache.pool.repository.meta.db.BerkeleyDBMetaDataRepository
pool.queues=ftp,dcap,nfs4,xrootd,'''xrootd-wan''',webdav
'''# dc040_1Domain - xrootd configuration for cms pools'''
'''pool.mover.xrootd.plugins=edu.uchicago.monitor'''
'''xrootd.monitor.detailed=cms-aaa-eu-collector.cern.ch:9330:60'''
'''xrootd.monitor.site=T1_ES_PIC'''
'''xrootd.monitor.summary=xrootd.t2.ucsd.edu:9931:60'''
'''xrootd.monitor.vo=CMS'''
# Cell: pool
[${host.name}_1Domain/pool]
pool.name=${host.name}_1
pool.path=/dcpool/vpool1
pool.wait-for-files=${pool.path}/data
## End pool_1Domain
############################################
* Relevant packages that need to be installed are the following:
[root@dc040 ~]# rpm -qa | grep ^dcache- | sort
'''dcache-2.16.38-1.noarch'''
'''dcache-plugin-xrootd-monitor-7.0.0-0.noarch'''
dcache-srmclient-3.0.9-1.noarch

@@ Line 3: / Line 3: @@
 === storage-disk.xml file ===
 * Every CMS site has his own '''storage-disk.xml''' file.
-* This file '''should be the same for all servers running a CMS service'''.
+* This file '''''should be the same for all servers running a CMS service'''''.
 * Below, an example for the PIC site ''storage-disk.xml'' file:
   <storage-mapping>

@@ Line 2: / Line 2: @@
 == Requirements ==
 === storage-disk.xml file ===
-* Every CMS site has his own '''storage-disk.xml''' file. Here we show an example for the PIC site:
+* Every CMS site has his own '''storage-disk.xml''' file.
   <storage-mapping>
   <!-- PFN to LFN -->
@@ Line 51: / Line 53: @@
       path-match="gsidcap://gsidcap\.pic\.es:22128(.*)" result="$1"/>
   </storage-mapping>
 === XRootD Local Redirector ===

@@ Line 1: / Line 1: @@
 = XRootD CMS =
 == Requirements ==
 === XRootD Local Redirector ===
 * '''xrootd-cmst1.pic.es''' is an example of local CMS XRootD redirector at PIC.

@@ Line 4: / Line 4: @@
 * '''xrootd-cmst1.pic.es''' is an example of local CMS XRootD redirector at PIC.
 * Relevant services are '''xrootd''' and '''cmsd''' which should be up and running:
-  [root@xrootd01-cmst1 ~]# service xrootd status
+  [root@xrootd01-cmst1 ~]# service '''xrootd''' status
   [default] xrootd (pid  26682) is running...
-  [root@xrootd01-cmst1 ~]# service cmsd status
+  [root@xrootd01-cmst1 ~]# service '''cmsd''' status
   [default] cmsd (pid  26583) is running...
 * XRootD configuration files is locate at: '''/etc/xrootd/xrootd-clustered.cfg''' and should be defined as follows:
   [root@xrootd01-cmst1 ~]# cat /etc/xrootd/xrootd-clustered.cfg | grep -v '#' | grep .
-  xrd.port 1095
+  xrd.port '''1095'''
   all.role server
-  all.sitename T1_ES_PIC
+  all.sitename '''T1_ES_PIC'''
-  all.manager any xrootd-cms.infn.it+ 1213
+  all.manager any '''xrootd-cms.infn.it+''' 1213
-  xrootd.redirect xrootd-cmst1-door.pic.es:1094 /
+  '''xrootd.redirect xrootd-cmst1-door.pic.es:1094 /'''
   all.export / nostage
   cms.allow host *
@@ Line 22: / Line 22: @@
   xrd.trace conn
   cms.trace all
-  oss.namelib /usr/lib64/libXrdCmsTfc.so file:/etc/xrootd/storage-disk.xml?protocol=direct
+  oss.namelib /usr/lib64/libXrdCmsTfc.so file:'''/etc/xrootd/storage-disk.xml?protocol=direct'''
   xrootd.seclib /usr/lib64/libXrdSec.so
   xrootd.fslib /usr/lib64/libXrdOfs.so
@@ Line 29: / Line 29: @@
   cms.delay startup 10
   cms.fxhold 60s
-  xrd.report xrootd.t2.ucsd.edu:9931 every 60s all sync
+  xrd.report '''xrootd.t2.ucsd.edu:9931''' every 60s all sync
-  xrootd.monitor all fstat 60s lfn ops ssq xfr 5 ident 5m dest fstat info user CMS-AAA-EU-COLLECTOR.cern.ch:9330
+  xrootd.monitor all fstat 60s lfn ops ssq xfr 5 ident 5m dest fstat info user '''CMS-AAA-EU-COLLECTOR.cern.ch:9330'''
 * Relevant packages that need to be installed are the following:
   [root@xrootd01-cmst1 ~]# rpm -qa | grep xrootd | sort