Xrootd

xrootd 를 활용한 SE Data 사용 가이드

voms proxy 생성

• xrootd 프로토콜을 이용하여 SE에 접근하기 위해서는 반드시 Grid 인증을 사용해야합니다.

$ voms-proxy-init --voms

xrdcp client 사용하기

$ xrdcp root://cluster142.knu.ac.kr//store/user/(username)/test.file /some/local/path/test.file
  (참조 : http://xrootd.org/doc/prod/Syntax_config.htm)

$ xrdcp /some/local/path/test.file root://cluster142.knu.ac.kr//store/user/(username)/test.file

libXrdPosixPreload library 사용하기

$ export LD_PRELOAD=/usr/lib64/libXrdPosixPreload.so 
$ echo “This is a new test” >./test 

$ mkdir xroot://cluster142.knu.ac.kr:1094//store/user/(username)/testdir
$ cp ./test xroot://cluster142.knu.ac.kr:1094//store/user/(username)/testdir/test 

$ cp xroot://cluster142.knu.ac.kr:1094//store/user/(username)/testdir/test ./test1
$ diff ./test1 ./test 

$ rm xroot://cluster142.knu.ac.kr:1094//store/user/(username)/testdir/test
$ rmdir xroot://cluster142.knu.ac.kr:1094//store/user/(username)/testdir

XROOTD_VMP (xrootd virtual mount point) 사용하기 <- 추천

$ voms-proxy-init --voms    # grid proxy 생성 
$ export LD_PRELOAD=/usr/lib64/libXrdPosixPreload.so
$ export XROOTD_VMP=cluster142.knu.ac.kr:/xrootd=/store     # 가상의 마운트 포인트 /xroot에 root://cluster142.knu.knu.ar/store 로 설정 
$ ls /xrootd/user/test            # 가상의 mount point /xrootd

$ ls /xrootd
ReVal  backfill  data  express  generator  group  hidata  himc  hirelval  lumi  mc  relval  results  temp  test  unmerged  user

$ ls /xrootd/user/test
CE4D66EB-5AAE-E111-96D6-003048D37524.root  test.transfer.20181004101241  test.transfer.20190503114034  test.transfer.20200818153523
OrthDatasetHistos_7.root                   test.transfer.20190116161947  test.transfer.20190503114035  test.transfer.20200818154436
test                                       test.transfer.20190116174233  test.transfer.20190619153215  test.transfer.20200819171901
test.root                                  test.transfer.20190117150051  test.transfer.20190619153842  test2
test.transfer                              test.transfer.20190128143636  test.transfer.20190919        testdir
test.transfer.20180907142043               test.transfer.20190429141454  test.transfer.20191024200821
test.transfer.20180907142048               test.transfer.20190503114027  test.transfer.20191202153201

• 실재로 마운트된 상태가 아니기 때문에 다음과 같은 명령어는 동작하지 않습니다.

$ cd /xrootd/user/test
-bash: cd: /xrootd/user/test: 그런 파일이나 디렉터리가 없습니다

$ echo "AAA" > /xrootd/user/test/aaa.txt
-bash: /xrootd/user/test/aaa.txt: 그런 파일이나 디렉터리가 없습니다

xrdfs 사용하기

$ echo “This is a new test” >./test 
$ cat ./test
“This is a new test”
$ xrdcp ./test xroot://cluster142.knu.ac.kr:1094//store/user/(username)/test
$ xrdfs cluster142 ls /store/user/(username)
...
/store/user/(username)/test
...
$ xrdfs cluster142.knu.ac.kr cat /store/user/(username)/test
“This is a test”
   
$ xrdfs cluster142.knu.ac.kr
[cluster142.knu.ac.kr:1094] / > cd store/user/(username)
[cluster142.knu.ac.kr:1094] /store/user/(username) > ls
...
/store/user/(username)/test
...
[cluster142.knu.ac.kr:1094] /store/user/(username) > ls test
[ERROR] Server responded with an error: [3012] Error when processing list response: Not a directory: /pnfs/knu.ac.kr/data/cms/store/user/(username)/test

[cluster142.knu.ac.kr:1094] /store/user/(username) > cat test
“This is a test”

[cluster142.knu.ac.kr:1094] /store/user/(username) > mkdir testdir
[cluster142.knu.ac.kr:1094] /store/user/(username) > ls
...
/store/user/(username)/test
/store/user/(username)/testdir 
...

[cluster142.knu.ac.kr:1094] /store/user/(username) > mv test testdir
[ERROR] Server responded with an error: [3005] Will not overwrite existing file (File exists:/pnfs/knu.ac.kr/data/cms/store/user/(username)/testdir).

[cluster142.knu.ac.kr:1094] /store/user/(username) > mv test testdir/test
[cluster142.knu.ac.kr:1094] /store/user/(username) > cd testdir
[cluster142.knu.ac.kr:1094] /store/user/(username)/testdir > ls
/store/user/(username)/testdir/test

[cluster142.knu.ac.kr:1094] /store/user/(username)/testdir > rm test
[cluster142.knu.ac.kr:1094] /store/user/(username)/testdir > cd ..
[cluster142.knu.ac.kr:1094] /store/user/(username) > rmdir testdir
[cluster142.knu.ac.kr:1094] / > quit
Goodbye.

$ xrdfs --help
Usage:
   xrdfs [--no-cwd] host[:port]              - interactive mode
   xrdfs            host[:port] command args - batch mode

Available options:

  --no-cwd no CWD is being preset

Available commands:

  exit
    Exits from the program.

                              tpc           - support for third party copies
                              wan_port      - the port to use for wan copies
                              wan_window    - the wan_port window size
                              window        - the tcp window size
                              cms           - the status of the cmsd
                              role          - the role in a cluster
                              sitename      - the site name
                              version       - the version of the server
    checksumcancel <path>   File checksum cancellation
    checksum       <path>   File checksum
    opaque         <arg>    Implementation dependent
    opaquefile     <arg>    Implementation dependent
    space          <space>  Logical space stats
    stats          <what>   Server stats; <what> is a list
                            of letters indicating information
                            to be returned:
                              a - all statistics
                              p - protocol statistics
                              b - buffer usage statistics
                              s - scheduling statistics
                              d - device polling statistics
                              u - usage statistics
                              i - server identification
                              z - synchronized statistics
                              l - connection statistics
    xattr          <path>   Extended attributes

  rm <filename>
    Remove a file.

  rmdir <dirname>
    Remove a directory.

  truncate <filename> <length>
    Truncate a file.

  prepare [-c] [-f] [-s] [-w] [-p priority] filenames
    Prepare one or more files for access.
    -c co-locate staged files if possible
    -f refresh file access time even if the location is known
    -s stage the files to disk if they are not online
    -w the files will be accessed for modification
    -p priority of the request, 0 (lowest) - 3 (highest)

  cat [-o local file] file
    Print contents of a file to stdout.
    -o print to the specified local file

  tail [-c bytes] [-f] file
    Output last part of files to stdout.
    -c num_bytes out last num_bytes
    -f           output appended data as file grows

  spaceinfo path
    Get space statistics for given path.