CEA DSM Irfu StoRM sur Lustre Pierre-François Honoré Guillaume Rivière.

Présentation au sujet: "CEA DSM Irfu StoRM sur Lustre Pierre-François Honoré Guillaume Rivière."— Transcription de la présentation:

1 CEA DSM Irfu StoRM sur Lustre Pierre-François Honoré Guillaume Rivière

2 CEA DSM Irfu Lustre Lustre est un système de fichier distribué composé de 3 éléments : Des serveurs de métadonnées (MDS) hébergeant l'espace de nommage et permettant de déterminer la localisation des données (les objets qui constituent les fichiers) Des serveurs d'objets (OSS) hébergeant le contenu des fichiers Des clients accédant en parallèle aux fichiers stockés sur les OSS 24/06/2008P-F Honoré - [StoRM sur Lustre]2

3 CEA DSM Irfu Lustre de Carriocas 24/06/2008P-F Honoré - [StoRM sur Lustre]3 POP client 10G 8 à 12 Routeur Gbit/s ethernet Au moins 20 client 1G Serveur NFS Routeur 10 Gbit/s ethernet OSS MDS OSS Liaison 10 Gbit/s SR Liaison Gbit/s Serveurs de stockage Clients LUSTRE client 1G client 10G Liaison 10 Gbit/s CX4 ou SR Poste de contrôle Au moins 5

4 CEA DSM Irfu StoRM INFN : http://storm.forge.cnaf.infn.it/ http://storm.forge.cnaf.infn.it/ Interfaces : SRMv2, gsiftp, file Java + MySQL Filesysteme : GPFS, XFS, pour la comptabilité avec réservation d’espace ou posix : ext3, Lustre 24/06/2008P-F Honoré - [StoRM sur Lustre]4

5 CEA DSM Irfu 24/06/2008P-F Honoré - [StoRM sur Lustre]5

6 CEA DSM Irfu StoRM et SRMv2 : test de base Les commandes SRMv2 –srmping Mon Jun 23 14:21:20 CEST 2008: In SRMClient ExpectedName: host Mon Jun 23 14:21:20 CEST 2008: SRMClient(https,srm/managerv2,true) SRMClientV2 : user credentials are: /O=GRID-FR/C=FR/O=CEA/OU=IRFU/CN=Pierre-Francois Honore SRMClientV2 : WEBSERVICE_PATH srm/managerv2 SRMClientV2 : connecting to srm at httpg://wn044.datagrid.cea.fr:8444/srm/managerv2 SRMClientV2 : srmPing, contacting service httpg://wn044.datagrid.cea.fr:8444/srm/managerv2 Mon Jun 23 14:21:24 CEST 2008: received response Mon Jun 23 14:21:24 CEST 2008: VersionInfo : v2.2 backend_type:StoRM backend_version: –Srmmkdir srmmkdir -debug=true srm://wn044:8444/vo.dapnia.cea.fr/test_mkdir/ surl[0]=srm://wn044:8444/vo.dapnia.cea.fr/test_mkdir/ Mon Jun 23 14:54:06 CEST 2008: In SRMClient ExpectedName: host Mon Jun 23 14:54:06 CEST 2008: SRMClient(https,srm/managerv2,true) SRMClientV2 : user credentials are: /O=GRID-FR/C=FR/O=CEA/OU=IRFU/CN=Pierre-Francois Honore SRMClientV2 : WEBSERVICE_PATH srm/managerv2 SRMClientV2 : connecting to srm at httpg://wn044.datagrid.cea.fr:8444/srm/managerv2 SRMClientV2 : srmMkdir, contacting service httpg://wn044.datagrid.cea.fr:8444/srm/managerv2 STORM# ls -la /mnt/spfs/vo.dapnia.cea.fr/ total 36 drwxrwxr-x+ 5 storm storm 4096 Jun 23 14:53. drwxr-xr-x 4 root root 4096 Jun 10 16:49.. drwxrwx---+ 2 storm storm 4096 Jun 10 17:35 cl drwxrwx---+ 3 storm storm 4096 Jun 10 19:51 pfh drwxrwx---+ 2 storm storm 4096 Jun 23 14:53 test_mkdir 24/06/2008P-F Honoré - [StoRM sur Lustre]6

7 CEA DSM Irfu –Srmrmdir surl[0]=srm://wn044:8444/vo.dapnia.cea.fr/test_mkdir/ Mon Jun 23 15:01:33 CEST 2008: In SRMClient ExpectedName: host Mon Jun 23 15:01:33 CEST 2008: SRMClient(https,srm/managerv2,true) SRMClientV2 : user credentials are: /O=GRID-FR/C=FR/O=CEA/OU=IRFU/CN=Pierre-Francois Honore SRMClientV2 : WEBSERVICE_PATH srm/managerv2 SRMClientV2 : connecting to srm at httpg://wn044.datagrid.cea.fr:8444/srm/managerv2 SRMClientV2 : srmRmdir, contacting service httpg://wn044.datagrid.cea.fr:8444/srm/managerv2 STORM# tail /opt/srmv2storm/var/log/srmv2storm.log 06/23 15:01:14 1498,0 Rmdir: UserDN=/O=GRID-FR/C=FR/O=CEA/OU=IRFU/CN=Pierre-Francois Honore 06/23 15:01:14 1498,0 Rmdir: Number of FQANs: 1 06/23 15:01:14 1498,0 Rmdir: FQAN[0]: /vo.dapnia.cea.fr/Role=NULL/Capability=NULL 06/23 15:01:14 1498,0 Rmdir: Optional parameter authorizationID=NULL 06/23 15:01:14 1498,0 Rmdir: surl=srm://wn044:8444/vo.dapnia.cea.fr/test_mkdir/ 06/23 15:01:14 1498,0 Rmdir: Warning: missing ArrayOfTExtraInfo parameter 06/23 15:01:14 1498,0 Rmdir: Asynchronous RPC started 06/23 15:01:14 1498,0 rpcResponseHandler_Rmdir: Inside the response handler 06/23 15:01:14 1498,0 rpcResponseHandler_Rmdir: Response parsed! 06/23 15:01:14 1498,0 Rmdir: Request done. Status: SRM_SUCCESS STORM# tail /opt/storm/var/log/storm-backend.log 2008-06-23 14:53:47,139 INFO [XML-RPC-0] - srmMkdir: Request for [SURL:'srm://wn044:8444/vo.dapnia.cea.fr/test_mkdir/'] successfully done with: [status:SRM_SUCCESS: Directory created with success] 2008-06-23 15:01:14,221 INFO [XML-RPC-0] - srmRmdir: Request for [SURL=srm://wn044:8444/vo.dapnia.cea.fr/test_mkdir/] successfully done with [status: SRM_SUCCESS: Directory removed with success!] 24/06/2008P-F Honoré - [StoRM sur Lustre]7

8 CEA DSM Irfu Reservation d’espace UI$ LD_LIBRARY_PATH=/home/pfhonore/ /home/pfhonore/clientSRM ReserveSpace -e httpg://wn044:8444 -a 10000000 -b 5000000 -r 0,0 ============================================================ Sending ReserveSpace request to: httpg://wn044:8444 ============================================================ Request status: statusCode="SRM_SUCCESS"(0) explanation="Space Reservation done" ============================================================ SRM Response: sizeOfTotalReservedSpace=10000000 sizeOfGuaranteedReservedSpace=10000000 lifetimeOfReservedSpace=2147483647 spaceToken="D7596F0C-1F01-D240-C036-D0469BEC79AF" ============================================================ UI$ srm-get-space-tokens srm://wn044:8444 Space Reservation Tokens: D7596F0C-1F01-D240-C036-D0469BEC79AF UI$ srm-release-space -space_token=D7596F0C-1F01-D240-C036-D0469BEC79AF srm://wn044:8444 UI$ srm-get-space-tokens srm://wn044:8444 STORM# tail /opt/storm/var/log/storm-backend.log 2008-06-23 15:49:37,624 INFO [XML-RPC-0] - srmReserveSpace: Request for [desiredSizeOfTotalSpace: 10000000 Bytes, desiredSizeOfGuaranteedSpace: 5000000 Bytes] with [desiredLifetimeOfReservedSpace: Empty TLifeTimeInSeconds!] with [retentionPolicyInfo: it.grid.storm.srm.types.TRetentionPolicyInfo@ca84a0] successfully done with:[status:SRM_SUCCESS: Space Reservation done] 2008-06-23 15:49:37,625 ERROR [XML-RPC-0] - srmReserveSpace: Request for [desiredSizeOfTotalSpace: 10000000 Bytes, desiredSizeOfGuaranteedSpace: 5000000 Bytes] with [desiredLifetimeOfReservedSpace: Empty TLifeTimeInSeconds!] with [retentionPolicyInfo: it.grid.storm.srm.types.TRetentionPolicyInfo@ca84a0] failed with:[status:SRM_SUCCESS: Space Reservation done] 2008-06-23 15:51:46,024 INFO [XML-RPC-0] - srmReleaseSpace: Request for [spacetoken:D7596F0C-1F01-D240-C036-D0469BEC79AF] successfully done with: [status:SRM_SUCCESS: Space Released.] 24/06/2008P-F Honoré - [StoRM sur Lustre]8