Add RSE configuration guide for operators #710
Conversation
Documentation on setting up and configuring Rucio Storage Elements (RSEs) from an operator's perspective. Includes: - Overview of RSE types (POSIX, WebDAV, Disk, Tape) - Two setup methods: CLI and Python API with side-by-side examples - Configuration examples for each RSE type - WebDAV setup with Apache configuration and davs protocol - EOS disk RSE with https and root protocols - CTA tape RSE configuration with staging timeouts - RSE attributes, protocols, and account limits reference - Best practices and common pitfalls - Quick reference commands The examples use the latest `rucio` CLI commands Co-authored-by: Nikita Avdeev <[email protected]> Co-authored-by: Luis Antonio Obis Aparicio <[email protected]>
|
|
||
| ```python | ||
| #!/usr/bin/env python | ||
| from rucio.core.rse import add_rse, add_protocol, get_rse_id, add_rse_attribute |
There was a problem hiding this comment.
This should be done through the client, not core
| 2. **Python API**: Using the Rucio core Python library | ||
|
|
||
| Both methods achieve the same result. Choose the method that best fits your workflow. | ||
|
|
There was a problem hiding this comment.
A set of definitions for limits, protocols, and attributes here would be helpful.
There was a problem hiding this comment.
We should also link the rse attributes section of config parameters and the protocols docs
| rucio rse add $RUCIO_RSE_NAME | ||
|
|
||
| # Set RSE type and attributes | ||
| rucio rse update $RUCIO_RSE_NAME --key rse_type --value TAPE |
There was a problem hiding this comment.
This ends up being a little confusing with the disk rse being set with
rucio rse update $RUCIO_RSE_NAME --key rse_type --value DISK
rucio rse attribute add $RUCIO_RSE_NAME --key istape --value False
The disiction between istape and rse_type should be made more clear
| 'scheme': 'https', | ||
| 'hostname': 'eospublic.cern.ch', | ||
| 'port': 8444, | ||
| 'prefix': '//eos/workspace/e/experiment/tmp', |
There was a problem hiding this comment.
The urls and prefixes should probably be made more generic
| - Typically only `https` protocol is defined (avoid `root` protocol for tape) | ||
|
|
||
| **Testing Environment:** | ||
| At CERN, the path `//eos/ctaatlaspps/archivetest/experiment/tmp` can be used for tape testing. This is a temporary space where data is not expected to be kept long-term. |
There was a problem hiding this comment.
This is experiment specific (ish) and shouldn't be in the general docs
|
|
||
| --- | ||
|
|
||
| ## RSE Attributes |
There was a problem hiding this comment.
Link to https://rucio.cern.ch/documentation/operator/configuration_parameters#rse-attributes instead of making a new table for this
|
|
||
| Protocols define how clients access data on an RSE. Multiple protocols can be defined for the same RSE. | ||
|
|
||
| ### Common Protocol Schemes |
There was a problem hiding this comment.
Link to these instead - https://rucio.cern.ch/documentation/html/transfer_protocols/posix.html
|
Overall all these changes are really good!! Only thing I didn't comment on directly in the body of the review is that we might want to mention how rse specific limits work vs account only limits |
2 participants
Closes #709
Documentation on setting up and configuring Rucio Storage Elements (RSEs) from an operator's perspective. Includes:
The examples use the latest
rucioCLI commands