Document approaches for site builds on top of EESSI by casparvl · Pull Request #778 · EESSI/docs

casparvl · 2026-06-04T11:18:09Z

No description provided.

… the mkdocs.yml

ocaisa

Found some typos

first attempt at writing the shared fs section

…, after a reboot, the stratum 0 overlay won't mount

…cript. Also, added a full, copyable script combining all the previous components.

Neves-P

Looks really nice! This is my first pass of comments, as of know I only focused on the text. This week I will follow the documentation in practice and review again.

Neves-P · 2026-06-23T08:49:10Z

+
+Another thing to consider is to create a secondary user with `aws iam create-user --user-name <name>` and attach a very limited policy to it (e.g. only read/write/list on buckets, nothing else). Then, create credentials for this user with `aws iam create-access-key --user-name <name>` and provide those credentials to the EESSI build bot and Stratum 0 machines. That way, if that token is compromized, the impact is minimized (e.g. the token can at least not be used to create new IAM idententies, etc).
+
+## Setting up the EESSI build bot


I'd consider separating this into a new page. The build bot is somewhat self contained and the page is quite long already.

casparvl · 2026-07-22T14:34:04Z

+
+Another thing to consider is to create a secondary user with `aws iam create-user --user-name <name>` and attach a very limited policy to it (e.g. only read/write/list on buckets, nothing else). Then, create credentials for this user with `aws iam create-access-key --user-name <name>` and provide those credentials to the EESSI build bot and Stratum 0 machines. That way, if that token is compromised, the impact is minimized (e.g. the token can at least not be used to create new IAM idententies, etc).
+
+## Setting up the EESSI build bot


I realize these docs lack one crucial thing: setting up a site_config_script that sets EESSI_SITE_INSTALL_FORCE and EESSI_SITE_SOFTWARE_PATH_PREFIX

I.e. at SURF, our site_config_script looks like this:

# To build on top of EESSI, we need to software.eessi.io repository to be mounted next to our own repository # The bot/build.sh script does this when the EESSI_SITE_INSTALL_FORCE environment variable is set # Other build scripts will also respect this variable where needed in order to make sure that 'building on top' # of EESSI is possible export EESSI_SITE_INSTALL_FORCE=1 echo "Value of EESSI_SITE_INSTALL_FORCE: $EESSI_SITE_INSTALL_FORCE" # We also need to set a prefix that our installations should end up in # The build scripts should take this prefix, and construct the final EESSI_SITE_SOFTWARE_PATH out of it # that the EESSI-extend module expects export EESSI_SITE_SOFTWARE_PATH_PREFIX=/cvmfs/software.surf.nl/versions/2025.06/ echo "Value of EESSI_SITE_SOFTWARE_PATH_PREFIX: $EESSI_SITE_SOFTWARE_PATH_PREFIX"

One thing to figure out might be how to avoid hard-coding the version in here :)

Scratch that, this should not be needed anymore... I think we do this differently now... We automatically set these here https://github.com/EESSI/software-layer-scripts/blob/ca929cd7ef32a9fcd79fafd4e0d5c362a1fff452/bot/build.sh#L156 . That happens if the repo_name in the repos config file is unequal to software.eessi.io or dev.eess.io essentially.

I think this does warrant a small explanation of why EESSI_SITE_SOFTWARE_PREFIX, EESSI_SITE_INSTALL and friends are not needed with this workflow. Maybe we don't even need an explanation, but just a mention.

I'm saying this because going through the whole docs - especially the shared file system installs - you get the impression that at least doing EESSI_SITE_INSTALL is necessary for site installations (which it is, but that happens behind the scenes).

Neves-P

Just two comments, suggestions. I would have liked to have been more thorough, but overall I did two passes to the text and managed to get to the point of installing the bot (had to wait a bit for the internal S3 and maintenance).

In my mind, after addressing these comments we can either wait a little longer and I spin up a new test CVMFS infrastructure to also test that part, or merge before that and address any thing else we spot in a later PR.

Overall I think the instructions are very nice! I'm not sure I am able to put myself in the shoes of a complete novice to EESSI, but they seem very appropriate for that target audience.

Neves-P · 2026-08-04T10:16:59Z

+
+Another thing to consider is to create a secondary user with `aws iam create-user --user-name <name>` and attach a very limited policy to it (e.g. only read/write/list on buckets, nothing else). Then, create credentials for this user with `aws iam create-access-key --user-name <name>` and provide those credentials to the EESSI build bot and Stratum 0 machines. That way, if that token is compromised, the impact is minimized (e.g. the token can at least not be used to create new IAM idententies, etc).
+
+## Setting up the EESSI build bot


I think this does warrant a small explanation of why EESSI_SITE_SOFTWARE_PREFIX, EESSI_SITE_INSTALL and friends are not needed with this workflow. Maybe we don't even need an explanation, but just a mention.

I'm saying this because going through the whole docs - especially the shared file system installs - you get the impression that at least doing EESSI_SITE_INSTALL is necessary for site installations (which it is, but that happens behind the scenes).

Neves-P · 2026-08-04T15:38:05Z

+``` { .bash .copy }
+curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
+unzip awscliv2.zip
+sudo ./aws/install


This can be installed in userland, and ideally I'd do so. That's how I normally install it in the dev.eessi.io bots and it works well.

Neves-P · 2026-08-04T15:40:02Z

+
+The standard deployment method for the EESSI build bot is to stage tarballs in an S3 bucket. While the bot's functionality may be extended in the future (the [function](https://github.com/EESSI/eessi-bot-software-layer/blob/29dc5e9aa339c323a900dc1454d39246def73984/tasks/deploy.py#L689) actually uploading the tarballs could easily be altered to deploy to a central directory on a local filesystem, for example), for now this means we need an S3 bucket if we want to use the bot's deployment functionality.
+
+There is extensive documentation on how to interact with S3 buckets available online - here we only list the few commands that you would commonly need to set up a staging bucket. Note that you can create the bucket from anywhere - but you'll need the AWS commands to be available on the machine where you [install the EESSI build bot](#install-eessi-build-bot-on-a-machine), so you might as well set it up there directly.


Suggested change

There is extensive documentation on how to interact with S3 buckets available online - here we only list the few commands that you would commonly need to set up a staging bucket. Note that you can create the bucket from anywhere - but you'll need the AWS commands to be available on the machine where you [install the EESSI build bot](#install-eessi-build-bot-on-a-machine), so you might as well set it up there directly.

There is extensive documentation on how to interact with S3 buckets available online - here we only list the few commands that you would commonly need to set up a staging bucket. Note that you can create the bucket from anywhere - but you'll need the AWS commands to be available on the machine where you [install the EESSI build bot](#install-eessi-build-bot-on-a-machine), so you might as well set it up there directly. You don't necessarily need to use Amazon AWS S3 buckets, for example, if you have access to object storage on a Ceph cluster. The only requirement is that the `aws` tool can read and write to your endpoint.

Add initial files for each site-building approach

a4d5de8

casparvl changed the title ~~Add initial files for each site-building approach~~ Document approaches for site builds on top of EESSI Jun 4, 2026

Caspar van Leeuwen and others added 5 commits June 4, 2026 17:12

Made a start with the site builds as a CVMFS repo

259e935

Started to write Stratum 0 install docs. Work in progress

6aebcec

Slight rephrasing of site-build overview section and add new pages to…

dcf695f

… the mkdocs.yml

Merge branch 'main' into site_builds

87ccf16

Corrected typo

c887a09

ocaisa requested changes Jun 9, 2026

View reviewed changes

Caspar van Leeuwen added 2 commits June 9, 2026 15:20

More or less completed the docs on how to setup a Stratum 0

729cf88

Finished description for Stratum 0 and Stratum 1

c31746d

casparvl commented Jun 10, 2026

View reviewed changes

Comment thread docs/site_build/site_cvmfs.md

Caspar van Leeuwen and others added 9 commits June 11, 2026 10:09

Finished a few more chapters

e4fac7e

Small update

b9205ca

Started section on installing EESSI build bot on a machine

7b027fc

Further work on deploying bot on machine

3f49e7b

first attempt at writing the shared fs section

8fe6e19

Merge pull request #5 from bedroge/site_builds_shared_fs

0cddb6b

first attempt at writing the shared fs section

Make sure we add instructions to stop _and disable_ autofs. Otherwise…

ed6f41a

…, after a reboot, the stratum 0 overlay won't mount

Merge branch 'site_builds' of github.com:casparvl/docs into site_builds

5aa289b

Add some debugging instructions

d200d12

Neves-P mentioned this pull request Jun 30, 2026

Add documentation on how to build on top of EESSI as a site - and include the use of EESSI_SITE_SOFTWARE_PREFIX in that #768

Open

Caspar van Leeuwen added 2 commits June 30, 2026 16:23

Added an image, rewrote some parts, fixed some formatting

92f27e0

Fixed some typos

6ea3bef

casparvl marked this pull request as ready for review June 30, 2026 14:36

Caspar van Leeuwen and others added 2 commits July 1, 2026 17:36

Added logic to do signature verification to the automatic ingestion s…

acfcb9e

…cript. Also, added a full, copyable script combining all the previous components.

Fixed codespell issues

2bba8e8

Neves-P linked an issue Jul 7, 2026 that may be closed by this pull request

Add documentation on how to build on top of EESSI as a site - and include the use of EESSI_SITE_SOFTWARE_PREFIX in that #768

Open

Neves-P self-requested a review July 7, 2026 08:16

Neves-P requested changes Jul 20, 2026

View reviewed changes

Fix typo

e0724fa

Neves-P added 5 commits July 20, 2026 16:04

Fix missing word

7bef7b8

Fix markdown

51a0d0a

Add line break

0cc3ec7

Bash highlight

bf3f6f8

Use monospaced font for path

e18965a

casparvl commented Jul 22, 2026

View reviewed changes

Neves-P requested changes Aug 4, 2026

View reviewed changes


		Another thing to consider is to create a secondary user with `aws iam create-user --user-name <name>` and attach a very limited policy to it (e.g. only read/write/list on buckets, nothing else). Then, create credentials for this user with `aws iam create-access-key --user-name <name>` and provide those credentials to the EESSI build bot and Stratum 0 machines. That way, if that token is compromized, the impact is minimized (e.g. the token can at least not be used to create new IAM idententies, etc).

		## Setting up the EESSI build bot


		The standard deployment method for the EESSI build bot is to stage tarballs in an S3 bucket. While the bot's functionality may be extended in the future (the [function](https://github.com/EESSI/eessi-bot-software-layer/blob/29dc5e9aa339c323a900dc1454d39246def73984/tasks/deploy.py#L689) actually uploading the tarballs could easily be altered to deploy to a central directory on a local filesystem, for example), for now this means we need an S3 bucket if we want to use the bot's deployment functionality.

		There is extensive documentation on how to interact with S3 buckets available online - here we only list the few commands that you would commonly need to set up a staging bucket. Note that you can create the bucket from anywhere - but you'll need the AWS commands to be available on the machine where you [install the EESSI build bot](#install-eessi-build-bot-on-a-machine), so you might as well set it up there directly.

Uh oh!

Conversation

casparvl commented Jun 4, 2026

Uh oh!

ocaisa left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Neves-P left a comment

Choose a reason for hiding this comment

Uh oh!

Neves-P Jun 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

casparvl Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

casparvl Jul 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

casparvl Jul 22, 2026

Choose a reason for hiding this comment

Uh oh!

Neves-P Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Neves-P left a comment

Choose a reason for hiding this comment

Uh oh!

Neves-P Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Neves-P Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

Neves-P Aug 4, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

casparvl Jul 22, 2026 •

edited

Loading

Neves-P Aug 4, 2026 •

edited

Loading

Neves-P Aug 4, 2026 •

edited

Loading