Documentation refactoring

[Clare72]

rearrangement and some modifications to docs
I moved ontologies to their own section (so not bloating releases section) - I guess something needs changing to generate stuff in the right place?
I didn't end up moving the content report and I linked to the list of scRNAseq datasets instead of trying to build a page for it.

[Robbie1977]

Issues Identified

CRITICAL ISSUE 1: update_vfb_terms_to_static_site Jenkins job will break

The problem: The update_vfb_terms_to_static_site job runs vfbterms.py with hardcoded paths:

bash

# Build step 1 (line 1021-1023 in Background.txt):
cd "$WORKSPACE/content/en/blog/releases/ontologies/"
python3 "$WORKSPACE/vfbterms.py" "$WORKSPACE/../update_vfb_static_site/content/en/blog/releases/ontologies/"

The script receives content/en/blog/releases/ontologies/ as the base path, then uses relative subdirectories like fbbt/, go/, vfb/, etc. from that base.

After PR #398, the ontology directories will be at content/en/blog/ontologies/ instead. Both the cd command and the argument to vfbterms.py will fail because content/en/blog/releases/ontologies/ will no longer exist.

Files involved:

• Jenkins job: update_vfb_terms_to_static_site - Build Step 1
• Script: /Users/rcourt/GIT/VFB2/vfbterms.py (lines 878-904) - the relative paths within the script (fbbt/, go/, etc.) are fine, but the base path passed as argument needs updating
• Also note: the Datasets group in vfbterms.py uses ../datasets/ relative path (line 885), which currently resolves to blog/releases/datasets/. After the PR, blog/releases/datasets/ is deleted, so Datasets term generation will also break.

Fix required: Update Jenkins job to use:

bash

cd "$WORKSPACE/content/en/blog/ontologies/"
python3 "$WORKSPACE/vfbterms.py" "$WORKSPACE/../update_vfb_static_site/content/en/blog/ontologies/"

And either create a blog/datasets/ directory or adjust the Datasets relative path in vfbterms.py.

CRITICAL ISSUE 2: update_vfb_terms_to_static_site post-build symlink will break

The problem: Build Step 2 of the same job creates a symlink for SEO:

bash

cd "$WORKSPACE/../update_vfb_static_site/public"
if [ ! -d term ]
then
    ln -s blog/2022/01/01 term
fi

The ontology term pages currently have date 2022-01-01 in their frontmatter, so Hugo generates them at public/blog/2022/01/01/<term_id>/. The symlink term → blog/2022/01/01 makes them accessible at /term/<id>/.

After the PR, the ontology pages move to blog/ontologies/ which is not under blog/releases/. The new _index.md at blog/ontologies/ has date: 2021-12-24. The generated term pages' Hugo output path will change depending on the cascade section type configuration. The new _index.md sets type: "docs" in a cascade, which may change how Hugo resolves the URL.

Impact: The ln -s blog/2022/01/01 term symlink will likely point to a directory that no longer contains the ontology term pages. All /term/FBbt_* etc. URLs (used in sitemaps and external links) would 404.

Fix required: Determine what Hugo output path the moved ontology term pages will generate, and update the symlink target accordingly. Also update the sitemap generation loop.

ISSUE 3: update_vfb_static_site Jenkins job - content_report.md path

The problem: Build Step 1 writes to:

bash

echo '...' > en/blog/releases/content_report.md
sed '...' content_report.md >> en/blog/releases/content_report.md

PR #398 does not move or delete content_report.md from blog/releases/ (it was never in the repo, only generated at build time). However, the PR does change blog/releases/_index.md description, and the releases section is being slimmed down. The content report will still be generated into blog/releases/ which should still work, but it's worth verifying the intent - should it also move?

Severity: Low - this will likely still work, but may be orphaned content.

ISSUE 4: update_vfb_static_site Jenkins job - geppetto-vfb.md

The problem: The build script manipulates en/blog/releases/geppetto-vfb.md:

bash

cat en/blog/releases/geppetto-vfb.md | grep -v "..." > en/blog/releases/geppetto-vfb1.md
mv -vf en/blog/releases/geppetto-vfb1.md en/blog/releases/geppetto-vfb.md
curl ... >> en/blog/releases/geppetto-vfb.md

PR #398 moves blog/releases/geppetto/_index.md → about/geppetto.md, but blog/releases/geppetto-vfb.md (separate file with GitHub release links) still exists in the repo and is not moved by the PR. So this build step should still work.

Severity: Low - still works but the geppetto-vfb.md file is now somewhat orphaned from the moved geppetto page.

ISSUE 5: updateVFBcontributors job writes to content/en/about/contributors.md

The problem: This job generates content/en/about/contributors.md. PR #398 modifies content/en/about/collaborators.md but not contributors.md. The job pushes directly to master.

Severity: None - this job is unaffected. The path hasn't changed.

ISSUE 6: Merge conflicts

The PR currently shows "mergeable": "CONFLICTING". This must be resolved before merge.

ISSUE 7: Rancher hugo service volume mounts and hugo-server

Hugo build service: Mounts jenkins_home/workspace/update_vfb_static_site:/src, runs /src/deploy.sh which does hugo --enableGitInfo --gc -v. The deploy.sh installs npm deps and runs Hugo. This should work fine with the restructured content - Hugo will just build whatever content structure it finds.

Hugo-server service: Mounts jenkins_home/workspace/update_vfb_static_site/public:/usr/share/nginx/html and serves with nginx (rcourt/docker-nginx-fileserver:v1.1.3-p). This serves the public/ output. The key concern is the term symlink (Issue 2) - if it's broken, the nginx server will serve 404s for all term pages.

Severity of build service: None - Hugo is content-agnostic. Severity of server service: Tied to Issue 2 - broken symlink = broken term URLs.

---

Summary of Required Actions Before Merging PR #398

Priority	Issue	Action Required
CRITICAL	Jenkins update_vfb_terms_to_static_site paths	Update both cd and vfbterms.py argument from blog/releases/ontologies/ to blog/ontologies/
CRITICAL	Datasets path in vfbterms.py	The ../datasets/ relative path (line 885) needs updating since blog/releases/datasets/ is deleted
CRITICAL	Term symlink ln -s blog/2022/01/01 term	Determine new Hugo output path for ontology terms under blog/ontologies/ and update symlink + sitemap generation
HIGH	Merge conflicts	Resolve before merge
LOW	content_report.md placement	Verify intent - still generated into blog/releases/
LOW	geppetto-vfb.md orphaning	Consider whether this file should also be moved or removed

[Robbie1977]

Already Done

Merge conflict - Resolved and merged
vfbterms.py datasets path - Fixed '../datasets/' → 'datasets/' (now under blog/ontologies/datasets/)
blog/ontologies/datasets/_index.md - Created for Hugo to recognize the section
Jenkins update_vfb_terms_to_static_site - You updated the paths from blog/releases/ontologies/ → blog/ontologies/

No Changes Needed

Symlink (ln -s blog/2022/01/01 term) - Still works because Hugo's permalink rule blog = "/:section/:year/:month/:day/:slug/" renders by date, not directory structure
Sitemap generation - Same reason, term paths unchanged in public/
update_vfb_static_site Jenkins job - content_report.md and geppetto-vfb.md still write to blog/releases/ which still exists
updateVFBcontributors Jenkins job - Writes to content/en/about/contributors.md, unaffected
Rancher hugo/hugo-server services - Hugo builds whatever content it finds; nginx serves public/ which is unchanged in structure