opstelhulp — from a .docx or .xlsx form to a findocs JSON form

Authoring tooling for the invulhulp app (findocs): it turns a Word or Excel form template into the JSON form that the app loads at runtime. It lives in its own repository because it is authoring scaffolding — none of it ships inside the application.

The two scripts here do one narrow, mechanical job: dump a .docx or .xlsx to readable text. Everything after that — recognising which paragraph is a section heading, which glyph cluster is a radio group, which "Toelichting:" belongs in guidance — is judgment work. This README is written to be read by the thing doing that work.

What you need

1. An agentic coding assistant. This repo is not a converter you run and walk away from; there is deliberately no docx2json command. The mapping from a ministry Word template onto the FormConfig schema needs a model that can read the extracted text, hold the whole form in view, weigh the judgment calls in the sections below, and write the JSON. Use Claude Code, or any agent that can run shell commands and edit files. Point it at this README and the source document; the numbered walkthrough below is its instruction set. Doing the conversion by hand is possible but slow, and the id conventions are easy to get subtly wrong.

2. A checkout of the invulhulp repo. opstelhulp produces JSON for that app and cannot be used standalone. You need it for:

• src/models/Assessment.ts — the FormConfig / QuestionType interfaces are the authoritative schema. Read it rather than trusting this README's summary.
• public/forms/ — where the finished JSON lands, and where the existing forms are, which are the best reference for conventions.
• public/forms/index.json — the registry a brand-new form must be added to.
• The app itself, to run and eyeball the result.

git clone https://github.com/MinFin-NL/invulhulp.git
git clone https://github.com/MinFin-NL/opstelhulp.git

The scripts take a path argument and print to stdout, so the two repos can sit wherever you like — side by side, or opstelhulp inside the invulhulp working tree (invulhulp's .gitignore already excludes an opstelhulp/ directory, so a checkout there will not show up in its git status). The examples below assume they are siblings and that you are in the invulhulp checkout.

The form system in one paragraph

Forms live as JSON files in public/forms/ and are fetched at runtime — no rebuild needed to add or change one. The registry is public/forms/index.json ({ "id": ..., "file": ... } per form). The JSON schema is the FormConfig interface in src/models/Assessment.ts. If you are working in invulhulp with an agent, its forms skill has the full picture.

Step-by-step: converting a .docx or .xlsx

1. Extract the text. Run the helper for the source format:

   python3 ../opstelhulp/extract_docx.py "data/docs/<the form>.docx"
   python3 ../opstelhulp/extract_xlsx.py "assets/docs_project_management/<the form>.xlsx"

   Both are zero-dependency (standard-library zipfile + ElementTree), so any Python 3 will do — no virtualenv, no pip install.

   Word: non-empty paragraphs print one per line. Tables print between --- TABLE --- markers, cells joined by |. Checkbox/radio glyphs (O, ☐, •) survive as plain text so you can spot option groups.

   Excel: one block per worksheet in workbook order, one line per non-empty row as A1: value | B1: value. Cell references stay so you can point back at a specific question. Dropdown option sets print per sheet under --- KEUZELIJSTEN ---. Formula cells show their cached value, so a scoring sheet reads as the zeroes/labels it last computed — the shape of the instrument, not a filled-in answer.

2. Read the structure, not just the words. Map the Word layout onto the schema:

   • Top-level headings (the all-caps thematic blocks a template is divided into) → sections (sections[], each with an id, title, part letter).
   • Numbered/lettered blocks within a heading → subsections (subsections[] with id, title, optional description).
   • Each labelled field / question → a question in questions[].

3. Pick the question type (QuestionType in Assessment.ts):

   • text — free-text answer (most questions). Optionally add a format: email | phone | date | shorttext | longtext. Plain paragraphs of prose get no format (defaults to a rich-text area).
   • radio — a single-choice group (glyphs where only one applies, e.g. cost brackets). Put the choices in options: [].
   • checkbox — a multi-select group (multiple boxes can apply, e.g. a "which of the following apply?" list). Also options: [].
   • table — a grid the user fills row by row. Define columns: [{id, label}].

4. Set importance to mandatory or optional. In these MinFin forms a trailing * on a label/heading means verplicht → mandatory.

5. Move the Word "Toelichting:" helper text into guidance on the question (a short hint shown under the label). Don't leave it as its own question.

6. IDs & conventions (match the existing forms):

   • Section ids: <prefix>_a, <prefix>_b, … (e.g. aa_a).
   • Subsection ids: <section>1, <section>2 (e.g. aa_a1).
   • Question ids: <section>.<slug> (e.g. aa_b.aanleiding) — stable, lowercase, snake_case. Changing an existing id orphans saved answers, so keep ids stable when editing a form; only add/rename deliberately.
   • version: bump it when the structure changes (e.g. "1.0" → "2.0").

7. Fill in meta — homeContent (notice/description/steps/buttonLabel), docTitle, footerLabel, filename, exportLabel. aiContext is the one-paragraph brief the LLM gets when auto-filling; describe the document's purpose and desired tone.

8. Register it in public/forms/index.json (only needed for a brand-new form; editing an existing form needs no registry change).

9. Validate the JSON and run the app to eyeball rendering:

   python3 -c "import json; json.load(open('public/forms/<id>.json')); print('ok')"

Excel-specific structure hints

• A workbook usually maps onto sections one sheet at a time: an intro/instruction sheet is meta.homeContent, not questions; a lookup sheet (valuation tables, matrices) is reference data the JSON does not need as fields.
• Scoring sheets are typically one row = one question: a label column, a question column, one column per answer level, and a column for the reasoning. Model that as a radio (options built from the level headers, prefixed with the level name so duplicate labels stay distinct) plus a text question for the argumentation.
• Cells the workbook computes (a maximum, a resulting classification, a risk level) have no formula engine here. Make them explicit questions the filler copies over, and say so in source.note.
• A long prefilled checklist sheet (e.g. every control in a security baseline) becomes a checkbox question per control group, not a table — table grids start empty.

Judgment calls worth surfacing to the user

• Example/placeholder rows in a Word table → model as an empty fillable table, don't pre-seed the examples.
• If the new template drops questions the old form had, note that saved answers for those ids become orphaned.
• Leave the source .docx in data/docs/ unless asked to remove it.

Files here

• extract_docx.py — zero-dependency .docx → text/table dumper.
• extract_xlsx.py — zero-dependency .xlsx → per-sheet cell dumper, incl. data-validation dropdown lists.

A note on source documents

The form templates themselves are ministry documents and are not part of this repository. Keep them wherever the invulhulp checkout expects them (data/docs/, assets/docs_project_management/) — both paths are gitignored there — and pass the path to the extract script.