Keep documentation on your computer

AI tools are better when they can look something up instead of guessing. This project downloads official documentation onto your computer, keeps those copies up to date, and lets you (or an AI assistant) search them without going online every time.

Think of it as a filing cabinet. You choose which websites to save. The copies live in a folder called reference/ on your machine. This repository is only the toolkit that fills that cabinet — the downloaded pages are created on your computer, and they are not stored in git.

You do not need to be a programmer. You can pick numbers from a menu, or hand this file to an AI assistant and let it do the work.

---

Have an AI do this for you

Open this project (or this README) in Claude, Codex, Cursor, Copilot, or any similar assistant, and paste:

Set up this documentation toolkit on my computer and download the starter docs.
Follow the section "If you are an AI agent" in README.md.
Do not copy files to other machines unless I ask.

If you also want a particular website saved, add its address to that message. Then let the assistant run the commands. You should not need the menu.

If you are an AI agent

You are setting this toolkit up for a human. You are not developing it. Do not edit source, tests, or gitignored config. Do not run the interactive menu (./bin/scanner) — it waits for keyboard input.

1. Confirm python3 --version works. If it does not, tell the human to install Python 3 and stop. Do not pip install anything. This project uses the standard library only.

2. If this folder is not already ai-reference-docs, clone https://github.com/theglove44/ai-reference-docs.git and work inside it.

3. Default job — fill the starter library:

   python3 scripts/update_local_docs.py --only claude-code --only openai-codex --only pi

   Download only those three. Do not run a bare python3 scripts/update_local_docs.py — this machine may have extra private sources, and they are not part of first-time setup. Do not copy config/sources.local.json from anyone else.

   That hits live documentation sites and can take several minutes. A failed source leaves its previous copy in place.

4. If the human named one starter source, refresh only that one:

   python3 scripts/update_local_docs.py --only SOURCE_ID

5. If the human gave a documentation URL, pre-check it, then run the command it prints (Next:):

   python3 scripts/precheck_url.py --json THE_URL

   Use --source they chose, or the suggested name. Keep --max-pages at the pre-check default. Do not crawl a whole domain. If the report says rendered_browser and there is no Chrome, Chromium, or Edge, stop and say so. If there is no Next: command, stop and explain what the report recommended — do not invent a scrape, and do not overwrite config/sources.local.json.

6. Success is folders appearing under reference/. List them. Tell the human they can search with rg "topic" reference/<name>/ and that those files must not be edited by hand.

Never run scripts/push_reference_docs.py or scripts/refresh_and_push.py unless the human clearly asked to copy docs onto other machines. Never commit config/deploy_fleet.json, config/sources.local.json, or anything under reference/.

AGENTS.md is for people changing this project. Ignore it during setup.

---

What you need

• A Mac or Linux computer
• An internet connection
• Git (to copy this project onto your computer)
• Python 3 (already present on many Macs)

You do not install extra Python packages.

Chrome, Chromium, or Edge is only needed later, for a few modern sites that do not show their text without a browser.

Copying the filing cabinet to other computers is optional. Skip that until this computer is working.

Check Python in Terminal:

python3 --version

If that fails, install Python 3 from python.org.

---

Set it up

On a Mac, open Terminal (press Command-Space, type Terminal, press Return).

1. Copy this project onto your computer:

git clone https://github.com/theglove44/ai-reference-docs.git
cd ai-reference-docs

If you already have the folder, skip the clone and cd into it instead.

2. Start the menu:

./bin/scanner

If that does not run:

python3 scripts/interactive_menu.py

You should see a numbered list. Type a number and press Return. That is the whole interface.

You are set up when the menu appears. The reference/ folder is still empty until you download something.

---

Download your first docs

Right after a clone, nothing has been saved yet. In the menu, choose Update existing docs, then Refresh all managed sources.

That downloads the starter set that ships with the project: Claude Code, OpenAI Codex, and Pi. Anything else is optional — you add it yourself. It can take a while. If one site fails, the last good copy of that site is left alone.

When it finishes, open the reference/ folder. You should see one folder per product, full of readable files.

Or save a site that is not on the starter list

Back at the menu, choose Scrape a new URL (“scrape” here just means download pages and save them as files).

1. Paste the address of a documentation page.
2. Answer the questions. The menu checks the site and picks a method.
3. Give the copy a short name, like my-product. That becomes reference/my-product/.

A longer walkthrough is in docs/scrape-your-own-docs.md.

---

Look things up

Search the copies on your computer. If you have ripgrep installed (rg), this is the usual way:

rg "authentication" reference/claude-code/

Finder works too: open reference/ and search inside it.

Point your AI tools at the reference/ folder and tell them to search there first. Do not edit those files by hand. Download them again when you want a fresh copy, so the next update can rebuild the same result.

---

Other computers (optional)

Only do this if several machines should share the same local copies, and those machines are already on your Tailscale network.

1. In the scanner menu, choose Onboard a new machine.
2. Pick the computer from the Tailscale list, confirm the login user, and let it copy reference/ across.

That writes config/deploy_fleet.json on this computer (keep it out of git) and copies the files in reference/. The other computers do not need your private source list (config/sources.local.json).

You can still create the fleet file by hand from config/deploy_fleet.example.json if you prefer. Later copies use Push local docs to other machines.

Details: docs/device-deployment.md.

---

If something goes wrong

The scanner will not start. Make sure Terminal is inside the ai-reference-docs folder, then try python3 scripts/interactive_menu.py.

Python is missing. python3 --version should print a number. If it does not, install Python 3 and open a new Terminal window.

A download fails. Try that source again later. The previous copy stays on disk.

A site comes back almost empty. Install Chrome (or Chromium or Edge) and try that site again. A few documentation sites need a real browser.

You would rather type commands than use the menu. Use the short command list in docs/quick-use-guide.md. The menu is still the easier first stop.

---

Longer guides

• Save a site that is not on the starter list
• Day-to-day commands
• Copy docs to other computers
• How AI tools should search the copies