From 3a17dcf1f5fee460600403f3b991d63044825aa1 Mon Sep 17 00:00:00 2001 From: zm Date: Tue, 4 Aug 2026 02:56:12 -0700 Subject: [PATCH] Fix: add documentation directory indexes Add landing pages for the hardware, troubleshooting, Remote L3, and device error-code guide collections. Organize each index by reader intent, connect related references, register the pages in navigation, and point directory entry links at explicit README files. Promote unrecognized links to warnings so strict MkDocs builds reject future dead directory links. Fixes #1608 --- docs/README.md | 6 +-- docs/dfx/README.md | 2 +- docs/hardware/README.md | 28 ++++++++++++++ docs/remote-l3-worker-design/README.md | 37 +++++++++++++++++++ docs/troubleshooting/README.md | 30 +++++++++++++++ .../device-error-codes/README.md | 27 ++++++++++++++ docs/user/how-to/debug-a-failed-run.md | 2 +- mkdocs.yml | 13 ++++++- 8 files changed, 138 insertions(+), 7 deletions(-) create mode 100644 docs/hardware/README.md create mode 100644 docs/remote-l3-worker-design/README.md create mode 100644 docs/troubleshooting/README.md create mode 100644 docs/troubleshooting/device-error-codes/README.md diff --git a/docs/README.md b/docs/README.md index bcaefdff48..8d51742fd7 100644 --- a/docs/README.md +++ b/docs/README.md @@ -43,7 +43,7 @@ changing simpler's own internals. | [Orchestrator](orchestrator.md) | DAG submission: TensorMap, Scope, Ring, task state machine | | [Scheduler](scheduler.md) | DAG dispatch: wiring / ready / completion queues, dispatch loop | | [Worker Manager](worker-manager.md) | Worker pool, THREAD/PROCESS modes, fork + mailbox mechanics | -| [hardware/](hardware/) | Hardware substrate: chip architecture, cache coherency, MMIO performance, CANN source references | +| [hardware/](hardware/README.md) | Hardware substrate: chip architecture, cache coherency, MMIO performance, CANN source references | ## Kernels and task authoring @@ -74,7 +74,7 @@ changing simpler's own internals. | [L3-L2 Message Queue](l3-l2-message-queue.md) | The queue channel between an L3 host and L2 | | [Directed NEXT_LEVEL Scheduling](directed-next-level-scheduling.md) | Targeting a specific next-level child instead of any free one | | [Remote L3 Worker Design](remote-l3-worker-design.md) | L4 host-to-host workers — protocol, transports, status | -| [remote-l3-worker-design/](remote-l3-worker-design/) | Full design set: protocol, buffers and transports, implementation plan and record | +| [remote-l3-worker-design/](remote-l3-worker-design/README.md) | Full design set: protocol, buffers and transports, implementation plan and record | ## Profiling, logging, and DFX @@ -97,5 +97,5 @@ changing simpler's own internals. | Document | What it covers | | -------- | -------------- | -| [troubleshooting/](troubleshooting/) | Device error codes, local timeout defaults, AICPU shared-SO faults, sim oversubscription hangs, macOS build issues, cpput ABI issues | +| [troubleshooting/](troubleshooting/README.md) | Device error codes, local timeout defaults, AICPU shared-SO faults, sim oversubscription hangs, macOS build issues, cpput ABI issues | | [investigations/](investigations/README.md) | Considered-and-dropped proposals and measured dead ends. **Check here before proposing an optimization or refactor** | diff --git a/docs/dfx/README.md b/docs/dfx/README.md index 3f9b687787..80bc8111eb 100644 --- a/docs/dfx/README.md +++ b/docs/dfx/README.md @@ -47,4 +47,4 @@ Analysis CLIs that consume these outputs are documented in | Document | What it covers | | -------- | -------------- | | [Log System](../logging.md) | Log levels and sinks — the host/device logging path, distinct from profiling collectors | -| [troubleshooting/](../troubleshooting/) | Device error codes and timeout defaults, for when a run fails rather than runs slowly | +| [troubleshooting/](../troubleshooting/README.md) | Device error codes and timeout defaults, for when a run fails rather than runs slowly | diff --git a/docs/hardware/README.md b/docs/hardware/README.md new file mode 100644 index 0000000000..820bffac6f --- /dev/null +++ b/docs/hardware/README.md @@ -0,0 +1,28 @@ +# Hardware + +Every hardware-level reference lives in this directory. Start with the chip +architecture overview if you need the host/chip model, or jump to the +cache-coherency and MMIO references when working on a data or notification +path. + +## Architecture and memory + +| Document | What it covers | +| -------- | -------------- | +| [Ascend Chip Architecture](chip-architecture.md) | Host, AICPU, and AICore execution tiers; chip generations; task flow; memory hierarchy | +| [Cache Coherency](cache-coherency.md) | GM visibility across AICore, AICPU, DMA, and SDMA paths, including required barriers and cache operations | + +## Control-path investigation + +| Document | What it covers | +| -------- | -------------- | +| [MMIO Performance](mmio-performance.md) | AIC control-register memory attributes, measured access costs, concurrency limits, and notification-channel tradeoffs | +| [CANN Source References](cann-source-references.md) | Upstream driver, HCCL, and HCOMM source locations used to investigate hardware and communication behavior | + +## Related, outside this directory + +| Document | What it covers | +| -------- | -------------- | +| [Chip-Level Architecture](../chip-level-arch.md) | The three-program software model layered onto the host, AICPU, and AICore tiers | +| [Task Flow](../task-flow.md) | End-to-end callable, argument, and task flow across the execution tiers | +| [AICore Kernel Programming](../aicore-kernel-programming.md) | Kernel authoring rules for the compute tier described here | diff --git a/docs/remote-l3-worker-design/README.md b/docs/remote-l3-worker-design/README.md new file mode 100644 index 0000000000..34472b712f --- /dev/null +++ b/docs/remote-l3-worker-design/README.md @@ -0,0 +1,37 @@ +# Remote L3 Worker Design References + +Detailed protocol, data-movement, delivery, and audit references live in this +directory. Start with the +[Remote L3 Worker Design](../remote-l3-worker-design.md) for the architecture +and current implementation status, then use the references below for a +specific layer or delivery stage. + +## Protocol and data movement + +| Document | What it covers | +| -------- | -------------- | +| [Remote L3 Protocol](protocol.md) | Versioned wire frames, payloads, ordering, controls, and bounds validation | +| [Remote L3 Buffers and Transports](buffers-and-transports.md) | Remote buffer ownership and lifetime, tensor sidecars, transport profiles, and simulation behavior | + +## Delivery status + +| Document | What it covers | +| -------- | -------------- | +| [Remote L3 Implementation Plan](implementation-plan.md) | Incremental delivery sequence, required tests, hardware-gated work, and open decisions | +| [Remote L3 Implementation Record](implementation-record.md) | Feature-by-feature implementation status and verification evidence | + +## PR split and audit + +| Document | What it covers | +| -------- | -------------- | +| [PR Split and Audit Plan](pr-split-and-audit-plan.md) | Strategy for splitting and reviewing the remote L3 stack without losing contract coverage | +| [PR Split and Audit Artifacts](pr-split-and-audit-artifacts.md) | Recorded split inputs, compliance matrix, risk register, and verification log | + +## Related, outside this directory + +| Document | What it covers | +| -------- | -------------- | +| [Callable Identity Registration](../callable-identity-registration.md) | Stable callable identity and target-private execution slots used by remote routing | +| [Dynamic Callable Registration over IPC](../callable-ipc-dynamic-register.md) | Registration lifecycle and control semantics shared with hierarchical workers | +| [Python Callable Serialization](../python-callable-serialization.md) | Serialization contract for negotiated remote Python callable payloads | +| [Communication Domains](../comm-domain.md) | Deferred-release communication resources that remote domains build on | diff --git a/docs/troubleshooting/README.md b/docs/troubleshooting/README.md new file mode 100644 index 0000000000..2edf3a62fe --- /dev/null +++ b/docs/troubleshooting/README.md @@ -0,0 +1,30 @@ +# Troubleshooting + +Every troubleshooting reference lives in this directory. Start with the device +error-code reference when a run fails, or jump to the environment-specific +guide when the failure happens during build, test setup, or simulation. + +## Runtime and device failures + +| Document | What it covers | +| -------- | -------------- | +| [Device Error Codes](device-error-codes.md) | Runtime and CANN error classification, first-response triage, minimal reproductions, and links to focused diagnostic guides | +| [Local Runtime Timeouts](local-timeout-defaults.md) | Local and CI timeout defaults, override variables, and the required ordering for onboard watchdogs | +| [a2a3 AICPU Shared-SO Device Fault](a2a3-507899-aicpu-shared-so-fault.md) | Diagnosing mass `507899`/`507018` cascades caused by an AICPU shared-library device fault | +| [Sim CPU Oversubscription](sim-oversubscription-hang.md) | Simulation hangs and false timeouts when AICPU and AICore host threads are CPU-starved | + +## Build and test environments + +| Document | What it covers | +| -------- | -------------- | +| [macOS Build and Lint Toolchain](macos-build.md) | Compiler, SDK, sysroot, and `clang-tidy` behavior in the macOS build pipeline | +| [macOS libomp Collision](macos-libomp-collision.md) | Duplicate OpenMP runtime crashes in single-process simulation CI and their mitigation | +| [C++ Unit-Test gtest ABI Mismatch](ut-cpp-gtest-abi.md) | Link failures caused by a system GoogleTest library built with the old libstdc++ ABI | + +## Related, outside this directory + +| Document | What it covers | +| -------- | -------------- | +| [DFX and Profiling](../dfx/README.md) | Timing, traces, dependencies, and resource-usage evidence for runs that complete or make partial progress | +| [Testing](../testing.md) | Test types, commands, platform selection, and failure-reproduction workflows | +| [Debug a Failed Run](../user/how-to/debug-a-failed-run.md) | User-facing first-response path from the surfaced error to the focused guides in this directory | diff --git a/docs/troubleshooting/device-error-codes/README.md b/docs/troubleshooting/device-error-codes/README.md new file mode 100644 index 0000000000..68cda7a43a --- /dev/null +++ b/docs/troubleshooting/device-error-codes/README.md @@ -0,0 +1,27 @@ +# Device Error Code Diagnostics + +Focused procedures for chasing runtime and device failures live in this +directory. Start with [Device Error Codes](../device-error-codes.md) to identify +the reported code or watchdog signature, then use the matching guide below. + +## Capacity and progress failures + +| Document | What it covers | +| -------- | -------------- | +| [Capacity Codes](capacity.md) | Diagnosing codes 1, 2, and 4 with scope resource peaks and distinguishing structural deadlock from stalled reclaim | +| [Stalls](stall.md) | Diagnosing scheduler timeout code 100 and tensor-wait code 8 by ordering watchdogs and locating the stuck task | + +## Core faults and defensive codes + +| Document | What it covers | +| -------- | -------------- | +| [AICore Faults](aicore-fault.md) | Separating kernel addressing faults from control-flow corruption using device logs and static inspection | +| [Codes Without End-to-End Tests](untested.md) | Why codes 10, 11, 103 and stall classes S4, S5, and unknown cannot be triggered through the public API | + +## Related, outside this directory + +| Document | What it covers | +| -------- | -------------- | +| [Local Runtime Timeouts](../local-timeout-defaults.md) | Default watchdog ordering and local overrides used to expose the most useful failure code | +| [Debug a Failed Run](../../user/how-to/debug-a-failed-run.md) | User-facing first-response workflow for collecting and classifying failure evidence | +| [L0 Swimlane Profiling](../../dfx/l0-swimlane-profiling.md) | Intra-core task timing for kernels that run but are unexpectedly slow | diff --git a/docs/user/how-to/debug-a-failed-run.md b/docs/user/how-to/debug-a-failed-run.md index cf0ab9bc57..ace016877d 100644 --- a/docs/user/how-to/debug-a-failed-run.md +++ b/docs/user/how-to/debug-a-failed-run.md @@ -25,7 +25,7 @@ mechanism before you theorize: 2. **Then look up the code** in [device-error-codes](../../troubleshooting/device-error-codes.md), the repo's own table, with per-mechanism notes in - [`device-error-codes/`](../../troubleshooting/device-error-codes/): + [`device-error-codes/`](../../troubleshooting/device-error-codes/README.md): [aicore-fault](../../troubleshooting/device-error-codes/aicore-fault.md), [capacity](../../troubleshooting/device-error-codes/capacity.md), [stall](../../troubleshooting/device-error-codes/stall.md). diff --git a/mkdocs.yml b/mkdocs.yml index e93ee0ea71..5019f34f40 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -35,6 +35,9 @@ validation: omitted_files: warn not_found: warn links: + # A directory link needs a README.md or index.md landing page. Otherwise + # MkDocs leaves the URL unchanged and the published site serves a 404. + unrecognized_links: warn # Defaults to `info`, which --strict ignores, so a link to a heading that # was renamed or never existed builds clean and 404s in the reader's # browser. Note the two slug dialects differ: GitHub keeps the doubled @@ -106,6 +109,7 @@ nav: - A5 AICPU core selection: design/a5-fg-pg-core-selection.en.md - Worker Manager: worker-manager.md - Hardware: + - Overview: hardware/README.md - Chip architecture: hardware/chip-architecture.md - Cache coherency: hardware/cache-coherency.md - MMIO performance: hardware/mmio-performance.md @@ -127,7 +131,9 @@ nav: - L3-L2 Orchestrator Comm: l3-l2-orch-comm.md - L3-L2 Message Queue: l3-l2-message-queue.md - Directed NEXT_LEVEL Scheduling: directed-next-level-scheduling.md - - Remote L3 Worker Design: remote-l3-worker-design.md + - Remote L3 Worker Design: + - Overview: remote-l3-worker-design.md + - Design document index: remote-l3-worker-design/README.md - Profiling and DFX: - Overview: dfx/README.md - Profiling Framework: dfx/profiling-framework.md @@ -154,7 +160,10 @@ nav: - Sanitizers: sanitizers.md - Sim Multi-Device Isolation: sim-multi-device-isolation.md - Troubleshooting: - - Device error codes: troubleshooting/device-error-codes.md + - Overview: troubleshooting/README.md + - Device error codes: + - Overview and code reference: troubleshooting/device-error-codes.md + - Diagnostic guide index: troubleshooting/device-error-codes/README.md - Local timeout defaults: troubleshooting/local-timeout-defaults.md - a2a3 507899 AICPU shared-SO fault: troubleshooting/a2a3-507899-aicpu-shared-so-fault.md - Sim oversubscription hang: troubleshooting/sim-oversubscription-hang.md