Skip to content

Commit c2394de

Browse files
jiangxinjiangxin
committed
docs(l10n): add AI agent instructions to review translations
Add a new "Reviewing po/XX.po" section to po/AGENTS.md that provides comprehensive guidance for AI agents to review translation files. Translation diffs lose context, especially for multi-line msgid and msgstr entries. Some LLMs ignore context and cannot evaluate translations accurately; others rely on scripts to search for context in source files, making the review process time-consuming. To address this, git-po-helper implements the compare subcommand, which extracts new or modified translations with full context (complete msgid/msgstr pairs), significantly improving review efficiency. A limitation is that the extracted content lacks other already-translated content for reference, which may affect terminology consistency. This is mitigated by including a glossary in the PO file header. git-po-helper-generated review files include the header entry and glossary (if present) by default. The review workflow leverages git-po-helper subcommands: - git-po-helper compare: Extract new or changed entries between two versions of a PO file into a valid PO file for review. Supports multiple modes: * Compare HEAD with the working tree (local changes) * Compare a commit's parent with the commit (--commit) * Compare a commit with the working tree (--since) * Compare two arbitrary revisions (-r) - git-po-helper msg-select: Split large review files into smaller batches by entry index range for manageable review sessions. Supports range formats like "-50" (first 50), "51-100", "101-" (to end). Evaluation with the Qwen model: git-po-helper agent-run review --commit 2000abe --agent qwen Benchmark results: | Metric | Value | |------------------|----------------------------------| | Turns | 22 | | Input tokens | 537263 | | Output tokens | 4397 | | API duration | 167.84 s | | Review score | 96/100 | | Total entries | 63 | | With issues | 4 (1 critical, 2 major, 1 minor) | Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
1 parent bef8cd9 commit c2394de

1 file changed

Lines changed: 193 additions & 1 deletion

File tree

‎po/AGENTS.md‎

Lines changed: 193 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ most commonly used housekeeping tasks:
1010
1. Generating or updating po/git.pot
1111
2. Updating po/XX.po
1212
3. Translating po/XX.po
13+
4. Reviewing translation quality
1314

1415

1516
## Background knowledge for localization workflows
@@ -731,6 +732,191 @@ and fuzzy entry; do not stop before the loop completes.
731732
```
732733

733734

735+
### Task 4: Review translation quality
736+
737+
Review may target the full `po/XX.po`, a specific commit, or changes since a
738+
commit. When asked to review, follow the steps below. **Note**: This task uses
739+
`git-po-helper compare`; if `git-po-helper` is not available, the task
740+
cannot be performed.
741+
742+
1. **Check for existing review**: Evaluate the following in order:
743+
744+
- If `po/review-input.po` does **not** exist, proceed to step 2 regardless
745+
of any other files (e.g., batch or JSON files).
746+
- If both `po/review-input.po` and `po/review-result.json` exist, go
747+
directly to step 5 (Merge and summary) and display the report.
748+
Do **not** check for batch or other temporary files; no further review
749+
steps are needed.
750+
- If `po/review-input.po` exists but `po/review-result.json` does not,
751+
go to step 4 (Process one batch) to continue the previous review.
752+
753+
2. **Extract entries**: Run `git-po-helper compare` with the desired range and
754+
redirect the output to `po/review-input.po`. Do not use `git show` or
755+
`git diff`—they can fragment or lose PO context (see "Comparing PO files
756+
for translation and review" under git-po-helper).
757+
758+
3. **Prepare review batches**: Run the script below to clean up any leftover
759+
files from previous reviews and split `po/review-input.po` into one or
760+
more `po/review-input-<N>.json` files (dynamic batch sizing). Run as a
761+
single script (define the function, then call it):
762+
763+
```shell
764+
review_split_batches () {
765+
min_batch_size=${1:-50}
766+
rm -f po/review-input-*.json
767+
rm -f po/review-result-*.json
768+
rm -f po/review-result.json
769+
rm -f po/review-output.po
770+
771+
ENTRY_COUNT=$(grep -c '^msgid ' po/review-input.po 2>/dev/null || true)
772+
ENTRY_COUNT=$((ENTRY_COUNT > 0 ? ENTRY_COUNT - 1 : 0))
773+
774+
if test "$ENTRY_COUNT" -gt $min_batch_size
775+
then
776+
if test "$ENTRY_COUNT" -gt $((min_batch_size * 8))
777+
then
778+
NUM=$((min_batch_size * 2))
779+
elif test "$ENTRY_COUNT" -gt $((min_batch_size * 4))
780+
then
781+
NUM=$((min_batch_size + min_batch_size / 2))
782+
else
783+
NUM=$min_batch_size
784+
fi
785+
BATCH_COUNT=$(( (ENTRY_COUNT + NUM - 1) / NUM ))
786+
for i in $(seq 1 "$BATCH_COUNT")
787+
do
788+
START=$(((i - 1) * NUM + 1))
789+
END=$((i * NUM))
790+
if test "$END" -gt "$ENTRY_COUNT"
791+
then
792+
END=$ENTRY_COUNT
793+
fi
794+
if test "$i" -eq 1
795+
then
796+
git-po-helper msg-select --json --range "-$NUM" \
797+
-o "po/review-input-$i.json" po/review-input.po
798+
elif test "$END" -ge "$ENTRY_COUNT"
799+
then
800+
git-po-helper msg-select --json --range "$START-" \
801+
-o "po/review-input-$i.json" po/review-input.po
802+
else
803+
git-po-helper msg-select --json --range "$START-$END" \
804+
-o "po/review-input-$i.json" po/review-input.po
805+
fi
806+
done
807+
else
808+
git-po-helper msg-cat --json \
809+
-o po/review-input-1.json po/review-input.po
810+
fi
811+
}
812+
# Parameter controls batch size; reduce if the batch file is too large for
813+
# the Agent to process.
814+
review_split_batches 20
815+
```
816+
817+
4. **Process one batch (repeat until none left)**:
818+
819+
a. If no `po/review-input-*.json` files exist, proceed to step 5.
820+
821+
b. Select the smallest remaining index N (e.g. `po/review-input-1.json`).
822+
The current batch is `po/review-input-<N>.json`.
823+
824+
c. Review translation quality in the current batch: Read the current
825+
batch file (`po/review-input-<N>.json`) and:
826+
- Consult the "Background knowledge for localization workflows" section
827+
for PO format, JSON format, placeholder rules, and terminology. If the
828+
current batch file has a glossary in the `header_comment` field, add
829+
it to your context for consistent terminology.
830+
- Do not review or modify the header entry (in PO format: empty `msgid`
831+
with metadata in `msgstr`; in JSON format: `header_comment` and
832+
`header_meta`).
833+
- For all other entries, check the quality of translations in `msgstr`
834+
(singular form) and `msgstr_plural` (plural forms) against `msgid` and
835+
`msgid_plural`. See the "Quality checklist" above for criteria.
836+
837+
d. After reviewing all entries in the current batch, write the issues you
838+
found to `po/review-result-<N>.json` using the format described in the
839+
"Review result JSON format" section below. If no issues are found, write
840+
`{"issues": []}` to `po/review-result-<N>.json`. Always write this file;
841+
it marks the batch as complete.
842+
843+
e. Delete the current batch file (`po/review-input-<N>.json`).
844+
845+
f. Return to step 4a.
846+
847+
This loop is resumable: remaining `po/review-input-*.json` files indicate
848+
batches still to process.
849+
850+
5. **Merge and summary**: Run the command below to merge all
851+
`po/review-result-*.json` files into `po/review-result.json`, apply the
852+
result to `po/review-output.po`, and display the report.
853+
854+
```shell
855+
git-po-helper agent-run report
856+
```
857+
858+
**Do not delete** `po/review-result.json`, `po/review-output.po`, or
859+
`po/review-input.po`.
860+
861+
**Review result JSON format**:
862+
863+
The **Review result JSON** format defines the structure for translation
864+
review reports. For each entry with translation issues, create an issue
865+
object as follows:
866+
867+
- Copy the original entry's `msgid`, `msgstr`, `msgid_plural` and
868+
`msgstr_plural` (if present) to the corresponding fields in the
869+
result issue object.
870+
- Write a summary of all issues found for this entry in `description`.
871+
- Set `score` according to the severity of issues found for this entry,
872+
from 0 to 3 (3 = perfect, no issues; 0 = critical, 1 = major, 2 = minor).
873+
- Place the suggested translation in `suggest_msgstr` (singular) or
874+
`suggest_msgstr_plural` (plural).
875+
- Include only entries with issues (score less than 3). When no issues are
876+
found in the batch, write `{"issues": []}`.
877+
878+
Example review result (with issues):
879+
880+
```json
881+
{
882+
"issues": [
883+
{
884+
"msgid": "commit",
885+
"msgid_plural": "",
886+
"msgstr": "委托",
887+
"msgstr_plural": [],
888+
"suggest_msgstr": "提交",
889+
"suggest_msgstr_plural": [],
890+
"score": 0,
891+
"description": "Terminology error: 'commit' should be translated as '提交'"
892+
},
893+
{
894+
"msgid": "repository",
895+
"msgid_plural": "repositories",
896+
"msgstr": "",
897+
"msgstr_plural": ["版本库", "版本库"],
898+
"suggest_msgstr": "",
899+
"suggest_msgstr_plural": ["仓库", "仓库"],
900+
"score": 2,
901+
"description": "Consistency issue: '版本库' and '仓库' are used interchangeably; suggest using '仓库' consistently"
902+
}
903+
]
904+
}
905+
```
906+
907+
Field descriptions for each issue object (element of the `issues` array):
908+
909+
- `msgid` (and `msgid_plural` for plural entries): Original source text.
910+
- `msgstr` (and `msgstr_plural` for plural entries): Original translation.
911+
- `suggest_msgstr`: Suggested translation for the singular form.
912+
- `suggest_msgstr_plural`: Array of suggested translations for plural forms;
913+
`suggest_msgstr` is empty for plural-only entries.
914+
- `score`: 0–3 (see scale below).
915+
- `description`: Brief summary of the issue.
916+
- Score scale: 0 = critical (must fix before release), 1 = major (should fix),
917+
2 = minor (improve later), 3 = perfect.
918+
919+
734920
## Human translators remain in control
735921

736922
Git translation is human-driven; language team leaders and contributors are
@@ -743,7 +929,13 @@ responsible for:
743929
- Building and maintaining language glossaries
744930
- Reviewing and approving all changes before submission
745931

746-
AI tools, if used, only accelerate routine tasks.
932+
AI tools, if used, only accelerate routine tasks:
933+
934+
- First-draft translations for new or updated messages
935+
- Finding untranslated or fuzzy entries
936+
- Checking consistency with glossary and existing translations
937+
- Detecting technical errors (placeholders, formatting)
938+
- Reviewing against quality criteria
747939

748940
AI-generated output should always be treated as rough drafts requiring human
749941
review, editing, and approval by someone who understands both the technical

0 commit comments

Comments
 (0)