Repository navigation
[libc][ci] Add automated pre-commit and post-commit coverage bots - #215269
Draft
tapiwagonga wants to merge 9 commits into
Draft
tapiwagonga wants to merge 9 commits into
tapiwagonga wants to merge 9 commits into
Conversation
|
Hello @tapiwagonga 👋 Thank you for submitting a Pull Request (PR) to the LLVM Project. Since this is your first PR, here are a few useful links covering our main contribution policies and review practices.
Please reply to this message to confirm that you have read these policies, especially the LLVM AI Tool Use Policy, and that any AI tool usage has been noted in the PR description. Frequently asked questionsHow do I add reviewers? This PR will be automatically labeled, and the relevant teams will be notified. For some parts of the project, reviewers may also be added automatically. You can also add reviewers manually using the Reviewers section on this page. If you cannot use that section, it is probably because you do not have write permissions for the repository. In that case, you can request a review by tagging reviewers in a comment using What if there are no comments? If you have not received any comments on your PR after a week, you can request a review by pinging the PR with a comment such as “Ping”. The common courtesy ping rate is once a week. Please remember that you are asking for volunteer time from other developers. Are any special GitHub settings required to contribute to LLVM? We only require contributors to have a public email address associated with their GitHub commits, see this section of LLVM Developer Policy for details. If you have questions, feel free to leave a comment on this PR, or ask on LLVM Discord or LLVM Discourse. Thank you, |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
added 8 commits
August 19, 2026 16:20
| persist-credentials: false | ||
|
|
||
| - name: Setup Compiler Cache (sccache) | ||
| uses: hendrikmuhs/ccache-action@v1.2.23 |
| - name: Deploy Coverage Dashboard to GitHub Pages | ||
| if: github.repository != 'llvm/llvm-project' | ||
| continue-on-error: true | ||
| uses: peaceiris/actions-gh-pages@v4 |
| persist-credentials: false | ||
|
|
||
| - name: Setup Compiler Cache (sccache) | ||
| uses: hendrikmuhs/ccache-action@v1.2.23 |
| shell: bash | ||
| steps: | ||
| - name: Checkout Code | ||
| uses: actions/checkout@v4 |
| persist-credentials: false | ||
|
|
||
| - name: Setup Compiler Cache (sccache) | ||
| uses: hendrikmuhs/ccache-action@v1.2.23 |
| - name: Deploy Coverage Dashboard to GitHub Pages | ||
| if: github.repository != 'llvm/llvm-project' | ||
| continue-on-error: true | ||
| uses: peaceiris/actions-gh-pages@v4 |
| shell: bash | ||
| steps: | ||
| - name: Checkout Code | ||
| uses: actions/checkout@v4 |
| persist-credentials: false | ||
|
|
||
| - name: Setup Compiler Cache (sccache) | ||
| uses: hendrikmuhs/ccache-action@v1.2.23 |
|
|
||
| - name: Post or Update Sticky PR Comment | ||
| if: always() && github.event_name == 'pull_request' | ||
| uses: actions/github-script@v7 |
You can test this locally with the following command:git-clang-format --diff origin/main HEAD --extensions cpp -- libc/src/ctype/isalnum.cpp libc/src/string/strlen.cpp --diff_from_common_commit
View the diff from clang-format here.diff --git a/libc/src/ctype/isalnum.cpp b/libc/src/ctype/isalnum.cpp
index d49589804..aca4ba6e4 100644
--- a/libc/src/ctype/isalnum.cpp
+++ b/libc/src/ctype/isalnum.cpp
@@ -16,7 +16,8 @@
namespace LIBC_NAMESPACE_DECL {
LLVM_LIBC_FUNCTION(int, isalnum, (int c)) {
- const int max_uchar = static_cast<int>(cpp::numeric_limits<unsigned char>::max());
+ const int max_uchar =
+ static_cast<int>(cpp::numeric_limits<unsigned char>::max());
if (c < 0 || c > max_uchar)
return 0;
return static_cast<int>(internal::isalnum(static_cast<char>(c)));
|
You can test this locally with the following command:darker --check --diff -r origin/main...HEAD libc/utils/coverage/full_report.py libc/utils/coverage/patch_report.py libc/utils/coverage/test_coverage_tools.py
View the diff from darker here.--- full_report.py 2026-08-20 16:19:10.000000 +0000
+++ full_report.py 2026-08-28 18:26:44.298839 +0000
@@ -19,13 +19,11 @@
def render_full_report(cov_data: dict) -> None:
if "data" not in cov_data or not cov_data["data"]:
print("## LLVM-libc Full Codebase Coverage Report\n")
print("> [!WARNING]")
print("> ### No Coverage Data Detected")
- print(
- "> The test execution completed but no coverage profiles were exported."
- )
+ print("> The test execution completed but no coverage profiles were exported.")
return
subsystems: Dict[str, Dict[str, Any]] = {}
total_lines_cov = 0
@@ -102,20 +100,14 @@
subsystems[subsystem]["mcdc_cov"] += mcdc_cov
subsystems[subsystem]["mcdc_tot"] += mcdc_tot
subsystems[subsystem]["decisions_tot"] += file_decisions_tot
subsystems[subsystem]["decisions_full"] += file_decisions_full
- line_pct = (
- (total_lines_cov / total_lines_tot * 100) if total_lines_tot > 0 else 0
- )
- func_pct = (
- (total_func_cov / total_func_tot * 100) if total_func_tot > 0 else 0
- )
+ line_pct = (total_lines_cov / total_lines_tot * 100) if total_lines_tot > 0 else 0
+ func_pct = (total_func_cov / total_func_tot * 100) if total_func_tot > 0 else 0
has_mcdc = total_mcdc_tot > 0
- mcdc_pct = (
- (total_mcdc_cov / total_mcdc_tot * 100) if total_mcdc_tot > 0 else 0
- )
+ mcdc_pct = (total_mcdc_cov / total_mcdc_tot * 100) if total_mcdc_tot > 0 else 0
decisions_pct = (
(total_decisions_full / total_decisions_count * 100)
if total_decisions_count > 0
else 0
)
@@ -143,13 +135,11 @@
)
print(
f"> Tested **{total_lines_cov:,} / {total_lines_tot:,}** executable lines and **{total_mcdc_cov:,} / {total_mcdc_tot:,}** boolean conditions across **{total_decisions_count:,}** decisions."
)
print("")
- print(
- f"- **Coverage Dashboard:** [{mcdc_pages_url}]({mcdc_pages_url})"
- )
+ print(f"- **Coverage Dashboard:** [{mcdc_pages_url}]({mcdc_pages_url})")
else:
print(f"> ### Overall Codebase Coverage: **{line_pct:.2f}%**")
print(
f"> Tested **{total_lines_cov:,} / {total_lines_tot:,}** executable lines across all LLVM-libc subsystems."
)
@@ -197,13 +187,11 @@
(data["lines_cov"] / data["lines_tot"] * 100)
if data["lines_tot"] > 0
else 0
)
s_func_pct = (
- (data["func_cov"] / data["func_tot"] * 100)
- if data["func_tot"] > 0
- else 0
+ (data["func_cov"] / data["func_tot"] * 100) if data["func_tot"] > 0 else 0
)
missed_lines = data["lines_tot"] - data["lines_cov"]
if has_mcdc:
s_mc_pct = (
--- patch_report.py 2026-08-20 16:19:10.000000 +0000
+++ patch_report.py 2026-08-28 18:26:44.451127 +0000
@@ -342,13 +342,11 @@
else:
print("> [!WARNING]")
print(
f"> ### Patch Coverage: **{coverage_percent:.2f}%** ({total_missed} Missed Lines)"
)
- print(
- f"> **{total_missed}** unexecuted lines detected in patch."
- )
+ print(f"> **{total_missed}** unexecuted lines detected in patch.")
print("")
# Commit metadata and targets executed
if base_sha and head_sha and base_branch and head_branch:
print(
@@ -396,13 +394,11 @@
f_mc_pct = f_mc_cov / f_mc_tot * 100
mcdc_cell = f"**{f_mc_pct:.1f}%** ({f_mc_cov}/{f_mc_tot})"
else:
mcdc_cell = "N/A"
- dec_cell = (
- f"**{f_dec_ver} / {f_dec_tot}**" if f_dec_tot > 0 else "N/A"
- )
+ dec_cell = f"**{f_dec_ver} / {f_dec_tot}**" if f_dec_tot > 0 else "N/A"
diag_cell = "<br>".join(diag_list) if diag_list else "None"
print(
f"| {file_link} | **{f_pct:.2f}%** ({len(f_covered)}/{f_total}) | {mcdc_cell} | {dec_cell} | {len(f_missed)} | {diag_cell} |"
)
@@ -440,11 +436,13 @@
if l_type == "+":
if l_num in f_missed:
print(f"- {text} // [MISSED]")
elif l_num in unverified_lines:
unverified_conds = ", ".join(unverified_lines[l_num])
- print(f"! {text} // [PARTIAL MC/DC: {unverified_conds} unverified]")
+ print(
+ f"! {text} // [PARTIAL MC/DC: {unverified_conds} unverified]"
+ )
elif l_num in f_covered:
print(f"+ {text}")
else:
print(f" {text}")
elif l_type == " ":
--- test_coverage_tools.py 2026-08-20 16:19:10.000000 +0000
+++ test_coverage_tools.py 2026-08-28 18:26:44.544357 +0000
@@ -77,11 +77,15 @@
[15, 1, 0, True, True, False],
[20, 1, 0, False, False, False],
],
"summary": {
"lines": {"count": 20, "covered": 10, "percent": 50.0},
- "functions": {"count": 1, "covered": 1, "percent": 100.0},
+ "functions": {
+ "count": 1,
+ "covered": 1,
+ "percent": 100.0,
+ },
},
}
]
}
]
@@ -109,22 +113,31 @@
"mcdc_records": [
[12, 5, 12, 25, 1, 1, 0, 0, 5, [True, False, False]]
],
"summary": {
"lines": {"count": 10, "covered": 10, "percent": 100.0},
- "functions": {"count": 1, "covered": 1, "percent": 100.0},
- "mcdc": {"count": 3, "covered": 1, "notcovered": 2, "percent": 33.33},
+ "functions": {
+ "count": 1,
+ "covered": 1,
+ "percent": 100.0,
+ },
+ "mcdc": {
+ "count": 3,
+ "covered": 1,
+ "notcovered": 2,
+ "percent": 33.33,
+ },
},
}
]
}
]
}
diff_files = {"libc/src/string/memchr.cpp": []}
matrix = CoverageJSONParser.extract_patch_matrix(sample_mcdc_data, diff_files)
-
+
decisions = matrix["libc/src/string/memchr.cpp"]["mcdc_decisions"]
self.assertEqual(len(decisions), 1)
self.assertEqual(decisions[0]["line_start"], 12)
self.assertEqual(decisions[0]["total"], 3)
self.assertEqual(decisions[0]["covered"], 1)
@@ -271,25 +284,43 @@
"mcdc_records": [
[10, 1, 10, 20, 1, 1, 0, 0, 5, [True, True]],
],
"summary": {
"lines": {"count": 10, "covered": 10, "percent": 100.0},
- "functions": {"count": 1, "covered": 1, "percent": 100.0},
- "mcdc": {"count": 2, "covered": 2, "notcovered": 0, "percent": 100.0},
+ "functions": {
+ "count": 1,
+ "covered": 1,
+ "percent": 100.0,
+ },
+ "mcdc": {
+ "count": 2,
+ "covered": 2,
+ "notcovered": 0,
+ "percent": 100.0,
+ },
},
},
{
"filename": "/root/llvm-project/libc/src/math/sin.cpp",
"mcdc_records": [
[20, 1, 20, 30, 1, 1, 0, 0, 5, [False, False]],
],
"summary": {
"lines": {"count": 20, "covered": 15, "percent": 75.0},
- "functions": {"count": 1, "covered": 1, "percent": 100.0},
- "mcdc": {"count": 2, "covered": 0, "notcovered": 2, "percent": 0.0},
+ "functions": {
+ "count": 1,
+ "covered": 1,
+ "percent": 100.0,
+ },
+ "mcdc": {
+ "count": 2,
+ "covered": 0,
+ "notcovered": 2,
+ "percent": 0.0,
+ },
},
- }
+ },
]
}
]
}
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This patch adds automated GitHub Actions workflows and supporting Python tools to generate code coverage reports for LLVM-libc.
Pre-commit Patch Coverage Bot (.github/workflows/libc-pre-commit-coverage.yml):
When a pull request modifies C/C++ files under libc/src/, libc/include/, or libc/test/, the workflow parses the modified file list, maps each source file to its corresponding Ninja unit test target, and executes only those specific tests. The workflow then diffs the patch against the base commit, extracts statement execution data from llvm-cov export, and posts a line-level coverage report to the job summary and PR comments. Pull requests modifying only documentation, benchmarks, or Markdown files are ignored.
Post-commit Global Coverage Bot (.github/workflows/libc-post-commit-coverage.yml):
Runs on pushes to main and scheduled nightly builds. It executes the full LLVM-libc unit test suite (1,920+ tests), generates an interactive HTML report using llvm-cov show with directory grouping (--show-directory-coverage) and branch execution counts (--show-branches=count), and deploys the static dashboard to GitHub Pages. It creates .nojekyll to prevent 404 errors on internal directories starting with underscores (libc/src/__support/).
Coverage Report Generation:
libc/utils/coverage/patch_report.py compares git diff against llvm-cov JSON output to evaluate line coverage for newly added or modified lines.
libc/utils/coverage/full_report.py aggregates global line and function coverage across all LLVM-libc subsystems.