feat(test): [Test] Add mutation testing suite for Soroban Smart Contracts using Cargo Mutants - #1053
Merged
Ceejaytech25 merged 2 commits intoSep 2, 2026
Conversation
…racts Add automated mutation testing with cargo-mutants for the core Soroban contracts (sep41-token, staking, liquid_staking) and enforce it in CI. The dormant mutants.toml used an outdated cargo-mutants schema and was never run; rewrite it for the current schema and add a contracts-workspace config, a CI job that fails on any missed mutant, and CONTRIBUTING guidelines. Add edge-case tests (balance history, zero amounts, negative string inputs, lock/withdraw boundaries) so no mutants survive: 101 root workspace mutants and 95 liquid_staking mutants all caught. Also fix pre-existing merge-artifact breakage blocking CI: duplicate imports in src/utils/events.rs, duplicated tests and bool asserts in src/security_registry/lib.rs, an uncompilable create_vault in src/liquidation, and pin ed25519-dalek 2.x in the contracts workspace. Closes ceejaylaboratory#1021 Generated with Codebuff 🤖 Co-Authored-By: Codebuff <noreply@codebuff.com>
Contributor
|
Nice implementation, LGTM! |
…add-mutation-testing-suite Co-authored-by: Cursor <cursoragent@cursor.com> # Conflicts: # contracts/liquid_staking/src/lib.rs # src/liquidation/src/lib.rs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #1021
This PR turns the existing (dormant)
mutants.tomlinto a real, CI-enforced mutation testing pipeline usingcargo-mutantsfor the core Soroban contracts:sep41-token(src/token),staking(src/staking), andliquid_staking(contracts/liquid_staking).The repo already shipped a
mutants.toml, but it was written for an old cargo-mutants config schema and was never auto-discovered (cargo-mutants only reads.cargo/mutants.toml), and no CI ran it. This PR makes mutation testing executable locally and enforced in CI, and hardens the test suites so that no mutants survive.Mutation testing results
sep41-token+staking)liquid_staking)Before this PR, the same suites left 18 missed mutants in the root workspace and 14 missed in
liquid_staking.What changed
1. Configuration (
mutants.toml,contracts/mutants.toml)mutants.tomlfor the current cargo-mutants schema:examine_globs(file-path globs) instead of the removed string-formexamine_re,minimum_test_timeoutinstead of the removedtimeout, andcap_lints = trueso deny-level lints don't make mutants unviable.contracts/mutants.tomlfor the separate contracts workspace, scoped toliquid_staking/**.exclude_relist (e.g.reward > 0→>= 0guards where a zero-value Stellar Asset Contract transfer is a no-op, and an < 0vsn <= 0sign guard that only differs atn == 0, which early-returns). Each exclusion is commented with its justification so it stays honest.2. Edge-case tests added to kill surviving mutants
src/token/lib.rs(was 6 missed → 0)test_get_past_balance_no_transfer_history— fallback-to-current-balance and zero-balance paths forget_past_balancetest_get_past_balance_snapshots— checkpointing across two transfers at controlled ledger sequences (kills_write_checkpointremoval,!=→==,>=→<, and the return-value substitutions)test_operator_approval— added negative assertions (!is_approved_for_all) before approval is grantedsrc/staking/lib.rs(was 12 missed → 0)test_pending_rewards_zero_without_stake/test_pending_rewards_after_time— cover the previously untestedpending_rewardsquery (kills return-value and arithmetic mutants)test_stake_accumulates_rewards— second stake after a time advance banks accrued rewards intoaccumulated_rewards(kills+=→-=/*=)test_stake_same_lock_different_tier_keeps_multiplier— staking a second tier with an identical lock duration must not extend the lock or swap the multiplier (kills>→>=on the lock-extension guard)test_withdraw_at_exact_lock_end_no_penalty— withdrawing exactly atlock_endapplies no penalty (kills<→<=)contracts/liquid_staking/src/lib.rs(was 14 missed → 0)test_i128_to_string_helper/test_u64_to_string_helper— direct unit tests for the string-conversion helpers including negative numbers and zero (kills the negative-path and digit-loop mutants)test_deposit_rewards_with_no_stake— depositing rewards with zero total staked must not touch the rpt accumulator (killstotal_staked > 0→>= 0, which would divide by zero)test_claim_zero_rewards_emits_no_claim_event— a zero-reward claim must not emit aclaimedevent (killsreward > 0→>= 0inclaim)test_unstake_with_zero_rewards,test_emergency_withdraw_with_zero_fee— zero-amount edge-case behavior tests3. CI (
.github/workflows/rust.yml)Added a dedicated
mutation-testingjob that runs both suites and fails the build if any mutant is missed:cargo mutants --config mutants.toml -p sep41-token -p staking -j 2cd contracts && cargo mutants --config mutants.toml -p liquid_staking --in-place(--in-placeis required because the contracts workspace has a path dependency pointing outside itself)4. Documentation (
CONTRIBUTING.md)Added a Mutation Testing section under Testing: what it is, how to install cargo-mutants, exact commands for both workspaces, how to interpret
missed/unviable, and guidelines for killing mutants with edge-case tests vs. documenting equivalent ones.Pre-existing CI breakage fixed along the way
Running the full CI suite locally surfaced unrelated breakage on
main(all introduced by the "Resolve merge conflicts with main" commit) that was blocking every pipeline. Fixed so CI is green on this PR:src/utils/events.rs— removed a duplicateduse soroban_sdk::{…}line that failed clippy with-D warningssrc/security_registry/lib.rs— removed two duplicated test functions and replacedassert_eq!(…, true/false)withassert!/assert!(!…)(clippybool_assert_comparisonunder-D warnings)src/liquidation/src/lib.rs— fixedcreate_vault, which did not compile (cannot assign twice to immutable variable): removed the duplicate vault store and the double increment ofNextVaultId, restoring the original pre-merge semantics (store atid, bump counter toid + 1, returnid)contracts/Cargo.toml— pinneded25519-dalek = "2.2"in the contracts workspace. The workspace'sCargo.lockis gitignored, so fresh resolution pullsed25519-dalek 3.x, which is API-incompatible withsoroban-env-host 22.xtestutils (ChaCha20Rngno longer satisfies theCryptoRngbound) and breakscargo test -p liquid_staking. Also added the missingtestutils::Eventstrait import that the liquid_staking test module needed to compile.Evidence of local test runs
cargo mutants --config mutants.toml -p sep41-token -p staking -j 2→ 101 mutants: 92 caught, 9 unviable, 0 missed (exit 0)cd contracts && cargo mutants --config mutants.toml -p liquid_staking --in-place→ 95 mutants: 89 caught, 6 unviable, 0 missed (exit 0)cargo test -p sep41-token→ 33 passed;cargo test -p staking→ 15 passed;cd contracts && cargo test -p liquid_staking→ 23 passedcargo clippy -p anchorpoint-utils -p anchorpoint-security-registry -p event-hub --all-targets --all-features -- -D warnings→ cleancargo build --target wasm32v1-none --release→ cleancargo test -p upgradeableandcargo test -p anchorpoint-utils -p anchorpoint-security-registry -p event-hub --all-features→ all passed./scripts/security-audit.sh --warn-only→ completed successfullyNote
package-lock.jsonhas unrelated pre-existing working-tree modifications from before this PR and is intentionally not included.cargo test -p liquidationhas 4 pre-existing failures inliquidate/partial_liquidatecaused by the contract invoking the oracle'sget_pricewith no arguments while the mock oracle expects anassetargument. This is unrelated to the compile fix included here (which only restorescreate_vault's intended behavior) and is not part of the CI test steps; it is left for a separate fix.