Skip to content

Clean up GC tests #9751

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
arielshaqed wants to merge 3 commits into
base: master
Choose a base branch
from
+54 −12
Open

Clean up GC tests #9751

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d4778a5
Close GCStartingPointIterator in tests
arielshaqed Dec 2, 2025
37cb112
Scramble IDs in TestActiveCommits
arielshaqed Dec 2, 2025
6059b51
[CR] Copilot fixes
arielshaqed Dec 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 53 additions & 11 deletions pkg/graveler/retention/active_commits_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import (
"context"
"errors"
"fmt"
"hash/fnv"
"sort"
"testing"
"time"
Expand Down Expand Up @@ -165,12 +166,44 @@ func TestCommitsMap(t *testing.T) {
}
}

type testRepoCommits struct {
commits map[string]testCommit
headsRetentionDays map[string]int32
expectedActiveIDs []string
}

Copy link

Copilot AI Dec 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The fingerprint function lacks documentation explaining its purpose and behavior. Consider adding a comment describing that it creates a deterministic hash-prefixed string to scramble commit IDs for testing purposes.

Suggested addition:

// fingerprint creates a hash-prefixed version of the input string to scramble
// commit IDs in tests, ensuring they are not alphabetically ordered in a way
// that could accidentally match their time ordering.
func fingerprint[T ~string](s T) T {
Suggested change
// fingerprint creates a hash-prefixed version of the input string to scramble
// commit IDs in tests, ensuring they are not alphabetically ordered in a way
// that could accidentally match their time ordering.

Copilot uses AI. Check for mistakes.
func fingerprint[T ~string](s T) T {
h := fnv.New64a()
h.Write([]byte(s)) // Write cannot fail for a hash func.
fingerprint := h.Sum64()
return T(fmt.Sprintf("%016x-%s", fingerprint, s))
Comment on lines +178 to +179
Copy link

Copilot AI Dec 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The local variable fingerprint shadows the function name, which can be confusing. Consider renaming the variable to something like hash or fpHash.

Suggested fix:

func fingerprint[T ~string](s T) T {
    h := fnv.New64a()
    h.Write([]byte(s)) // Write cannot fail for a hash func.
    hash := h.Sum64()
    return T(fmt.Sprintf("%016x-%s", hash, s))
}
Suggested change
fingerprint := h.Sum64()
return T(fmt.Sprintf("%016x-%s", fingerprint, s))
hash := h.Sum64()
return T(fmt.Sprintf("%016x-%s", hash, s))

Copilot uses AI. Check for mistakes.
}
Comment on lines +175 to +180
Copy link
Contributor

@nopcoder nopcoder Dec 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit; can keep it simple while accepting strings


Copy link

Copilot AI Dec 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The scrambleIDs function lacks documentation explaining its purpose. Consider adding a comment describing that it applies fingerprinting to all IDs in the test data structure.

Suggested addition:

// scrambleIDs applies fingerprinting to all commit IDs, head IDs, and expected
// active IDs in the test data to ensure they are not alphabetically ordered.
func scrambleIDs(tst testRepoCommits) testRepoCommits {
Suggested change
// scrambleIDs applies fingerprinting to all commit IDs, head IDs, and expected
// active IDs in the test data to ensure they are not alphabetically ordered.

Copilot uses AI. Check for mistakes.
func scrambleIDs(tst testRepoCommits) testRepoCommits {
commits := make(map[string]testCommit, len(tst.commits))
for id, commit := range tst.commits {
for idx, parent := range commit.parents {
commit.parents[idx] = fingerprint(parent)
}
Comment on lines +185 to +187
Copy link

Copilot AI Dec 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The commit.parents slice is being modified, but since commit is a value copy from the range loop, these modifications won't be reflected in the commit value stored in the commits map on line 188. The parents should be modified before storing the commit.

Suggested fix:

for id, commit := range tst.commits {
    parents := make([]graveler.CommitID, len(commit.parents))
    for idx, parent := range commit.parents {
        parents[idx] = fingerprint(parent)
    }
    commit.parents = parents
    commits[fingerprint(id)] = commit
}
Suggested change
for idx, parent := range commit.parents {
commit.parents[idx] = fingerprint(parent)
}
parents := make([]graveler.CommitID, len(commit.parents))
for idx, parent := range commit.parents {
parents[idx] = fingerprint(parent)
}
commit.parents = parents

Copilot uses AI. Check for mistakes.
commits[fingerprint(id)] = commit
}
headsRetentionDays := make(map[string]int32, len(tst.headsRetentionDays))
for id, days := range tst.headsRetentionDays {
headsRetentionDays[fingerprint(id)] = days
}
expectedActiveIDs := make([]string, len(tst.expectedActiveIDs))
for idx, id := range tst.expectedActiveIDs {
expectedActiveIDs[idx] = fingerprint(id)
}
return testRepoCommits{
commits: commits,
headsRetentionDays: headsRetentionDays,
expectedActiveIDs: expectedActiveIDs,
}
}

func TestActiveCommits(t *testing.T) {
tests := map[string]struct {
commits map[string]testCommit
headsRetentionDays map[string]int32
expectedActiveIDs []string
}{
tests := map[string]testRepoCommits{
"two_branches": {
commits: map[string]testCommit{
"a": newTestCommit(15),
Expand Down Expand Up @@ -330,7 +363,7 @@ func TestActiveCommits(t *testing.T) {
expectedActiveIDs: []string{"h1", "h2", "h3", "e1", "e2", "e3"},
},
}
for name, tst := range tests {
for name, tstBase := range tests {
t.Run(name, func(t *testing.T) {
now := time.Now()
ctrl := gomock.NewController(t)
Expand All @@ -339,6 +372,11 @@ func TestActiveCommits(t *testing.T) {
repositoryRecord := &graveler.RepositoryRecord{
RepositoryID: "test",
}

// Shuffle startingPoints (below) by replacing all CommitIDs with their
// fingerprints. Otherwise they tend to be alphabetically ordered,
// which can match their time ordering better than will actually happen.
tst := scrambleIDs(tstBase)
garbageCollectionRules := &graveler.GarbageCollectionRules{DefaultRetentionDays: 5, BranchRetentionDays: make(map[string]int32)}
var branches []*graveler.BranchRecord
for head, retentionDays := range tst.headsRetentionDays {
Expand Down Expand Up @@ -369,12 +407,16 @@ func TestActiveCommits(t *testing.T) {

refManagerMock.EXPECT().ListCommits(ctx, repositoryRecord).Return(testutil.NewFakeCommitIterator(commitsRecords), nil).MaxTimes(1)

gcCommits, err := GetGarbageCollectionCommits(ctx, NewGCStartingPointIterator(
startingPoints := NewGCStartingPointIterator(
testutil.NewFakeCommitIterator(findMainAncestryLeaves(now, tst.headsRetentionDays, tst.commits)),
testutil.NewFakeBranchIterator(branches)), &repositoryCommitGetter{
refManager: refManagerMock,
repository: repositoryRecord,
}, garbageCollectionRules)
testutil.NewFakeBranchIterator(branches))
defer startingPoints.Close()

gcCommits, err := GetGarbageCollectionCommits(ctx, startingPoints,
&repositoryCommitGetter{
refManager: refManagerMock,
repository: repositoryRecord,
}, garbageCollectionRules)
if err != nil {
t.Fatalf("failed to find expired commits: %v", err)
}
Expand Down
2 changes: 1 addition & 1 deletion pkg/graveler/retention/starting_point_iterator_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ func TestStartingPointIterator(t *testing.T) {
branchIterator := testutil.NewFakeBranchIterator(branchRecords)
commitIterator := testutil.NewFakeCommitIterator(commitRecords)
it := NewGCStartingPointIterator(commitIterator, branchIterator)
defer it.Close()
i := 0
for it.Next() {
val := it.Value()
Expand All @@ -89,7 +90,6 @@ func TestStartingPointIterator(t *testing.T) {
if it.Err() != nil {
t.Fatalf("unexpected error: %v", it.Err())
}
it.Close()
if i != len(expected) {
t.Fatalf("got unexpected number of results. expected=%d, got=%d", len(expected), i)
}
Expand Down
Loading