Skip to content

IMDS credential scanner/refresher updates - #5145

Open
singholt wants to merge 7 commits into
aws:devfrom
singholt:imds-credential-delivery-status
Open

singholt wants to merge 7 commits into
aws:devfrom
singholt:imds-credential-delivery-status

Conversation

@singholt

@singholt singholt commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Summary

This PR updates how the ECS agent reads rotated task credentials from IMDS to match the new info-file contract.

The info file now maps each taskID-roleType to a delivery status ("0" delivered, "1" the provider could not assume the role). The role ARN will no longer be delivered in IMDS. Its not needed as the agent gets that from ECS control-plane.

The change is split into small commits to make review easier.

Functional commits

  • imds: parse credential delivery status from info file: read the per-credential status, fetch a credential file only for delivered ("0") entries, and drop the role ARN from the info-file model and the scanned credential.
  • imds: track roles the provider could not assume: surface "1" roles from the scanner in a ScanResult, and track them in the credentials manager (cleared when credentials arrive or the id is removed).
  • imds: drop resolved load-testing TODOs: remove the TODOs now that the scan rate and interval are validated. Values are unchanged.

Test commits

  • imds: update tests for delivery-status info file: update the scanner and refresher unit tests, the mock IMDS server, and the refresher integration test for the new contract.
  • imds: tests for unassumable-role tracking: cover the "1" path in the scanner, the credentials manager, and end-to-end in the integration test.
  • imds: run credential scanner and refresher tests in parallel: mark the tests and subtests with t.Parallel().

Vendor commit

  • vendor: regenerate agent vendor for IMDS credential changes: isolated vendor sync so the functional and test commits stay vendor-free.

Implementation details

  • The scanner switches on the info-file status: it fetches and returns delivered credentials, records roles marked "1" (no credential file exists for them), and skips unrecognized statuses.
  • The role ARN is no longer read from the info file. The refresher stores the task's own role ARN (from task state) with the credential, so the previous scanned-vs-task ARN cross-check is removed.
  • The Docker agent only logs the unassumable roles; it does not track them in its credentials manager, since no metric is wired for emission there.

Testing

New tests cover the changes: yes, added/updated unit and integ tests

Description for the changelog

Enhancement - IMDS credentials refresher: read task credential delivery status from IMDS and skip roles the provider could not assume. (disabled in production)

Additional Information

Does this PR include breaking model changes? If so, Have you added transformation functions? No

Does this PR include the addition of new environment variables in the README? No

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

The info file reports a per-credential delivery status keyed by
"<taskID>-<roleType>". A value of "0" means the provider assumed the
role and wrote a credential file; any other value means no credential
file was written for that entry. Parse this status and fetch a
credential file only for entries marked delivered.

Drop the role ARN from the info-file model and from TaskCredential,
since it is no longer published in the info file. The refresher stores
the task's own role ARN with the credential.
Update the scanner unit tests, refresher tests, and the mock IMDS
server to the delivery-status info file: entries map to a status
string ("0" for delivered) instead of a role ARN, and the credential
model no longer carries a role ARN. Add a case covering an entry not
marked delivered being skipped, and drop the role-ARN cross-check
cases that no longer apply.
The info file marks a credential with status "1" when the provider
could not assume the role, so no credential file was written. Surface
these roles from the scanner in a ScanResult alongside the retrieved
credentials, so a consumer can distinguish a stale credential the
customer must remediate from a broken delivery path.

Add credentials-manager tracking keyed by credentials id
(SetAssumeRoleFailedCredentials / IsCredentialsAssumeRoleFailed). A
role marked failed leaves any credentials already held in place, and
is cleared when credentials arrive for the id or the id is removed.

The Docker-based agent only logs this for debugging; it does not track
the roles in its credentials manager, since no metric is wired for
emission.
Update the scanner tests to the ScanResult return type and cover a
status "1" entry being recorded as an unassumable role rather than
fetched, plus an unrecognized status being skipped. Add credentials
manager tests for the failed-role tracking and its clearing on
credential delivery or removal, and adapt the refresher test to the
ScanResult scan output.
The IMDS query rate and scan interval were validated by load testing;
remove the TODOs noting the values were pending it. The values are
unchanged.
Mark the IMDS scanner, credentials manager, and refresher unit tests
(and their subtests) with t.Parallel(). Each test and subtest builds
its own mocks and manager and shares no mutable state, so they are
safe to run concurrently. Verified with -race.
@singholt
singholt force-pushed the imds-credential-delivery-status branch from d04f465 to aea766c Compare September 18, 2026 21:32
@singholt
singholt marked this pull request as ready for review September 18, 2026 23:14
@singholt
singholt requested a review from a team as a code owner September 18, 2026 23:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants