Pushing Signed images in Azure Pipeline - Throws couldn't add target to targets/

[Monishguru]

When we try to push signed images as per the documentation from the Azure pipeline, to Azure Container Registry we are facing the following issue :

##[error]time="2021-06-29T10:07:28Z" level=error msg="couldn't add target to targets/zzzzz: could not find necessary signing keys, at least one of these keys must be available: xxxxxxxxxxxxxxxxxxxxxxxxxxxx"
##[error]failed to sign repo/test/pipeline-templates_tests_docker_build_v3:20210629100550.62491.521a451: could not find necessary signing keys, at least one of these keys must be available: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
##[error]The process '/usr/bin/docker' failed with exit code 1

Followed the following steps:

• The private keys are generated using docker trust key generate and the private key from ~./.docker/trust/private are kept in the Secure Files and copied as mentioned
  script: |
  mkdir -p $(DOCKER_CONFIG)/trust/private
  cp $(privateKey.secureFilePath) $(DOCKER_CONFIG)/trust/private
• Tried initializing the repo with docker trust signer add --key xyz.pub xyz registry.example.com/admin/demo
• We ran docker trust inspect on the image repo and we are able to see the signer with root and repo key.
• We couldn't execute the notary delegation list due to internal complications.
• Note: ACRImage signer role is added for the service connection.

With this set up we are getting the above error.

Can you please help with this issue?

Reference:
Azure DevOps Documentation
Docker Content Trust Documentation

[shizhMSFT]

To troubleshoot, I assume you followed the official documentation from the Azure pipeline.

What command did you run to produce the above error messages? docker trust signer add, docker trust sign, or docker push?
From the error message, it seems that those error messages are produced by docker trust signer add where the repository key is missing in ~/.docker/trust/private folder. Please double check if you have copied the right key.

[Monishguru]

@shizhMSFT

Yes, I followed the documentation... The error message was produced by docker push task.

Tried the following:
Step 1:
Enabled Content Trust in Azure portal navigating to ACR -Content trust
Step 2:
Generated a Delegate key pair in my local machine using docker trust key generate testuser

Got two files testuser.pub and private key in. /.docker/trust/private of the local machine

Step 3: Created a pipeline with a bash script to initiate the repo as follows

docker trust signer add --key testuser.pub testuser reponame.azurecr.io/xxx/yyy

Note: public key has been placed in a secure file and fed to the above command.

Passphrase has been set in variables export DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE and similarly root PASSPHRASE

Then executed docker trust inspect --pretty reponame.azurecr.io/xxx/yyy

Got the administrative keys and signer details. It showed all the data.

Step 4
Assuming the repository initialized as above. Proceeded with the same document as you provided in the link

Copied the private delegate key which was generated in my local machine as mentioned below from the secure file
mkdir -p $(DOCKER_CONFIG)/trust/private
cp $(privateKey.secureFilePath) $(DOCKER_CONFIG)/trust/private

When the docker login task is executed, the $DOCKER_CONFIG variable is set in the VM. And the private keys are placed at /data/opt/vstsagent/_work/_temp/DockerConfig_27820625(some.random number) /trust/private.

I tried to place even the root key and repo key in addition to the private delegate key. Still am facing the same issue when the docker push task is executed

Please help

[Monishguru]

I guess i figured out the issue :

task: Docker@2
inputs:
command: push
containerRegistry: $(containerRegistryServiceConnection)
repository: $(imageRepository)
tags: |
$(tag)
arguments: '--disable-content-trust=false'
env:
DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE: $(DOCKER_CONTENT_TRUST_REPOSITORY_PASSPHRASE)
The issue is with the passphrase which is mentioned in the documentation. The documentation states the passphrase to be used after initialization is Repository Passphrase, whereas when we initialize the image repo with a private delegate key, it looks for the private key's passphrase, but not the repository passphrase. Please correct this in documentation, as this is misleading.

[shizhMSFT]

@Monishguru I've forwarded your feedback at https://github.com/MicrosoftDocs/azure-devops-docs/issues/11057. Please track the documenetation issue there.

[shizhMSFT]

Besides, in step 3, the root key and the repository key will be generated when the repo is initialized. It is suggested to keep those keys locally as you may need them later.

Note: The loss of the root key is not recovable.

[prahlad141992]

Do we need to initialize every individual created repository in ACR using docker trust signer add command?
ex: we have 2 repositories in same ACR contoso.azurecr.io/xyz & contoso.azurecr.io/abc