[BUG]: ldd: ./bin/libcoreclr.so: No such file or directory

[angaaruriakhil]

What happened?

Environment Details

• Agent Version: 4.261.0
• Ubuntu Version: 22.04
• Architecture: x64
• Deployment: Self-hosted VMSS on Azure DevOps
• Image Source: actions/runner-images

Problem Description

We are experiencing intermittent provisioning failures with Azure DevOps agents in our VMSS pool. The agent setup fails during the configuration phase in config.sh with missing .NET runtime files.

We are using self-hosted VMSS on Azure DevOps running Ubuntu 22.04. The images are built using runner-images.

We intermittently get issues reported in the Diagnostics tab where the agent can't provision as the set-up in config.sh fails.

The key point from the logs provided below is:

ldd: ./bin/libcoreclr.so: No such file or directory

Root Cause Analysis

At first, I thought that this was an issue with the way dotnet is being installed on the OS, then I read that the Azure agent is supposed to be self-contained and include its own .NET 6 runtime as per the official documentation:

"The agent software runs on .NET 6, but installs its own version of .NET so there is no .NET prerequisite."

Possible Cause

I believe that, intermittently, somewhere upstream in the agent installation process (possibly enableagent.sh or config.sh) the relevant files are not being extracted properly for some reason. I believe it then relies on a fallback to look for system dotnet in /usr/share/dotnet. Here, it won't be able to find libcoreclr.so, as multiple versions of dotnet are installed to that directory by the runner-images team, so the file won't exist in the root of that directory. So the local agent install of dotnet needs to work reliably.

Impacts

This causes provisioning issues which result in queues as agents can't provision in the pool.

Versions

Agent v4.261.0

Environment type (Please select at least one enviroment where you face this issue)

• Self-Hosted
• Microsoft Hosted
• VMSS Pool
• Container

Azure DevOps Server type

dev.azure.com (formerly visualstudio.com)

Azure DevOps Server Version (if applicable)

Azure DevOps SaaS

Operation system

Ubuntu 22.04

Version controll system

No response

Relevant log output

Azure Pipelines Extension failed with message: The Extension failed to execute: Pipeline script execution failed with exit code 100.

2025-10-28 08:59:01 version 16
2025-10-28 08:59:01 Url is {obscured}
2025-10-28 08:59:01 Pool is {obscured}
2025-10-28 08:59:01 RunArgs is --once
2025-10-28 08:59:01 ForceUser is false
2025-10-28 08:59:01 Directory is /agent
2025-10-28 08:59:01 AzDevOps account already exists
2025-10-28 08:59:01 Zipfile is /agent/vsts-agent-linux-x64-4.261.0.tar.gz

2025-10-28 08:59:01 Installing dependencies
++ id -u
+ user_id=0
+ '[' 0 -ne 0 ']'
+ '[' -e /etc/os-release ']'
+ filepath=/etc/os-release
+ '[' -e /etc/os-release ']'
+ echo '--------OS Information--------'
+ cat /etc/os-release
+ echo ------------------------------
+ '[' -e /etc/debian_version ']'
+ echo 'The current OS is Debian based'
+ echo '--------Debian Version--------'
+ cat /etc/debian_version
+ echo ------------------------------
+ command -v apt
+ '[' 0 -eq 0 ']'

+ apt update

+ apt install -y libkrb5-3 zlib1g debsums
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

+ apt install -y liblttng-ust1
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

+ '[' 0 -ne 0 ']'
++ wget -qO- http://security.ubuntu.com/ubuntu/pool/main/o/openssl/
++ grep -oP '(libssl1.1_1.1.1f.*?_amd64.deb)'
++ head -1
+ package=libssl1.1_1.1.1f-1ubuntu2.24_amd64.deb

+ apt install -y libssl3
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

+ '[' 0 -ne 0 ']'
+ apt install -y libicu74
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
E: Package 'libicu74' has no installation candidate

+ apt install -y libicu70
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

+ '[' 0 -ne 0 ']'
+ echo -----------------------------
+ echo ' Finish Install Dependencies'
+ echo -----------------------------

2025-10-28 08:59:08 Dependencies installation succeeded
2025-10-28 08:59:08 Configuring build agent
2025-10-28 08:59:08 Configuring agent
2025-10-28 08:59:08 ldd: ./bin/libcoreclr.so: No such file or directory
ldd: ./bin/libSystem.IO.Compression.Native.so: No such file or directory

A fatal error was encountered. The library 'libhostpolicy.so' required to execute the application was not found in '/usr/share/dotnet'.
Failed to run as a self-contained app.
  - The application was run as a self-contained app because '/agent/bin/Agent.Listener.runtimeconfig.json' was not found.
  - If this should be a framework-dependent app, add the '/agent/bin/Agent.Listener.runtimeconfig.json' file and specify the appropriate framework.

2025-10-28 08:59:08 Build agent configuration failed

[raujaiswal]

@angaaruriakhil , could you please try adding the following under variables in yml pipeline and can confirm whether it works?

LD_LIBRARY_PATH: '/usr/share/dotnet/packs/Microsoft.NETCore.App.Host.linux-x64/9.0.9/runtimes/linux-x64/native:/usr/share/dotnet/packs/Microsoft.NETCore.App.Host.linux-x64/8.0.20/runtimes/linux-x64/native'

Example:

[angaaruriakhil]

@angaaruriakhil , could you please try adding the following under variables in yml pipeline and can confirm whether it works?

LD_LIBRARY_PATH: '/usr/share/dotnet/packs/Microsoft.NETCore.App.Host.linux-x64/9.0.9/runtimes/linux-x64/native:/usr/share/dotnet/packs/Microsoft.NETCore.App.Host.linux-x64/8.0.20/runtimes/linux-x64/native'

Example:

Hi, thanks for your reply. This issue is manifesting intermittently before we even run a pipeline, in the Diagnostics tab in the VMSS pool in Azure DevOps. It is during agent provisioning for the pool (VM provisioning and installation of Azure agent) which is automatically controlled and executed by Azure DevOps via an Azure extension.

If an agent has successfully provisioned in the VMSS pool and is available to run a pipeline, then its already past this issue and is working fine as its been able to find those relevant dotnet binaries in that case... so I am not sure what we would achieve by running a pipeline with this variable defined. Happy to be corrected here.

[raujaiswal]

@angaaruriakhil , the solution I suggested was based on the assumption that the pipeline was failing because the dynamic linker on Linux couldn’t automatically locate the required .NET native library. Adding the path to LD_LIBRARY_PATH would help the linker find it. However, since you mentioned the issue is related to provisioning the VMSS pool, could you confirm if the problem still exists or if it’s resolved now?

[angaaruriakhil]

@angaaruriakhil , the solution I suggested was based on the assumption that the pipeline was failing because the dynamic linker on Linux couldn’t automatically locate the required .NET native library. Adding the path to LD_LIBRARY_PATH would help the linker find it. However, since you mentioned the issue is related to provisioning the VMSS pool, could you confirm if the problem still exists or if it’s resolved now?

Hi, we have a lot of scaling events so the Diagnostics tab only shows the latest ones...as the issue is intermittent and the agent installation process is controlled by Azure DevOps, I can't reproduce the issue reliably.

Would this fix be more appropriate to put into the agent installation code? Or maybe a retry of the local .NET install if those dependencies aren't found, e.g. if it fails to extract properly/networking hiccup, before configuring the agent further.