🏥 CI Failure Investigation - Run #254

Summary

Windows CI build failing due to socket permission error when HTTP test server tries to bind to localhost on dynamically allocated port.

Failure Details

• Run: 17346334863
• Commit: 26b0dde
• Trigger: pull_request
• PR: Daily Perf Improver: Optimize isNumChar function in JSON parser #1543 "Daily Perf Improver: Optimize isNumChar function in JSON parser"

Root Cause Analysis

The Windows build job failed while the Ubuntu build succeeded. The specific failure:

Failed Test: Setting timeout in customizeHttpRequest overrides timeout argument
Location: tests/FSharp.Data.Core.Tests/Http.fs:211
Error: System.Net.Sockets.SocketException (10013): An attempt was made to access a socket in a way forbidden by its access permissions

Technical Details

1. Problem: The test's startHttpLocalServer() function cannot bind to a dynamically allocated localhost port (64840 in this case)
2. Platform: Windows-specific issue - Ubuntu build passed all tests ✅
3. Root Error: Windows socket permission denied when Kestrel server tries to bind
4. Stack Trace: Failure in ASP.NET Core Kestrel binding process

Failed Jobs and Errors

Windows Build (build-windows) ❌

• Job ID: 49246818190
• Duration: ~3.5 minutes
• Test Results: 1 failed, 2267 passed, 0 skipped
• Key Error: Socket binding permission denied

Ubuntu Build (build-ubuntu) ✅

• Job ID: 49246818175
• Duration: ~5 minutes
• Test Results: All passed

Investigation Findings

HTTP Test Server Analysis

The failing test uses a local HTTP server implementation in Http.fs:

let startHttpLocalServer() =
    // ... 
    let freePort =
        let random = new System.Random()
        let mutable port = random.Next(10000, 65000) // Random high port
        // Check for port availability...
    
    let baseAddress = $"http://localhost:{freePort}"
    let workerTask = app.RunAsync(baseAddress) // FAILS HERE on Windows

The Issue:

• Uses dynamic port allocation (10000-65000 range)
• Port availability check passes, but actual binding fails
• Windows GitHub Actions runner has restricted socket permissions
• Error occurs during Kestrel server startup

Historical Context

• No previous similar issues found in repository history
• First occurrence of this specific socket permission error
• Windows CI builds generally stable historically

Recommended Actions

Immediate Solutions

• Skip the failing test on Windows CI using conditional compilation:

  [<Test>]
  [<Conditional("NOT_WINDOWS_CI")>]
  let ``Setting timeout in customizeHttpRequest overrides timeout argument`` () =

• Mock the HTTP server instead of using real socket binding for this test

• Use a different port range or implement retry logic with multiple port attempts

Better Long-term Solutions

• Refactor test architecture to use in-memory HTTP testing without real sockets
• Create platform-specific test configurations in CI workflow
• Implement HTTP client timeout tests without requiring server binding

Code Changes Required

File: tests/FSharp.Data.Core.Tests/Http.fs:211
Function: startHttpLocalServer()
Options:

1. Add Windows CI detection and test skipping
2. Implement socket binding retry logic
3. Replace with mock HTTP server

Prevention Strategies

• Add Windows-specific CI test exclusions for socket-dependent tests
• Use containerized test environments to avoid host permission issues
• Implement comprehensive mocking for external dependency tests
• Add platform detection in test setup and configuration

AI Team Self-Improvement

Additional prompting instructions for AI coding agents:

When writing HTTP server tests:
1. Always check for platform compatibility (Windows vs Linux socket permissions)
2. Prefer mocking over real socket binding in unit tests  
3. Implement retry logic for dynamic port allocation
4. Use conditional compilation for platform-specific test exclusions
5. Avoid hardcoded ports - always use dynamic allocation with availability checks
6. Consider using TestContainers or in-memory servers for integration tests

Historical Context

This is the first occurrence of this specific Windows socket permission error in the repository. The failure appears to be related to:

• GitHub Actions Windows runner environment changes
• Stricter socket permission policies on Windows runners
• Random port collision with system-reserved or restricted ports

The issue is not related to the PR changes (JSON parser optimization) but rather to the CI environment.

AI-generated content by CI Failure Doctor may contain mistakes.