-
Notifications
You must be signed in to change notification settings - Fork 1.2k
Description
Describe the bug
We are experiencing intermittent Docker socket communication failures across multiple self-hosted GitHub Actions runner scale sets running in Kubernetes with Docker-in-Docker (dind).
Issue started immediately after upgrading from runner v2.328.0 → v2.329.0
The Problem:
- Upgraded runner from v2.328.0 to v2.329.0 around late October 2025
- Errors began appearing on October 31, 2025 (shortly after upgrade)
- No issues with v2.328.0 - runners were stable before the upgrade
- Random workflow failures with "broken pipe" errors when the runner tries to attach to container exec stdout streams
- 5-10% of jobs fail randomly during peak hours
Error Pattern:
The dind container logs show socket write failures when trying to communicate with /var/run/docker.sock. All scale sets experience errors simultaneously during spike events, suggesting this is more than just resource exhaustion.
Runner Version and Platform
Runner:
Version: v2.329.0
Deployment Method: actions-runner-controller (gha-runner-scale-set)
Controller Version: 0.12.1
Chart Version: 0.12.1
Kubernetes:
Platform: Amazon EKS
EKS Version: 1.33
EKS AMI: 1.33.5-20251023 (October 2025 release)
Docker (in dind container):
Docker Engine: 29.0.1 (built November 14, 2025)
API version: 1.52
containerd: v2.1.5
runc: 1.3.3
Image: docker:dind (latest tag from Docker Hub)
Runner Logs
Error Messages - Primary Error (from dind container stderr): Error running exec <container_id> in container: exec attach failed:
error attaching stdout stream: write unix /var/run/docker.sock->@: write: broken pipe
Secondary Error: http2: server: error reading preface from client @: read unix /var/run/docker.sock->@: read: connection reset by peer
Sample Logs from dind Container
2025-11-14T07:01:11.063364452Z level=error msg="Error running exec 2bdbf255470b5a38 in container: exec attach failed: error attaching stdout stream: write unix /var/run/docker.sock->@: write: broken pipe"
2025-11-14T07:04:31.631887068Z level=error msg="Error running exec 06cffe9160ae9 in container: exec attach failed: error attaching stdout stream: write unix /var/run/docker.sock->@: write: broken pipe"
2025-11-14T07:04:34.408Z level=warning msg="ShouldRestart failed, container will not be restarted" daemonShuttingDown=false error="" execDuration=3m10.311722009s exitStatus="{1 2025-11-14 07:04:34.390005475 +0000 UTC}" hasBeenManuallyStopped=true restartCount=0
2025-11-14T07:04:54.908Z http2: server: error reading preface from client @: read unix /var/run/docker.sock->@: read: connection reset by peer
Runner and Worker's Diagnostic Logs
No errors appear in the runner container logs. All errors are only visible in the dind (Docker-in-Docker) container stderr logs. The runner container shows normal operation.
Questions
- What changed in v2.329.0 related to Docker socket handling?
- Is there a known regression in v2.329.0 with dind setups?
- Are there recommended socket configurations for v2.329.0 with dind?
- Could Docker v29.0.1 (Nov 14, 2025) + runner v2.329.0 combination be problematic?
We would appreciate guidance on!
Thank you