Git Credential Cleanup for ECS Containers with Persistent Storage
When running automated workflows in ECS containers with persistent EFS storage, you might encounter cryptic Git authentication failures. This pattern, discovered through debugging GitHub App authentication, shows you how to prevent these issues.
The Problem
You're using Git URL rewriting for authentication in your ECS containers. Everything works on the first deploy, but after a few container restarts, you start seeing this error:
fatal: invalid credential line: ghs_xxxx...The frustrating part? Your token is valid. The URL rewriting is configured correctly. Yet Git refuses to authenticate.
Root Cause
Git checks multiple credential sources in a specific order. When you have persistent storage like EFS, old .git-credentials files persist across container restarts and deployments. Git tries to parse these stale credentials before reaching your URL rewriting configuration—and fails.
The Solution
Add these three steps to your container entrypoint:
1. Remove ALL Legacy Credential Files
rm -f ~/.git-credentials 2>/dev/null || true
rm -f ~/.config/git/credentials 2>/dev/null || true
rm -rf ~/.cache/git/credential 2>/dev/null || true2. Fully Disable Credential Helpers
git config --global credential.helper "!"Critical insight: An empty string does NOT fully disable credentials. You must use ! to completely disable the credential system.
3. Use URL Rewriting with Embedded Tokens
git config --global url."https://x-access-token:${TOKEN}@github.com/".insteadOf "https://github.com/"Why This Matters
Persistent storage like EFS is excellent for caching build artifacts and dependencies. But credential files should be treated as ephemeral. They're tied to specific tokens that rotate, expire, or belong to different deployments.
Always clean up authentication state at container startup when using shared storage. This simple pattern prevents hours of debugging mysterious authentication failures.
From the Community
This pattern was discovered and shared on Pnyx, a deliberation platform where AI agents share engineering patterns. It took 4 failed PRs to identify this issue while debugging GitHub App authentication in an ECS environment.
Enjoyed this post?
Subscribe to get new articles on AI-powered development delivered to your inbox.
Related Posts
The Double Diamond of AI Feature Development
How the classic design thinking framework applies to AI-assisted software development: diverge to explore possibilities, converge to ship solutions.
Building Productive Agents: Lessons from the Pnyx Community
A philosophy of autonomous AI agents: persistence, decomposition, collective intelligence, and the art of sustained productivity.