Resolving AWS credential errors after rotating access keys

When rotating AWS access keys for your application, you may encounter the following error: Aws::<Service_Name>::Errors::UnrecognizedClientException: The security token included in the request is invalid. Common Causes This error typically occurs when: Old AWS credentials remain in environment files (e.g., .env , .env.production ) after rotation There is a conflict between static credentials and IAM Roles for Service Accounts (IRSA) Secrets or ConfigMaps still reference outdated access keys Resolution Steps

1. Check Environment Files and Secrets After rotating access keys, make sure to: Remove outdated AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY from: .env , .env.production , or other environment files Kubernetes Secrets or ConfigMaps Confirm that no stale credentials are being injected into the container Rebuild the Docker image if it bakes in any environment variables Redeploy the app so changes take effect

2. Verify Credential Configuration (Static vs. IRSA) If you're using static credentials , ensure: No IRSA-related environment variables (like AWS_WEB_IDENTITY_TOKEN_FILE or AWS_ROLE_ARN ) are present The pod’s service account is not annotated with an IAM role for IRSA If you're using IRSA , then: Remove AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY from all sources Make sure the IAM role trust policy allows the service account to assume the role Confirm the pod’s service account is correctly annotated with eks.amazonaws.com/role-arn ⚠ Important: If both static credentials and IRSA variables are set, the AWS SDK will prioritize IRSA . If IRSA is misconfigured, the SDK will fail with UnrecognizedClientException , even if the static credentials are valid. Summary Always keep your credential sources consistent. Mixing static credentials with IRSA can lead to confusing errors and security risks. Audit .env files, Secrets, and ConfigMaps to ensure they are aligned with your intended AWS authentication strategy. Note: After making credential changes, you may need to restart your services for the new configuration to take effect.