Troubleshooting Deployments

Deploys fail for a variety of reasons: failed Docker builds, CPU architecture incompatibility, programming language version mismatches, and failed health checks to name a few. In this guide we'll help you narrow down the issue you're facing and do our best to help you resolve it.

Check deployment logs

Deployment logs include information about the status of your build and CloudFormation stack event history.

Docker build errors

When your Docker build fails, you'll likely see logs similar to this:

ERROR: failed to solve...

When you encounter this error, it means one of the steps in your Dockerfile failed. More often than not there will be an accompanying stack trace which can help you figure out where exactly the failure occurred. To resolve this more quickly, run a docker build command locally until the step passes.

CloudFormation errors

Deployment logs output a stream of events containing stack update information. These events will tell you which infrastructure deployments are in progress, completed, or failed. In some cases, such as when a CloudFront distribution or ECS cluster fails to create, you may be able to resolve the issue by re-deploying your component or environment. Other cases, such as failed ECS rollouts may require additional action on your part in order to resolve.

When an ECS Service has an UPDATE_FAILED or CREATE_FAILED state, there will be an accompanying message such as Service ... did not stabilize. When you see this error, it means that the rollout of your service's task failed. Continue below for guidance on debugging the error.

Monitor service events and rollout status

When components are deploying, you can monitor their rollout status and any events emitted by the container service in real-time. This view can be found on the component overview dashboard.

There you'll be able to track the health of tasks as they roll out and view the progress of the rollout as it completes.

For example, under "Recent events" in the screenshot below, you'll notice that a task was started 18 seconds ago with the id afe198.... That ID corresponds to the task with the same ID under the "2 tasks" heading.

Common service event error messages

Capacity is unavailable at this time. Please try again later or in a different availability zone.

This occurs when Amazon ECS doesn't have the capacity to launch a task. This is most commonly run into when using Spot pricing which is enabled by default in development and staging environments. To fix this, you can opt to either disable spot pricing or you can re-deploy your component and try again.

Service ... did not stabilize

The service failed to rollout your desired number tasks, either because essential task containers exited or the service was unable to reach a healthy state.

...failed container health checks

A task in your service failed to pass container health checks. Confirm that the command that you pass to the container is correct. Test the health check locally to ensure it is working properly. Lastly, check the container logs in the "Logs" tab of the component to ensure there aren't any errors being thrown by the container.

Monitor component logs

Check your container logs in the Logs tab of the failing component to ensure there aren't any errors being thrown by the application. A typical example is something like SUPABASE_KEY is undefined indicating a missing environment variable as a likely culprit. You can check the logs for an individual task by clicking on the task from the component overview page or by clicking on the task ID on the logs page.

Verify your configuration is correct

• Make sure your application is receiving all of the environment variables it needs and is configured to use the correct host names, ports, and credentials. Some popular libraries for this include: envsafe (JavaScript) and env (Golang).
• If you're deploying a component with port bindings such a web service, check that the port your web server listens on is the same port number configured in the Deploy tab of your component. As a safeguard, you can configure your web server to listen to the port defined in the PORT environment variable which is always set by FlexStack.
• If you're using a compiled language like Golang or Rust, make sure your container is configured to build for the right platform. Docker's cross-compilation guide has more information on this.
• Ensure the version of your runtime image is compatible with the version of the language your code is written in. Using our Dockerfile generator alongside tools like asdf can help with this.
• Ensure you're installing all of the dependencies your application requires and that these dependencies have been copied to the runtime image.

Seek community guidance

For help debugging, you can try asking our Discord community which includes members of the FlexStack team.