Run on AWS
Let’s run the AI API in production on AWS.
AWS Setup
Add Credentials
To run on AWS, you need one of the following:
- The
~/.aws/credentialsfile with your AWS credentials AWS_ACCESS_KEY_ID+AWS_SECRET_ACCESS_KEYenvironment variables
To create the credentials file, install the aws cli and run aws configure
Add Region and Subnets
Add 2 Subnets to the workspace/settings.py file (required for ECS services)
ws_settings = WorkspaceSettings(
...
# -*- AWS settings
# Add your Subnet IDs here
subnet_ids=["subnet-xyz", "subnet-xyz"],
...
)
Please check that the subnets belong to the selected aws_region
Update Secrets
RDS database password
Update the RDS database password in workspace/secrets/prd_db_secrets.yml
# Secrets used by prd RDS database
MASTER_USERNAME: api
MASTER_USER_PASSWORD: "api9999!!"
API Secrets
Add any other secrets used by your api to workspace/secrets/prd_api_secrets.yml
SECRET_KEY: "very_secret"
# OPENAI_API_KEY: "sk-***"
Create AWS resources
Create AWS resources using:
This will create:
- ECS Cluster for the application.
- ECS Task Definitions and Services that run the application on the ECS cluster.
- LoadBalancer to route traffic to the application.
- Security Groups that control incoming and outgoing traffic.
- Secrets for managing application and database secrets.
- RDS Database for Knowledge Base and Storage.
Press Enter to confirm and grab a cup of coffee while the resources spin up.
- The RDS database takes about 5 minutes to activate.
- These resources are defined in the
workspace/prd_resources.pyfile. - Use the ECS console to view services and logs.
- Use the RDS console to view the database instance.
Production FastApi
Enable FastApi
Update the workspace/settings.py file and set prd_api_enabled=True
...
ws_settings = WorkspaceSettings(
...
# Uncomment the following line
prd_api_enabled=True,
...
Create FastApi resources
Press Enter to confirm
View API Endpoints
- Open the LoadBalancer DNS + the
/docsendpoint to view the API Endpoints. - Load the knowledge base using
/v1/assitants/load-knowledge-base - Test the
v1/assitants/chatendpoint with
{
"message": "How do I make pad thai?",
"assistant": "AUTO_PDF"
}
- Integrate with your front-end or product.
Update Production
Follow this guide to update your production application. You'll need to:
- Create a new image
- Update the ECS Task Definition and Services.
Delete AWS resources
Play around and then delete AWS resources using:
Next
Congratulations on running your AI API on AWS. Next Steps:
- Read how to update workspace settings
- Read how to create a git repository for your workspace
- Read how to manage the production application
- Read how to format and validate your code
- Read how to add python libraries
- Read how to add a custom domain and HTTPS
- Read how to implement CI/CD
- Chat with us on discord
Was this page helpful?