8000 Setup auto deploy for test app by VGLoic · Pull Request #1 · VGLoic/aws-exploration · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Setup auto deploy for test app #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
94 changes: 94 additions & 0 deletions .github/workflows/aws.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
# This workflow will build and push a new container image to Amazon ECR,
# and then will deploy a new task definition to Amazon ECS, when there is a push to the "main" branch.
#
# To use this workflow, you will need to complete the following set-up steps:
#
# 1. Create an ECR repository to store your images.
# For example: `aws ecr create-repository --repository-name my-ecr-repo --region us-east-2`.
# Replace the value of the `ECR_REPOSITORY` environment variable in the workflow below with your repository's name.
# Replace the value of the `AWS_REGION` environment variable in the workflow below with your repository's region.
#
# 2. Create an ECS task definition, an ECS cluster, and an ECS service.
# For example, follow the Getting Started guide on the ECS console:
# https://us-east-2.console.aws.amazon.com/ecs/home?region=us-east-2#/firstRun
# Replace the value of the `ECS_SERVICE` environment variable in the workflow below with the name you set for the Amazon ECS service.
# Replace the value of the `ECS_CLUSTER` environment variable in the workflow below with the name you set for the cluster.
#
# 3. Store your ECS task definition as a JSON file in your repository.
# The format should follow the output of `aws ecs register-task-definition --generate-cli-skeleton`.
# Replace the value of the `ECS_TASK_DEFINITION` environment variable in the workflow below with the path to the JSON file.
# Replace the value of the `CONTAINER_NAME` environment variable in the workflow below with the name of the container
# in the `containerDefinitions` section of the task definition.
#
# 4. Store an IAM user access key in GitHub Actions secrets named `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY`.
# See the documentation for each action used below for the recommended IAM policies for this IAM user,
# and best practices on handling the access key credentials.

name: Deploy to Amazon ECS

on:
push:
branches: [ "main" ]

env:
AWS_REGION: eu-west-3 # set this to your preferred AWS region, e.g. us-west-1
ECR_REPOSITORY: aws-guide-repo # set this to your Amazon ECR repository name
ECS_SERVICE: sample-fargate-service # set this to your Amazon ECS service name
ECS_CLUSTER: AwsFargateGuideCluster # set this to your Amazon ECS cluster name
ECS_TASK_DEFINITION: aws/task-def.json # set this to the path to your Amazon ECS task definition
# file, e.g. .aws/task-definition.json
CONTAINER_NAME: fargate-app # set this to the name of the container in the
# containerDefinitions section of your task definition

permissions:
contents: read

jobs:
deploy:
name: Deploy
runs-on: ubuntu-latest

steps:
- name: Checkout
uses: actions/checkout@v3

- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ env.AWS_REGION }}

- name: Login to Amazon ECR
id: login-ecr
uses: aws-actions/amazon-ecr-login@v1

# - name: Build, tag, and push image to Amazon ECR
# id: build-image
# env:
# ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
# IMAGE_TAG: ${{ github.sha }}
# run: |
# # Build a docker container and
# # push it to ECR so that it can
# # be deployed to ECS.
# docker build -t $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG .
# docker push $ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG
# echo "image=$ECR_REGISTRY/$ECR_REPOSITORY:$IMAGE_TAG" >> $GITHUB_OUTPUT

# - name: Fill in the new image ID in the Amazon ECS task definition
# id: task-def
# uses: aws-actions/amazon-ecs-render-task-definition@v1
# with:
# task-definition: ${{ env.ECS_TASK_DEFINITION }}
# container-name: ${{ env.CONTAINER_NAME }}
# image: ${{ steps.build-image.outputs.image }}

- name: Deploy Amazon ECS task definition
uses: aws-actions/amazon-ecs-deploy-task-definition@v1
with:
# task-definition: ${{ steps.task-def.outputs.task-definition }}
task-definition: ${{ env.ECS_TASK_DEFINITION }}
service: ${{ env.ECS_SERVICE }}
cluster: ${{ env.ECS_CLUSTER }}
wait-for-service-stability: true
90 changes: 90 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -665,6 +665,96 @@ So I go to the security group part in AWS and I go for modifying my default secu

I save this and try to hit on my load balancer DNS (with `/health`) and it works!

### 6. Integrating a CI/CD - following the blog post

I would now like to have some automation and try to automatically deploy my new code when I push it on the `main` branch. I found this [blog post](https://aws.amazon.com/blogs/opensource/github-actions-aws-fargate/) of AWS that introduce the main AWS Github Actions, let's dig a bit.


The blog post describes four Github Actions:
> We have open sourced the following actions at github.com/aws-actions:
> - github.com/aws-actions/configure-aws-credentials – Retrieves AWS credentials from GitHub secrets and makes them available for use by the rest of the actions in the workflow.
> - github.com/aws-actions/amazon-ecr-login – Lets you log into your ECR registry so you can pull images from and push them to your ECR repositories.
> - github.com/aws-actions/amazon-ecs-render-task-definition – Inserts the ID of a container image into the task definition for ECS.
> - github.com/aws-actions/amazon-ecs-deploy-task-definition – Deploys an updated task definition and image to your ECS service.


Looks exactly what I need, so let's try to integrate these.

I'll first follow what they do in the blog post, then I'll adapt it to my codebase.

#### Understanding a bit more IAM

So I previously created a user `AwsGuide`.

This user has a set of permissions, and as said by AWS directly
> Permissions are defined by policies attached to the user directly or through groups.

I don't want to handle `groups` for now so I'll go directly with the `policy`. There is a huge list of all the possible policies and it's not very easy for me to navigate in this. There are two ways to attach policies:
1. create an `inline policy` for the user: in this case, I directly browse the policies for one or more services (like ECR or EC2) and I add the ones I need,
2. attach a `managed policy` directly, as I understand, AWS has prepared some packages of policies that are commonly used together.

I will try to use the `managed policy` in my case. It is actually advised to give policy to `group` and not to `user` directly, so I'll go create my group `aws-fargate-ci` then.

I can directly attach some permissions policies. I find on [this page](https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies_job-functions.html#jf_developer-power-user) the `Developer power user job function` with the managed policy `PowerUserAccess`, it seems too broad for my case but I will start with that! I will then add my user to this group.

At the beginning of this work, I had added an inline policy to my user with full access to ECR, I will remove it as I expect my new managed policy to also provide this (and it does). Taking a look at this managed policy in AWS is actually not so easy because it contains some lower level managed policies, so I understand that it has a lot of things, but I don't grasp all of it. I know however that it can not create or manage accounts, which is good.

I will continue with the set up in the blog post and go back to this wheen needed.

The blog post contains some setup, I'll try to follow with the AWS CLI this time. Additionally, I'll need to do some actions in the CI so I will not be able to rely on our root user anymore.

#### Configuration of the ECR

I already have an ECR so I will not create a new one, but I still want to be able to read from it and ultimately push image in it.

I can query my repositories using `aws ecr describe-repositories --profile <my profile>` so it's all good on this side.

#### Task definition

I'll copy their task definition in `./aws/task-def.json`. The task is really simple at it directly echoes an HTML page. If I compare to my previous task definitions, there are a lot less things, but we'll try to underestand this better a bit later.

I'll register it using their command
```console
aws ecs register-task-definition --region eu-west-3 --cli-input-json file://./aws/task-def.json --profile <my profile>
```

I can see it on my AWS console so all good on this side!

#### Service creation

The blog post contains the creation of a cluster but I'll try to use the one I already have. Let's jump to the service.

I needed to adapt a bit the service creation command as I needed to point to my desired cluster and change the names of the subnets and security group. I still use my default VPC so I don't need to input this I guess.
```console
aws ecs create-service --region eu-west-3 --cluster AwsFargateGuideCluster --service-name sample-fargate-service --task-definition sample-fargate:2 --desired-count 2 --launch-type "FARGATE" --network-configuration "awsvpcConfiguration={subnets=[<my three subnets>],securityGroups=[<my default security group>]}" --profile aws-guide
```
PS: I did a bad manipulation so my revision number is 2 instead of 1.

So my service has been created but the tasks were failing. The error was something like `CannotPullContainerError: pull image manifest has been retried 5 time(s): failed to resolve ref docker.io/library/httpd:2.4: failed to do request: Head "https://registry-1.docker.io/v2/library/httpd/manifests/2.4": dial tcp <IP>: i/o timeout`

I have found this [post](https://stackoverflow.com/questions/76398247/cannotpullcontainererror-pull-image-manifest-has-been-retried-5-times-failed) with a similar error. So I am using a public subnet but I have neither a gateway, neither set `assignPublicIp` to `true`, so my task can't connect to the image registry in order to get its Docker image.

I will shut down my service and create a new one with this option enabled:
```console
aws ecs create-service --region eu-west-3 --cluster AwsFargateGuideCluster --service-name sample-fargate-service --task-definition sample-fargate:2 --desired-count 2 --launch-type "FARGATE" --network-configuration "awsvpcConfiguration={subnets=[<my three subnets>],securityGroups=[<my default security group>],assignPublicIp=ENABLED}" --assignPublicIp ENABLED --profile aws-guide
```

Okay now I have my service running, my tasks running, and I can visit the public IP of my tasks in order to get the setup HTML. Good stuff.

By the way, in order to make this work, I modified the inbound rules of my default security group in order to add inbound rules for PORT 80.

And since my service does not have any load balancer, I can not access my "service" directly, I need to go to the task directly. So it seems way more serious to have a load balancer in this case.

#### Setting up my repository

I'll add two secrets for my CI in the Github repository directly: `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` that I obtain from my `AwsGuide` user on IAM. During the creation, AWS recommended another alternative for such a use case, I'll start with this but it would be interested to learn more about this alternative.

Then I'll add the workflows, for this I simply go to `actions` tab of my repository, GitHub recommends me a few and a select the `Deploy to Amazon ECS`. Instead of commiting this to `main`, I'll copy the content in `.github/workflows/aws.yaml` and commit it in a dedicated PR. I update the workflow to my needs:
- replacing the correct environment for my AWS resources,
- removing the `environment: production` as I don't have any environment in my repository,
- commenting the steps `build-image` and `task-def` as I don't build any Docker image for now, I then change in the last job the line `task-definition` to `task-definition: ${{ env.ECS_TASK_DEFINITION }}` in order to rely on my local file directly.

I create a PR with that, I also modify the `task-def.json` content to add a change to the HTML.

## Development

Expand Down
30 changes: 30 additions & 0 deletions aws/task-def.json
52AC
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"family": "sample-fargate",
"networkMode": "awsvpc",
"containerDefinitions": [
{
"name": "fargate-app",
"image": "httpd:2.4",
"portMappings": [
{
"containerPort": 80,
"hostPort": 80,
"protocol": "tcp"
}
],
"essential": true,
"entryPoint": [
"sh",
"-c"
],
"command": [
"/bin/sh -c \"echo '<html> <head> <title>Amazon ECS Sample App !Version 2!</title> <style>body {margin-top: 40px; background-color: #333;} </style> </head><body> <div style=color:white;text-align:center> <h1>Amazon ECS Sample App</h1> <h2>Congratulations!</h2> <p>Your application is now running on a container in Amazon ECS.</p> </div></body></html>' > /usr/local/apache2/htdocs/index.html && httpd-foreground\""
]
}
],
"requiresCompatibilities": [
"FARGATE"
],
"cpu": "256",
"memory": "512"
}
0