8000 Feat 465 add ARM support by eldadfux · Pull Request #726 · appwrite/appwrite · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Feat 465 add ARM support #726

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 42 commits into from
Apr 19, 2021
Merged

Feat 465 add ARM support #726

merged 42 commits into from
Apr 19, 2021

Conversation

eldadfux
Copy link
Member
@eldadfux eldadfux commented Nov 3, 2020

What does this PR do?

Adding ARM support for all Appwrite Docker images. This will also enable support for Raspberry PI.

Test Plan

Added support for multicore tests on Travis CI.

Related PRs and Issues

#465

Have you read the Contributing Guidelines on issues?

Yes.

@eldadfux eldadfux changed the title Feat 465 add arm support Feat 465 add ARM support Nov 4, 2020
@eldadfux eldadfux linked an issue Jan 4, 2021 that may be closed by this pull request
@eldadfux
Copy link
Member Author
eldadfux commented Jan 4, 2021

ARM Support Update

Seems like we've managed to solve 99% of all open issues. The only thing still failing right now is the test for executing a Deno cloud function. Seems like the Deno Docker image lacks ARM support: denoland/deno_docker#100

< 8000 form class="js-comment-update" id="issuecomment-753874939-edit-form" data-turbo="false" action="/appwrite/appwrite/issue_comments/753874939" accept-charset="UTF-8" method="post">

@mlucas-NU
Copy link

Thank you for all the hard work here, @eldadfux .

Are there instructions for building the ARM64 image to launch locally while we wait for Deno Docker support?

@eldadfux
Copy link
Member Author

@mlucas-NU building the repo from source should do the trick (from the ARM branch). You can learn how here:
https://github.com/appwrite/appwrite/blob/master/CONTRIBUTING.md#setup-from-source

You can also see how to push to Docker hub here:
https://github.com/appwrite/appwrite/blob/master/CONTRIBUTING.md#build

We also decided not to wait for Deno or Dart (just added it 😁). We've added a new architecture detection phase and we enable only cloud functions envs, that are supported to your specific hardware. Here is a spoiler from the new docs:
image

@mlucas-NU
Copy link
mlucas-NU commented Jan 31, 2021

Looks fantastic.

I was able to get a local container running off of this branch. Four of the upstream appwrite docker images are not arm64-compatible yet, so here are the steps I took in case it helps anyone:

1. Build local arm64 docker images

cd some_working_directory
git clone git@github.com:appwrite/appwrite.git build_arm64/appwrite
cd build_arm64/appwrite
g
8000
it checkout feat-465-add-arm-support
docker buildx build --platform linux/arm64 -t appwrite/appwrite_arm64:latest .

Repeat the same process for each project in {smtp, mariadb, clamav, telegraf}. The latest appwrite images don't support ARM64 architectures yet and will crash repeatedly on startup.

cd some_working_directory
git clone git@github.com:appwrite/docker-{project}.git build_arm64/{project}
cd build_arm64/{project}
docker buildx build --platform linux/arm64 -t appwrite/{project}_arm64:latest .

Running docker images now shows the local arm64 images:

REPOSITORY                                     TAG               IMAGE ID       CREATED             SIZE
appwrite/clamav_arm64                          latest            2dceaac02882   58 minutes ago      17.7MB
appwrite/telegraf_arm64                        latest            1c2dd1ea9613   59 minutes ago      87.2MB
appwrite/mariadb_arm64                         latest            77071ccfecd7   About an hour ago   392MB
appwrite/smtp_arm64                            latest            ca7a8cc3bcdb   About an hour ago   9.13MB
appwrite/appwrite_arm64                        latest            fa138942e967   3 hours ago         380MB

2. Start a new appwrite project.

Create a new directory and download appwrite's docker-compose template.

mkdir some_project_name
cd some_project_name
wget https://appwrite.io/docker-compose.yml

3. Update docker-compose.yml to point to the local arm64 images:

Change the appwrite service's image to the locally built appwrite/appwrite_arm64 (latest).

services:
...
    appwrite:
        image: appwrite/appwrite_arm64

Repeat this change for the other 4 services (e.g. mariadb's image should be appwrite/mariadb_arm64).

4. Launch appwrite normally.

cd some_project_name
docker-compose up -d

@mlucas-NU
Copy link
mlucas-NU commented Feb 18, 2021

I made some arm64 migration notes for appwrite 0.6.3 -> 0.7.0

Warnings:

  • I'm running this on a single-board computer. I can reach it via a local network but it doesn't have a public-facing IP or domain name, so some of my configuration reflects that.
  • I'm building arm64-compatible images from the master branch of each repository. A more stable approach is to find version-tagged releases for each branch, but pulling master is working for me.

1. Shut down previous instance

If you are still running the previous version of appwrite, shut it down by running docker-compose down in the directory of your old docker-compose.yml configuration. If you prefer to wait until your configuration is complete, copy your old docker-compose file to a temporary directory and run docker-compose down from that directory when you are ready.

2. Get the new docker-compose template

I'm working from a docker-compose.yml template for appwrite v0.7 shared by @eldadfux here: https://gist.github.com/eldadfux/977869ff6bdd7312adfd4e629ee15cc5

  1. Back up your existing docker-compose.yml file somewhere secure (it has passwords/keys).
  2. Save Eldad's .env and docker-compose.yml templates to your project directory.
  3. Transfer your environment variables from your previous docker-compose.yml to .env. I made these changes:
    • Commented out one of the environments (# _APP_ENV=development)
    • Added values for _APP_SYSTEM_EMAIL_NAME, _APP_SYSTEM_EMAIL_ADDRESS, and _APP_SYSTEM_SECURITY_EMAIL_ADDRESS.
    • Copied the key over for _APP_OPENSSL_KEY_V1 from the old docker-compose.yml
    • Set _APP_DOMAIN and _APP_DOMAIN_TARGET to localhost
    • Copied values for _APP_DB_USER and _APP_DB_PASS
    • Added values for _APP_SMTP_SECURE, _APP_SMTP_USERNAME, and _APP_SMTP_PASSWORD
    • Added _APP_CONSOLE_WHITELIST_EMAILS at the end to prevent admin registrations.

3. Build arm64-compatible docker images

You can save this as a shell script in your project directory to build images. Otherwise, I posted a manual version of these instructions in a previous comment that provide more detail.

The general idea is that I've identified the list of appwrite projects that don't have arm64-compatible images available. This script pulls the master branch for each project, builds its docker image with --platform linux/arm64 set, and saves the image locally as a appwrite/{project}:latest.

build_arm64.sh

WORKDIR=.arm64_builds
CLEANUP=true  # Delete $WORKDIR at the end of the script -- built docker images will remain locally

mkdir -p $WORKDIR
cd $WORKDIR
git clone git@github.com:appwrite/appwrite.git appwrite
(cd appwrite && docker buildx build --platform linux/arm64 -t appwrite/appwrite_arm64:latest . )

for project in smtp telegraf; do
    git clone git@github.com:appwrite/docker-$project.git $project
    (cd $project && docker buildx build --platform linux/arm64 -t appwrite/${project}_arm64:latest . )
done

if [ $CLEANUP ] then
    rm -r $WORKDIR
fi

4. Modify docker-compose.yml

  1. Switch to the base influxdb image since it has an arm64 build

    • influxdb:1.8-alpine -> influxdb:1.8 (Line 342)
  2. Replace docker images with arm64-compatible ones (we will build those images ourselves in Step 5).

    • appwrite/appwrite:0.7.0 -> appwrite/appwrite_arm64:latest (Lines 30, 85, 102, 122, 143, 164, 187, 211, 241, 265, 283)
    • appwrite/smtp:1.0.1 -> appwrite/smtp_arm64:latest (Line 314)
    • appwrite/telegraf:1.0.0 -> appwrite/telegraph_arm64:latest (Line 351)

5. Start your appwrite 0.7.0 server up

After running docker-compose up -d, your server should begin spinning up its various docker containers.

Optionally, watch pretty logs with docker-compose logs -f.
Optionally, navigate to your server in a web browser. I was able to log in at this point and see information about my existing project. There were some weird page loading issues initially, but they disappeared. Maybe it was incompatible older cookies causing problems and getting replaced?

5. Run the appwrite migration tool

The Appwrite docs suggests using the migration tool. Let's see what it does!!

$ docker-compose exec appwrite migrate
Starting Data Migration
Migrating project: Appwrite (console)
Migrating: 0 / 8
Fetched 1/1 projects...
Migrating project: TestProject (601908fabaeb8)
Migrating: 0 / 29
Data Migration Completed

It didn't seem to change any behavior on the server, but maybe it did something nice behind the scenes.

@eldadfux eldadfux marked this pull request as ready for review March 23, 2021 09:16
@TorstenDittmann TorstenDittmann mentioned this pull request Mar 29, 2021
@eldadfux eldadfux changed the base branch from 0.7.x to 0.8.x April 16, 2021 10:56
Copy link
Contributor
@TorstenDittmann TorstenDittmann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@eldadfux Have run the CI multiple times now 👍🏻

This is ready for more review and merge 🎉

@eldadfux eldadfux merged commit c6af32f into 0.8.x Apr 19, 2021
@eldadfux
Copy link
Member Author
eldadfux commented Apr 19, 2021

This was one of our greatest ever community effort! Big KUDOS to everyone who helped, I'm proud to finally merge it and confirm it will be part of the next Appwrite release.

@eldadfux eldadfux deleted the feat-465-add-arm-support branch May 15, 2021 07:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support for ARM platform
5 participants
0