This script migrates files uploaded to RocketChat from the default GridFS storage to FileSystem or AmazonS3.
migrate -c [command] -d [s3 bucket|output directory] -r [dbname] -t [target]
Run ./migrate.py -h to see all available options.
- dump: Exports files from GridFS to the specified folder/S3 bucket and generates a log file.
- updatedb: Updates database entries to point to the new storage instead of GridFS.
- removeblobs: Removes migrated files from GridFS.
-
python3 (e.g.
sudo apt install python3 python3-pip) -
Install dependencies with:
pip install -r requirements.txtOr manually:
pip install pymongo gridfs tqdm boto3
- If using Amazon S3, set credentials as environment variables (Boto3 configuration guide).
- Export
PYTHONIOENCODING=utf-8to avoid issues with non-ASCII filenames.
To speed up file lookups in GridFS, create the following index in your Rocket.Chat database:
db.rocketchat_uploads.chunks.createIndex({ files_id: 1, n: 1 }, { unique: true })
Not having this index can make migration very slow on large databases.
-
When installed via snap, the default database name is
partiesand usually there is no username or password. -
Example usage:
./migrate.py -c dump -r parties -t FileSystem -d /app/uploads
To speed up migration, you can configure the script to use multiple workers. Adjust according to your CPU core count. Example:
./migrate.py -c dump -r parties -t FileSystem -d /app/uploads --max-workers 8
-
Backup your MongoDB database so you don't lose any data in case of issues. (MongoDB Backup Methods)
-
Change the Storage Type in Rocket.Chat under
Administration > File UploadtoFileSystemorAmazonS3. Update the relevant configuration settings. -
Start copying files to the new storage:
-
File System
./migrate.py -c dump -r rocketchat -t FileSystem -d /app/uploads -
S3
./migrate.py -c dump -r rocketchat -t AmazonS3 -d S3bucket_name
-
-
Update the database to use the new storage (use
-t AmazonS3if migrating to S3):./migrate.py -c updatedb -d /app/uploads -r rocketchat -t FileSystem
-
Check if everything is working correctly and ensure there are no missing files.
-
Remove obsolete data from GridFS:
./migrate.py -c removeblobs -d /app/uploads -r rocketchat
In some configurations, it may help to add the parameters directconnection=True and connect=False to the MongoClient constructor, for example:
MongoClient(..., retryWrites=False, directconnection=True, connect=False)[self.db]
This ensures the connection uses Single topology.