-
-
Notifications
You must be signed in to change notification settings - Fork 2.9k
Repair MongoDB
Lauri Ojansivu edited this page Aug 10, 2023
·
12 revisions
sudo su
snap stop wekan
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/snap/wekan/current/lib/x86_64-linux-gnu
export PATH="/snap/wekan/current/bin:$PATH"
mongod --dbpath "/var/snap/wekan/common" --repair >> /root/repairlog.txt
snap start wekan
exit
Logs from Snap:
https://github.com/wekan/wekan/issues/5073
At step 12 below is actual repair MongoDB command.
Some customer of WeKan Commercial Support ordered restore and repair of WeKan MongoDB database at UCS appliance. This was needed when changing back to previous UCS 4.x major version, when in new major UCS 5.x version Docker containers were all the time crashing and restarting, or something else got broken. Here are commands that were used while repairing.
Similarly, MongoDB may require repair, if:
- MongoDB does not recover from sudden power failure cleanly
- Disk comes full, that is worst case. Recommended are daily backups, in this case sometimes repair does not work well enough.
- SSH to UCS appliance as root:
ssh root@192.168.0.100
- If backup or raw database files is at external USB harddrive, look where it is inserted:
ls /dev
fdisk -l /dev/sdb
fdisk -l /dev/sdc
fdisk -l /dev/sdd
- If external USB drive has NTFS file format, and mount complains about not having NTFS support, install NTFS support to Linux:
apt install ntfs-3g
- Mount USB drive to /mnt
mount /dev/sdd1 /mnt
- Stop WeKan Docker containers. If copying raw database files, WeKan and MongoDB should not be running, so that MongoDB would not be even more corrupted. (If MongoDB is running, and you have mongodump backup, you can Backup with mongorestore
docker stop wekan-db wekan-app
- Find WeKan database raw files. MongoDB has various database engines, like WiredTiger, how to save raw data compressed, or other formats. WiredTiger is file format to save compressed MongoDB data. Similar like MySQL has ISAM, InnoDB etc.
apt -y install mlocate
updatedb
locate WiredTiger.wt
It can show for example USB drive and Docker container directory:
root@ucs:/mnt/wekan/data/db# locate WiredTiger.wt
/mnt/wekan/data/db/WiredTiger.wt
/var/lib/univention-appcenter/apps/wekan/data/db/WiredTiger.wt
- Change to newly installed MongoDB data directory:
cd /var/lib/univention-appcenter/apps/wekan/data/db/
- Look at directory files owner permissions:
ls -lah
There could be like this:
-rw-r--r-- 1 tss tss 1004 Jan 25 13:09 WiredTiger.turtle
-rw-r--r-- 1 tss tss 392K Jan 25 13:09 WiredTiger.wt
- Move that probably empty newly installed MongoDB data elsewhere:
mkdir /root/new-empty-wekan-data
mv * /root/new-empty-wekan-data
- Copy MongoDB raw database files from USB harddrive to docker container directory:
root@ucs-bdc:/var/lib/univention-appcenter/apps/wekan/data/db# cp -pR /mnt/wekan/data/db/* .
- Like looked at step 7, change file owner permissions to be correct, and change directory
chown -R tss:tss *
cd /root
- Find mongod command:
locate /usr/bin/mongod
- Repair Docker WeKan with version of Docker MongoDB that WeKan uses, change mongod path to below. Repairing logged to textfile.
root@ucs:~# /var/lib/docker/overlay2/7b58483a16a2f67ee50486c00ec669940f7a95d460ee8188966fee0096e81fa2/diff/usr/bin/mongod --dbpath "/var/lib/univention-appcenter/apps/wekan/data/db" --repair >> repairlog.txt
- Look what containers are running:
docker ps
- Look what containers are not running:
docker ps -a
- Start MongoDB first:
docker start wekan-db
- Look what happends at MongoDB container:
docker logs wekan-db
- Start WeKan next.
docker start wekan-app
- Look what happends at MongoDB container.
docker logs wekan-app
- Check are both wekan-db and wekan-app containers running, and not restarting:
docker ps
- Try to login to WeKan with webbrowser.
- Backup WeKan database now after repair:
docker exec -it wekan-db bash
cd /data
rm -rf dump
mongodump
exit
docker cp wekan-db:/data/dump .
zip -r wekan-backup-YEAR-MONTH-DATE-TIME-HERE.zip dump
Now backup is at wekan-backup-YEAR-MONTH-DATE-TIME-HERE.zip file.
- At your local computer terminal (not at UCS server), transfer file to your local computer with scp:
scp root@192.168.0.100:/root/wekan-backup-YEAR-MONTH-DATE-TIME-HERE.zip .
- Copy backup to external USB harddrive, change YEAR-MONTH-DATE-TIME-HERE to current date and time:
cp /root/wekan-backup-YEAR-MONTH-DATE-TIME-HERE.zip /mnt/
- Look what drives are mounted:
df -h
- Sync and save unwritted data to disk, and unmount external USB harddrive safely:
sync
umount /dev/sdd1
- Remove external USB harddisk from server.
- Login to WeKan, check do all WeKan boards work. In this case, all did work.
Wekan - OpenSource Kanban
- Deep Dive Into WeKan
- Meteor WeKan Roadmap - board at Wekan demo
- Multiverse WeKan Roadmap
- Docs/Manual
- Change Language
- Forgot Password
- About
- Test Edge
- WeKan Design Principles
- FAQ
- IRC FAQ - answers to questions asked at IRC
- Team
- Press
- Blog
- Wekan vs Trello vs Restyaboard
- Results of Survey 2020-01
- Allow private boards only: Disable Public Boards
- Security Disclosure and details of Security in Wekan
- Security issues
- Password Hashing
- Cron: Hourly restart WeKan, because of memory leaks
- Maybe: Add more RAM to Node.js to prevent crash
- Clustering AWS etc
- Scaling
- Kubernetes
- Redis Oplog
- Meteor Scaling at Meteor Cloud
- Scaling at Meteor forums
- From Previous Export, paste big WeKan JSON
- Progress: Import/Export/Sync
- From CSV/TSV
- From Trello
- From Jira
- From Asana
- From Zenkit
- From old Wekan manually
- Converting Meteor Stylus to CSS
- Repair MongoDB
- Using Meteor MongoDB to repair files
- If board does not open and keeps loading
- Repair Docker
- Wekan Markdown
- Emoji
- Mermaid Diagram DOES NOT WORK ANYMORE
- Numbered text
- Automatic login
- Disable Password Login
- Forgot Password
- Admin: Impersonate user
- Adding Users
- Active users Presence
- Accounts Lockout: Brute force login protection
- LDAP
- LDAP AD Simple Auth
- Keycloak
- Google login
- Azure
- OAuth2, Auth0, GitLab, RocketChat
- Oracle OIM on premise using OAuth2
- ADFS 4.0 using OAuth2 and OpenID
- Azure AD B2C using OAuth2
- Nextcloud
- CAS Please test
- SAML Please test
- Zitadel
- Multiline
- Linked Cards
- Drag Drop on Mobile and Desktop
- Rclone: Store attachments to cloud storage like S3, MinIO, etc
- Python based features
- Burndown and Velocity Chart
- Wait Spinners
- Translations
- Default Language for All Users
- Roadmap
- Features
- Planning Poker
- Scaling
- Custom Logo
- Subtasks <== Has fix
- Templates
- Cover
- Archive and Delete
- Custom Fields
- Fix Export board menu not visible on some boards
- RAM usage
- Demo
- Swimlane Documentation
- Customize Translations
- Download Wekan for various Platforms: Supported by xet7, Operating Systems, NAS, Cloud
- Helm Chart for Kubernetes
- Caddy
- Nginx
- Apache
- OpenLiteSpeed
- Local self signed TLS
- Let's Encrypt and Google Auth
- TLS with Node.js
- Traefik and self-signed SSL certs
- Example: New card with Python3 and REST API
- Python client to REST API
- Go client to REST API
- Java
- Wekan Sandstorm cards to CSV using Python
- Excel and VBA
- Global Webhook
- Limiting Webhook data
- Receiving Webhooks
- Outgoing Webhook to Discord/Slack/RocketChat/Riot
- Outgoing Webhook to NodeRed
- Outgoing Webhook to PowerShell
- Security: Webhook and CA
- Outgoing Webhooks Data
- Outgoing Webhooks and Let's Encrypt
- Outgoing Webhooks Original Pull Request, multiple Webhooks, more parameters and response order