Disk space check during mongo storage upgrade to wiredtiger failing and dropping database
When this happens:
2019-05-29T15:59:51.497-0400 Failed: error writing data for collection `pulp_database.units_rpm` to disk: error writing to file: write /var/tmp/mongodb_engine_upgrade/pulp_database/units_rpm.bson: no space left on device
mongodump --host localhost --out /var/tmp/mongodb_engine_upgrade failed! Check the output for error!
Then immediately followed by:
mv: error writing ‘/var/tmp/mongodb_backup/diagnostic.data/metrics.2019-05-28T14-59-39Z-00000’: No space left on device
mv: failed to extend ‘/var/tmp/mongodb_backup/diagnostic.data/metrics.2019-05-28T14-59-39Z-00000’: No space left on device
mv /var/lib/mongodb/* /var/tmp/mongodb_backup failed! Check the output for error!
sed -i.bak -e 's/mmapv1/wiredTiger/g' /etc/opt/rh/rh-mongodb34/mongod.conf finished successfully!
mv: cannot create regular file ‘/var/tmp/mongodb_backup/mongod.conf.bak’: No space left on device
mv /etc/opt/rh/rh-mongodb34/mongod.conf.bak /var/tmp/mongodb_backup failed! Check the output for error!
mongorestore --host localhost --db=pulp_database --drop --quiet --dir=/var/tmp/mongodb_engine_upgrade/pulp_database failed! Check the output for error!
Result is no database.
1. There should be a check and warning that space is required in /var/tmp/mongodb_backup to hold the database
2. A mongorestore with --drop should ONLY be attempted if the prior steps complete successfully
Where are you experiencing the behavior? What environment?
Satellite 6.5 upgrading to wiredtiger
#1 Updated by The Foreman Bot over 3 years ago
- Status changed from New to Ready For Testing
- Pull request https://github.com/theforeman/foreman-installer/pull/388 added
#2 Updated by The Foreman Bot over 3 years ago
- Fixed in Releases 1.24.0 added
#3 Updated by Chris Roberts over 3 years ago
- Triaged changed from No to Yes
- Target version set to 1.24.0
- Status changed from Ready For Testing to Closed
Fixes #27826 - Update Mongo upgrade recovery logic