Bug #8995
closedDocument space requirement for mongodb
Description
Possibly an upstream issue, but mongod does not start during katello-installer on CentOS7.
I'm installing Katello 2.1
Output:
[root@foreman ~]# katello-installer
Could not start Service[mongodb]: Execution of '/usr/share/katello-installer/modules/service_wait/bin/service-wait mongod start' returned 1: Redirecting to /bin/systemctl start mongod.service
/Stage[main]/Mongodb::Server::Service/Service[mongodb]/ensure: change from stopped to running failed: Could not start Service[mongodb]: Execution of '/usr/share/katello-installer/modules/service_wait/bin/service-wait mongod start' returned 1: Redirecting to /bin/systemctl start mongod.service
/Stage[main]/Apache::Service/Service[httpd]: Failed to call refresh: Could not start Service[httpd]: Execution of '/usr/share/katello-installer/modules/service_wait/bin/service-wait httpd start' returned 1: Redirecting to /bin/systemctl start httpd.service
/Stage[main]/Apache::Service/Service[httpd]: Could not start Service[httpd]: Execution of '/usr/share/katello-installer/modules/service_wait/bin/service-wait httpd start' returned 1: Redirecting to /bin/systemctl start httpd.service
/Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: Failed to call refresh: /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0]
/Stage[main]/Foreman::Database/Foreman::Rake[db:seed]/Exec[foreman-rake-db:seed]: /usr/sbin/foreman-rake db:seed returned 1 instead of one of [0]
/Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[foreman.rhel7.local]: Failed to call refresh: Could not load data from https://foreman.rhel7.local
/Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[foreman.rhel7.local]: Could not load data from https://foreman.rhel7.local
Installing Done [100%] [............................................................................................]
Something went wrong! Check the log for ERROR-level output
The full log is at /var/log/katello-installer/katello-installer.log
[root@foreman ~]#
When I try starting mongod manually, it also fails:
[root@foreman ~]# systemctl start mongod.service
Job for mongod.service failed. See 'systemctl status mongod.service' and 'journalctl -xn' for details.
[root@foreman ~]# systemctl status mongod.service
mongod.service - High-performance, schema-free document-oriented database
Loaded: loaded (/usr/lib/systemd/system/mongod.service; disabled)
Active: failed (Result: exit-code) since Fri 2015-01-16 11:53:14 GMT; 10s ago
Process: 32023 ExecStart=/usr/bin/mongod $OPTIONS run (code=exited, status=100)
Jan 16 11:53:14 foreman.rhel7.local systemd1: Starting High-performance, schema-free document-oriented database...
Jan 16 11:53:14 foreman.rhel7.local mongod32023: about to fork child process, waiting until server is ready for connections.
Jan 16 11:53:14 foreman.rhel7.local mongod32023: forked process: 32025
Jan 16 11:53:14 foreman.rhel7.local mongod32023: ERROR: child process failed, exited with error number 100
Jan 16 11:53:14 foreman.rhel7.local systemd1: mongod.service: control process exited, code=exited status=100
Jan 16 11:53:14 foreman.rhel7.local systemd1: Failed to start High-performance, schema-free document-oriented database.
Jan 16 11:53:14 foreman.rhel7.local systemd1: Unit mongod.service entered failed state.
[root@foreman ~]#
I'm following the install notes at: http://www.katello.org/docs/2.1/installation/index.html rather than the katello-deploy method in order to test the installation method. I have fixed the yum repositories to make sure I am getting the right software:
[root@foreman ~]# yum repolist
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: mirror.as29550.net
* epel: mirror.bytemark.co.uk
* extras: mirror.bytemark.co.uk
* updates: mirror.bytemark.co.uk
repo id repo name status
base/7/x86_64 CentOS-7 - Base 8,465
epel/x86_64 Extra Packages for Enterprise Linux 7 - x86_64 7,031
extras/7/x86_64 CentOS-7 - Extras 103
foreman/x86_64 Foreman 1.7 308
foreman-plugins/x86_64 Foreman plugins 1.7 158
katello/7/x86_64 Katello Stable 62
katello-candlepin/7/x86_64 An open source entitlement management system. 6
katello-pulp/7/x86_64 Pulp Community Releases 41
puppetlabs-deps/x86_64 Puppet Labs Dependencies El 7 - x86_64 17
puppetlabs-products/x86_64 Puppet Labs Products El 7 - x86_64 123
rhscl-ruby193-epel-7-x86_64 Ruby193 - epel-7-x86_64 405
rhscl-v8314-epel-7-x86_64 V8314 - epel-7-x86_64 23
updates/7/x86_64 CentOS-7 - Updates 1,568
repolist: 18,310
[root@foreman ~]#
Updated by Duncan Innes over 9 years ago
I'm aware of Issue #8478 and have tried both the workarounds listed in http://projects.theforeman.org/issues/8478#note-4
Neither of these appear to work for me however.
My test machine is rebuilt clean from PXE boot for every test, so I don't think I've got any old edits hanging around.
Updated by Justin Sherrill about 9 years ago
apologies if i missed it, but you can confirm the exact version of mongodb-server you are running?
Updated by Duncan Innes about 9 years ago
Justin Sherrill wrote:
apologies if i missed it, but you can confirm the exact version of mongodb-server you are running?
Just reinstalled the test box again and I currently have:
[root@foreman ~]# rpm -qa | grep mongo
mongodb-server-2.6.5-2.el7.x86_64
python-pymongo-2.5.2-2.el7.x86_64
mongodb-2.6.5-2.el7.x86_64
You have mail in /var/spool/mail/root
[root@foreman ~]#
Going to run katello-installer now (no flags given - using defaults all round). Will post mongo versions after the install as well.
Updated by Duncan Innes about 9 years ago
Getting the same version of mongodb after installation now too:
[root@foreman ~]# rpm -qa | grep mongo
mongodb-server-2.6.5-2.el7.x86_64
python-pymongo-2.5.2-2.el7.x86_64
mongodb-2.6.5-2.el7.x86_64
[root@foreman ~]#
The repositories available to the machine are:
[root@foreman ~]# yum repolist
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
* base: mirrors.coreix.net
* epel: mirror.bytemark.co.uk
* extras: mirrors.coreix.net
* updates: mirrors.coreix.net
repo id repo name status
base/7/x86_64 CentOS-7 - Base 8,465
epel/x86_64 Extra Packages for Enterprise Linux 7 - x86_64 7,143
extras/7/x86_64 CentOS-7 - Extras 104
foreman/x86_64 Foreman 1.7 324
foreman-plugins/x86_64 Foreman plugins 1.7 160
katello/7/x86_64 Katello Stable 63
katello-candlepin/7/x86_64 An open source entitlement management system. 6
katello-pulp/7/x86_64 Pulp Community Releases 41
puppetlabs-deps/x86_64 Puppet Labs Dependencies El 7 - x86_64 17
puppetlabs-products/x86_64 Puppet Labs Products El 7 - x86_64 125
rhscl-ruby193-epel-7-x86_64 Ruby193 - epel-7-x86_64 405
rhscl-v8314-epel-7-x86_64 V8314 - epel-7-x86_64 23
updates/7/x86_64 CentOS-7 - Updates 1,615
repolist: 18,491
[root@foreman ~]#
Updated by Justin Sherrill about 9 years ago
Hey Duncan,
Can you upload a foreman-debug to the rsync server (instructions given at the end of the command)? and then paste here the filename.
-Justin
Updated by Duncan Innes about 9 years ago
Justin Sherrill wrote:
Hey Duncan,
Can you upload a foreman-debug to the rsync server (instructions given at the end of the command)? and then paste here the filename.
-Justin
Try foreman-debug-HDOMh.tar.xz
I opted for y at the end of the command to rsync it for me. I then tried to do it manually afterwards, but keep getting errors.
If that fails, I renamed it to foreman-debug-Duncan.tar.xz and managed to upload that one successfully.
D
Updated by Justin Sherrill about 9 years ago
- Target version deleted (
66)
Debugging with a user on IRC with a similar issue, it ended up that /var/run/mongodb didn't exist. Can you see if this is your case too?
Updated by Duncan Innes about 9 years ago
Left 'Katello' laptop at work, so can't check until Monday. But will test that as soon as I can Monday. Will try to get on IRC too if possible.
Updated by Duncan Innes about 9 years ago
Eric,
The mongodb test is passed now, but there are other failures in my default install. I don't remember the Katello packages being at version 2.2 last week (could be wrong). Should I close this and open new bugs to track the issue I find?
Duncan
Updated by Duncan Innes about 9 years ago
Right, getting confused about what changes have been going on in the last week.
I checked Bug #9163 which shows install issues similar to what I now find myself facing. But downgrading puppet as suggest in that issue now gets me back to facing the issue I reported here.
I'm looking at /var/run/mogodb at the moment though - so that exists in my current install, although I can't figure if it got there because of the install that failed due to the high version number of puppet.
Updated by Duncan Innes about 9 years ago
OK - after getting a bit misdirected by the arrival of 2.2 into the nightlies, I've amended my OS install to fetch the Katello 2.1 repo RPM:
The install guide needs to be updated to reflect this.
Running another install now to see what the result is.
D
Updated by Duncan Innes about 9 years ago
I'm still getting a failure at the mongodb point:
[root@foreman ~]# katello-installer
Could not start Service[mongodb]: Execution of '/usr/share/katello-installer/modules/service_wait/bin/service-wait mongod start' returned 1: Redirecting to /bin/systemctl start mongod.service
/Stage[main]/Mongodb::Server::Service/Service[mongodb]/ensure: change from stopped to running failed: Could not start Service[mongodb]: Execution of '/usr/share/katello-installer/modules/service_wait/bin/service-wait mongod start' returned 1: Redirecting to /bin/systemctl start mongod.service
/Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[foreman.rhel7.local]: Failed to call refresh: Could not load data from https://foreman.rhel7.local
/Stage[main]/Foreman_proxy::Register/Foreman_smartproxy[foreman.rhel7.local]: Could not load data from https://foreman.rhel7.local
Installing Done [100%] [........................................]
Something went wrong! Check the log for ERROR-level output
The full log is at /var/log/katello-installer/katello-installer.log
[root@foreman ~]#
With attempts to start mongodb producing:
[root@foreman mongodb]# systemctl restart mongod.service
Job for mongod.service failed. See 'systemctl status mongod.service' and 'journalctl -xn' for details.
[root@foreman mongodb]# systemctl status mongod.service
mongod.service - High-performance, schema-free document-oriented database
Loaded: loaded (/usr/lib/systemd/system/mongod.service; disabled)
Active: failed (Result: exit-code) since Mon 2015-02-02 16:39:24 GMT; 5s ago
Process: 8612 ExecStart=/usr/bin/mongod $OPTIONS run (code=exited, status=100)
Feb 02 16:39:24 foreman.rhel7.local systemd1: Starting High-performance, schema-free document-oriented database...
Feb 02 16:39:24 foreman.rhel7.local mongod8612: about to fork child process, waiting until server is ready for con...ions.
Feb 02 16:39:24 foreman.rhel7.local mongod8612: forked process: 8614
Feb 02 16:39:24 foreman.rhel7.local mongod8612: ERROR: child process failed, exited with error number 100
Feb 02 16:39:24 foreman.rhel7.local systemd1: mongod.service: control process exited, code=exited status=100
Feb 02 16:39:24 foreman.rhel7.local systemd1: Failed to start High-performance, schema-free document-oriented database.
Feb 02 16:39:24 foreman.rhel7.local systemd1: Unit mongod.service entered failed state.
Hint: Some lines were ellipsized, use -l to show in full.
[root@foreman mongodb]# rpm -qa | grep mongod
mongodb-server-2.6.5-2.el7.x86_64
mongodb-2.6.5-2.el7.x86_64
[root@foreman mongodb]# ls -laFs /var/run/mongodb/
total 0
0 drwxr-xr-x. 2 mongodb root 40 Feb 2 16:39 ./
0 drwxr-xr-x. 33 root root 1060 Feb 2 16:28 ../
[root@foreman mongodb]#
Updated by Duncan Innes about 9 years ago
Have uploaded: foreman-debug-64zYP.tar.xz to the debug-incoming folder.
Updated by Duncan Innes about 9 years ago
Have just rebuilt test server from scratch and tried to enable/start mongod.service before any attempt at katello-install:
[root@foreman ~]# rpm -qa | grep mongo
mongodb-server-2.6.5-2.el7.x86_64
python-pymongo-2.5.2-2.el7.x86_64
mongodb-2.6.5-2.el7.x86_64
[root@foreman ~]# systemctl status mongod.service
mongod.service - High-performance, schema-free document-oriented database
Loaded: loaded (/usr/lib/systemd/system/mongod.service; disabled)
Active: inactive (dead)
[root@foreman ~]# systemctl enable mongod.service
ln -s '/usr/lib/systemd/system/mongod.service' '/etc/systemd/system/multi-user.target.wants/mongod.service'
[root@foreman ~]# systemctl status mongod.service
mongod.service - High-performance, schema-free document-oriented database
Loaded: loaded (/usr/lib/systemd/system/mongod.service; enabled)
Active: inactive (dead)
[root@foreman ~]# systemctl start mongod.service
Job for mongod.service failed. See 'systemctl status mongod.service' and 'journalctl -xn' for details.
[root@foreman ~]# systemctl status mongod.service
mongod.service - High-performance, schema-free document-oriented database
Loaded: loaded (/usr/lib/systemd/system/mongod.service; enabled)
Active: failed (Result: exit-code) since Tue 2015-02-03 10:43:49 GMT; 4s ago
Process: 1147 ExecStart=/usr/bin/mongod $OPTIONS run (code=exited, status=100)
Feb 03 10:43:49 foreman.rhel7.local mongod1147: about to fork child process, waiting until server is ready for connections.
Feb 03 10:43:49 foreman.rhel7.local mongod1147: forked process: 1149
Feb 03 10:43:49 foreman.rhel7.local mongod1147: ERROR: child process failed, exited with error number 100
Feb 03 10:43:49 foreman.rhel7.local systemd1: mongod.service: control process exited, code=exited status=100
Feb 03 10:43:49 foreman.rhel7.local systemd1: Failed to start High-performance, schema-free document-oriented database.
Feb 03 10:43:49 foreman.rhel7.local systemd1: Unit mongod.service entered failed state.
[root@foreman ~]#
Is this an issue with mongod rather than Katello?
Updated by Duncan Innes about 9 years ago
Looks like it's me being a bit dense and the Hardware Requirements perhaps needing a bit of clarification.
Reason: We split the disk into various partitions:
[root@foreman ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_root-lv_root 20G 2.2G 18G 11% /
devtmpfs 3.9G 0 3.9G 0% /dev
tmpfs 3.9G 0 3.9G 0% /dev/shm
tmpfs 3.9G 8.4M 3.9G 1% /run
tmpfs 3.9G 0 3.9G 0% /sys/fs/cgroup
/dev/mapper/vg_root-lv_tmp 1014M 33M 982M 4% /tmp
/dev/mapper/vg_root-lv_home 1014M 33M 982M 4% /home
/dev/mapper/vg_root-lv_var 2.0G 245M 1.8G 13% /var
/dev/mapper/vg_root-lv_pulp 32G 33M 32G 1% /var/lib/pulp
/dev/vda2 509M 104M 405M 21% /boot
[root@foreman ~]#
Which seems to cover the 30Gb for /var/lib/pulp and the 20Gb for root that is called for. We run into problems here though because the /var provides too little space for /var/lib/mongodb (which I've now rectified)
I now run into http startup problems, but I'll log those elsewhere.
Do you want me to close this an open a new one asking for clarification in the Hardware Requirements section of the Install notes? Or deal with that in this ticket?
Updated by Justin Sherrill about 9 years ago
- Category changed from Installer to Documentation
- Difficulty set to easy
ahh, we can deal with it in this issue. Changing category and title.
Updated by Justin Sherrill about 9 years ago
- Subject changed from katello-installer: mongod service failing to Document space requirement for mongodb
Updated by Stephen Benjamin about 9 years ago
- Status changed from New to Closed
- Pull request https://github.com/Katello/katello.org/pull/113 added
- Pull request deleted (
)