Project

General

Profile

Troubleshooting » History » Version 85

Dominic Cleal, 06/11/2015 04:02 AM
installer logging parameters

1 3 Ohad Levy
{{toc}}
2 3 Ohad Levy
3 43 Lukas Zapletal
h1. How to provide enough info to get help?
4 43 Lukas Zapletal
5 43 Lukas Zapletal
Foreman is pretty complex software project with many components and extensive configuration. Always try to describe best your specific case (OS, version, ruby and foreman version, database, specific configuration). From Foreman version 1.2 you can use _foreman-debug_ script which collects all these information and also filters out things like passwords or tokens. For earlier Foreman releases you can use this bash script directly from git: https://github.com/theforeman/foreman/blob/develop/script/foreman-debug
6 43 Lukas Zapletal
7 67 Dominic Cleal
h1. ERF error code
8 67 Dominic Cleal
9 67 Dominic Cleal
If you see an ERF error code, such as ERF12-7740, look it up on [[ErrorCodes]].
10 67 Dominic Cleal
11 1 Ohad Levy
h1. rake aborted!
12 1 Ohad Levy
13 1 Ohad Levy
I get the following error while running rake db:migrate
14 1 Ohad Levy
<pre>
15 1 Ohad Levy
rake aborted!
16 1 Ohad Levy
undefined method `reenable' for <Rake::Task db:schema:dump => [environment]>:Rake::Task
17 1 Ohad Levy
</pre>
18 1 Ohad Levy
19 1 Ohad Levy
this means you have an old version of rake (older than 0.87), you can verify it by typing:
20 1 Ohad Levy
<pre>
21 1 Ohad Levy
rake --version
22 1 Ohad Levy
</pre>
23 1 Ohad Levy
24 1 Ohad Levy
Usually this error is "safe" to ignore, however, it is recommended to upgrade to rake >= 0.87
25 2 Ohad Levy
26 10 Corey Osman
27 20 Florian Koch
h1. Unable to save: failed to detect boot server: getaddrinfo: Name or service not known
28 1 Ohad Levy
29 20 Florian Koch
if you have registred your smart-proxys via name, and these name are not resolvable, you get this error, you can add the name to your dns server or set 
30 20 Florian Koch
:tftp_servername: in smart-proxy settings to the ip of the smart-proxy.
31 10 Corey Osman
32 7 Corey Osman
h1. I use puppet enterprise, what now?
33 7 Corey Osman
34 7 Corey Osman
As of PE version 1.0, foreman won't install easily without a few modifications.  Since PE ships with its own version of Ruby you will need to install a few gems.
35 12 Corey Osman
Basically you need to tell foreman to use the PE provided ruby library instead.  Your not required to run the puppet version of ruby either as many distros now ship with ruby 1.8.7.
36 7 Corey Osman
37 8 Ohad Levy
# You first need to install the pe-ruby-devel package that ships with PE.  You can find this in your packages folder wherever you extracted PE.
38 1 Ohad Levy
      a. rpm -ivh pe-ruby-devel-1.8.7.302-1.el5.i386.rpm 
39 8 Ohad Levy
# Update your path variable to use the ruby version PE ships with (export PATH=/opt/puppet/bin:$PATH)
40 8 Ohad Levy
# Check to make sure your PATH was updated ("which gem" should return /opt/puppet/bin/gem)
41 8 Ohad Levy
# gem install json
42 8 Ohad Levy
# gem install rest-client
43 7 Corey Osman
(install the gem database driver you use)
44 8 Ohad Levy
# gem install sqlite3 or gem install mysql  ( you may have additional dependencies that would need to be resolved first)
45 7 Corey Osman
46 7 Corey Osman
Some of the gem dependencies can be resolved by installing the development library for that package (example: mysql-devel)
47 7 Corey Osman
48 7 Corey Osman
I found it difficult to install the sqlite3 gem because of dependency hell so I just setup mysql instead which was far easier to setup.  Additionally mysql is much more scalable than sqlite.
49 7 Corey Osman
50 7 Corey Osman
You can get a list of the currently installed gems by using: gem list.
51 1 Ohad Levy
52 10 Corey Osman
If your running the foreman init script or /etc/sysconfig/foreman configuration file.  You may need to update apache/passenger configurations that point to the PE ruby version.
53 6 Corey Osman
54 6 Corey Osman
55 2 Ohad Levy
h1. error: Entry 'some file' not uptodate. Cannot merge.
56 2 Ohad Levy
57 2 Ohad Levy
If you downloaded Foreman from source (git), it could be that some of the files you have modified (or were modified automatically) conflicts with another file Foreman wants to upgrade.
58 2 Ohad Levy
59 2 Ohad Levy
If you don't think that you edited this file (e.g. db/schema.rb) it is usually safe to do:
60 2 Ohad Levy
<pre>
61 2 Ohad Levy
git checkout 'some file'
62 2 Ohad Levy
</pre> 
63 2 Ohad Levy
64 2 Ohad Levy
This will revert the file to its previous condition (as in the repo at the time of checkout) and now you should be able to get the latest version by:
65 2 Ohad Levy
<pre>
66 2 Ohad Levy
git pull
67 2 Ohad Levy
</pre>
68 3 Ohad Levy
69 3 Ohad Levy
h1. Strange errors with passenger
70 3 Ohad Levy
71 3 Ohad Levy
Passenger executes foreman, based on the owner of the config/environmnets.rb file, make sure that this use can actually access:
72 3 Ohad Levy
* puppet configuration files
73 3 Ohad Levy
* sqlite database (if using sqlite)
74 3 Ohad Levy
* write to log, tmp directories
75 3 Ohad Levy
76 3 Ohad Levy
see source:extras/puppet/foreman for complete puppet based setup for Foreman.
77 4 Bash Shell
78 4 Bash Shell
79 4 Bash Shell
80 4 Bash Shell
h1. error: Could not send facts to Foreman: wrong Content-Length format (RuntimeError)
81 4 Bash Shell
82 4 Bash Shell
You might see this error if you run the HTTP push facts script or the sample external nodes script.
83 4 Bash Shell
This is most likely due to older version of the mongrel gem. Please try and update your gems.
84 4 Bash Shell
85 4 Bash Shell
<pre>
86 4 Bash Shell
gem update mongrel
87 4 Bash Shell
</pre>
88 5 Lukas Zapletal
89 5 Lukas Zapletal
h1. Is my Foreman instance running?
90 5 Lukas Zapletal
91 5 Lukas Zapletal
There is simple status service that returns JSON with "result" message "ok" when the instance is up and running. It also "pings" database and returns lag. Example:
92 5 Lukas Zapletal
93 5 Lukas Zapletal
<pre>
94 5 Lukas Zapletal
$ curl -k -H "Accept: application/json" http://instance:3000/status
95 5 Lukas Zapletal
{"status":200,"result":"ok","db_duration_ms":"5"}
96 5 Lukas Zapletal
</pre>
97 9 Dis Connect
98 9 Dis Connect
h1. How do I enable debugging?
99 9 Dis Connect
100 79 Dominic Cleal
h2. On Foreman 1.8 or older
101 79 Dominic Cleal
102 72 Ashton Davis
Edit ~foreman/config/environments/production.rb and uncomment this line:
103 71 Ashton Davis
<pre>
104 71 Ashton Davis
config.log_level = :debug
105 71 Ashton Davis
</pre>
106 1 Ohad Levy
107 72 Ashton Davis
And reload the config (nicely):
108 72 Ashton Davis
<pre>
109 72 Ashton Davis
touch ~foreman/tmp/restart.txt
110 72 Ashton Davis
</pre>
111 79 Dominic Cleal
112 79 Dominic Cleal
h2. On Foreman 1.9 or newer
113 79 Dominic Cleal
114 79 Dominic Cleal
Edit /etc/foreman/settings.yaml and either uncomment or add these lines:
115 79 Dominic Cleal
116 79 Dominic Cleal
<pre>
117 79 Dominic Cleal
:logging:
118 79 Dominic Cleal
  :level: debug
119 79 Dominic Cleal
</pre>
120 79 Dominic Cleal
121 79 Dominic Cleal
And reload the config (nicely):
122 79 Dominic Cleal
<pre>
123 79 Dominic Cleal
touch ~foreman/tmp/restart.txt
124 79 Dominic Cleal
</pre>
125 79 Dominic Cleal
126 80 Dominic Cleal
h3. Enabling more specific logs
127 80 Dominic Cleal
128 80 Dominic Cleal
More types of log messages can be enabled from settings.yaml:
129 80 Dominic Cleal
130 80 Dominic Cleal
* @app@ - web requests and all general application logs (default: true)
131 83 Dominic Cleal
* @ldap@ - high level LDAP queries (e.g. find users in group) and LDAP operations performed (default: false)
132 84 Dominic Cleal
* @permissions@ - evaluation of user roles, filters and permissions when loading pages (default: false)
133 81 Dominic Cleal
* @sql@ - SQL queries made through Rails ActiveRecord, only debug (default: true)
134 80 Dominic Cleal
135 80 Dominic Cleal
Uncomment or add a @:loggers@ block to enable or disable loggers:
136 80 Dominic Cleal
137 80 Dominic Cleal
<pre>
138 80 Dominic Cleal
:loggers:
139 80 Dominic Cleal
  :app:
140 80 Dominic Cleal
    :enabled: true
141 83 Dominic Cleal
  :ldap:
142 84 Dominic Cleal
    :enabled: false
143 84 Dominic Cleal
  :permissions:
144 83 Dominic Cleal
    :enabled: false
145 80 Dominic Cleal
  :sql:
146 80 Dominic Cleal
    :enabled: true
147 80 Dominic Cleal
</pre>
148 80 Dominic Cleal
149 82 Dominic Cleal
Some plugins may add their own loggers.  See the configuration files in @/etc/foreman/plugins/@ which should list possibilities.
150 82 Dominic Cleal
151 85 Dominic Cleal
h3. From foreman-installer
152 85 Dominic Cleal
153 85 Dominic Cleal
When using Foreman installer or "our installer modules":https://github.com/theforeman/puppet-foreman, you can specify the following parameters to change the logging config:
154 85 Dominic Cleal
155 85 Dominic Cleal
* @foreman-installer --foreman-logging-level=debug@ or set @foreman::logging_level@ to @debug@ etc.
156 85 Dominic Cleal
* @foreman-installer --foreman-loggers=ldap:true@ or set @foreman::loggers@ to a hash, e.g. @{"ldap" => true}@
157 85 Dominic Cleal
158 79 Dominic Cleal
h2. Compute resource debugging
159 71 Ashton Davis
160 72 Ashton Davis
*Do not run this command unless specifically told to do so.*
161 71 Ashton Davis
To enable debugging of Compute Resources HTTP requests, you should pass some environment variables:
162 71 Ashton Davis
<pre>
163 71 Ashton Davis
  cd ~foreman
164 71 Ashton Davis
  EXCON_DEBUG=true DEBUG=true ./script/rails s
165 71 Ashton Davis
</pre>
166 13 Corey Osman
167 78 Dominic Cleal
h1. How do I enable smart proxy debugging?
168 78 Dominic Cleal
169 78 Dominic Cleal
Edit /etc/foreman-proxy/settings.yml and change or uncomment the log_level parameter, setting it to DEBUG:
170 78 Dominic Cleal
171 78 Dominic Cleal
    :log_level: DEBUG
172 78 Dominic Cleal
173 78 Dominic Cleal
Then restart the service:
174 78 Dominic Cleal
175 78 Dominic Cleal
    service foreman-proxy restart
176 78 Dominic Cleal
177 78 Dominic Cleal
Logs will go to /var/log/foreman-proxy/proxy.log and access.log by default.
178 78 Dominic Cleal
179 78 Dominic Cleal
If the service is crashing immediately, failing to start up or continually returns 500 Internal Server error, start it up in foreground mode.  To do this, disable the daemonize parameter in /etc/foreman-proxy/settings.yml:
180 78 Dominic Cleal
181 78 Dominic Cleal
    :daemonize: false
182 78 Dominic Cleal
183 78 Dominic Cleal
And then run the smart proxy process:
184 78 Dominic Cleal
185 78 Dominic Cleal
    sudo -u foreman-proxy /usr/share/foreman-proxy/bin/smart-proxy
186 78 Dominic Cleal
187 78 Dominic Cleal
Make the request from Foreman again, and logging should be shown on the console.  Don't forget to change daemonize back to true before starting the service.
188 78 Dominic Cleal
189 13 Corey Osman
h1. [[Unattended Provisioning Troubleshooting]]
190 14 Florian Koch
191 14 Florian Koch
h1. Puppet and Storeconfigs: err: Could not retrieve catalog from remote server: Error 400 on SERVER: stack level too deep
192 14 Florian Koch
193 14 Florian Koch
Your activerecord Version is to new, you need activrecord 3.0.10 see http://projects.puppetlabs.com/issues/9290
194 14 Florian Koch
195 14 Florian Koch
<pre>
196 14 Florian Koch
gem uninstall activerecord
197 14 Florian Koch
gem install activerecord -v 3.0.10
198 14 Florian Koch
</pre>
199 15 Romain Vrignaud
200 15 Romain Vrignaud
201 15 Romain Vrignaud
202 15 Romain Vrignaud
h1. How do I fix this error: "Unable to find directory $confdir/environments/$environment/modules in environment master" ?
203 15 Romain Vrignaud
204 15 Romain Vrignaud
If you've followed http://docs.puppetlabs.com/guides/environment.html#configuring-environments-on-the-puppet-master to 
205 15 Romain Vrignaud
create a multi-environment puppet configuration, you're initial rake may look something like this:
206 15 Romain Vrignaud
<pre>
207 15 Romain Vrignaud
rake puppet:import:puppet_classes RAILS_ENV=production
208 15 Romain Vrignaud
(in /usr/share/foreman)
209 15 Romain Vrignaud
Evaluating possible changes to your installation
210 15 Romain Vrignaud
Problems were detected during the evaluation phase
211 15 Romain Vrignaud
212 15 Romain Vrignaud
Unable to find directory $confdir/environments/$environment/modules in environment master
213 15 Romain Vrignaud
214 15 Romain Vrignaud
Please fix these issues and try again
215 15 Romain Vrignaud
</pre>
216 15 Romain Vrignaud
This is because Foreman does not currently evaluate $confdir.  There is a feature request open on this issue.  In the 
217 15 Romain Vrignaud
meantime, hard code your $confdir in your puppet.conf.
218 15 Romain Vrignaud
219 21 Romain Vrignaud
h1. Smart-proxy do not recognize my puppet environment
220 23 Romain Vrignaud
221 21 Romain Vrignaud
If I query smart-proxy and it return empty puppet environment :
222 22 Romain Vrignaud
<pre>
223 21 Romain Vrignaud
curl -k -H "Content-Type:application/json" -H "Accept:application/json" http://puppet:8443/puppet/environments
224 21 Romain Vrignaud
=> []
225 22 Romain Vrignaud
</pre>
226 21 Romain Vrignaud
227 21 Romain Vrignaud
You may have to add in your puppetmaster's puppet.conf environment definition like :
228 21 Romain Vrignaud
229 22 Romain Vrignaud
<pre>
230 21 Romain Vrignaud
[production]
231 21 Romain Vrignaud
    manifest   = /etc/puppet/manifests/site.pp
232 21 Romain Vrignaud
    modulepath = /etc/puppet/modules
233 21 Romain Vrignaud
    fileserverconfig = /etc/puppet/fileserver.conf
234 21 Romain Vrignaud
    templatedir=/etc/puppet/templates/
235 21 Romain Vrignaud
236 21 Romain Vrignaud
[preprod]
237 21 Romain Vrignaud
    manifest   = /etc/puppet/preprod/manifests/site.pp
238 21 Romain Vrignaud
    modulepath = /etc/puppet/preprod/modules
239 21 Romain Vrignaud
    fileserverconfig = /etc/puppet/preprod/fileserver.conf
240 21 Romain Vrignaud
    templatedir=/etc/puppet/preprod/templates
241 21 Romain Vrignaud
242 21 Romain Vrignaud
[development]
243 21 Romain Vrignaud
    manifest   = /etc/puppet/development/manifests/site.pp
244 21 Romain Vrignaud
    modulepath = /etc/puppet/development/modules
245 21 Romain Vrignaud
    fileserverconfig = /etc/puppet/development/fileserver.conf
246 21 Romain Vrignaud
    templatedir=/etc/puppet/development/templates/
247 21 Romain Vrignaud
248 21 Romain Vrignaud
[test]
249 21 Romain Vrignaud
    manifest   = /etc/puppet/test/manifests/site.pp
250 21 Romain Vrignaud
    modulepath = /etc/puppet/test/modules
251 21 Romain Vrignaud
    fileserverconfig = /etc/puppet/test/fileserver.conf
252 21 Romain Vrignaud
    templatedir=/etc/puppet/test/templates/
253 22 Romain Vrignaud
</pre>
254 21 Romain Vrignaud
255 21 Romain Vrignaud
256 15 Romain Vrignaud
h1. How do I fix the puppet class import problem with multiple puppet environment ?
257 15 Romain Vrignaud
258 15 Romain Vrignaud
If Foreman don't seems to be aware of all your environment and is able to import only your "production" environment (see http://theforeman.org/issues/1159),
259 34 Christoph Witzany
try to add ":puppet_conf: /etc/puppet/puppet.conf" in your settings.yaml.
260 17 Martin Hovmöller
261 17 Martin Hovmöller
h1. Unable to connect to Hypervisor?
262 18 Martin Hovmöller
263 74 Neil Katin
Make sure the user that's actually running foreman can connect to your remote hypervisor (for instance by running sudo -u foreman virsh -c qemu+ssh://root@<host>/system list).
264 19 Martin Hovmöller
265 17 Martin Hovmöller
If you are running foreman through phusion passenger, ps auxwwf | grep "R[a]ils: /usr/share/foreman" | awk '{ print $1 }' will give you the user that's running foreman. If you find that it's not being run by foreman, do the following:
266 17 Martin Hovmöller
cd ~foreman
267 17 Martin Hovmöller
chown foreman config/environment.rb
268 17 Martin Hovmöller
touch ~foreman/tmp/restart.txt
269 24 chris alfonso
270 24 chris alfonso
h1. Routing errors when running rake test?
271 24 chris alfonso
272 24 chris alfonso
Edit the config/settings.yaml and set the :login: setting to true
273 25 Ohad Levy
274 25 Ohad Levy
h1. cannot load such file -- facter / puppet (LoadError)
275 25 Ohad Levy
276 52 Dominic Cleal
If you running foreman under RVM or custom ruby installation, you would need to get facter installed as a gem (as facter packages are not in your ruby search path).
277 25 Ohad Levy
278 1 Ohad Levy
simply add another file to FOREMAN_DIR/bundler.d
279 52 Dominic Cleal
<pre>
280 52 Dominic Cleal
echo "gem 'facter'" > bundler.d/Gemfile.local.rb
281 52 Dominic Cleal
</pre>
282 52 Dominic Cleal
283 52 Dominic Cleal
h2. Pre-Foreman 1.3
284 52 Dominic Cleal
285 52 Dominic Cleal
Prior to 1.3, puppet was also a dependency, so instead use:
286 52 Dominic Cleal
287 25 Ohad Levy
<pre>
288 37 Dominic Cleal
echo "gem 'puppet'" > bundler.d/Gemfile.local.rb
289 1 Ohad Levy
</pre>
290 37 Dominic Cleal
291 37 Dominic Cleal
Edit the config/settings.yaml and add: @:puppetgem: true@
292 28 Bash Shell
293 28 Bash Shell
h1. Causes of "Error 400 Bad Request", and other smart-proxy related errors in the Foreman interface:
294 28 Bash Shell
295 29 Bash Shell
* Check the sudoers file on the proxy, if the user "foreman-proxy" can run "puppetca". The command puppetca is un-available in puppet 3.0. Workaround is to create a wrapper script.
296 29 Bash Shell
* From the cmd line, check if the user can run "puppetca" and "nsupdate" properly.
297 28 Bash Shell
* Check if Bind is listening on 127.0.0.1. The proxy connects to localhost only, so this is required. 
298 28 Bash Shell
* Check if the foreman-proxy user "foreman-proxy" can read the Bind rndc keys.
299 29 Bash Shell
* In Ubuntu, you will have to tell apparmor to allow Bind to write to zone files and journals. If your zone files are in /etc/bind/zones/, add "/etc/bind/zones/** rw," to /etc/apparmor.d/usr.sbin.named.
300 28 Bash Shell
* If using Ubuntu Libvirt, and the "Virtual Machine" tab is empty, then you most likely need to create a default storage pool:
301 28 Bash Shell
<pre>
302 28 Bash Shell
cat /etc/libvirt/storage/default.xml 
303 28 Bash Shell
<pool type='dir'>
304 28 Bash Shell
  <name>default</name>
305 28 Bash Shell
  <uuid>35e0bbce-4019-ca88-6dec-b1a0c1be774b</uuid>
306 28 Bash Shell
  <capacity>0</capacity>
307 28 Bash Shell
  <allocation>0</allocation>
308 28 Bash Shell
  <available>0</available>
309 28 Bash Shell
  <source>
310 28 Bash Shell
  </source>
311 28 Bash Shell
  <target>
312 28 Bash Shell
    <path>/var/lib/libvirt/images</path>
313 28 Bash Shell
    <permissions>
314 28 Bash Shell
      <mode>0700</mode>
315 28 Bash Shell
      <owner>-1</owner>
316 28 Bash Shell
      <group>-1</group>
317 28 Bash Shell
    </permissions>
318 28 Bash Shell
  </target>
319 28 Bash Shell
</pool>
320 28 Bash Shell
</pre>
321 28 Bash Shell
* On Ubuntu Libvirt, you may have to change /etc/libvirt/libvirtd.conf to listen on TCP.
322 31 Benjamin Papillon
323 31 Benjamin Papillon
h1. Using Webrick you get : Error 400 on SERVER: Could not find node '<node fqdn>'; cannot compile
324 31 Benjamin Papillon
325 32 Benjamin Papillon
If you are using Foreman with webrick and you get an error downloading your catalog, maybe you encouter the issue noted in bug #1507
326 31 Benjamin Papillon
Edit the node.rb script and replace the following :
327 31 Benjamin Papillon
328 31 Benjamin Papillon
<pre>
329 31 Benjamin Papillon
<br />SETTINGS = {<br /> :url => "http://<node fqdn>:3000*/*",<br />
330 31 Benjamin Papillon
</pre>
331 31 Benjamin Papillon
332 31 Benjamin Papillon
by this :
333 31 Benjamin Papillon
334 31 Benjamin Papillon
<pre>
335 31 Benjamin Papillon
<br />SETTINGS = {<br /> :url => "http://<node fqdn>:3000",<br />
336 31 Benjamin Papillon
</pre>
337 35 Charles Gomes
338 35 Charles Gomes
h1. DHCP Provisioning Fails with: "dhcpctl_connect: not found"
339 35 Charles Gomes
340 35 Charles Gomes
Depending on the version of ISC DHCP you are using a wrong key will return "dhcpctl_connect: not found" , this misleads to a connection problem.
341 35 Charles Gomes
To solve basically check if your KEY NAME matches your proxy settings.yml and also matches on your dhcpd.conf
342 38 Charles Gomes
343 38 Charles Gomes
h1. Foreman is showing : Error message: Could not find json-1.5.5 in any of the sources
344 38 Charles Gomes
345 38 Charles Gomes
If you have a problem after doing a yum update ruby bundler needs to re-run.
346 38 Charles Gomes
This could happen after an update or chane of ruby packages.
347 39 Charles Gomes
348 39 Charles Gomes
<pre>
349 38 Charles Gomes
rm ~foreman/Gemfile.lock
350 1 Ohad Levy
cd ~foreman 
351 38 Charles Gomes
bundle install --local
352 39 Charles Gomes
</pre>
353 40 Greg Sutcliffe
354 40 Greg Sutcliffe
h1. VNC console in the Foreman UI refuses to connect to my VM: Server Disconnected
355 40 Greg Sutcliffe
356 42 Greg Sutcliffe
Check what version of python you have installed - noVNC requires python2. If you have multiple python versions installed, you can edit `extras/noVNC/websockify.py` and change the shebang on line 1
357 44 Dominic Cleal
358 44 Dominic Cleal
h1. Unable to upgrade to Puppet 3.2 due to rubygem-net-ldap dependency issues
359 44 Dominic Cleal
360 44 Dominic Cleal
There are conflicting version requirements with rubygem-net-ldap between the Foreman and Puppet Labs dependency repos since Puppet 3.2.  Since Foreman 1.1, the rubygem-net-ping package is no longer required for the proxy, so first remove this and you should either be able to upgrade or remove rubygem-net-ldap altogether if there's no dependency any more.
361 45 Dominic Cleal
362 45 Dominic Cleal
h1. How do I change the FQDN of the Foreman host?
363 45 Dominic Cleal
364 45 Dominic Cleal
# Some settings in the Foreman UI use the hostname, update under More, Settings, e.g.
365 45 Dominic Cleal
#* General, foreman_url
366 45 Dominic Cleal
#* Provisioning, ssl_certificate and ssl_priv_key
367 45 Dominic Cleal
# The registered smart proxy URL if you've installed it, this can be edited via More>Configuration>Smart Proxies
368 45 Dominic Cleal
# Puppet SSL certs: generate new ones with @puppet cert generate FQDN@
369 45 Dominic Cleal
# Apache configs: update conf.d/{foreman,puppet}.conf with new SSL cert filenames, ServerName and VirtualHost IP addresses if applicable
370 45 Dominic Cleal
# Smart proxy configuration: update SSL cert filenames
371 46 Dominic Cleal
# URLs in the puppetmaster's ENC script (/etc/puppet/node.rb) and report script (foreman.rb)
372 47 Stephen Hoekstra
373 47 Stephen Hoekstra
h1. My deleted host keeps reappearing in the Hosts tab
374 47 Stephen Hoekstra
375 47 Stephen Hoekstra
After a period of time a deleted host reappears in the Host tab although the host no longer exists.  A potential reason for this could be the --push-facts cron job is pushing the facts to the Foreman and the Foreman expects the host to check in but never does.
376 47 Stephen Hoekstra
377 47 Stephen Hoekstra
To fix:
378 47 Stephen Hoekstra
<pre>
379 47 Stephen Hoekstra
rm /var/lib/puppet/yaml/facts/$hostname.yaml
380 47 Stephen Hoekstra
rm /var/lib/puppet/yaml/node/$hostname.yaml
381 47 Stephen Hoekstra
</pre>
382 48 Samarendra M Hedaoo
383 48 Samarendra M Hedaoo
h1. My node's environment is being reset to 'X' even though the puppet.conf on the host has environment = 'Y'. The Foreman settings have default_puppet_environment set as 'X' and enc_environment as False. I am using Puppet 3 on the node.
384 48 Samarendra M Hedaoo
385 48 Samarendra M Hedaoo
Foreman expects the node to specify it's own environment, if enc_environment is False. If no environment is returned by the node, it will override it with the value of default_puppet_environment. This was good till Puppet 3. But Puppet 3 no longer returns environment as a fact and hence Foreman assumes that the environment is not set.
386 49 Dominic Cleal
387 49 Dominic Cleal
h1. No A and/or PTR records are created in DNS for new hosts
388 49 Dominic Cleal
389 49 Dominic Cleal
First ensure you have a smart proxy registered with the DNS feature, with a DNS server installed and configured for the appropriate forward and reverse DNS zones.
390 49 Dominic Cleal
391 49 Dominic Cleal
In Foreman, under More, Provisioning, Domains, edit your domain, change the DNS proxy setting to the appropriate proxy server.  This will enable A records to be created for hosts built in that domain.
392 49 Dominic Cleal
393 49 Dominic Cleal
Next, under More, Provisioning, Subnets, edit your subnet, change the DNS proxy to the appropriate proxy for the reverse DNS zone.  This will cause PTR records to be added for hosts with NICs in that subnet.
394 49 Dominic Cleal
395 49 Dominic Cleal
Both parts are needed for a complete DNS setup.
396 49 Dominic Cleal
397 49 Dominic Cleal
h1. No TFTP menus or files are created for new hosts
398 49 Dominic Cleal
399 49 Dominic Cleal
This requires a proxy server registered with the TFTP feature, and a TFTP daemon running on it.  Foreman will write directly to the TFTP root directory, as configured in the proxy settings.  Do ensure the "tftp_servername" setting is also correct, this is the IP address that gets passed to the DHCP server for the @next-server@ option.
400 49 Dominic Cleal
401 49 Dominic Cleal
In Foreman, under More, Provisioning, Subnets, edit your subnet, change the TFTP proxy to the appropriate proxy server for the subnet.  This will cause hosts built in this subnet to receive a PXELinux menu for their primary MAC address.
402 50 Dominic Cleal
403 50 Dominic Cleal
h1. After Foreman 1.3 upgrade, reports and ENC fact uploads no longer work
404 50 Dominic Cleal
405 51 Lukas Zapletal
We changed how the fact and report uploads work in the 1.3 release. During 1.3 upgrade, read "release notes":http://theforeman.org/manuals/1.3/index.html#Upgradenotes and update both the report processor (foreman.rb) and ENC script (node.rb), otherwise errors will be logged in the production.log (return code 400):
406 51 Lukas Zapletal
    
407 51 Lukas Zapletal
    DEPRECATION: /reports/create is deprecated, update your report processor to POST to /api/reports
408 51 Lukas Zapletal
    See the Foreman 1.3 release notes for a new example report processor
409 51 Lukas Zapletal
410 51 Lukas Zapletal
In the release candidates, the error was:
411 50 Dominic Cleal
412 50 Dominic Cleal
    ActionController::RoutingError (No route matches [POST] "/fact_values/create")
413 50 Dominic Cleal
    ActionController::RoutingError (No route matches [POST] "/reports/create")
414 53 Lukas Zapletal
415 53 Lukas Zapletal
h1. Unprocessable Entity error during installation
416 53 Lukas Zapletal
417 53 Lukas Zapletal
Foreman installer registers proxy which is being deployed automatically. If it is, for any reason, already present but under different name, Foreman reject to register the proxy with HTTP 422 error. Proxy registration can be either turned off, or name can be changed using @--foreman-proxy-register-in-foreman@ or @--foreman-proxy-registered-name@ installer options.
418 54 Dominic Cleal
419 54 Dominic Cleal
h1. Installation errors on Puppet 3.4.x or 2.7.24
420 54 Dominic Cleal
421 54 Dominic Cleal
When using Puppet 3.4.0 or 3.4.1, installation errors will be seen and running "puppet --version" after install will show the following:
422 54 Dominic Cleal
423 54 Dominic Cleal
    Could not intialize global default settings: undefined method `mode=' for #<Puppet::Settings::AutosignSetting:0x7fa026ad44e0>
424 54 Dominic Cleal
425 58 Dominic Cleal
This is caused by Puppet bug "PUP-1015":https://tickets.puppetlabs.com/browse/PUP-1015.  On 3.4.1 and 2.7.24, additionally the /etc/puppet/manifests/site.pp file will be set to mode 0600 rather than the default 0644.  This is "PUP-1255":https://tickets.puppetlabs.com/browse/PUP-1255.
426 57 Dominic Cleal
427 58 Dominic Cleal
Puppet 3.4.2 and 2.7.25 fix both of these issues, please ensure you use these versions instead of 3.4.0/1 or 2.7.24.
428 59 Dominic Cleal
429 59 Dominic Cleal
h1. Yum fails to install foreman-installer, Requires: rubygem(rest-client)
430 59 Dominic Cleal
431 59 Dominic Cleal
If you get this error, it indicates the EPEL repository is missing:
432 59 Dominic Cleal
433 59 Dominic Cleal
    Error: Package: rubygem-foreman_api-0.1.9-1.el6.noarch (foreman)
434 59 Dominic Cleal
           Requires: rubygem(rest-client) >= 1.6.1
435 59 Dominic Cleal
436 59 Dominic Cleal
This is a requirement for the installer and Foreman.  Download and install the epel-release RPM from here: http://dl.fedoraproject.org/pub/epel/6/x86_64/repoview/epel-release.html
437 60 Lukas Zapletal
438 60 Lukas Zapletal
h1. Foreman proxy fails to start with Are the values correct in settings.yml and do permissions allow reading?: Permission denied
439 60 Lukas Zapletal
440 60 Lukas Zapletal
If you are using puppet CA see here: http://theforeman.org/manuals/latest/index.html#4.3.7SSL
441 61 Stephen Benjamin
442 77 Tommy McNeely
h1. FreeIPA realm proxy fails with SSLv3 read server certificate B: certificate verify failed
443 77 Tommy McNeely
444 77 Tommy McNeely
If you see error messages like this in proxy.log:
445 77 Tommy McNeely
446 77 Tommy McNeely
    E, [2014-11-21T06:09:59.630126 #5631] ERROR -- : SSL_connect returned=1 errno=0 state=SSLv3 read server certificate B: certificate verify failed
447 77 Tommy McNeely
448 77 Tommy McNeely
The certificate authority (CA) that signed your IPA CA cert is not trusted by the proxy host. This is fairly common if the "foreman proxy" host is not an IPA server. You can trust the CA certificate of the IPA server...
449 77 Tommy McNeely
450 77 Tommy McNeely
    cp /etc/ipa/ca.crt /etc/pki/ca-trust/source/source
451 77 Tommy McNeely
    update-ca-trust
452 77 Tommy McNeely
453 77 Tommy McNeely
454 61 Stephen Benjamin
h1. FreeIPA realm proxy fails with "generic preauthentication failure"
455 61 Stephen Benjamin
456 61 Stephen Benjamin
If you see error messages like this in your proxy logs:
457 61 Stephen Benjamin
 
458 61 Stephen Benjamin
    E, [2014-05-15T19:28:08.211121 #3595] ERROR -- : Failed to initialise credential cache from keytab: krb5_get_init_creds_keytab:  Generic preauthentication failure
459 61 Stephen Benjamin
    E, [2014-05-15T19:28:08.211515 #3595] ERROR -- : Failed to initailize credentials cache from keytab: krb5_get_init_creds_keytab: Generic preauthentication failure
460 61 Stephen Benjamin
    D, [2014-05-15T19:28:08.211614 #3595] DEBUG -- : /usr/share/foreman-proxy/bin/../lib/proxy/kerberos.rb:13:in `init_krb5_ccache'
461 61 Stephen Benjamin
462 76 Tommy McNeely
First, check permissions, the /etc/foreman-proxy/freeipa.keytab file should be owned by foreman-proxy and mode 600.
463 76 Tommy McNeely
464 61 Stephen Benjamin
You may be running a much newer version of FreeIPA than the client which provides some unknown encryption types.
465 61 Stephen Benjamin
466 61 Stephen Benjamin
Run `klist -etk /etc/foreman-proxy/freeipa.keytab` and you'll see some unnamed enryption types like this:
467 61 Stephen Benjamin
468 61 Stephen Benjamin
    1 05/14/14 21:14:17 realm-proxy@EXAMPLE.COM (etype 25)
469 61 Stephen Benjamin
    1 05/14/14 21:14:17 realm-proxy@EXAMPLE.COM (etype 26)
470 61 Stephen Benjamin
471 61 Stephen Benjamin
To fix it, delete /etc/foreman-poxy/freeipa.keytab and refetch it, specifying only the enctypes your system knows about:
472 61 Stephen Benjamin
473 62 Stephen Benjamin
    ipa-getkeytab -s ipa.example.com -p realm-proxy@EXAMPLE.COM -k /etc/foreman-proxy/freeipa.keytab --enctypes=aes256-cts-hmac-sha1-96,aes128-cts-hmac-sha1-96,des3-cbc-sha1,arcfour-hmac
474 62 Stephen Benjamin
475 62 Stephen Benjamin
h1. FreeIPA realm proxy fails with "LoadError: no such file to load -- ffi_c"
476 62 Stephen Benjamin
477 62 Stephen Benjamin
This is due to a broken rubygem-ffi that ships from Puppetlabs: https://jira-web03-dev.puppetlabs.com/browse/CPR-26
478 62 Stephen Benjamin
479 63 Stephen Benjamin
We need the EPEL one (although with the exact same version).
480 62 Stephen Benjamin
481 62 Stephen Benjamin
Ensure you have the latest (rubygem-ffi-1.0.9-11.el6.x86_64):
482 62 Stephen Benjamin
483 62 Stephen Benjamin
    yum upgrade rubygem-ffi
484 62 Stephen Benjamin
485 62 Stephen Benjamin
Now downgrade, but disable the puppetlabs-dependecies repo so we grab the package from EPEL:
486 62 Stephen Benjamin
487 62 Stephen Benjamin
    yum downgrade --disablerepo=puppetlabs-dependencies rubygem-ffi-1.0.9-9.el6
488 62 Stephen Benjamin
489 62 Stephen Benjamin
Restart the proxy.
490 65 Lukas Zapletal
491 65 Lukas Zapletal
h1. How to log REST client calls
492 65 Lukas Zapletal
493 65 Lukas Zapletal
Sometimes it is useful to see content of calls that are being made by Foreman using rest-client library (compute resources, proxy communication etc). This is as easy as dropping the following file and restarting Foreman:
494 65 Lukas Zapletal
495 65 Lukas Zapletal
<pre>
496 65 Lukas Zapletal
$ cat >/usr/share/foreman/config/initializers/00_rest_client.rb <<'EOT'
497 65 Lukas Zapletal
require 'rest_client'
498 65 Lukas Zapletal
RestClient.log =
499 65 Lukas Zapletal
  Object.new.tap do |proxy|
500 65 Lukas Zapletal
    def proxy.<<(message)
501 65 Lukas Zapletal
      Rails.logger.info message
502 65 Lukas Zapletal
    end
503 65 Lukas Zapletal
  end
504 65 Lukas Zapletal
EOT
505 65 Lukas Zapletal
</pre>
506 66 Fergus Nelson
507 66 Fergus Nelson
h1. I'm getting connection timeouts when Foreman tries to connect to ec2 when running behind a http proxy
508 66 Fergus Nelson
509 66 Fergus Nelson
The error can look like this.
510 66 Fergus Nelson
<pre>
511 66 Fergus Nelson
<div id="backtrace" class="alert alert-block alert-danger base in fade hide">
512 66 Fergus Nelson
  <em>Excon::Errors::Timeout</em><br>
513 66 Fergus Nelson
  <strong>connect timeout reached</strong><br>
514 66 Fergus Nelson
  app/models/compute_resources/foreman/model/ec2.rb:59:in `regions'
515 66 Fergus Nelson
<br>app/models/compute_resources/foreman/model/ec2.rb:72:in `test_connection'
516 66 Fergus Nelson
<br>app/models/compute_resource.rb:120:in `new_vm'
517 66 Fergus Nelson
<br>app/views/hosts/_compute.html.erb:1:in `_app_views_hosts__compute_html_erb__2073528223_70084372690700'
518 66 Fergus Nelson
<br>app/controllers/hosts_controller.rb:139:in `compute_resource_selected'
519 66 Fergus Nelson
<br>app/models/taxonomy.rb:48:in `as_taxonomy'<br>app/models/concerns/foreman/thread_session.rb:143:in `as_location'
520 66 Fergus Nelson
<br>app/models/taxonomy.rb:47:in `as_taxonomy'<br>app/models/concerns/foreman/thread_session.rb:108:in `as_org'<br>app/models/taxonomy.rb:46:in `as_taxonomy'
521 66 Fergus Nelson
<br>app/controllers/hosts_controller.rb:135:in `compute_resource_selected'<br>app/models/concerns/foreman/thread_session.rb:33:in `clear_thread'
522 66 Fergus Nelson
<br>lib/middleware/catch_json_parse_errors.rb:9:in `call'
523 66 Fergus Nelson
</div>
524 66 Fergus Nelson
<p><a href="/" data-id="aid_not_defined">Back</a></p>
525 66 Fergus Nelson
</pre>
526 66 Fergus Nelson
527 68 Dominic Cleal
The problem is that fog isn't picking up your environment's proxy settings, so the solution is for apache to pass them to it.
528 1 Ohad Levy
529 68 Dominic Cleal
On Debian/Ubuntu, edit /etc/apache2/envvars to include the following
530 68 Dominic Cleal
531 66 Fergus Nelson
export http_proxy=http://<host>:<port>
532 66 Fergus Nelson
export https_proxy=https://<host>:<port>
533 1 Ohad Levy
534 68 Dominic Cleal
On Red Hat OSes, edit /etc/sysconfig/httpd instead.
535 68 Dominic Cleal
536 1 Ohad Levy
Restart apache.
537 68 Dominic Cleal
538 68 Dominic Cleal
More information at https://www.phusionpassenger.com/documentation/Users%20guide%20Apache.html#_apache.
539 73 Dominic Cleal
540 73 Dominic Cleal
h1. How to take a thread dump of running Foreman processes
541 73 Dominic Cleal
542 73 Dominic Cleal
It's possible to trigger running Foreman applications to log thread dumps - if they're not responding, this can help determine which piece of code they're running.
543 73 Dominic Cleal
544 73 Dominic Cleal
If you're using Passenger (default setup with foreman-installer), run @pkill -ABRT -f RackApp@ to send an abort signal to Passenger.  This will be written to /var/log/httpd/error_log (EL/Fedora) or /var/log/apache2/error.log (Debian/Ubuntu).
545 73 Dominic Cleal
546 73 Dominic Cleal
If you're using Foreman standalone (WEBrick or other), @pkill -TTIN -f rails@ (or TTIN signal to whatever container you're using) will print the thread dump to stdout - you may have to run Foreman in the foreground to see this.
547 75 Stephen Benjamin
548 75 Stephen Benjamin
h1. "ERROR -- : Wrong size. Was 315, should be 196" when using Realm Proxy on EL7
549 75 Stephen Benjamin
550 75 Stephen Benjamin
A bug in Ruby 2.0 prevents the Realm proxy from working correctly: https://bugs.ruby-lang.org/issues/8182
551 75 Stephen Benjamin
552 75 Stephen Benjamin
Commenting these out on lines 505-506 in /usr/share/ruby/xmlrpc/client.rb is a temporary workaround:
553 75 Stephen Benjamin
554 75 Stephen Benjamin
      <pre>
555 75 Stephen Benjamin
      #elsif expected != "<unknown>" and expected.to_i != data.bytesize and resp["Transfer-Encoding"].nil?
556 75 Stephen Benjamin
      #  raise "Wrong size. Was #{data.bytesize}, should be #{expected}"
557 75 Stephen Benjamin
      </pre>
558 75 Stephen Benjamin
559 75 Stephen Benjamin
This is expected to be fixed in one of the next releases of EL7.