Bug #17395

@host.params are not interpreted when was used in ptable kickstart template.

Added by Tomasz Gromowski 3 months ago. Updated 2 months ago.

Status:Closed
Priority:Normal
Assigned To:Dominic Cleal
Category:Templates
Target version:-
Difficulty: Bugzilla link:
Found in release:1.13.1 Pull request:https://github.com/theforeman/foreman/pull/4097, https://github.com/theforeman/foreman/pull/4083
Story points-
Velocity based estimate-
Release1.13.3Release relationshipAuto

Description

I try to interpolate some variables in partition table template.
And when i put in ptable for example:
<%= @host.params['vg_name'] %>
then any of param aren't interpolated by renderer.

However when i checked generated template for host in
Host > template> provisioning template -> review,
everything looks fine, variables are interpolated

But when i GET
http://foreman.local/unattended/provision?token=a9e59aec-f66d-4558-8a9e-3252d62ee428
All variables are default in whole kickstart file

It's issue only for RedHat family OS because for Debian everything works great.
I attach my ptable templates.

kickstart-lvm.txt Magnifier (1.72 KB) Tomasz Gromowski, 11/18/2016 11:14 AM

provision_hostname.txt Magnifier (4.46 KB) Tomasz Gromowski, 11/22/2016 07:54 AM

provision_token.txt Magnifier (4.46 KB) Tomasz Gromowski, 11/22/2016 07:54 AM

provision_template_get.log (153 KB) Tomasz Gromowski, 11/23/2016 09:02 AM

production.log (413 KB) Kevin Carrasco, 11/29/2016 06:25 PM

production.log (606 KB) Maurice Mouw, 12/02/2016 09:02 AM


Related issues

Related to Foreman - Bug #15599: Unattended controller can't access host parameters in kic... Closed 07/07/2016
Related to Foreman - Bug #13164: view_params is missing Closed 01/12/2016

Associated revisions

Revision f8a2b268
Added by Dominic Cleal 2 months ago

fixes #17395 - use admin user for load_template_vars

When `load_template_vars` is called from before_action, the current user
must already be set to ensure caching of parameters inside Host#params
is done with consistent permissions. Otherwise when Foreman::Renderer's
kickstart_attributes (or similar) renders a ptable that uses params,
the method will cache an empty parameters hash.

History

#1 Updated by Dominic Cleal 3 months ago

What do you mean by "default"? Is the template not interpreted at all, or is it just the parameter values aren't correct?

#2 Updated by Tomasz Gromowski 3 months ago

Dominic Cleal wrote:

What do you mean by "default"? Is the template not interpreted at all, or is it just the parameter values aren't correct?

Sorry, that wasn't so clear,

when i try to interpolate some values from foreman params in ptable,
then all foreman params are ignored, as if they not there.

#3 Updated by Dominic Cleal 3 months ago

Could you attach the rendered template please?

#4 Updated by Tomasz Gromowski 3 months ago

Templates attached,
As I wrote before i got another template when i refer to it with hostname and another when with token.
I have attached both.

#5 Updated by Dominic Cleal 3 months ago

Thanks, that's a little clearer. The "lang" line near the top is also different - in the previewed file it's pl_PL and in the token-fetched file it's en_US.UTF-8, suggesting @host.params is returning different results. I think this points to the parameters not working generally, not only in ptable templates.

Where are lang and the root_size parameters set? On the host itself or something like a domain, OS or globally?

If you could make the token request while running tail -f /var/log/foreman/production.log and attach the log, that may be helpful. It would be great if you could enable debugging first with "permissions" and "sql" logs enabled by following https://theforeman.org/manuals/1.13/index.html#7.2Debugging.

Thanks for providing this info.

#6 Updated by Tomasz Gromowski 3 months ago

Dominic Cleal wrote:

Thanks, that's a little clearer. The "lang" line near the top is also different - in the previewed file it's pl_PL and in the token-fetched file it's en_US.UTF-8, suggesting @host.params is returning different results. I think this points to the parameters not working generally, not only in ptable templates.

Yes but only did not work when i use some forman_params in ptable, then whole template was not propertly rendered.

Where are lang and the root_size parameters set? On the host itself or something like a domain, OS or globally?

I try to set them on group, and on host.

If you could make the token request while running tail -f /var/log/foreman/production.log and attach the log, that may be helpful. It would be great if you could enable debugging first with "permissions" and "sql" logs enabled by following https://theforeman.org/manuals/1.13/index.html#7.2Debugging.

Thanks for providing this info.

Logs attached.

#7 Updated by Kevin Carrasco 3 months ago

I'm affected by this too. For me it happens on any template (ptable or kickstart). I created a test Kickstart template with the following contents:

<%#
kind: provision
name: Host Params Bug Test
-%>
<%= @host.params %>

Here are the results from the different URLs. Notice that ?token is missing all params shown in the other URLs except for remote execution stuff for some reason:

http://foreman01.corp.bardel.ca/unattended/provision?token=620640d5-a57d-4b61-84b4-2309213cb846

{"remote_execution_ssh_keys"=>["ssh-rsa <HIDDEN> puppet-foreman01"], "remote_execution_ssh_user"=>"root", "remote_execution_effective_user_method"=>"sudo"}

http://foreman01.corp.bardel.ca/unattended/provision?spoof=10.49.78.64

{"bootloader-append"=>"nofb splash=quiet crashkernel=auto nomodeset quiet net.ifnames=0", "domain-shortname"=>"BARDEL", "enable-puppetlabs-repo"=>"true", "force-puppet"=>"true", "join-domain-password"=>"<HIDDEN>", "join-domain-user"=>"<HIDDEN>", "md2-mount-point"=>"", "ntp-server"=>"ntp.corp.bardel.ca", "os-drive-type"=>"single", "time-zone"=>"America/Vancouver", "x-server"=>"false", "remote_execution_ssh_keys"=>["ssh-rsa <HIDDEN> puppet-foreman01"], "remote_execution_ssh_user"=>"root", "remote_execution_effective_user_method"=>"sudo"}

http://foreman01.corp.bardel.ca/unattended/provision?hostname=kc-centos7-01.corp.bardel.ca

{"bootloader-append"=>"nofb splash=quiet crashkernel=auto nomodeset quiet net.ifnames=0", "domain-shortname"=>"BARDEL", "enable-puppetlabs-repo"=>"true", "force-puppet"=>"true", "join-domain-password"=>"<HIDDEN>", "join-domain-user"=>"<HIDDEN>", "md2-mount-point"=>"", "ntp-server"=>"ntp.corp.bardel.ca", "os-drive-type"=>"single", "time-zone"=>"America/Vancouver", "x-server"=>"false", "remote_execution_ssh_keys"=>["ssh-rsa <HIDDEN> puppet-foreman01"], "remote_execution_ssh_user"=>"root", "remote_execution_effective_user_method"=>"sudo"}

I've also attached my production.log with sql and permission logging.

Thanks.

#8 Updated by Kevin Carrasco 3 months ago

Kevin Carrasco wrote:

I'm affected by this too. For me it happens on any template (ptable or kickstart). I created a test Kickstart template with the following contents:

[...]

Here are the results from the different URLs. Notice that ?token is missing all params shown in the other URLs except for remote execution stuff for some reason:

http://foreman01.corp.bardel.ca/unattended/provision?token=620640d5-a57d-4b61-84b4-2309213cb846
[...]

http://foreman01.corp.bardel.ca/unattended/provision?spoof=10.49.78.64
[...]

http://foreman01.corp.bardel.ca/unattended/provision?hostname=kc-centos7-01.corp.bardel.ca
[...]

I've also attached my production.log with sql and permission logging.

Thanks.

EDIT: The production.log attached is not for this sample template, it's for my real template. I made this sample template later to show the bug manifesting very clearly.

#9 Updated by Dominic Cleal 3 months ago

Do you have any plugins installed? (foreman-rake plugin:list)

I can see the issue in both log files - parameter lookups are happening before the correct user is assigned (added in #15599), but I can't reproduce this. It looks like something is triggering a very early lookup of parameters before rendering begins, which is then cached.

#10 Updated by Tomasz Gromowski 3 months ago

List of plugins used by me.

Foreman plugin: foreman-tasks, 0.8.2, Ivan Nečas, The goal of this plugin is to unify the way of showing task statuses across the Foreman instance.
It defines Task model for keeping the information about the tasks and Lock for assigning the tasks
to resources. The locking allows dealing with preventing multiple colliding tasks to be run on the
same resource. It also optionally provides Dynflow infrastructure for using it for managing the tasks.
Foreman plugin: foreman_ansible, 1.2.1, Daniel Lobato Garcia, Ansible integration with Foreman
Foreman plugin: foreman_discovery, 7.0.0, Amos Benari,Bryan Kearney,ChairmanTubeAmp,Daniel Lobato García,Dominic Cleal,Eric D. Helms,Frank Wall,Greg Sutcliffe,Imri Zvik,Joseph Mitchell Magen,Lukas Zapletal,Lukáš Zapletal,Marek Hulan,Martin Bačovský,Matt Jarvis,Michael Moll,Nick,Ohad Levy,Ori Rabin,Petr Chalupa,Phirince Philip,Scubafloyd,Shlomi Zadok,Stephen Benjamin,Yann Cézard, MaaS Discovery Plugin engine for Foreman
Foreman plugin: foreman_hooks, 0.3.12, Dominic Cleal, Plugin engine for Foreman that enables running custom hook scripts on Foreman events
Foreman plugin: foreman_setup, 5.0.0, Dominic Cleal, Plugin for Foreman that helps set up provisioning.

Additionally i use memcached plugin but is not listed above.

#11 Updated by Maurice Mouw 3 months ago

We have the same problem with a foreman server this has been recently migrated from 1.12.4 to 1.13.2
We have a automated setup for deploying foreman servers and when I build a foreman server with 1.13.2 directly installed on it I cannot reproduce this (yet).
The only two notable differences between the servers I can think of right now are:
  • LDAP is configured on the foreman server that has been upgraded.
  • The migration itself from 1.12.4 to 1.13.2

I've added the production.log set for debugging with sql and permissions enabled.

Below the installed plugins:

Foreman plugin: foreman-tasks, 0.8.2, Ivan Nečas, The goal of this plugin is to unify the way of showing task statuses across the Foreman instance.
It defines Task model for keeping the information about the tasks and Lock for assigning the tasks
to resources. The locking allows dealing with preventing multiple colliding tasks to be run on the
same resource. It also optionally provides Dynflow infrastructure for using it for managing the tasks.
Foreman plugin: foreman_bootdisk, 8.0.2, Dominic Cleal, Plugin for Foreman that creates iPXE-based boot disks to provision hosts without the need for PXE infrastructure.
Foreman plugin: foreman_discovery, 7.0.0, Amos Benari,Bryan Kearney,ChairmanTubeAmp,Daniel Lobato García,Dominic Cleal,Eric D. Helms,Frank Wall,Greg Sutcliffe,Imri Zvik,Joseph Mitchell Magen,Lukas Zapletal,Lukáš Zapletal,Marek Hulan,Martin Bačovský,Matt Jarvis,Michael Moll,Nick,Ohad Levy,Ori Rabin,Petr Chalupa,Phirince Philip,Scubafloyd,Shlomi Zadok,Stephen Benjamin,Yann Cézard, MaaS Discovery Plugin engine for Foreman
Foreman plugin: foreman_hooks, 0.3.12, Dominic Cleal, Plugin engine for Foreman that enables running custom hook scripts on Foreman events
Foreman plugin: foreman_salt, 7.0.2, Stephen Benjamin, Foreman Plug-in for Salt
Foreman plugin: foreman_setup, 5.0.0, Dominic Cleal, Plugin for Foreman that helps set up provisioning.
Foreman plugin: foreman_templates, 3.1.0, Greg Sutcliffe, Engine to synchronise provisioning templates from GitHub

#12 Updated by Dominic Cleal 3 months ago

  • Release set to 1.13.3

Thanks, I can reproduce it now - no plugins necessary.

This looks like it only affects the Redhat and Solaris OS families, as Foreman::Renderer calls @host.diskLayout in the jumpstart/kickstart_attributes methods prior to the main template being rendered from UnattendedController. Calling diskLayout causes the renderer to be used and for .params to be cached on the host model.

There is no user set while diskLayout is called, it's only set (from the fix in #15599) during the main template render in UnattendedController.

The user must also be set during load_template_vars in the UnattendedController.

#13 Updated by Maurice Mouw 3 months ago

For those who require a rather 'dirty' workaround. Clone your normal provisioning template, remove all the disklayout references in the clone and copy the partition table in the %pre (at the end) of the provisioning template (dont forget removing the ruby header from the template). Ones 1.13.3 comes around switch your provisioning template back to the original one. In the mean time you can provision hosts.

#14 Updated by Kevin Carrasco 3 months ago

Maurice's workaround doesn't work for me. If you look at my test Kickstart template on comment 7, It does nothing else other than dump @host.params. There are no references to any other template or snippets.

#15 Updated by Dominic Cleal 3 months ago

  • Status changed from New to Assigned
  • Assigned To set to Dominic Cleal

#16 Updated by The Foreman Bot 3 months ago

  • Status changed from Assigned to Ready For Testing
  • Pull request https://github.com/theforeman/foreman/pull/4083 added

#17 Updated by Dominic Cleal 3 months ago

  • Related to Bug #15599: Unattended controller can't access host parameters in kickstart templates added

#18 Updated by Dominic Cleal 3 months ago

  • Related to Bug #13164: view_params is missing added

#19 Updated by Kevin Carrasco 3 months ago

Thanks Dominic, I've manually applied your code changes to app/controllers/unattended_controller.rb and that fixes the bug for me.

#20 Updated by Dominic Cleal 2 months ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

#21 Updated by The Foreman Bot 2 months ago

  • Pull request https://github.com/theforeman/foreman/pull/4097 added

Also available in: Atom PDF