Bug #11434
closed
Foreman 1.7.x -> 1.8.2 (Puppet collects/adds Lots of bogus interface entries)
Description
We've noticed an issue with some existing hosts after upgrading from Foreman 1.7.2 -> 1.8.2
where there are sometimes dozens of new interfaces populated under interfaces, this causes updates to the host like changing hostgroup, toggling build to reprovision etc to fail.
Via hammer changing the hostgroup would render:
Started PUT "/api/hosts/92" for 127.0.0.1 at 2015-08-19 13:11:35 +0000
2015-08-19 13:11:35 [I] Processing by Api::V2::HostsController#update as JSON
2015-08-19 13:11:35 [I] Parameters: {"host"=>{"name"=>"host29-rack06.scale.openstack.engineering.XXXXXX.com", "hostgroup_id"=>"183", "interfaces_attributes"=>{}, "host_parameters_attributes"=>{}, "compute_attributes"=>{"volumes_attributes"=>{}}}, "apiv"=>"v2", "id"=>"92"}
2015-08-19 13:11:35 [I] Authorized user admin(Admin User)
2015-08-19 13:11:36 [E] Unprocessable entity Host::Managed (id: 92):
Interfaces some interfaces are invalid
Attempting to remove the non-primary interfaces listed, toggling to unmanaged and performing this or removing attributes from the primary interface (and deleting the generated non-primary interfaces) would yield:
NoMethodError undefined method `changed?' for nil:NilClass app/models/host/managed.rb:910:in `trigger_nic_orchestration' app/models/concerns/orchestration.rb:48:in `valid?' app/models/concerns/foreman/sti.rb:29:in `save_with_type' app/controllers/hosts_controller.rb:100:in `block in update' app/models/taxonomy.rb:54:in `block in no_taxonomy_scope' app/models/taxonomy.rb:61:in `block (2 levels) in as_taxonomy' app/models/concerns/foreman/thread_session.rb:147:in `as_location' app/models/taxonomy.rb:60:in `block in as_taxonomy' app/models/concerns/foreman/thread_session.rb:112:in `as_org' app/models/taxonomy.rb:59:in `as_taxonomy' app/models/taxonomy.rb:53:in `no_taxonomy_scope' app/controllers/hosts_controller.rb:99:in `update' app/controllers/concerns/application_shared.rb:13:in `set_timezone' app/models/concerns/foreman/thread_session.rb:32:in `clear_thread' lib/middleware/catch_json_parse_errors.rb:9:in `call'
We have found a workaround with manually removing the interfaces via hammer CLI
and skipping the primary/provision interface: this lets us further edit and manage the host.
problemhost=yourhostFQDN
skip_id=$(hammer host info --name $problemhost | grep -B 3 "nterface (primary, provision" | grep Id: | awk '{ print $NF }')
hammer host info --name $problemhost > /tmp/$problemhost
for interface in $(grep Id /tmp/$problemhost | grep ')' | grep -v $skip_id | awk '{ print $NF }') ; do \
hammer host interface delete --host $problemhost --id $interface \
done
Files
| foreman_issue_11434_1.png | View foreman_issue_11434_1.png | 98.2 KB | |||
| foreman_issue_11434_1.yaml | foreman_issue_11434_1.yaml | 7.94 KB | YAML output of facter |
Updated by
Updated by Marek Hulán over 9 years ago
Could you please attach the output of facter for this host? A screenshot of interfaces from webui would also help.
Updated by Will Foster over 9 years ago
Marek Hulán wrote:
Could you please attach the output of facter for this host? A screenshot of interfaces from webui would also help.
Hi Marek,
Here is facter output from that host and screenshot from the UI. Note this is another host that has this problem (I believe they all seem to in this set of hostgroups from 1.7.3 --> 1.8.2). The one filed in this report previously we used the above shell workaround to fix.
This is the state where they cannot be edited in the UI unless we manually remove the other non-primary, non-provisioning interfaces via hammer CLI above.
[root@host27-rack06 ~]# facter
Error: NetworkManager is not running.
architecture => x86_64
augeasversion => 1.1.0
bios_release_date => 03/07/2013
bios_vendor => Dell Inc.
bios_version => 1.6.0
blockdevice_sda_model => PERC H710P
blockdevice_sda_size => 999653638144
blockdevice_sda_vendor => DELL
blockdevice_sdb_model => PERC H710P
blockdevice_sdb_size => 999653638144
blockdevice_sdb_vendor => DELL
blockdevice_sdc_model => PERC H710P
blockdevice_sdc_size => 999653638144
blockdevice_sdc_vendor => DELL
blockdevice_sdd_model => PERC H710P
blockdevice_sdd_size => 999653638144
blockdevice_sdd_vendor => DELL
blockdevices => sda,sdb,sdc,sdd
boardmanufacturer => Dell Inc.
boardproductname => 0KCKR5
boardserialnumber => ..CN747512CJ0256.
domain => scale.openstack.engineering.XXXXX.com
facterversion => 2.4.1
filesystems => ext2,ext3,ext4,xfs
fqdn => host27-rack06.scale.openstack.engineering.XXXXXX.com
gid => root
hardwareisa => x86_64
hardwaremodel => x86_64
hostname => host27-rack06
id => root
interfaces => bond0,bond0_9,bond0_30,br1,em1,em2,em3,em4,lo,virbr0,virbr0_nic,vnet0,vnet1
ipaddress => 10.1.64.51
ipaddress_bond0 => 10.1.64.51
ipaddress_bond0_30 => 10.1.247.100
ipaddress_lo => 127.0.0.1
ipaddress_virbr0 => 192.168.122.1
is_virtual => false
kernel => Linux
kernelmajversion => 3.10
kernelrelease => 3.10.0-229.1.2.el7.x86_64
kernelversion => 3.10.0
macaddress => b8:ca:3a:63:89:38
macaddress_bond0 => b8:ca:3a:63:89:38
macaddress_bond0_30 => b8:ca:3a:63:89:38
macaddress_bond0_9 => b8:ca:3a:63:89:38
macaddress_br1 => 96:ec:0f:24:fa:d2
macaddress_em1 => B8:CA:3A:63:89:38
macaddress_em2 => B8:CA:3A:63:89:3A
macaddress_em3 => b8:ca:3a:63:89:3c
macaddress_em4 => b8:ca:3a:63:89:3d
macaddress_virbr0 => 52:54:00:bd:ec:b3
macaddress_virbr0_nic => 52:54:00:bd:ec:b3
macaddress_vnet0 => fe:54:00:80:56:4d
macaddress_vnet1 => fe:54:00:f1:c9:87
manufacturer => Dell Inc.
memoryfree => 53.62 GB
memoryfree_mb => 54905.31
memorysize => 62.72 GB
memorysize_mb => 64220.45
mtu_bond0 => 9000
mtu_bond0_30 => 9000
mtu_bond0_9 => 9000
mtu_br1 => 1500
mtu_em1 => 9000
mtu_em2 => 9000
mtu_em3 => 1500
mtu_em4 => 1500
mtu_lo => 65536
mtu_virbr0 => 1500
mtu_virbr0_nic => 1500
mtu_vnet0 => 1500
mtu_vnet1 => 1500
netmask => 255.255.192.0
netmask_bond0 => 255.255.192.0
netmask_bond0_30 => 255.255.255.224
netmask_lo => 255.0.0.0
netmask_virbr0 => 255.255.255.0
network_bond0 => 10.1.64.0
network_bond0_30 => 10.1.247.96
network_lo => 127.0.0.0
network_virbr0 => 192.168.122.0
operatingsystem => RedHat
operatingsystemmajrelease => 7
operatingsystemrelease => 7.1
os => {"name"=>"RedHat", "family"=>"RedHat", "release"=>{"major"=>"7", "minor"=>"1", "full"=>"7.1"}}
osfamily => RedHat
partitions => {"sda1"=>{"size"=>"134217728", "label"=>"1", "filesystem"=>"linux_raid_member"}, "sda2"=>{"size"=>"1048576", "label"=>"0", "filesystem"=>"linux_raid_member"}, "sda3"=>{"size"=>"1817180160", "label"=>"2", "filesystem"=>"linux_raid_member"}, "sdb1"=>{"size"=>"134217728", "label"=>"1", "filesystem"=>"linux_raid_member"}, "sdb2"=>{"size"=>"1048576", "label"=>"0", "filesystem"=>"linux_raid_member"}, "sdb3"=>{"size"=>"1817180160", "label"=>"2", "filesystem"=>"linux_raid_member"}, "sdc1"=>{"size"=>"1952446464", "label"=>"3", "filesystem"=>"linux_raid_member"}, "sdd1"=>{"size"=>"1952446464", "label"=>"3", "filesystem"=>"linux_raid_member"}}
path => /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin:/sbin
physicalprocessorcount => 2
processor0 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor1 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor10 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor11 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor12 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor13 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor14 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor15 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor16 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor17 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor18 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor19 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor2 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor20 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor21 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor22 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor23 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor3 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor4 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor5 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor6 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor7 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor8 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processor9 => Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz
processorcount => 24
processors => {"models"=>["Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz", "Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz"], "count"=>24, "physicalcount"=>2}
productname => PowerEdge R620
ps => ps -ef
puppetversion => 3.7.4
rubyplatform => x86_64-linux
rubysitedir => /usr/local/share/ruby/site_ruby/
rubyversion => 2.0.0
selinux => true
selinux_config_mode => permissive
selinux_config_policy => unknown
selinux_current_mode => permissive
selinux_enforced => false
selinux_policyversion => 28
serialnumber => 3HNNQW1
swapfree => 63.97 GB
swapfree_mb => 65503.87
swapsize => 63.97 GB
swapsize_mb => 65503.87
system_uptime => {"seconds"=>3942327, "hours"=>1095, "days"=>45, "uptime"=>"45 days"}
timezone => UTC
type => Rack Mount Chassis
uniqueid => 010a3340
uptime => 45 days
uptime_days => 45
uptime_hours => 1095
uptime_seconds => 3942327
uuid => 4C4C4544-0048-4E10-804E-B3C04F515731
virtual => physical
vlans => 30,9
Updated by Will Foster over 9 years ago
Will Foster wrote:
Marek Hulán wrote:
Could you please attach the output of facter for this host? A screenshot of interfaces from webui would also help.
Hi Marek,
Also attaching the YAML output of facter
Here is facter output from that host and screenshot from the UI. Note this is another host that has this problem (I believe they all seem to in this set of hostgroups from 1.7.3 --> 1.8.2). The one filed in this report previously we used the above shell workaround to fix.
This is the state where they cannot be edited in the UI unless we manually remove the other non-primary, non-provisioning interfaces via hammer CLI above.
[...]
Updated by Marek Hulán over 8 years ago
- Related to Feature #11972: Implement regex for option ignore_puppet_facts_for_provisioning added
Updated by Marek Hulán over 8 years ago
- Status changed from New to Assigned
- Assignee set to Marek Hulán
There are two problems in this case. One is invalid eth0 vs bond0, second is that vdsmdummy interfaces being created. The first one I believe is because after you provision a host on eth0 you start a bond which becomes new primary interface but this was not updated in Foreman DB, that's why they have same IP which I believe is the cause for eth0 being invalid. Second issue is that vdsmdummy are temporary interfaces created by oVirt and they should not be imported in the first place. Since Foreman 1.10 we have setting for selectively ignoring interfaces based on their identifier so to fix this we can simply add vdsmdummy to the list.
Updated by Marek Hulán over 8 years ago
- Status changed from Assigned to Closed
- Translation missing: en.field_release set to 136
- Pull request https://github.com/theforeman/foreman/pull/3429 added