Bug #10805
closed
VLAN interface attached to bond is wrongly created as Bond
Description
Hi, I've been using Foreman successfully since 1.2, to manage an
engineering lab, so great job guys!
We recently got new switches, and had been looking forward to Foreman
1.8 to start doing some more advanced networking. It's mostly working,
but we're still running into some rough edges.
In this particular scenario I'm setting up a CentOS 7 KVM hypervisor
(with latest updates as of somewhere this week). I used Foreman and a
slightly adapted version of the kickstart_networking_setup.erb snippet
to set up:
- eno1 as the provisioning interface
- eno2 and eno4 as bond0, which is the primary interface. We use
bonding mode 802.3ad.
So these are the four managed interfaces I have configured in
Foreman. eno3 gets autocreated (unmanaged) after the first puppet run,
but is not relevant further to the discussion.
I then use puppet-network
(https://forge.puppetlabs.com/razorsedge/network) to create a VLAN
interface on top of bond0 (bond0.3130) with a corresponding bridge
(br3130):
network::bridge { 'br3130':
ensure => 'up',
stp => true,
delay => '0',
}network::if::bridge { 'bond0.3130':
ensure => 'up',
bridge => 'br3130'
}This works, and the intention is that this bridge is used to connect
the virtual machines (also created by Foreman, and that works too,
great!). Note that on the hypervisor/host, neither the bridge nor the
VLAN interface has an IP address: this VLAN is intended only for the
guest VM's.
The problem is that after puppet runs, a number of interfaces get
autocreated (unmanaged):
- a second bond0 (Bond/virtual), with the current MAC address of the
bonding group (the original managed bond0 has no MAC address
configured).
- bond0.3130 (Bond/virtual): This isn't strictly speaking a bonding
interface. The VLAN ID is also not discovered. Not sure either of
these two should be considered a bug.
- br3130 (Interface/virtual)
See the screenshot foreman-autocreated-interfaces.png for an overview
The bug is that there are now two interfaces called "bond0", which is
invalid according to Foreman's model: whenever I now try to save the
host, I get an error for the unmanaged bond0: identifier bond0 has
already been taken (see foreman-bad-interface-bond0.png)
And what's worse: it's now even impossible to delete that duplicate
bond0 by hand: it disappears when I click the corresponding Delete
button, but when I click Submit to save the host, it re-appears, with
the same error. The only way out is to manually delete it from the
database (I'm probably going to install a cronjob for that as a
workaround).
It's a bit similar to the ticket created by my collegue Julien, but
the context here is somewhat more complex:
http://projects.theforeman.org/issues/10546
I've also attached the output of ifconfig and facter:
- facter.txt
- ifconfig.txt
Files
Updated by Davy Verstappen almost 10 years ago
- File 4-duplicate-bridge-interfaces.png 4-duplicate-bridge-interfaces.png4-duplicate-bridge-interfaces.png added
I now managed to get no less than 4 duplicate auto-created bridge
interfaces: see 4-duplicate-bridge-interfaces.png
This happened because of an erroneous situation in the first place:
the bond0.3130 interface wouldn't start initially, because the kernel
module 8021q wasn't loaded (presumably because I was missing VLAN=yes
in /etc/sysconfig/network, but not 100% sure of that). I think every
time I ran puppet, it tried restarting the network, and then the
bridge got a new MAC address.
Granted, this is definitely an error on my side, but Foreman shouldn't
get itself into a state where it violates its own model. Again, I can
only get out of this by manually deleting the bad nics from the
database.
Updated by Davy Verstappen almost 10 years ago
Of course, after a clean kickstart of the hypervisor, I cannot reproduce the original problem anymore either. So it probably happened during my experiments with setting up the bridge/vlan interface.
Updated by Marek Hulán almost 10 years ago
- Related to Bug #10546: Reduce the number of interfaces duplicates added
Updated by Andrew Wagner almost 10 years ago
- File Foreman Duplicate NIC Issue.png Foreman Duplicate NIC Issue.pngForeman Duplicate NIC Issue.png added
Just a note that I'm seeing this issue also on 1.8.1 although it was present in 1.8.0 as well. Somehow our hosts are getting multiple duplicate nics auto-created for them with the same MAC address. The issue is not related solely to bond interfaces for us, but any type of virtual interface. Some of our foreman hosts have users using libvirt on them and creating virtual interfaces that end up with the same nic name (virbr0) but different mac address. There are also some physical nics that have the same MAC address listed for them but have also been duplicated. See the attached picture.
This breaks importing classes for us and is a very serious issue given the need to then manually delete the extraneous nics.
Updated by Marek Hulán almost 10 years ago
- Related to Bug #10888: duplicate interfaces when using bond device added
Updated by Marek Hulán almost 10 years ago
- Subject changed from auto-created duplicate bond0 interface to VLAN interface attached to bond is wrongly created as Bond
So there are couple of issue mentioned in this report.
1) a second bond0 (Bond/virtual), with the current MAC address of the
bonding group (the original managed bond0 has no MAC address
configured).
- definitely an issue caused by detection based on mac address, we couldn't match bond since no bond's mac was set before provisioning (sometimes it's not even known at this time), this should be fixed by #10607, after it's appliend, bonds are matched based only on their identifier
2) bond0.3130 (Bond/virtual): This isn't strictly speaking a bonding interface.
- definitely an issue, seems like we detect bond type even if it's actually vlan
3) The bug is that there are now two interfaces called "bond0", which is
invalid according to Foreman's model: whenever I now try to save the
host, I get an error for the unmanaged bond0: identifier bond0 has
already been taken (see foreman-bad-interface-bond0.png)
- this is the problem descried in 1), another workaround is to use API (so you don't have to go to DB directly), after 1) is fixed this shouldn't happen anymore
4) duplicated bridges
- this is a tricky one, Foreman does not support bridges yet (tracked under #7463) so we can't apply the same thing as we do for bonds. I see two ways out of this, either add a hack for bridge based on it's identifier to ignore it's MAC during matching or do the full implementation. I'd be glad to help with the later if someone provides a provisioning template snippet that would configure a bridge correctly on rpm-based distros but thinking about also UI changes I think the first option should be implemented anyway to prevent the issue meanwhile. I created a separate issue (#10932) for this as it was reported as later comment and is not strictly related to the original issue report.
Changing this issue to cover only 2)
Updated by Marek Hulán almost 10 years ago
- Related to Bug #10932: Bridges should not be matched based on mac address during fact imports added
Updated by Marek Hulán almost 10 years ago
- Related to Bug #10607: bonding on ubuntu causes foreman to create an interface after each ubuntu server reboot added
Updated by The Foreman Bot over 9 years ago
- Status changed from New to Ready For Testing
- Pull request https://github.com/theforeman/foreman/pull/2704 added
- Pull request deleted (
)
Updated by Dominic Cleal over 9 years ago
- Assignee set to Marek Hulán
- Translation missing: en.field_release set to 88
Updated by Marek Hulán over 9 years ago
- Status changed from Ready For Testing to Closed
- % Done changed from 0 to 100
Applied in changeset 44dac2f9d9d55c858a4556fd04a603ccc5f851d4.