Project

General

Profile

Actions

Bug #11106

closed

Foreman 1.8.2 with VMware errors with: Could not find virtual machine network interface matching x.x.x.x

Added by Seth Doty over 8 years ago. Updated over 5 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Compute resources - VMware
Target version:
Difficulty:
Triaged:
Fixed in Releases:
Found in Releases:

Description

I have been nagging the IRC channel with this one and the consensus is that its a bug. When building a VMware guest I receive: Could not find virtual machine network interface matching x.x.x.x.

Logs come back with : 2015-07-13 12:40:20 [W] Orchestration::Compute: Could not match network interface #<Nic::Managed id: nil, mac: nil, ip: "10.21.3.245", type: "Nic::Managed", name: "test.domain.local", host_id: nil, subnet_id: 1, domain_id: 1, attrs: {}, created_at: nil, updated_at: nil, provider: nil, username: nil, password: nil, virtual: false, link: true, identifier: "", tag: "", attached_to: "", managed: true, mode: "balance-rr", attached_devices: "", bond_options: "", primary: true, provision: false, compute_attributes: {"type"=>"VirtualVmxnet3", "network"=>"pg-vlan-3"}>
2015-07-13 12:40:20 [W] Could not find virtual machine network interface matching 10.21.3.245
2015-07-13 12:40:20 [W] Rolling back due to a problem: [Query instance details for test.domain.local 4 failed [#<Host::Managed id: nil, name: "test.domain.local", last_compile: nil, last_freshcheck: nil, last_report: nil, updated_at: nil, source_file_id: nil, created_at: nil, root_pass: "", serial: nil, puppet_status: 0, architecture_id: 1, operatingsystem_id: 1, environment_id: 2, ptable_id: 14, medium_id: 8, build: true, comment: "", disk: "", installed_at: nil, model_id: nil, hostgroup_id: 1, owner_id: 3, owner_type: "User", enabled: true, puppet_ca_proxy_id: 1, managed: true, use_image: nil, image_file: nil, uuid: "5032e796-3769-ee7b-3ed4-d2b036c20cb0", compute_resource_id: 1, puppet_proxy_id: 1, certname: nil, image_id: nil, organization_id: 1, location_id: 5, type: "Host::Managed", otp: nil, realm_id: nil, compute_profile_id: 4, provision_method: "build", content_source_id: 1, grub_pass: "", content_view_id: 1, lifecycle_environment_id: 1, discovery_rule_id: nil>, :setComputeDetails]]
2015-07-13 12:40:20 [I] Removing Compute instance for test.domain.local.

This is a fresh install of foreman 1.8.2 with katello.


Files

production.log production.log 261 KB Seth Doty, 07/14/2015 09:19 AM
production.log production.log 528 KB Sandro Roth, 06/16/2016 09:18 AM
Actions #1

Updated by Seth Doty over 8 years ago

I also just uploaded a debug file for reference to the debug-incomming directory: foreman-debug-GB7sy.tar.xz

Actions #2

Updated by Dominic Cleal over 8 years ago

  • Category changed from Orchestration to Compute resources - VMware
  • Status changed from New to Need more information

Could you please enable debugging and try again? Some extra information showing which NICs were created on the VM will be printed to /var/log/foreman/production.log which you can attach to this ticket.

http://projects.theforeman.org/projects/foreman/wiki/Troubleshooting#How-do-I-enable-debugging

Actions #3

Updated by Seth Doty over 8 years ago

Here you go. I ran a build twice, first setting the interface name in foreman and then I ran it again leaving it empty, only setting an IP address on the primary interface. It doesn't change much as far as errors, but I'm hoping it provides a bit more complete of a test.

Actions #4

Updated by Dominic Cleal over 8 years ago

  • Status changed from Need more information to New

Extract from log:

2015-07-14 08:09:58 [D] Orchestration::Compute: Trying to match network interfaces from fog   <Fog::Compute::Vsphere::Interfaces
    server_id="50328ecb-196d-a21d-b661-88b38e822f0d" 
    [
                  <Fog::Compute::Vsphere::Interface
        mac="00:50:56:b2:7a:23",
        network="dvportgroup-87629",
        name="Network adapter 1",
        status="untried",
        summary="DVSwitch: 12 a9 12 50 6e 04 89 81-1c ec 7d 3a 09 1b 6b 0e",
        type=VirtualVmxnet3,
        key=4000,
        virtualswitch=nil,
        server_id="50328ecb-196d-a21d-b661-88b38e822f0d" 
      >, 
                  <Fog::Compute::Vsphere::Interface
        mac="00:50:56:b2:20:11",
        network="dvportgroup-87636",
        name="Network adapter 2",
        status="untried",
        summary="DVSwitch: 12 a9 12 50 6e 04 89 81-1c ec 7d 3a 09 1b 6b 0e",
        type=VirtualVmxnet3,
        key=4001,
        virtualswitch=nil,
        server_id="50328ecb-196d-a21d-b661-88b38e822f0d" 
      >    
    ]
  >
2015-07-14 08:09:58 [W] Orchestration::Compute: Could not match network interface #<Nic::Managed id: nil, mac: nil, ip: "10.21.3.245", type: "Nic::Managed", name: "corp-testforeman.hammocks.local", host_id: nil, subnet_id: 1, domain_id: 1, attrs: {}, created_at: nil, updated_at: nil, provider: nil, username: nil, password: nil, virtual: false, link: true, identifier: "", tag: "", attached_to: "", managed: true, mode: "balance-rr", attached_devices: "", bond_options: "", primary: true, provision: false, compute_attributes: {"type"=>"VirtualVmxnet3", "network"=>"pg-vlan-3"}>
2015-07-14 08:09:58 [W] Could not find virtual machine network interface matching 10.21.3.245

Looks like it's struggling to match the requested network "pg-vlan-3" against either of the networks on the two created NICs, "dvportgroup-87629" and "dvportgroup-87636".

Actions #5

Updated by Seth Doty over 8 years ago

To provide a bit of history on this, we had a semi-working install of foreman 1.7 running. We hit a lot of katello side bugs, but the build process worked ok most of the time. Something changed with the upgrade to 1.8 though, aside from the obvious, since this occurs with all builds I have tested now in different data centers with different networks/port groups, although it is the same vcenter instance.

I rebuilt this foreman/katello server since the old box was a bit of a Frankenstein install, so nothing is in this box yet if we need to poke around a bit more. This box is only a 3 or 4 days old at this point so its all VERY recent data.

As far as the interface match struggles, I have a co-worker who wrote some powershell scripts for the windows side and had similar issues querying interfaces due to the order he queried the vmware switching component. He couldn't directly query for the port group, but had to query for the distributed switch first and work his way down through port groups or he would never get an interface match either. I'm not sure if this is pertinent information at this point, but it is the process we had to go through before for the windows side of the world.

Actions #6

Updated by Seth Doty over 8 years ago

As an FYI, I just upgraded to the latest release and this bug still persists.

Actions #7

Updated by Przemysław Ciesielski over 8 years ago

The same problem with Foreman 1.9.2 and vCenter 6.0.

Actions #8

Updated by Sandro Roth almost 8 years ago

Just ran into this after vCenter upgrade from 5.5 to 6.0.
Tested with Foreman 1.11.2 and 1.12 RC1.

Attached log from 1.12 RC1 with debug enabled.

Actions #9

Updated by salman butt over 7 years ago

I'm also hitting this bug with foreman 1.11.2, 1.12-RC1.

Provisioning exists with:

"2016-07-27 19:17:06 [app] [E] Unprocessable entity Host::Managed (id: new):

 |   Could not find virtual machine network interface matching eth0
 | 
2016-07-27 19:17:06 [app] [I]   Rendered api/v2/errors/unprocessable_entity.json.rabl within api/v2/layouts/error_layout (1.1ms)
2016-07-27 19:17:06 [app] [D] Body: {
 |   "error": {"id":null,"errors":{"base":["Could not find virtual machine network interface matching eth0"]},"full_messages":["Could not find virtual machine network interface matching eth0"]}
 | }
 | 
"

This started after a recent upgrade to vcenter 6.0.

Actions #10

Updated by salman butt over 7 years ago

Actually, looks like this works fine in 6.0 vcenter build number 3018524, but not with 6.0 vcenter build number 3634793. I'm not sure where in-between this might have broken. Up until last week we were using the 3018524 build, and provisioning was working fine.

Actions #11

Updated by salman butt over 7 years ago

We have found a potential fix for this in our environment (YMMV). It looks like something internal to VCenter changed in our all of our VDS, during the upgrade. Due to this, provisioning against the existing VDS keeps failing. A coworker performed a separate test on a newly created, temporary VDS, and provisioning worked fine against the VLANs defined there.

We're going to do some work internally to create a brand new VDS layout here, move VMs around, and hopefully that will fix this problem.

Actions #12

Updated by Bryan Kearney over 7 years ago

  • Bugzilla link set to 1394290
Actions #13

Updated by The Foreman Bot over 7 years ago

  • Status changed from New to Ready For Testing
  • Assignee set to Ivan Necas
  • Pull request https://github.com/theforeman/foreman/pull/2714 added
Actions #14

Updated by Ivan Necas over 7 years ago

  • Assignee deleted (Ivan Necas)

I think we've found the reason for this issue, the problem was we were matching `interface.network` value against network `id` in case of portgroup, while the `network.key` should be the thing to match against

Actions #15

Updated by Ivan Necas over 7 years ago

  • Assignee set to Ivan Necas
Actions #16

Updated by Ivan Necas over 7 years ago

  • Target version set to 1.15.3
Actions #17

Updated by The Foreman Bot over 7 years ago

  • Pull request https://github.com/theforeman/foreman/pull/4074 added
Actions #18

Updated by Ivan Necas over 7 years ago

  • Pull request deleted (https://github.com/theforeman/foreman/pull/4074)
Actions #19

Updated by Ivan Necas over 7 years ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100
Actions #20

Updated by Dominic Cleal over 7 years ago

  • translation missing: en.field_release set to 210
Actions #21

Updated by Oliver Neumann over 6 years ago

Hi There,

I ran into the same Problem. Creating a VM in vCenter leads to the error message of this ticket. vCenter-Version is 6.5 an theforman is on Version 1.15.6.

The production.log give the following message:

017-10-24T15:24:44 d7225981 [app] [I] Could not find Vsphere network for {"network"=>"dvportgroup-xxxx", "type"=>"VirtualVmxnet3"}
2017-10-24T15:24:44 d7225981 [app] [I] Started GET "/tasks/ea1944c7-05ea-490d-a512-6cbdbbb1cdfc" for xxx.xxx.xxx.xxx at 2017-10-24 15:24:44 +0200
2017-10-24T15:24:44 d7225981 [app] [I] Processing by TasksController#show as */*

Actions #22

Updated by Marek Hulán over 6 years ago

Could you double check that the given network is in the cluster you've selected in Foreman? We don't limit resources according to cluster in Foreman UI which could result in such errors.

Actions #23

Updated by Oliver Neumann over 6 years ago

Double checked. The Network is available in the selected Cluster.

Actions #24

Updated by Oliver Neumann over 6 years ago

My problem is fixed with upgrade from 1.15.6. to 1.16.0

Actions

Also available in: Atom PDF