Project

General

Profile

Actions

Bug #3541

closed

ec2 provisioning failure in setSSHProvision (when no CA available?)

Added by David Schmitt about 11 years ago. Updated over 6 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Compute resources
Target version:
Difficulty:
Triaged:
Fixed in Releases:
Found in Releases:

Description

Hi,

I've a fresh 1.3 foreman connected to aws. Deploying new machines works flawlessly, except for this failure:

Rolling back due to a problem: [Configure instance c06-test3.example.com via SSH   2003    failed  [#<Host::Managed id: 7, name: "c06-test3.example.com", ip: "54.211.196.93", last_compile: nil, last_freshcheck: nil, last_report: nil, updated_at: "2013-10-29 14:19:45", source_file_id: nil, created_at: "2013-10-29 14:19:45", mac: nil, root_pass: nil, serial: nil, puppet_status: 0, domain_id: 1, architecture_id: 1, operatingsystem_id: 1, environment_id: 11, subnet_id: nil, ptable_id: 1, medium_id: nil, build: true, comment: "", disk: "", installed_at: nil, model_id: nil, hostgroup_id: nil, owner_id: nil, owner_type: nil, enabled: true, puppet_ca_proxy_id: nil, managed: true, use_image: nil, image_file: nil, uuid: "i-953e16f0", compute_resource_id: 1, puppet_proxy_id: 1, certname: nil, image_id: 1, organization_id: nil, location_id: 1, type: "Host::Managed">, :setSSHProvision]]
ActiveRecord::Rollback

The finish script has already run successfully (checked by manually connecting to the new instance). setSSHProvision seems to fail on the "respond_to?(:initialize_puppetca,true) && initialize_puppetca && delAutosign if puppetca?" step. This system has no CA proxy configured and the test image/script has never tried to connect to the puppetmaster.

There were multiple factors which combined into making this an ugly issue to debug, I've no idea how easy those are to fix or which of those should be fixed:

  • no default finish template for EC2: this would have helped establishing the fact that a successful puppet run is required to finish the provisioning
  • output of the finish script not displayed: echoing the output somewhere on the foreman side would have easily established the success of the script without having to connect to the instance (which required extracting the key from the database)
    • related: unclear output from the ssh provisioner on default log levels. Has the following succeeded?
      SSH connection established to 54.205.211.252 - executing template
      negotiating protocol version
      got KEXINIT from server
      sending KEXINIT
      negotiating algorithms
      could not connect to ssh-agent
      channel_open_confirmation: 0 0 0 32768
      sending channel request "exec" 
      channel_window_adjust: 0 +2097152
      channel_success: 0
      channel_data: 0 1b
      channel_data: 0 1b
      channel_data: 0 1b
      channel_eof: 0
      channel_request: 0 exit-status false
      channel_close: 0
      closing remaining channels (0 open)
      

      (yes, it has, but it doesn't say so)
  • no error from
    respond_to?(:initialize_puppetca,true) && initialize_puppetca && delAutosign if puppetca?
    in setSSHProvision: this can fail the provisioning, but neither "respond_to?(:initialize_puppetca,true) == false" nor "not puppetca?" would report any specific error.
  • The "Rolling back due to a problem" message does not point to the source location actually causing the error: this would have helped with debugging * related: "Configure instance .* failed" only matches .mo and .js files, also not helping in locating the failing code: indicating the separate fields of this message ("[Configure instance c06-test3.example.com via SSH] [prio:2003] failed") would have helped with debugging

I've added a simple "true" after the initialize_puppetca line and thus was able to successfully provision to EC2.


Related issues 6 (2 open4 closed)

Related to Foreman - Bug #2693: unattended installation without Puppet CA is failingClosedDominic Cleal06/21/2013Actions
Related to Foreman - Bug #3222: Disabling UUID certificates leaves UUID certname in place on newly created hostsClosed10/08/2013Actions
Related to Foreman - Bug #13770: SSH provisioning fails when no puppetca is assigned to the hostDuplicate02/17/2016Actions
Related to Foreman - Tracker #14002: Orchestration build around ActiveRecord hooks trackerNew03/02/2016

Actions
Related to Foreman - Bug #14004: After_commit failure doesn't handle the rollback for the actions performed from the pre_commit phase and active record objectsNew03/02/2016Actions
Has duplicate Foreman - Bug #9414: The SSH config step of provisioning assumes a Puppet CA proxy is specified, fails otherwiseDuplicate02/17/2015Actions
Actions #1

Updated by Dominic Cleal about 11 years ago

  • Category set to Compute resources
Actions #2

Updated by Dominic Cleal about 11 years ago

Just as a note, I think this is the same problem as #2693 but I didn't fix it in the SSH part as I didn't realise the same code existed. We should review and combine these, as #3222 probably isn't addressed for SSH provisioning either.

Actions #3

Updated by Dominic Cleal about 11 years ago

  • Related to Bug #2693: unattended installation without Puppet CA is failing added
Actions #4

Updated by Dominic Cleal about 11 years ago

  • Related to Bug #3222: Disabling UUID certificates leaves UUID certname in place on newly created hosts added
Actions #5

Updated by Dominic Cleal over 8 years ago

  • Related to Bug #13770: SSH provisioning fails when no puppetca is assigned to the host added
Actions #6

Updated by Dominic Cleal over 8 years ago

  • Has duplicate Bug #9414: The SSH config step of provisioning assumes a Puppet CA proxy is specified, fails otherwise added
Actions #7

Updated by Ivan Necas over 8 years ago

  • Related to Tracker #14002: Orchestration build around ActiveRecord hooks tracker added
Actions #8

Updated by Ivan Necas over 8 years ago

  • Related to Bug #14004: After_commit failure doesn't handle the rollback for the actions performed from the pre_commit phase and active record objects added
Actions #9

Updated by The Foreman Bot over 8 years ago

  • Status changed from New to Ready For Testing
  • Assignee set to Ivan Necas
  • Pull request https://github.com/theforeman/foreman/pull/3269 added
Actions #10

Updated by Dominic Cleal almost 8 years ago

  • Status changed from Ready For Testing to New
  • Assignee deleted (Ivan Necas)
  • Pull request deleted (https://github.com/theforeman/foreman/pull/3269)

Issue likely still present, but PR closed due to inactivity.

Actions #11

Updated by The Foreman Bot almost 8 years ago

  • Status changed from New to Ready For Testing
  • Assignee set to Ivan Necas
  • Pull request https://github.com/theforeman/foreman/pull/4262 added
Actions #12

Updated by Ivan Necas almost 8 years ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100
Actions #13

Updated by Dominic Cleal almost 8 years ago

  • Translation missing: en.field_release set to 221
Actions

Also available in: Atom PDF