Project

General

Profile

Actions
Copy link

Bug #18447

closed

Route 53 record creation broken after upgrade to 1.14.1

Added by Paul Smyth about 7 years ago. Updated about 7 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
-
Difficulty:
Triaged:
Bugzilla link:
Pull request:
Fixed in Releases:
Found in Releases:
Red Hat JIRA:

Description

Creating new instances now fails on route 53 record creation with the following error:

Create IPv4 DNS record for broken-foreman-test.aws-eu-west.[private].com task failed with the following error: ERF12-2357 [ProxyAPI::ProxyException]: Unable to set DNS entry ([RestClient::NotFound]: 404 Not Found) for proxy https://foreman.[private].aws:8443/dns

Proxy log level is set to debug, log for this incident:

D, [2017-02-09T14:57:48.781537 ] DEBUG -- : accept: 172.31.63.188:32808
D, [2017-02-09T14:57:48.783599 ] DEBUG -- : Rack::Handler::WEBrick is invoked.
D, [2017-02-09T14:57:48.797753 ] DEBUG -- : verifying remote client 172.31.63.188 against trusted_hosts ["foreman.[private].aws"]
D, [2017-02-09T14:57:48.800806 ] DEBUG -- : Found puppetca at /usr/bin/puppet
D, [2017-02-09T14:57:48.800981 ] DEBUG -- : Found sudo at /usr/bin/sudo
D, [2017-02-09T14:57:48.801051 ] DEBUG -- : Executing /usr/bin/sudo S /usr/bin/puppet cert --ssldir /var/lib/puppet/ssl --clean a2b35939-f08d-4fa4-a08a-5402b8f73d96
D, [2017-02-09T14:57:49.699518 ] DEBUG - : Attempt to remove nonexistent client certificate for a2b35939-f08d-4fa4-a08a-5402b8f73d96
E, [2017-02-09T14:57:49.700034 ] ERROR -- : Attempt to remove nonexistent client certificate for a2b35939-f08d-4fa4-a08a-5402b8f73d96
D, [2017-02-09T14:57:49.700084 ] DEBUG -- : Attempt to remove nonexistent client certificate for a2b35939-f08d-4fa4-a08a-5402b8f73d96
I, [2017-02-09T14:57:49.700627 ] INFO -- : 172.31.63.188 - - [09/Feb/2017 14:57:49] "DELETE /puppet/ca/a2b35939-f08d-4fa4-a08a-5402b8f73d96 HTTP/1.1" 404 89 0.9030

D, [2017-02-09T14:57:49.740814 ] DEBUG -- : close: 172.31.63.188:32808
D, [2017-02-09T14:57:49.763909 ] DEBUG -- : accept: 172.31.63.188:32809
D, [2017-02-09T14:57:49.765290 ] DEBUG -- : Rack::Handler::WEBrick is invoked.
D, [2017-02-09T14:57:49.766848 ] DEBUG -- : verifying remote client 172.31.63.188 against trusted_hosts ["foreman.[private].aws"]
D, [2017-02-09T14:57:49.767820 ] DEBUG -- : Added a2b35939-f08d-4fa4-a08a-5402b8f73d96 to autosign
I, [2017-02-09T14:57:49.768231 ] INFO -- : 172.31.63.188 - - [09/Feb/2017 14:57:49] "POST /puppet/ca/autosign/a2b35939-f08d-4fa4-a08a-5402b8f73d96 HTTP/1.1" 200 - 0.0024

D, [2017-02-09T14:57:49.768973 ] DEBUG -- : close: 172.31.63.188:32809
D, [2017-02-09T14:58:08.195674 ] DEBUG -- : accept: 172.31.63.188:32819
D, [2017-02-09T14:58:08.197090 ] DEBUG -- : Rack::Handler::WEBrick is invoked.
I, [2017-02-09T14:58:08.238079 ] INFO -- : 172.31.63.188 - - [09/Feb/2017 14:58:08] "POST /dns/ HTTP/1.1" 404 27 0.0005

D, [2017-02-09T14:58:08.276545 ] DEBUG -- : close: 172.31.63.188:32819
D, [2017-02-09T14:58:08.553359 ] DEBUG -- : accept: 172.31.63.188:32822
D, [2017-02-09T14:58:08.554369 ] DEBUG -- : Rack::Handler::WEBrick is invoked.
D, [2017-02-09T14:58:08.554873 ] DEBUG -- : verifying remote client 172.31.63.188 against trusted_hosts ["foreman.[private].aws"]
D, [2017-02-09T14:58:08.555239 ] DEBUG -- : Found puppetca at /usr/bin/puppet
D, [2017-02-09T14:58:08.555364 ] DEBUG -- : Found sudo at /usr/bin/sudo
D, [2017-02-09T14:58:08.555431 ] DEBUG -- : Executing /usr/bin/sudo S /usr/bin/puppet cert --ssldir /var/lib/puppet/ssl --clean a2b35939-f08d-4fa4-a08a-5402b8f73d96
D, [2017-02-09T14:58:09.467588 ] DEBUG - : Attempt to remove nonexistent client certificate for a2b35939-f08d-4fa4-a08a-5402b8f73d96
E, [2017-02-09T14:58:09.467897 ] ERROR -- : Attempt to remove nonexistent client certificate for a2b35939-f08d-4fa4-a08a-5402b8f73d96
D, [2017-02-09T14:58:09.467942 ] DEBUG -- : Attempt to remove nonexistent client certificate for a2b35939-f08d-4fa4-a08a-5402b8f73d96
I, [2017-02-09T14:58:09.468304 ] INFO -- : 172.31.63.188 - - [09/Feb/2017 14:58:09] "DELETE /puppet/ca/a2b35939-f08d-4fa4-a08a-5402b8f73d96 HTTP/1.1" 404 89 0.9135

D, [2017-02-09T14:58:09.508612 ] DEBUG -- : close: 172.31.63.188:32822
D, [2017-02-09T14:58:09.530962 ] DEBUG -- : accept: 172.31.63.188:32823
D, [2017-02-09T14:58:09.532185 ] DEBUG -- : Rack::Handler::WEBrick is invoked.
D, [2017-02-09T14:58:09.532753 ] DEBUG -- : verifying remote client 172.31.63.188 against trusted_hosts ["foreman.[private].aws"]
D, [2017-02-09T14:58:09.533147 ] DEBUG -- : Removed a2b35939-f08d-4fa4-a08a-5402b8f73d96 from autosign
I, [2017-02-09T14:58:09.533366 ] INFO -- : 172.31.63.188 - - [09/Feb/2017 14:58:09] "DELETE /puppet/ca/autosign/a2b35939-f08d-4fa4-a08a-5402b8f73d96 HTTP/1.1" 200 - 0.0007

D, [2017-02-09T14:58:09.534018 ] DEBUG -- : close: 172.31.63.188:32823

I am unable to glean anything at all from this log. Route 53 record creation is broken and we are now stuck with having to manually create dbs entries vastly increasing deployment times. I am marking this urgent because it's a show stopper for us. Please advise

Actions
Copy link
 #1

Updated by Dominic Cleal about 7 years ago

  • Project changed from Foreman to Plugins
  • Category deleted (Compute resources - EC2)

I, [2017-02-09T14:58:08.238079 ] INFO -- : 172.31.63.188 - - [09/Feb/2017 14:58:08] "POST /dns/ HTTP/1.1" 404 27 0.0005

A 404 here suggests that the DNS module is disabled, either through configuration or from an error at startup. The smart proxy startup should log information about the DNS module configuration and any errors that occurred.

Please note that the smart_proxy_dns_route53 plugin issue tracker is at https://github.com/theforeman/smart_proxy_dns_route53/issues.

Actions
Copy link
 #2

Updated by Paul Smyth about 7 years ago

I'll create an issue on github. It would be helpful if I didn't have to look at two different locations in order to fix problems though. I've found the issue:

D, [2017-02-09T14:53:33.957926 ] DEBUG -- : Providers ['dns_route53'] are going to be configured for 'dns'
E, [2017-02-09T14:53:33.964124 ] ERROR -- : Disabling all modules in the group ['dns_route53', 'dns'] due to a failure in one of them: cannot load such file -- smart_proxy_dns_plugin_template/dns_plugin_template_main
D, [2017-02-09T14:53:33.964165 ] DEBUG -- : ["/usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:53:in `require'", "/usr/lib/ruby/2.0.0/rubygems/core_ext/kernel_require.rb:53:in `require'", "/usr/lib/ruby/vendor_ruby/smart_proxy_dns_route53/dns_route53_configuration.rb:4:in `load_classes'", "/usr/share/foreman-proxy/lib/proxy/plugin_initializer.rb:294:in `load_classes'", "/usr/share/foreman-proxy/lib/proxy/plugin_initializer.rb:243:in `load_settings'", "/usr/share/foreman-proxy/lib/proxy/plugin_initializer.rb:81:in `block in load_provider_settings'", "/usr/share/foreman-proxy/lib/proxy/plugin_initializer.rb:80:in `each'", "/usr/share/foreman-proxy/lib/proxy/plugin_initializer.rb:80:in `load_provider_settings'", "/usr/share/foreman-proxy/lib/proxy/plugin_initializer.rb:155:in `block in initialize_plugins'", "/usr/share/foreman-proxy/lib/proxy/plugin_initializer.rb:155:in `each'", "/usr/share/foreman-proxy/lib/proxy/plugin_initializer.rb:155:in `initialize_plugins'", "/usr/share/foreman-proxy/lib/launcher.rb:134:in `launch'", "/usr/share/foreman-proxy/bin/smart-proxy:6:in `<main>'"]

Actions
Copy link
 #3

Updated by Dominic Cleal about 7 years ago

  • Status changed from New to Resolved
Actions
Copy link

Also available in: Atom PDF