Project

General

Profile

Actions

Bug #3606

closed

Fog error: InternalError with Foreman 1.3

Added by Gerard Ryan over 10 years ago. Updated over 6 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Compute resources - EC2
Target version:
-
Difficulty:
Triaged:
Fixed in Releases:
Found in Releases:

Description

I've not been able to spin up new VMs in EC2 using Foreman 1.3. My system is Ubuntu 12.04 LTS in EC2 and foreman was installed via the installer. The create host feature worked priot to upgrading to 1.3.

When I try to create a host I get the following error:

Fog error: InternalError => An internal error has occurred
/usr/share/foreman/vendor/ruby/1.8/gems/excon-0.28.0/lib/excon/middlewares/expects.rb:10:in `response_call'
/usr/share/foreman/vendor/ruby/1.8/gems/excon-0.28.0/lib/excon/middlewares/response_parser.rb:8:in `response_call'
/usr/share/foreman/vendor/ruby/1.8/gems/excon-0.28.0/lib/excon/connection.rb:361:in `response'
/usr/share/foreman/vendor/ruby/1.8/gems/excon-0.28.0/lib/excon/connection.rb:254:in `request'
/usr/share/foreman/vendor/ruby/1.8/gems/fog-1.18.0/lib/fog/xml/sax_parser_connection.rb:36:in `request'
/usr/share/foreman/vendor/ruby/1.8/gems/fog-1.18.0/lib/fog/core/deprecated/connection.rb:18:in `request'
/usr/share/foreman/vendor/ruby/1.8/gems/fog-1.18.0/lib/fog/aws/compute.rb:434:in `_request'
/usr/share/foreman/vendor/ruby/1.8/gems/fog-1.18.0/lib/fog/aws/compute.rb:429:in `request'
/usr/share/foreman/vendor/ruby/1.8/gems/fog-1.18.0/lib/fog/aws/requests/compute/run_instances.rb:121:in `run_instances'
/usr/share/foreman/vendor/ruby/1.8/gems/fog-1.18.0/lib/fog/aws/models/compute/server.rb:194:in `save'
/usr/share/foreman/vendor/ruby/1.8/gems/fog-1.18.0/lib/fog/core/collection.rb:52:in `create'
/usr/share/foreman/app/models/compute_resource.rb:126:in `create_vm'
/usr/share/foreman/app/models/compute_resources/foreman/model/ec2.rb:45:in `create_vm'
/usr/share/foreman/app/models/concerns/orchestration/compute.rb:57:in `setCompute'
/usr/share/foreman/app/models/concerns/orchestration.rb:148:in `send'
/usr/share/foreman/app/models/concerns/orchestration.rb:148:in `execute'
/usr/share/foreman/app/models/concerns/orchestration.rb:88:in `process'
/usr/share/foreman/app/models/concerns/orchestration.rb:80:in `each'
/usr/share/foreman/app/models/concerns/orchestration.rb:80:in `process'
/usr/share/foreman/app/models/concerns/orchestration.rb:18:in `on_save'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:627:in `_run__316199885__save__4__c$
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:405:in `send'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:405:in `__run_callback'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:385:in `_run_save_callbacks'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:81:in `send'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:81:in `run_callbacks'
/usr/share/foreman/vendor/ruby/1.8/gems/activerecord-3.2.14/lib/active_record/callbacks.rb:264:in `create_or_update'
/usr/share/foreman/vendor/ruby/1.8/gems/activerecord-3.2.14/lib/active_record/persistence.rb:84:in `save'
/usr/share/foreman/vendor/ruby/1.8/gems/activerecord-3.2.14/lib/active_record/validations.rb:50:in `save'
/usr/share/foreman/vendor/ruby/1.8/gems/activerecord-3.2.14/lib/active_record/attribute_methods/dirty.rb:22:in `save'
/usr/share/foreman/vendor/ruby/1.8/gems/activerecord-3.2.14/lib/active_record/transactions.rb:259:in `save_without_type'
/usr/share/foreman/vendor/ruby/1.8/gems/activerecord-3.2.14/lib/active_record/transactions.rb:313:in `with_transaction_returning$
/usr/share/foreman/vendor/ruby/1.8/gems/activerecord-3.2.14/lib/active_record/connection_adapters/abstract/database_statements.r$
/usr/share/foreman/vendor/ruby/1.8/gems/activerecord-3.2.14/lib/active_record/transactions.rb:208:in `transaction'
/usr/share/foreman/vendor/ruby/1.8/gems/activerecord-3.2.14/lib/active_record/transactions.rb:311:in `with_transaction_returning$
/usr/share/foreman/vendor/ruby/1.8/gems/activerecord-3.2.14/lib/active_record/transactions.rb:259:in `save_without_type'
/usr/share/foreman/vendor/ruby/1.8/gems/activerecord-3.2.14/lib/active_record/transactions.rb:270:in `rollback_active_record_sta$
/usr/share/foreman/vendor/ruby/1.8/gems/activerecord-3.2.14/lib/active_record/transactions.rb:258:in `save_without_type'
/usr/share/foreman/app/models/concerns/foreman/sti.rb:29:in `save'
/usr/share/foreman/app/controllers/hosts_controller.rb:85:in `create'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/abstract_controller/base.rb:167:in `process_action'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_controller/metal/rendering.rb:10:in `process_action'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/abstract_controller/callbacks.rb:18:in `process_action'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:527:in `_run__27127587__process_act$
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:215:in `_conditional_callback_aroun$
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:326:in `around'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:310:in `send'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:310:in `_callback_around_743'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:214:in `_conditional_callback_aroun$
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:526:in `_run__27127587__process_act$
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:215:in `_conditional_callback_aroun$
/usr/share/foreman/app/models/concerns/foreman/thread_session.rb:33:in `clear_thread'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:214:in `_conditional_callback_aroun$
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:415:in `_run__27127587__process_act$
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:215:in `_conditional_callback_aroun$
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:326:in `around'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:310:in `send'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:310:in `_callback_around_13'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:214:in `_conditional_callback_aroun$
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:414:in `_run__27127587__process_act$
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:405:in `send'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:405:in `__run_callback'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:385:in `_run_process_action_callbac$
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:81:in `send'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:81:in `run_callbacks'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/abstract_controller/callbacks.rb:17:in `process_action'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_controller/metal/rescue.rb:29:in `process_action'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/notifications.rb:123:in `instrument'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/notifications.rb:123:in `instrument'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_controller/metal/instrumentation.rb:29:in `process_action'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_controller/metal/params_wrapper.rb:207:in `process_action'
/usr/share/foreman/vendor/ruby/1.8/gems/activerecord-3.2.14/lib/active_record/railties/controller_runtime.rb:18:in `process_acti$
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/abstract_controller/base.rb:121:in `process'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/abstract_controller/rendering.rb:45:in `process'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_controller/metal.rb:203:in `dispatch'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_controller/metal/rack_delegation.rb:14:in `dispatch'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_controller/metal.rb:246:in `action'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_dispatch/routing/route_set.rb:73:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_dispatch/routing/route_set.rb:73:in `dispatch'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_dispatch/routing/route_set.rb:36:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/journey-1.0.4/lib/journey/router.rb:68:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/journey-1.0.4/lib/journey/router.rb:56:in `each'
/usr/share/foreman/vendor/ruby/1.8/gems/journey-1.0.4/lib/journey/router.rb:56:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_dispatch/routing/route_set.rb:608:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/apipie-rails-0.0.24/lib/apipie/static_dispatcher.rb:65:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_dispatch/middleware/best_standards_support.rb:17:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/rack-1.4.5/lib/rack/etag.rb:23:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/rack-1.4.5/lib/rack/conditionalget.rb:35:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_dispatch/middleware/head.rb:14:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_dispatch/middleware/params_parser.rb:21:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_dispatch/middleware/flash.rb:242:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:210:in `context'
/usr/share/foreman/vendor/ruby/1.8/gems/rack-1.4.5/lib/rack/session/abstract/id.rb:205:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_dispatch/middleware/cookies.rb:341:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/activerecord-3.2.14/lib/active_record/query_cache.rb:64:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/activerecord-3.2.14/lib/active_record/connection_adapters/abstract/connection_pool.rb:47$
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_dispatch/middleware/callbacks.rb:28:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:405:in `_run__1193688236__call__4__$
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:405:in `send'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:405:in `__run_callback'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:385:in `_run_call_callbacks'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:81:in `send'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/callbacks.rb:81:in `run_callbacks'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_dispatch/middleware/remote_ip.rb:31:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_dispatch/middleware/debug_exceptions.rb:16:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_dispatch/middleware/show_exceptions.rb:56:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/railties-3.2.14/lib/rails/rack/logger.rb:32:in `call_app'
/usr/share/foreman/vendor/ruby/1.8/gems/railties-3.2.14/lib/rails/rack/logger.rb:16:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/tagged_logging.rb:22:in `tagged'
/usr/share/foreman/vendor/ruby/1.8/gems/railties-3.2.14/lib/rails/rack/logger.rb:16:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_dispatch/middleware/request_id.rb:22:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/rack-1.4.5/lib/rack/methodoverride.rb:21:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/rack-1.4.5/lib/rack/runtime.rb:17:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/activesupport-3.2.14/lib/active_support/cache/strategy/local_cache.rb:72:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/rack-1.4.5/lib/rack/lock.rb:15:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/actionpack-3.2.14/lib/action_dispatch/middleware/static.rb:63:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/rack-cache-1.2/lib/rack/cache/context.rb:136:in `forward'
/usr/share/foreman/vendor/ruby/1.8/gems/rack-cache-1.2/lib/rack/cache/context.rb:143:in `pass'
/usr/share/foreman/vendor/ruby/1.8/gems/rack-cache-1.2/lib/rack/cache/context.rb:155:in `invalidate'
/usr/share/foreman/vendor/ruby/1.8/gems/rack-cache-1.2/lib/rack/cache/context.rb:71:in `call!'
/usr/share/foreman/vendor/ruby/1.8/gems/rack-cache-1.2/lib/rack/cache/context.rb:51:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/railties-3.2.14/lib/rails/engine.rb:484:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/railties-3.2.14/lib/rails/application.rb:231:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/railties-3.2.14/lib/rails/railtie/configurable.rb:30:in `send'
/usr/share/foreman/vendor/ruby/1.8/gems/railties-3.2.14/lib/rails/railtie/configurable.rb:30:in `method_missing'
/usr/share/foreman/vendor/ruby/1.8/gems/rack-1.4.5/lib/rack/urlmap.rb:64:in `call'
/usr/share/foreman/vendor/ruby/1.8/gems/rack-1.4.5/lib/rack/urlmap.rb:49:in `each'
/usr/share/foreman/vendor/ruby/1.8/gems/rack-1.4.5/lib/rack/urlmap.rb:49:in `call'
/usr/lib/ruby/1.8/phusion_passenger/rack/request_handler.rb:92:in `process_request'
/usr/lib/ruby/1.8/phusion_passenger/abstract_request_handler.rb:207:in `main_loop'
/usr/lib/ruby/1.8/phusion_passenger/rack/application_spawner.rb:118:in `run'
/usr/lib/ruby/1.8/phusion_passenger/rack/application_spawner.rb:65:in `spawn_application'
/usr/lib/ruby/1.8/phusion_passenger/utils.rb:184:in `safe_fork'
/usr/lib/ruby/1.8/phusion_passenger/rack/application_spawner.rb:58:in `spawn_application'
/usr/lib/ruby/1.8/phusion_passenger/rack/application_spawner.rb:41:in `spawn_application'
/usr/lib/ruby/1.8/phusion_passenger/spawn_manager.rb:159:in `spawn_application'
/usr/lib/ruby/1.8/phusion_passenger/spawn_manager.rb:287:in `handle_spawn_application'
/usr/lib/ruby/1.8/phusion_passenger/abstract_server.rb:352:in `__send__'
/usr/lib/ruby/1.8/phusion_passenger/abstract_server.rb:352:in `main_loop'
/usr/lib/ruby/1.8/phusion_passenger/abstract_server.rb:196:in `start_synchronously'
/usr/lib/phusion_passenger/passenger-spawn-server:61
Rolling back due to a problem: Set up compute instance test-server.ec2.internal 1 failed test-server.ec2.internalsetCompute

Actions #1

Updated by Dominic Cleal over 10 years ago

  • Subject changed from Fog issues with Foreman 1.3 to Fog error: InternalError with Foreman 1.3
  • Category set to Compute resources
Actions #2

Updated by Gerard Ryan over 10 years ago

I have created new virtual machines on ubuntu 12.04 lts in EC2. They are running puppetmaster 3.31 & foreman 1.2.3 and a separate instance of puppetmaster 3.31 and foreman 1.3.

In both cases I am unable to create compute resources in EC2. The Create Host action fails with a presumably Fog related error: Fog error: InternalError => An internal error has occurred.

Stack trace here: http://pastebin.com/mxJskxdj

Actions #3

Updated by Ohad Levy over 10 years ago

I've seen that with older versions of fog, maybe try updating, e.g. if you are on ubuntu try:

su - foreman -s /bin/bash
bundle update fog
touch tmp/restart.txt

Actions #4

Updated by Gerard Ryan over 10 years ago

Hi Ohad. I've tried su'ing to the foreman user and issuing a 'bundle update fog' in the /usr/share/foreman directory.

No affect.

Actions #5

Updated by Gerard Ryan over 10 years ago

Having issued 'bundle update fog' in /usr/share/foreman as the foreman user, the last few lines of output are:

Updating files in vendor/cache
 * multi_json-1.8.2.gem
 * excon-0.28.0.gem
 * fog-1.18.0.gem

However the same nondescript error comes up when trying to create a new EC2 instance:
Fog error: InternalError => An internal error has occurred

Actions #6

Updated by Gerard Ryan over 10 years ago

This was solved by selecting the availability zone in the Create Host screen. Previously it was a non-issue to leave the availability zone in its default state. Now doing so will make Fog error out with a very vague "internal error" message.

Actions #7

Updated by Dominic Cleal over 9 years ago

  • Category changed from Compute resources to Compute resources - EC2
Actions #8

Updated by Anonymous over 6 years ago

  • Status changed from New to Resolved

closing after three years... a lot of things changed since then.

Actions

Also available in: Atom PDF