Plugins » foreman-tasks

See https://github.com/ruby-concurrency/concurrent-ruby/issues/585 and https://github.com/rails/rails/issues/26847 for more details.

On rails 5, code executing in a run or finalize phase can deadlock the server due to autoloading classes. If you run in production (or really with config.eager_load config.cache_classes true), the problem does not occur. I have only seen it as part of sync() tasks within a controller action, but it likely happens with async() tasks too.

From my tests, wrapping individual steps (i.e. the run phase of an action) in the suggested methods does NOT resolve the issue, such as:

def run
  Rails.application.executor.wrap do
    ActiveSupport::Dependencies.interlock.permit_concurrent_loads do
      #auto load some constants
    end
  end
end

Wrapping the full call to sync_task() within the controller does fix it, such as within katello's candlepin_proxies_controller:

    #api :POST, "/environments/:environment_id/consumers", N_("Register a consumer in environment")
    def consumer_create
      Rails.application.executor.wrap do
        ActiveSupport::Dependencies.interlock.permit_concurrent_loads do
          content_view_environment = find_content_view_environment
          host = Katello::Host::SubscriptionFacet.find_or_create_host(content_view_environment.environment.organization, rhsm_params)

          sync_task(::Actions::Katello::Host::Register, host, rhsm_params, content_view_environment)
          host.reload

          update_host_registered_through(host, request.headers)

          render :json => Resources::Candlepin::Consumer.get(host.subscription_facet.uuid)
       end
      end
    end

to reproduce currently with katello:

1. Install a dev environment for katello and enable rails 5
2. Checkout this commit onto katello: https://github.com/Katello/katello/pull/6967
3. Enable rails 5 support in foreman and bundle install
4. start the rails server and try to register a system with subscription-manager

The server seems to hang during the run phase of the Actions::Katello::Host::Register action, during fact importing. It hangs when loading the FactValue class here: https://github.com/theforeman/foreman/blob/develop/app/services/fact_importer.rb#L66

It is 100% reproducible on a freshly started server.