Bug #22838

Double-initialization of dynflow in core and foreman-tasks causes race-condition with ActiveRecord database connection

Added by Ivan Necas 3 months ago. Updated 2 months ago.

Assigned To:Ivan Necas
Category:Foreman plugin
Target version:Foreman - Team Ivan Iteration 29
Difficulty: Bugzilla link:
Found in release: Pull request:https://github.com/theforeman/foreman-tasks/pull/317
Story points-
Velocity based estimate-


During the initialization process, we ensure the size of the db pool is sufficient for the executors.

However, we run this twice (once from foreman, second from foreman-tasks). What can happen is:

1. the foreman initializes the world, as part of initialization, there are already some tasks being triggered
2. in the meantime, the foreman-tasks initialization runs as well (https://github.com/theforeman/foreman-tasks/blob/7434c95681f56f2295c82d46ad4a9590fb4763ff/lib/foreman_tasks/engine.rb#L149), and part of the initialization is re-connection of the ActiveRecord::Connection https://github.com/Dynflow/dynflow/blob/cc7c854c5d4c7ee5db883d9f8e285c9db573d83b/lib/dynflow/rails/configuration.rb#L93
3. during the short time, between disconnecting and connecting, there can be something in the tasks running, that requires the active record connection
4. it leads to "No connection pool with 'primary' found" error

Associated revisions

Revision ac217465
Added by Ivan Necas 3 months ago

Fixes #22838 - avoid double-initialization of Dynflow runtime (#317)

Since Foreman 1.17, we're already doing it in core. Repeating this in
here was causing race-conditions around ActiveRecord connection.


#1 Updated by Ivan Necas 3 months ago

  • Target version set to Team Ivan Iteration 29
  • Assigned To set to Ivan Necas
  • Status changed from New to Assigned

#2 Updated by The Foreman Bot 3 months ago

  • Status changed from Assigned to Ready For Testing
  • Pull request https://github.com/theforeman/foreman-tasks/pull/317 added

#3 Updated by Ivan Necas 2 months ago

  • % Done changed from 0 to 100
  • Status changed from Ready For Testing to Closed

Also available in: Atom PDF