Bug #29429: Dynflow tables are tried to migrate in parallel - Foreman

Bug #29429

Dynflow tables are tried to migrate in parallel

Added by Anonymous 5 months ago. Updated 3 months ago.

Status:

Closed

Priority:

Normal

Assignee:

Adam Ruzicka

Category:

DB migrations

Target version:

Difficulty:

Triaged:

Bugzilla link:

Pull request:

https://github.com/Dynflow/dynflow/pull/353, https://github.com/theforeman/foreman/pull/7593

Fixed in Releases:

2.1.0

Found in Releases:

Nightly

Description

Suspision: dynflow tries to migrate the tables when it boots up. Maybe dynflow tries to migrate the db several times at once from different puma workers

Examples:

CentOS 7:

/opt/theforeman/tfm/root/usr/share/gems/gems/sequel-5.7.1/lib/sequel/adapters/postgres.rb:146:in `async_exec': PG::UniqueViolation: ERROR:  duplicate key value violates unique constraint "pg_type_typname_nsp_index" (Sequel::UniqueConstraintViolation)
DETAIL:  Key (typname, typnamespace)=(dynflow_schema_info, 2200) already exists.

Ubuntu 18.04:

dynflow-sidekiq@orchestrator[8451]: More than 1 row in migrator table
dynflow-sidekiq@orchestrator[8451]: /usr/share/foreman/vendor/ruby/2.5.0/gems/sequel-5.30.0/lib/sequel/extensions/migration.rb:620:in `schema_dataset'
dynflow-sidekiq@orchestrator[8451]: /usr/share/foreman/vendor/ruby/2.5.0/gems/sequel-5.30.0/lib/sequel/extensions/migration.rb:455:in `initialize'
dynflow-sidekiq@orchestrator[8451]: /usr/share/foreman/vendor/ruby/2.5.0/gems/sequel-5.30.0/lib/sequel/extensions/migration.rb:519:in `initialize'
dynflow-sidekiq@orchestrator[8451]: /usr/share/foreman/vendor/ruby/2.5.0/gems/sequel-5.30.0/lib/sequel/extensions/migration.rb:402:in `new'
dynflow-sidekiq@orchestrator[8451]: /usr/share/foreman/vendor/ruby/2.5.0/gems/sequel-5.30.0/lib/sequel/extensions/migration.rb:402:in `run'
dynflow-sidekiq@orchestrator[8451]: /usr/share/foreman/vendor/ruby/2.5.0/gems/dynflow-1.4.3/lib/dynflow/persistence_adapters/sequel.rb:263:in `migrate_db'

Associated revisions

Revision 7a3d995c (diff)
Added by Adam Ruzicka 3 months ago

Fixes #29429 - Provide rake tasks for Dynflow migrations

Historically Dynflow tried to migrate its database on startup. This became an
issue with the move to Puma, because multiple instance of Dynflow running in
separate Puma workers would try to migrate the database concurrently and the
workers would die because of unique key validation errors.

This change adds dynflow:abort_if_pending migrations rake task for checking if
the Dynflow database is fully migrated. It also extends the db:migrate and
db:abort_if_pending_migrations to run their Dynflow counterparts as well.

Revision 7378af98 (diff)
Added by Adam Ruzicka 3 months ago

Fixes #29429 - Bump dependency on Dynflow

History

#1 Updated by Ewoud Kohl van Wijngaarden 4 months ago

One solution is to create rake tasks to do this and let the installer handle it. For the regular DB it uses two tasks (db:migrate and db:abort_if_pending_migrations) to indicate whether a migration is needed and to actually migrate. This provides idempotency.