Project

General

Profile

Actions

Bug #26666

closed

restarting dynflowd with a task in planning phase can leave the task "planning" forever

Added by Mike McCune almost 5 years ago. Updated over 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Dynflow
Target version:
-
Difficulty:
Triaged:
No
Fixed in Releases:
Found in Releases:

Description

Cloned from https://bugzilla.redhat.com/show_bug.cgi?id=1687771

Description of problem:
There are scenarios (based on a race condition or maybe call flow used for triggering the task?) where a task is planning for a while, and if dynflowd is restarted that time, the task sits "planning" forever.

Particular example (very visible without fix for bz1673447): see reproducer steps.

It is assumed https://github.com/Dynflow/dynflow/pull/303 fixes this.

How reproducible:
very likely (scale the test more to have better chance)

Steps to Reproduce:
1. Have more repos, more LEs and few Capsules

2. Create and publish many CVs with even identical content (one or two small repos, e.g.)

3. Promote many CVs to next LE, e.g. via:

for i in $(seq 1 20); do
hammer content-view version promote --content-view CV_${i} --organization-id 1 --from-lifecycle-environment-id 1 --to-lifecycle-environment-id 2 --async &
sleep 1
done

4. monitor tasks status summary e.g. via:

sudo su - postgres -c "psql -d foreman -c 'select label,count(label),state,result from foreman_tasks_tasks where state <> '\''stopped'\'' group by label,state,result ORDER BY label;'"

5. Once there will be more Actions::Katello::CapsuleContent::Sync tasks in planning state, restart dynflowd:

service dynflowd restart

6. Monitor the tasks status summary until all Capsule Sync tasks terminate

Actual results:
6. is waiting for Godot

Expected results:
6. all Sync tasks successfully complete after a reasonable time

Additional info:

Actions #1

Updated by Mike McCune almost 5 years ago

  • Subject changed from restarting dynflowd with a task in planning phase can leave the task "planning" forever to restarting dynflowd with a task in planning phase can leave the task "planning" forever
  • Pull request https://github.com/Dynflow/dynflow/pull/303 added
Actions #2

Updated by Adam Ruzicka almost 5 years ago

  • Category set to Dynflow
  • Status changed from New to Closed
  • Assignee set to Adam Ruzicka
Actions #3

Updated by Adam Ruzicka over 4 years ago

  • Fixed in Releases dynflow-1.3.0 added
Actions

Also available in: Atom PDF