Project

General

Profile

Actions

Bug #16336

open

Need to address concurrency limits in various bulk actions

Added by Partha Aji over 7 years ago. Updated over 5 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Orchestration
Target version:
Difficulty:
easy
Triaged:
Fixed in Releases:
Found in Releases:

Description

There seems to be a possibility that bulk actions can get overloaded when there are concurrent tasks with too many hosts.
This is likely to affect pretty much all the bulk actions which are run concurrently
look at
https://bugzilla.redhat.com/show_bug.cgi?id=1368103#c2

The Dynflow runtime keeps the data of currently-running tasks in memory, while waiting for the events: the could correspond to the amount of memory consumption, if the tasks get stuck.

How the thing might be solved would require dynflow being able to drop some inline data and load that from database again on demand.

Another thing, that might help would be the Katello bulk actions setting some upper limit on concurrent tasks (dynflow has support for this), so we would not get into the level of concurrency, where the memory issues appear. Another question, if the timeout limit for the stack actions could be lower to finish the task sooner (to not take up that much memory)

Likely to be fixed with something like https://github.com/Dynflow/dynflow/blob/master/examples/sub_plan_concurrency_control.rb#L49

Actions #1

Updated by Justin Sherrill over 7 years ago

  • Category set to Orchestration
  • translation missing: en.field_release set to 114
  • Difficulty set to easy
Actions #2

Updated by John Mitsch over 6 years ago

  • Target version set to 160
Actions

Also available in: Atom PDF