Bug #25422
closedSequential Actions::Katello::Host::Update calls from subscription-manager can fail under load
Description
Cloned from https://bugzilla.redhat.com/show_bug.cgi?id=1643649
We previously switched from a sync task to an async task for host update API calls:
https://github.com/Katello/katello/pull/6709/files
This temporarily reduced a bottleneck at the passenger layer but introduces a new issue:
During high traffic situations it is likely and common for clients to issue sequential calls from the client, eg:
subscription-manager repos disable ..
subscription-manager repos enable ..
these calls end up firing off an async task but it is common that the second task will fail because the Host is locked from the 1st.
With our new ability to have multiple Dynflow executors, we should explore moving this back to a sync task so we can avoid repeated failed Tasks calling host update.
Longer term, we need to:
1) move this out of Foreman Tasks into a direct API call like we did for registration
2) A better retry method
3) Queuing per-host so we can process multiple calls for the same host and process them in order.