Plugins » Webhooks

When subscribing to webhook events, the generated tasks randomly fail to be generated correctly.
Instead of a "Deliver webhook <event>" task that gets executed properly, we end up with tasks where the action in the UI is shown as the ID of the task, and they are stuck in the state pending with result pending forever.
In the raw tab for that task, I see both "Raw Input" and "Raw Output" as "{}". This happened both in our test environment and our production environment when creating a webhook subscribed to "host_updated" events, but way more often in production where we got around 10 tasks generated per second. In testing, with only about 6 webhook tasks per minute (due to way lower host count) this only occured rarely while in prod we got several broken tasks per minute.
For further reference, including some log excerpts, see https://community.theforeman.org/t/migrating-to-webhooks-shellhooks-are-we-doing-things-wrong/32203.