Bug #33747
open
Concurrent fact uploads for a non-existing host may lead to duplicate host records being created
Description
When uploading facts for a non-existent host, we seem to create a new host, but save it after importing its facts. This opens quite a large window during which additional fact upload for the same host could happen.
Reproducer details (assumes running on a foreman machine foreman-ansible, although it is reproducible with anything that uploads facts):
# playbook.yml
---
- hosts: all
tasks:
- setup:
Run with
for i in `seq 5`; do sudo -u foreman-proxy ansible-playbook -i ,duplicate-host -c local playbook.yml & done
On my machine, this managed to create anything from 1 to 5 instances of duplicate-host.
Updated by
Updated by Lukas Zapletal over 3 years ago
Fact uploading process is totally busted, it is ineffective. We build new host model in memory, then we create all facts names, values and associations and because host model was not yet saved, we cannot use optimized things like upsert or other faster ActiveRecord constructs. Host is then saved at the very end of the "transaction".
This needs a huge refactoring from the ground up. However, if we choose to keep the current design and save host in advance, then please let's at least drop using full active record models and work just with SQL instead (or optimized Rails methods).