dynflow_executor memory usage continues to grow, causing performance degredation
|Assigned To:||Ivan Necas|
|Velocity based estimate||-|
|Release||Katello 3.1.0||Release relationship||Auto|
Cloned from https://bugzilla.redhat.com/show_bug.cgi?id=1283582
Description of problem:
The memory usage of dynflow_executor continues to grow over time.
As a conequence of this, operations like content view publishes
take longer and longer to complete.
Restarting the Satellite improves the performance, which then
continues to degrade over time
Version-Release number of selected component (if applicable):
Steps to Reproduce:
1. Use Satellite
dynflow_executor memory usage continues to grow over time
memory usage should be reach a steady state and rise and
fall as work is done.
I used the following script to collect process mem stats
every 15 minutes
cat /proc/$PID/status | grep ^Vm >> $LOGFILE
The I can review the VmPeak, VmData etc sizes.
The only time these figures go down is on a process restart.
It seems the way we use the QPID client library causes the memory leaks
described in https://issues.apache.org/jira/browse/QPID-3321
Steps to test:
while subscription-manager register --username admin\
--password changeme --org 'Default Organization'\
--environment Library --force; do I=$((I+1)); echo
In other shell:
watch "cat /proc/$PID/status | grep ^VmData"
and watch the number grow without the fix and stay still with the fix