Project

General

Profile

Bug #12650

dynflow_executor memory usage continues to grow, causing performance degredation

Added by Bryan Kearney almost 3 years ago. Updated 2 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Subscriptions
Target version:
Difficulty:
Triaged:
Yes
Bugzilla link:
Team Backlog:
Fixed in Releases:
Found in Releases:

Description

Cloned from https://bugzilla.redhat.com/show_bug.cgi?id=1283582
Description of problem:

The memory usage of dynflow_executor continues to grow over time.
As a conequence of this, operations like content view publishes
take longer and longer to complete.

Restarting the Satellite improves the performance, which then
continues to degrade over time

Version-Release number of selected component (if applicable):

6.1.3, 6.1.4

How reproducible:

100%

Steps to Reproduce:
1. Use Satellite
2.
3.

Actual results:

dynflow_executor memory usage continues to grow over time

Expected results:

memory usage should be reach a steady state and rise and
fall as work is done.

Additional info:

I used the following script to collect process mem stats
every 15 minutes

#!/bin/bash

LOGFILE=/var/log/dynflow_executor-memory-usage.log
PID=`pidof dynflow_executor`
cat /proc/$PID/status | grep ^Vm >> $LOGFILE

The I can review the VmPeak, VmData etc sizes.

The only time these figures go down is on a process restart.

Associated revisions

Revision 7f661597 (diff)
Added by Ivan Necas over 2 years ago

Fixes #12650 - make sure we sync on QPID acknowledge (#6065)

It seems the way we use the QPID client library causes the memory leaks
described in https://issues.apache.org/jira/browse/QPID-3321

Steps to test:

while subscription-manager register --username admin\
--password changeme --org 'Default Organization'\
--environment Library --force; do I=$((I+1)); echo
$I; done

In other shell:

PID=`/usr/sbin/pidof dynflow_executor`
watch "cat /proc/$PID/status | grep ^VmData"

and watch the number grow without the fix and stay still with the fix

History

#1 Updated by Ivan Necas over 2 years ago

  • Project changed from foreman-tasks to Katello
  • Category set to Subscriptions

#2 Updated by The Foreman Bot over 2 years ago

  • Status changed from New to Ready For Testing
  • Pull request https://github.com/Katello/katello/pull/6065 added

#3 Updated by Eric Helms over 2 years ago

  • Legacy Backlogs Release (now unused) set to 143

#4 Updated by Ivan Necas over 2 years ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

Also available in: Atom PDF