Project

General

Profile

Bug #9953

Dynflow comsumes CPU with no tasks executing

Added by Mike McCune over 4 years ago. Updated 3 months ago.

Status:
Rejected
Priority:
Normal
Assignee:
Category:
Orchestration
Target version:
-
Difficulty:
Triaged:
Yes
Bugzilla link:
Pull request:
Fixed in Releases:
Found in Releases:

Description

On my RHEL6 based VM running the latest nightly build of Katello I noticed that the dynflow process was consuming lots of CPU:

top - 14:25:03 up 3:56, 1 user, load average: 1.09, 1.12, 1.28
Tasks: 129 total, 2 running, 127 sleeping, 0 stopped, 0 zombie
Cpu(s): 84.2%us, 15.8%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si,
Mem: 7865288k total, 7257996k used, 607292k free, 201936k buffers
Swap: 2097144k total, 0k used, 2097144k free, 4675568k cached

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND    
2466 foreman 20 0 485m 104m 1144 R 97.5 1.4 24:44.50 ruby
1883 tomcat 20 0 2968m 266m 12m S 1.0 3.5 0:44.53 java
...

running strace on the process yields pages and pages of:

stat("/usr/share/foreman/tmp/pids/dynflow_executor.pid", {st_mode=S_IFREG|0644, st_size=5, ...}) = 0
open("/usr/share/foreman/tmp/pids/dynflow_executor.pid", O_RDONLY) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=5, ...}) = 0
fstat(5, {st_mode=S_IFREG|0644, st_size=5, ...}) = 0
ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fffc1de3b50) = -1 ENOTTY (Inappropriate ioctl for device)
read(5, "2566\n", 8192) = 5
close(5) = 0
kill(2566, SIG_0) = 0
stat("/usr/share/foreman/tmp/pids/dynflow_executor.pid", {st_mode=S_IFREG|0644, st_size=5, ...}) = 0
open("/usr/share/foreman/tmp/pids/dynflow_executor.pid", O_RDONLY) = 5
fstat(5, {st_mode=S_IFREG|0644, st_size=5, ...}) = 0
fstat(5, {st_mode=S_IFREG|0644, st_size=5, ...}) = 0
ioctl(5, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fffc1de3b50) = -1 ENOTTY (Inappropriate ioctl for device)
read(5, "2566\n", 8192) = 5
close(5) = 0
kill(2566, SIG_0) = 0
stat("/usr/share/foreman/tmp/pids/dynflow_executor.pid", {st_mode=S_IFREG|0644, st_size=5, ...}) = 0
...


Related issues

Has duplicate Katello - Bug #9954: Dynflow comsumes CPU with no tasks executingDuplicate2015-03-30

History

#1 Updated by Mike McCune over 4 years ago

  • Status changed from New to Closed

#2 Updated by Ivan Necas over 4 years ago

Any reason for closing this issue?

#3 Updated by Eric Helms over 4 years ago

  • Status changed from Closed to New
  • Triaged changed from No to Yes

#4 Updated by Eric Helms over 4 years ago

  • Related to Bug #9954: Dynflow comsumes CPU with no tasks executing added

#5 Updated by Eric Helms over 4 years ago

  • Related to deleted (Bug #9954: Dynflow comsumes CPU with no tasks executing)

#6 Updated by Eric Helms over 4 years ago

  • Has duplicate Bug #9954: Dynflow comsumes CPU with no tasks executing added

#7 Updated by Ivan Necas over 4 years ago

Any pointers on the reproducibility of this issue? Was this observed frequently, both rhel6 and rhel7?

There was no major change on the dynflow/foremant-tasks for this behavior. I'm suspecting
the qpid connections to cause some issues recently.

Is the machine with reproducer available? Also, I would be interested into /usr/share/foreman/tmp/pids/dynflow_executor.output.

#8 Updated by Mike McCune over 4 years ago

I forgot to mark it closed as a dupe of #9954, apologies

I don't have RHEL7 setup so I can't state if it is occurring there.

I have the machine available if you wish to access it, you can also get a copy of the dynflow executor here:

https://mmccune.fedorapeople.org/scratch/dynflow_executor.tar.gz

#9 Updated by Ivan Necas over 4 years ago

Seems like the issue with qpid connection and core-dumping

#10 Updated by Ivan Necas over 4 years ago

I wonder if that might be connected to the other qpid issues we had in candlepin and pulp

#11 Updated by Eric Helms almost 4 years ago

  • Legacy Backlogs Release (now unused) set to 114

#12 Updated by John Mitsch 3 months ago

  • Target version deleted (Katello Backlog)
  • Status changed from New to Rejected

Thanks for reporting this issue. This issue was created over 4 years ago and hasn't seen an update in 1 year. We are closing this in an effort to keep a realistic backlog. Please open up a new issue that includes a link to this issue if you feel this still needs to be addressed. We can then triage the new issue and reassess.

Also available in: Atom PDF