Project

General

Profile

Bug #29370

Syncing a large product leaves Pulp tasks "waiting".

Added by Ian Ballou 9 months ago. Updated about 2 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
Repositories
Target version:
Difficulty:
Triaged:
Yes
Bugzilla link:
Pull request:
Fixed in Releases:
Found in Releases:

Description

I've tried a few times to sync a product with ~200 repositories and it never completes successfully. There are always sync tasks left over that are stuck waiting on "pulp:action:publish". The tasks waiting are a mix of Docker and File repos. I've tried restarting Pulp services but that doesn't help. It seems this could be a Katello bug or a Pulp bug.

Detailed information about the server:

Repositories: 212
-> 154 Docker repos with a mix of BusyBox, Alpine, and Bash. These are Docker Hub official images.
-> 58 file repos. All except 6 have the upstream url http://quartet.usersys.redhat.com/pub/fake-repos/large_file/. 1 has https://repos.fedorapeople.org/repos/pulp/pulp/fixtures/file-many/ and the other 5 have no upstream url.

CPUs: 8
RAM: 32 GB
Pulp workers: 8


Related issues

Related to Installer - Tracker #28999: Deploy Foreman with Puma as the application server by defaultClosed

Has duplicate Katello - Bug #29458: Syncing of repos got stuck in foreman (katello nightly version )Duplicate

History

#1 Updated by Chris Roberts 9 months ago

  • Triaged changed from No to Yes
  • Target version set to Katello 3.16.0
  • Category set to Repositories

#2 Updated by Justin Sherrill 8 months ago

  • Has duplicate Bug #29458: Syncing of repos got stuck in foreman (katello nightly version ) added

#3 Updated by Suresh Thirugnanasambandan 8 months ago

Any update on this?

#4 Updated by Imaan please_edit_me 8 months ago

Hello,

This issue is blocking us to test the performance of the katello nightly with the puma web server. Is there any work-around for this issue?

Any suggestions/recommendations??

#5 Updated by Justin Sherrill 8 months ago

Would you be able to try a couple things:

1. in /etc/foreman/database.yml, change the 'pool' setting from '5' to '30':

production:
  adapter: postgresql
  database: foreman
  username: foreman
  password: "SMA42qaA3FFFm2ioiam8RrMJMxpBZ7CB" 
  pool: 30

2. Create a new file:
/usr/share/foreman/config/initializers/foreman-tasks-db-pool-size.rb

with this one line:

ForemanTasks.dynflow.config.db_pool_size = 30

3. run foreman-maintain service restart

can you see if that helps at all? I have seen some benefit (with ~40 syncs).

#6 Updated by Ewoud Kohl van Wijngaarden 8 months ago

There is an installer option --foreman-db-pool which can be used to change the pool size in /etc/foreman/database.yml. If we need to tune the DB pool size as well then we need a proper way to configure this.

#7 Updated by Suresh Thirugnanasambandan 8 months ago

  • Related to Tracker #28999: Deploy Foreman with Puma as the application server by default added

#8 Updated by Samir Jha 7 months ago

  • Target version changed from Katello 3.16.0 to 1223

#9 Updated by Jonathon Turel 7 months ago

  • Target version changed from 1223 to Katello 4.0.0

#10 Updated by Chris Roberts 4 months ago

  • Target version changed from Katello 4.0.0 to Katello 3.17.0

#11 Updated by Chris Roberts about 2 months ago

  • Target version changed from Katello 3.17.0 to Katello 3.18.0

Also available in: Atom PDF