Project

General

Profile

Bug #10107

Timeout requesting rpm_ids from pulp during sync/publish/promote

Added by Justin Sherrill over 6 years ago. Updated about 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Repositories
Target version:
Difficulty:
Triaged:
Yes
Bugzilla link:
Pull request:
Fixed in Releases:
Found in Releases:

Description

This has been seen by a couple different users. Pulp seems to be timing out. we could request fewer ids for each request to try to alleviate the problem.

https://github.com/Katello/runcible/blob/855d43cb74ab066621ed4d1b0c3274d39a426259/lib/runcible/extensions/repository.rb#L129

```

---
- /opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.35/lib/restclient_ext/request.rb:56:in
`rescue in transmit'
- /opt/rh/ruby193/root/usr/share/gems/gems/rbovirt-0.0.35/lib/restclient_ext/request.rb:11:in
`transmit'
- /opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:64:in
`execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/request.rb:33:in
`execute'
- /opt/rh/ruby193/root/usr/share/gems/gems/rest-client-1.6.7/lib/restclient/resource.rb:67:in
`post'
- /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/base.rb:91:in
`get_response'
- /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/base.rb:82:in
`call'
- /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/resources/repository.rb:161:in
`unit_search'
- /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:151:in
`block in rpm_ids'
- /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:139:in
`<<'
- /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:139:in
`block (2 levels) in rpm_ids'
- /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:138:in
`loop'
- /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:138:in
`block in rpm_ids'
- /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:145:in
`each'
- /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:145:in
`each'
- /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:145:in
`reduce'
- /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:145:in
`rescue in rpm_ids'
- /opt/rh/ruby193/root/usr/share/gems/gems/runcible-1.3.1/lib/runcible/extensions/repository.rb:130:in
`rpm_ids'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/models/katello/glue/pulp/repo.rb:244:in
`package_ids'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/models/katello/glue/pulp/repo.rb:746:in
`package_lists_for_publish'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/actions/pulp/repository/purge_empty_package_groups.rb:24:in
`invoke_external_task'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/action/polling.rb:70:in
`initiate_external_action'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/action/polling.rb:12:in
`run'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/action/cancellable.rb:9:in
`run'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/actions/pulp/abstract_async_task.rb:57:in
`run'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/action.rb:456:in
`block (3 levels) in execute_run'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/middleware/stack.rb:26:in
`call'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/middleware/stack.rb:26:in
`pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/dynflow-0.7.6/lib/dynflow/middleware.rb:16:in
`pass'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/actions/middleware/remote_action.rb:28:in
`block in run'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/actions/middleware/remote_action.rb:58:in
`block (2 levels) in as_remote_user'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/katello/util/thread_session.rb:84:in
`pulp_config'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/actions/middleware/remote_action.rb:44:in
`as_pulp_user'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/actions/middleware/remote_action.rb:57:in
`block in as_remote_user'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/katello/util/thread_session.rb:91:in
`cp_config'
- /opt/rh/ruby193/root/usr/share/gems/gems/katello-2.2.0/app/lib/actions/middleware/remote_action.rb:39:in
`as_cp_user'
```


Related issues

Has duplicate Katello - Bug #10543: Composite Content View Promotions Fail with Resource Not Found ErrorsDuplicate2015-05-18

History

#1 Updated by Eric Helms over 6 years ago

  • Legacy Backlogs Release (now unused) set to 23
  • Triaged changed from No to Yes

#2 Updated by Eric Helms over 6 years ago

  • Legacy Backlogs Release (now unused) changed from 23 to 51

#3 Updated by Eric Helms over 6 years ago

Greg Koch wrote:

Composite content view with 8 content views containing public yum repos and 1 content view with a private repo.

Public Repos include:
dell, epel, hp sdr, katello client 6, puppet el 6, rhel 6.6, rhel 6Server HA, vmware

Server specs:
32GB ram
24 hyper threaded cores (2x6 core chips)
2TB allocated to /var/lib/pulp

A few traces are included below:

935: Actions::Pulp::Repository::PurgeEmptyErrata (error) [ 10.76s / 10.76s ]  Skip
Started at: 2015-05-18 18:07:27 UTC

Ended at: 2015-05-18 18:07:38 UTC

Real time: 10.76s

Execution time (excluding suspended state): 10.76s

Input:

---
pulp_id: lr-rhel-6_6-x86_64-comp-cv-3_0-Red_Hat_Enterprise_Linux_Server-Red_Hat_Enterprise_Linux_6_Server_-_RH_Common_RPMs_x86_64_6_6
remote_user: gkoch-bb8a038a
remote_cp_user: gkoch
locale: en
Output:

--- {}
Error:

RestClient::ResourceNotFound

404 Resource Not Found

---

 98: Actions::Pulp::Repository::RemovePuppetModule (error) [ 0.39s / 0.39s ]  Skip
Started at: 2015-05-18 18:31:26 UTC

Ended at: 2015-05-18 18:31:26 UTC

Real time: 0.39s

Execution time (excluding suspended state): 0.39s

Input:

---
pulp_id: xxxxxxxx-dev-xxxxx-lr-rhel-6_6-x86_64-comp-cv-Elvis-Elvis_5_5_x86_64
remote_user: gkoch-bb8a038a
remote_cp_user: gkoch
locale: en
Output:

--- {}
Error:

RestClient::ResourceNotFound

404 Resource Not Found

---
 
101: Actions::Pulp::Repository::CopyRpm (error) [ 0.18s / 0.18s ]  Skip
Started at: 2015-05-18 19:09:22 UTC

Ended at: 2015-05-18 19:09:22 UTC

Real time: 0.18s

Execution time (excluding suspended state): 0.18s

Input:

---
source_pulp_id: xxxxxxxx-xxxxx-lr-rhel-6_6-x86_64-comp-cv-3_0-Elvis-Elvis_5_5_x86_64
target_pulp_id: xxxxxxxx-qa-xxxxx-lr-rhel-6_6-x86_64-comp-cv-Elvis-Elvis_5_5_x86_64
clauses: 
remote_user: gkoch-bb8a038a
remote_cp_user: gkoch
locale: en
Output:

--- {}
Error:

RestClient::ResourceNotFound

404 Resource Not Found

---

#4 Updated by Eric Helms over 6 years ago

  • Has duplicate Bug #10543: Composite Content View Promotions Fail with Resource Not Found Errors added

#5 Updated by Eric Helms over 6 years ago

Potential patch to test:

EL6: /usr/lib/python2.6/site-packages/pulp
EL7: /usr/lib/python2.7/site-packages/pulp

Apply the following patch:

https://patch-diff.githubusercontent.com/raw/pulp/pulp/pull/1864.patch

katello-service restart

#6 Updated by Eric Helms over 6 years ago

  • Legacy Backlogs Release (now unused) changed from 51 to 55

#7 Updated by Eric Helms over 6 years ago

  • Legacy Backlogs Release (now unused) changed from 55 to 61

#8 Updated by Eric Helms about 6 years ago

  • Legacy Backlogs Release (now unused) changed from 61 to 31

#9 Updated by Eric Helms about 6 years ago

  • Category set to 91
  • Status changed from New to Resolved

This should be available now in Pulp 2.6.2 which is in the nightly repositories and the 2.3 repositories. I am setting this to resolved, if this is seen again please re-open.

Also available in: Atom PDF