Project

General

Profile

Bug #30775

publishing a large content view with filters fails with error on multi-copy api

Added by Justin Sherrill 3 months ago. Updated about 2 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Content Views
Target version:
Difficulty:
Triaged:
Yes
Bugzilla link:

Description

When publishing a large content view with ~10K packages, with a filter to exclude a few, it fails with an error:

PulpRpmClient::ApiError

Error message: the server returns an error HTTP status code: 502 Response headers: {"date"=>"Thu, 03 Sep 2020 14:04:49 GMT", "server"=>"Apache", "content-length"=>"422", "connection"=>"close", "content-type"=>"text/html; charset=iso-8859-1"} Response body: <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN"> <html><head> <title>502 Proxy Error</title> </head><body> <h1>Proxy Error</h1> <p>The proxy server received an invalid response from an upstream server.<br /> The proxy server could not handle the request <em><a href="/pulp/api/v3/rpm/copy/">POST /pulp/api/v3/rpm/copy/</a></em>.<p> Reason: <strong>Error reading from remote server</strong></p></p> </body></html>

this is largely because the copy is not breaking the packages into chunks.

In addition, all package/errata href's are being saved in the input. This causes a couple of problems:
  • long planning time, so publishes take longer to get started
  • bloated tasks in the database

Associated revisions

Revision b64f6f7b (diff)
Added by Ian Ballou about 2 months ago

Fixes #30775 - publishing large cv w/ filters fails Pulp 3 yum (#8935)

  • Fixes #30775 - publishing large cv w/ filters fails Pulp 3 yum
  • Refs #30775 - refactor small-repo chunk skipping and more chunked copy tests added

History

#1 Updated by The Foreman Bot 3 months ago

  • Assignee set to Ian Ballou
  • Status changed from New to Ready For Testing
  • Pull request https://github.com/Katello/katello/pull/8935 added

#2 Updated by Samir Jha 3 months ago

  • Triaged changed from No to Yes

#3 Updated by The Foreman Bot about 2 months ago

  • Fixed in Releases Katello 4.0.0 added

#4 Updated by The Foreman Bot about 2 months ago

  • Pull request https://github.com/Katello/katello/pull/8969 added

#5 Updated by Anonymous about 2 months ago

  • Status changed from Ready For Testing to Closed

Also available in: Atom PDF