Project

General

Profile

Bug #33451

Many Postgres ERRORs (duplicate key) especially on RedHat repo sync

Added by Justin Sherrill 9 months ago. Updated 4 months ago.

Status:
Closed
Priority:
Normal
Category:
Repositories
Target version:
Difficulty:
Triaged:
Yes
Bugzilla link:
Fixed in Releases:
Found in Releases:
Red Hat JIRA:

Description

Cloned from https://bugzilla.redhat.com/show_bug.cgi?id=1991557

Description of problem:

As part of the Satellite 6.9 Pulp 3 High-Touch Beta, several duplicate key errors were observed on Red Hat repository syncs.

During this HTB, a Satellite 6.9 instance went through the process of migration to Pulp 3, a switchover to Pulp 3 was performed and content management workflows executed.

Version-Release number of selected component (if applicable):
6.9.1

Errors observed:

The errors on Thursday are between 03:00am-04:00am that was the first time I was running the RedHat Repositories sync after the pulp3 switchover

To give an idea on the tables and error i summarized the count per hour and error

root@hostname:/var/opt/rh/rh-postgresql12/lib/pgsql/data/log# grep 'UTC ERROR:' .log | cut c-1000 | sed 's/\(2021. [0-9].:\).*UTC/\1/' | sort | uniq -c
2 postgresql-Mon.log:2021-06-28 08: ERROR: duplicate key value violates unique constraint "index_katello_rpms_on_pulp_id"
843 postgresql-Thu.log:2021-07-01 03: ERROR: duplicate key value violates unique constraint "index_katello_errata_on_pulp_id"
2111 postgresql-Thu.log:2021-07-01 03: ERROR: duplicate key value violates unique constraint "index_katello_erratum_cves_on_erratum_id_and_cve_id_and_href"
3990 postgresql-Thu.log:2021-07-01 03: ERROR: duplicate key value violates unique constraint "katello_erratum_bz_eid_bid_href"
18171 postgresql-Thu.log:2021-07-01 03: ERROR: duplicate key value violates unique constraint "katello_erratum_packages_eid_nvrea_n_f"problem:

Additional info:

https://bugzilla.redhat.com/show_bug.cgi?id=1991527 - contains additional errors reported

See https://bugzilla.redhat.com/show_bug.cgi?id=1991527#c3

Associated revisions

Revision 215b0401 (diff)
Added by Justin Sherrill 4 months ago

Fixes #33451 - use upsert all for content indexing

to avoid duplicate key errors. This also improves performance.

History

#1 Updated by The Foreman Bot 9 months ago

  • Assignee set to Justin Sherrill
  • Status changed from New to Ready For Testing
  • Pull request https://github.com/Katello/katello/pull/9625 added

#2 Updated by Jonathon Turel 9 months ago

  • Triaged changed from No to Yes
  • Target version set to Katello 4.3.0

#3 Updated by Justin Sherrill 7 months ago

  • Target version changed from Katello 4.3.0 to Katello 4.4.0

#4 Updated by The Foreman Bot 4 months ago

  • Fixed in Releases Katello 4.5.0 added

#5 Updated by Justin Sherrill 4 months ago

  • Status changed from Ready For Testing to Closed

#6 Updated by Ian Ballou 4 months ago

  • Fixed in Releases Katello 4.4.0 added
  • Fixed in Releases deleted (Katello 4.5.0)
  • Pull request deleted (https://github.com/Katello/katello/pull/9625)

#7 Updated by Ian Ballou 4 months ago

  • Pull request https://github.com/Katello/katello/pull/9625 added

Also available in: Atom PDF