Project

General

Profile

Actions
Copy link

Bug #35657

closed

Remove orphan fails

Added by Lucy Fu almost 2 years ago. Updated over 1 year ago.

Status:
Closed
Priority:
Normal
Assignee:
Samir Jha
Category:
Tooling
Target version:
Katello 4.6.1
Difficulty:
Triaged:
Yes
Bugzilla link:
2163457
Pull request:
https://github.com/Katello/katello/pull/10340
Fixed in Releases:
Katello 4.7.1
Found in Releases:
Katello 4.7.0
Red Hat JIRA:

Description

reported in https://community.theforeman.org/t/remove-orphan-fails/30807

when i run manually foreman-rake katello:delete_orphaned_content RAILS_ENV=production
i see 2 tasks getting created
the first run is fine
the second one fails with the error:

undefined method `all?’ for :_created:Symbol
“/usr/share/gems/gems/pulp_rpm_client-3.17.12/lib/pulp_rpm_client/api/repositories_rpm_versions_api.rb:146:in
`list_with_http_info’”
“/usr/share/gems/gems/pulp_rpm_client-3.17.12/lib/pulp_rpm_client/api/repositories_rpm_versions_api.rb:109:in
`list’”
“/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/api/core.rb:194:in
`block in versions_list_for_repository’”
“/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/api/core.rb:245:in
`block in fetch_from_list’”
“/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/api/core.rb:240:in
`loop’”
“/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/api/core.rb:240:in
`fetch_from_list’”
“/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/api/core.rb:194:in
`versions_list_for_repository’”
“/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/smart_proxy_mirror_repository.rb:29:in
`block (2 levels) in orphan_repository_versions’”
“/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/smart_proxy_mirror_repository.rb:28:in
`collect’”
“/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/smart_proxy_mirror_repository.rb:28:in
`block in orphan_repository_versions’”
“/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/smart_proxy_mirror_repository.rb:25:in
`each’”
“/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/smart_proxy_mirror_repository.rb:25:in
`orphan_repository_versions’”
“/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/smart_proxy_repository.rb:39:in
`delete_orphan_repository_versions’”
“/usr/share/gems/gems/katello-4.6.0/app/lib/actions/pulp3/orphan_cleanup/delete_orphan_repository_versions.rb:10:in
`run’”
.....

Actions
Copy link
 #1

Updated by Ewoud Kohl van Wijngaarden almost 2 years ago

  • Project changed from Foreman to Katello
Actions
Copy link
 #2

Updated by Chris Roberts almost 2 years ago

  • Category set to Tooling
  • Target version set to Katello 4.8.0
Actions
Copy link
 #3

Updated by Chris Roberts almost 2 years ago

  • Triaged changed from No to Yes
Actions
Copy link
 #4

Updated by Gerald Vogt almost 2 years ago

  • Found in Releases Katello 4.6.0 added
Actions
Copy link
 #5

Updated by Gerald Vogt almost 2 years ago

  • Found in Releases deleted (Katello 4.6.0)

The remove orphan task for the main katello server works fine. The failing task is the one for the content proxy.

Actions
Copy link
 #6

Updated by The Foreman Bot almost 2 years ago

  • Status changed from New to Ready For Testing
  • Assignee set to Samir Jha
  • Pull request https://github.com/Katello/katello/pull/10340 added
Actions
Copy link
 #7

Updated by Samir Jha almost 2 years ago

  • Target version changed from Katello 4.8.0 to Katello 4.7.0
  • Triaged changed from Yes to No
Actions
Copy link
 #8

Updated by Samir Jha almost 2 years ago

  • Target version changed from Katello 4.7.0 to Katello 4.6.1
Actions
Copy link
 #9

Updated by The Foreman Bot almost 2 years ago

  • Fixed in Releases Katello 4.8.0 added
Actions
Copy link
 #10

Updated by Samir Jha almost 2 years ago

  • Status changed from Ready For Testing to Closed
Actions
Copy link
 #11

Updated by Samir Jha almost 2 years ago

  • Triaged changed from No to Yes
Actions
Copy link
 #12

Updated by Gerald Vogt almost 2 years ago

The pull request now cause the remove orphan task on the main server to fail in pulp_deb:

Actions::Pulp3::OrphanCleanup::DeleteOrphanRepositoryVersions

Input:

{"smart_proxy_id"=>1,
 "remote_user"=>"admin",
 "remote_cp_user"=>"admin",
 "current_request_id"=>nil,
 "current_timezone"=>"UTC",
 "current_organization_id"=>nil,
 "current_location_id"=>nil,
 "current_user_id"=>1}

Output:

{}

Exception:

PulpDebClient::ApiError: Error message: the server returns an error
HTTP status code: 500
Response headers: {"date"=>"Thu, 24 Nov 2022 08:12:11 GMT", "server"=>"gunicorn", "content-type"=>"text/html; charset=UTF-8", "x-frame-options"=>"DENY", "content-length"=>"145", "vary"=>"Cookie", "x-content-type-options"=>"nosniff", "referrer-policy"=>"same-origin", "correlation-id"=>"7b661436db6f4000b0320860313ab068", "access-control-expose-headers"=>"Correlation-ID", "via"=>"1.1 foreman8.example.com", "connection"=>"close"}
Response body: 
<!doctype html>
<html lang="en">
<head>
  <title>Server Error (500)</title>
</head>
<body>
  <h1>Server Error (500)</h1><p></p>
</body>
</html>

Backtrace:

/usr/share/gems/gems/pulp_deb_client-2.18.1/lib/pulp_deb_client/api_client.rb:83:in `call_api'
/usr/share/gems/gems/pulp_deb_client-2.18.1/lib/pulp_deb_client/api/repositories_apt_versions_api.rb:196:in `list_with_http_info'
/usr/share/gems/gems/pulp_deb_client-2.18.1/lib/pulp_deb_client/api/repositories_apt_versions_api.rb:109:in `list'
/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/api/core.rb:194:in `block in versions_list_for_repository'
/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/api/core.rb:245:in `block in fetch_from_list'
/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/api/core.rb:240:in `loop'
/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/api/core.rb:240:in `fetch_from_list'
/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/api/core.rb:194:in `versions_list_for_repository'
/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/api/core.rb:187:in `block in repository_versions'
/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/api/core.rb:186:in `collect'
/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/api/core.rb:186:in `repository_versions'
/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/smart_proxy_repository.rb:59:in `block in orphan_repository_versions'
/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/smart_proxy_repository.rb:57:in `each'
/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/smart_proxy_repository.rb:57:in `orphan_repository_versions'
/usr/share/gems/gems/katello-4.6.0/app/services/katello/pulp3/smart_proxy_repository.rb:39:in `delete_orphan_repository_versions'
/usr/share/gems/gems/katello-4.6.0/app/lib/actions/pulp3/orphan_cleanup/delete_orphan_repository_versions.rb:10:in `run'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:582:in `block (3 levels) in execute_run'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:32:in `run'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/katello-4.6.0/app/lib/actions/middleware/remote_action.rb:16:in `block in run'
/usr/share/gems/gems/katello-4.6.0/app/lib/actions/middleware/remote_action.rb:40:in `block in as_remote_user'
/usr/share/gems/gems/katello-4.6.0/app/models/katello/concerns/user_extensions.rb:21:in `cp_config'
/usr/share/gems/gems/katello-4.6.0/app/lib/actions/middleware/remote_action.rb:27:in `as_cp_user'
/usr/share/gems/gems/katello-4.6.0/app/lib/actions/middleware/remote_action.rb:39:in `as_remote_user'
/usr/share/gems/gems/katello-4.6.0/app/lib/actions/middleware/remote_action.rb:16:in `run'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/rails_executor_wrap.rb:14:in `block in run'
/usr/share/gems/gems/activesupport-6.1.6.1/lib/active_support/execution_wrapper.rb:91:in `wrap'
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/rails_executor_wrap.rb:13:in `run'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action/progress.rb:31:in `with_progress_calculation'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action/progress.rb:17:in `run'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/load_setting_values.rb:20:in `run'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_request_id.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_request_id.rb:52:in `restore_current_request_id'
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_request_id.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_timezone.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_timezone.rb:44:in `restore_curent_timezone'
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_timezone.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_taxonomies.rb:45:in `restore_current_taxonomies'
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_taxonomies.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:32:in `run'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:27:in `pass'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware.rb:19:in `pass'
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_user.rb:15:in `block in run'
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_user.rb:54:in `restore_curent_user'
/usr/share/gems/gems/foreman-tasks-7.0.0/app/lib/actions/middleware/keep_current_user.rb:15:in `run'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/stack.rb:23:in `call'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/middleware/world.rb:31:in `execute'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:581:in `block (2 levels) in execute_run'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:580:in `catch'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:580:in `block in execute_run'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:483:in `block in with_error_handling'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:483:in `catch'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:483:in `with_error_handling'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:575:in `execute_run'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/action.rb:296:in `execute'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:18:in `block (2 levels) in execute'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract.rb:167:in `with_meta_calculation'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:17:in `block in execute'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:32:in `open_action'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/execution_plan/steps/abstract_flow_step.rb:16:in `execute'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/director.rb:69:in `execute'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:11:in `block (2 levels) in perform'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/executors.rb:18:in `run_user_code'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:9:in `block in perform'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:25:in `with_telemetry'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/executors/sidekiq/worker_jobs.rb:8:in `perform'
/usr/share/gems/gems/dynflow-1.6.7/lib/dynflow/executors/sidekiq/serialization.rb:27:in `perform'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:192:in `execute_job'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:165:in `block (2 levels) in process'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/middleware/chain.rb:133:in `invoke'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:164:in `block in process'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:137:in `block (6 levels) in dispatch'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/job_retry.rb:109:in `local'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:136:in `block (5 levels) in dispatch'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq.rb:37:in `block in <module:Sidekiq>'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:132:in `block (4 levels) in dispatch'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:250:in `stats'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:127:in `block (3 levels) in dispatch'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/job_logger.rb:8:in `call'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:126:in `block (2 levels) in dispatch'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/job_retry.rb:74:in `global'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:125:in `block in dispatch'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/logging.rb:48:in `with_context'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/logging.rb:42:in `with_job_hash_context'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:124:in `dispatch'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:163:in `process'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:83:in `process_one'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/processor.rb:71:in `run'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/util.rb:16:in `watchdog'
/usr/share/gems/gems/sidekiq-5.2.10/lib/sidekiq/util.rb:25:in `block in safe_thread'
/usr/share/gems/gems/logging-2.3.1/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'

Actions
Copy link
 #13

Updated by The Foreman Bot almost 2 years ago

  • Pull request https://github.com/Katello/katello/pull/10375 added
Actions
Copy link
 #14

Updated by Samir Jha almost 2 years ago

  • Pull request deleted (https://github.com/Katello/katello/pull/10375)
Actions
Copy link
 #15

Updated by Stephen Collier almost 2 years ago

  • Fixed in Releases Katello 4.7.1 added
  • Fixed in Releases deleted (Katello 4.8.0)
  • Found in Releases Katello 4.7.0 added

this issue above is still happening in katello 4.7.0

Error message: the server returns an error
HTTP status code: 500
Response headers: {"date"=>"Tue, 03 Jan 2023 01:29:16 GMT", "server"=>"gunicorn", "content-type"=>"text/html; charset=UTF-8", "x-frame-options"=>"DENY", "content-length"=>"145", "vary"=>"Cookie", "x-content-type-options"=>"nosniff", "referrer-policy"=>"same-origin", "correlation-id"=>"9c2edeaf94ea462e88673ba6ff5589c0", "access-control-expose-headers"=>"Correlation-ID", "via"=>"1.1 XXXXXXXXXXXXXXXXXX", "connection"=>"close"}
Response body:
<!doctype html>
<html lang="en">
<head>
<title>Server Error (500)</title>
</head>
<body>
<h1>Server Error (500)</h1><p></p>
</body>
</html>

Actions
Copy link
 #16

Updated by Samir Jha almost 2 years ago

@Stephen: Could you provide the traceback from the foreman logs for the error you're seeing. Also can you confirm if this is on 4.7.1 in case you're seeing the same error?

Actions
Copy link
 #17

Updated by Samir Jha over 1 year ago

  • Bugzilla link set to 2163457
Actions
Copy link

Also available in: Atom PDF