Project

General

Profile

Actions

Bug #20854

closed

Katello repository export requires a shared filesystem with Pulp

Added by Ewoud Kohl van Wijngaarden over 6 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Repositories
Target version:
Difficulty:
Triaged:
Yes
Fixed in Releases:
Found in Releases:

Description

Copied from https://groups.google.com/forum/#!topic/foreman-dev/sofiVxnbLGQ

While working on splitting the Katello installation from its services I ran into the pulp_export_destination setting. That made me look at the Repository Export which assumes the pulp server and application server share the same filesystem.

That means I'm stuck in attempting to split services to different machines where I don't want to assume a shared filesystem. Looking at the containers it's not handled there either. I don't see a way to support this feature without a major rewrite of the code but I'm not familiar with the Katello code base.

Two suggestions:
  • This should be fixed in Pulp
  • The workaround should be in the Smart Proxy like is done in vanilla Foreman

Related issues 1 (1 open0 closed)

Related to Installer - Tracker #20850: Allow split deployments using the installerNew

Actions
Actions #1

Updated by Ewoud Kohl van Wijngaarden over 6 years ago

  • Related to Tracker #20850: Allow split deployments using the installer added
Actions #2

Updated by Justin Sherrill over 6 years ago

  • translation missing: en.field_release set to 114
Actions #3

Updated by Ewoud Kohl van Wijngaarden almost 3 years ago

  • Triaged set to No

This was correct for Pulp 2. It may no longer be an issue with Pulp 3. That requires verification.

Actions #4

Updated by Jonathon Turel almost 3 years ago

  • Status changed from New to Need more information

Ewoud, do you have a strategy to verify this? Although it was designed to work this way the setup doesn't seem trivial by any means.

Actions #5

Updated by Ewoud Kohl van Wijngaarden almost 3 years ago

What should be verified is that /var/lib/pulp is not readable by a user other than pulp. If an export can be performed correctly without the foreman and foreman-proxy user being able to read the files directly. I think that's good enough.

Actions #6

Updated by Chris Roberts almost 3 years ago

  • Category set to Repositories
  • Status changed from Need more information to New
  • Target version changed from Katello Backlog to Katello 4.2.0
  • Triaged changed from No to Yes
Actions #7

Updated by Chris Roberts almost 3 years ago

This is still not working, here are the results of my testing:

[root@katello ~]# hammer content-export complete library --organization-id 1
Could not export the library:
ERF42-3817 [Foreman::Exception]: Unable to export. 'pulpcore_export_destination' setting is not set to a valid directory.

[root@katello ~]# ll /var/lib/pulp/
total 24
drwxr-xr-x. 6 pulp pulp 4096 Jun 11 18:42 assets
drwxrwx---. 2 pulp pulp 4096 Jun 11 18:40 exports
drwxrwx---. 2 pulp pulp 4096 Jun 11 18:40 imports
drwxr-x---. 3 pulp pulp 4096 Jun 14 13:15 media
drwxrwx---. 2 pulp pulp 4096 Jun 11 18:40 sync_imports
drwxr-x---. 3 pulp pulp 4096 Jun 14 13:21 tmp

ALLOWED_IMPORT_PATHS = ["/var/lib/pulp/sync_imports", "/var/lib/pulp/imports"]
ALLOWED_EXPORT_PATHS = ["/var/lib/pulp/exports"]

2021-06-14T13:56:39 [E|app|5717a3e4] <FalseClass> false
2021-06-14T13:56:39 [I|bac|5717a3e4] Task {label: Actions::Pulp3::Orchestration::ContentViewVersion::ExportLibrary, id: e4304359-1832-47cb-80f7-892abf2f993a, execution_plan_id: 44cf2e55-38e5-4e8b-aae3-ddb49d393069} state changed: stopped result: error
2021-06-14T13:56:40 [I|bac|5717a3e4] Task {label: Actions::Pulp3::Orchestration::ContentViewVersion::ExportLibrary, id: e4304359-1832-47cb-80f7-892abf2f993a, execution_plan_id: 44cf2e55-38e5-4e8b-aae3-ddb49d393069} state changed: stopped result: error
2021-06-14T13:56:40 [E|app|5717a3e4] Foreman::Exception: ERF42-3817 [Foreman::Exception]: Unable to export. 'pulpcore_export_destination' setting is not set to a valid directory.
5717a3e4 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.0.1.1/app/lib/actions/pulp3/orchestration/content_view_version/export.rb:28:in `plan'
5717a3e4 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:514:in `block (3 levels) in execute_plan'

Actions #8

Updated by Partha Aji almost 3 years ago

@croberts
So master has this line removed. I am guessing we can remove https://github.com/Katello/katello/blob/KATELLO-4.0/app/lib/actions/pulp3/orchestration/content_view_version/export.rb#L26-L28 if we want this in 4.0

Actions #9

Updated by Chris Roberts almost 3 years ago

  • Status changed from New to Closed
  • Assignee set to Chris Roberts

Tested on Katello 4.1 with the new DRYRUN options from Pulp 3.11

This works correctly, here is the output showing the export completed ok and my /var/lib/pulp showing it mounted over NFS

[root@katello ~]# hammer content-export complete library --organization-id 1
[...................................................................................................................] [100%]
Generated /var/lib/pulp/exports/Default_Organization/Export-Library/1.0/2021-06-16T17-40-47-00-00/metadata.json

[root@katello ~]# findmnt
TARGET SOURCE FSTYPE OPTIONS
└─/var/lib/pulp nfs.toledo.satellite.lab.eng.rdu2.redhat.com:/pulp
nfs4 rw,relatime,vers=4.1,rsize=262144,wsize=262144,namlen=255,hard,p
Closing this as works.

Actions #10

Updated by Chris Roberts almost 3 years ago

Partha,

It looks like 6.10 GA will be based on Katello 4.1, so we are fine to close it out.

Actions #11

Updated by Chris Roberts almost 3 years ago

  • Fixed in Releases Katello 4.1.0 added
Actions #12

Updated by Justin Sherrill over 2 years ago

  • Status changed from Closed to Resolved
Actions #13

Updated by Chris Roberts over 2 years ago

  • Target version changed from Katello 4.2.0 to Katello Recycle Bin
Actions

Also available in: Atom PDF