Project

General

Profile

Bug #20854

Katello repository export requires a shared filesystem with Pulp

Added by Ewoud Kohl van Wijngaarden over 4 years ago. Updated 6 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
Repositories
Target version:
Difficulty:
Triaged:
Yes
Bugzilla link:
Pull request:
Fixed in Releases:
Found in Releases:
Red Hat JIRA:

Description

Copied from https://groups.google.com/forum/#!topic/foreman-dev/sofiVxnbLGQ

While working on splitting the Katello installation from its services I ran into the pulp_export_destination setting. That made me look at the Repository Export which assumes the pulp server and application server share the same filesystem.

That means I'm stuck in attempting to split services to different machines where I don't want to assume a shared filesystem. Looking at the containers it's not handled there either. I don't see a way to support this feature without a major rewrite of the code but I'm not familiar with the Katello code base.

Two suggestions:
  • This should be fixed in Pulp
  • The workaround should be in the Smart Proxy like is done in vanilla Foreman

Related issues

Related to Installer - Tracker #20850: Allow split deployments using the installerNew

History

#1 Updated by Ewoud Kohl van Wijngaarden over 4 years ago

  • Related to Tracker #20850: Allow split deployments using the installer added

#2 Updated by Justin Sherrill over 4 years ago

  • Legacy Backlogs Release (now unused) set to 114

#3 Updated by Ewoud Kohl van Wijngaarden about 1 year ago

  • Triaged set to No

This was correct for Pulp 2. It may no longer be an issue with Pulp 3. That requires verification.

#4 Updated by Jonathon Turel 12 months ago

  • Status changed from New to Need more information

Ewoud, do you have a strategy to verify this? Although it was designed to work this way the setup doesn't seem trivial by any means.

#5 Updated by Ewoud Kohl van Wijngaarden 12 months ago

What should be verified is that /var/lib/pulp is not readable by a user other than pulp. If an export can be performed correctly without the foreman and foreman-proxy user being able to read the files directly. I think that's good enough.

#6 Updated by Chris Roberts 12 months ago

  • Triaged changed from No to Yes
  • Target version changed from Katello Backlog to Katello 4.2.0
  • Status changed from Need more information to New
  • Category set to Repositories

#7 Updated by Chris Roberts 11 months ago

This is still not working, here are the results of my testing:

[root@katello ~]# hammer content-export complete library --organization-id 1
Could not export the library:
ERF42-3817 [Foreman::Exception]: Unable to export. 'pulpcore_export_destination' setting is not set to a valid directory.

[root@katello ~]# ll /var/lib/pulp/
total 24
drwxr-xr-x. 6 pulp pulp 4096 Jun 11 18:42 assets
drwxrwx---. 2 pulp pulp 4096 Jun 11 18:40 exports
drwxrwx---. 2 pulp pulp 4096 Jun 11 18:40 imports
drwxr-x---. 3 pulp pulp 4096 Jun 14 13:15 media
drwxrwx---. 2 pulp pulp 4096 Jun 11 18:40 sync_imports
drwxr-x---. 3 pulp pulp 4096 Jun 14 13:21 tmp

ALLOWED_IMPORT_PATHS = ["/var/lib/pulp/sync_imports", "/var/lib/pulp/imports"]
ALLOWED_EXPORT_PATHS = ["/var/lib/pulp/exports"]

2021-06-14T13:56:39 [E|app|5717a3e4] <FalseClass> false
2021-06-14T13:56:39 [I|bac|5717a3e4] Task {label: Actions::Pulp3::Orchestration::ContentViewVersion::ExportLibrary, id: e4304359-1832-47cb-80f7-892abf2f993a, execution_plan_id: 44cf2e55-38e5-4e8b-aae3-ddb49d393069} state changed: stopped result: error
2021-06-14T13:56:40 [I|bac|5717a3e4] Task {label: Actions::Pulp3::Orchestration::ContentViewVersion::ExportLibrary, id: e4304359-1832-47cb-80f7-892abf2f993a, execution_plan_id: 44cf2e55-38e5-4e8b-aae3-ddb49d393069} state changed: stopped result: error
2021-06-14T13:56:40 [E|app|5717a3e4] Foreman::Exception: ERF42-3817 [Foreman::Exception]: Unable to export. 'pulpcore_export_destination' setting is not set to a valid directory.
5717a3e4 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.0.1.1/app/lib/actions/pulp3/orchestration/content_view_version/export.rb:28:in `plan'
5717a3e4 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:514:in `block (3 levels) in execute_plan'

#8 Updated by Partha Aji 11 months ago

@croberts
So master has this line removed. I am guessing we can remove https://github.com/Katello/katello/blob/KATELLO-4.0/app/lib/actions/pulp3/orchestration/content_view_version/export.rb#L26-L28 if we want this in 4.0

#9 Updated by Chris Roberts 11 months ago

  • Assignee set to Chris Roberts
  • Status changed from New to Closed

Tested on Katello 4.1 with the new DRYRUN options from Pulp 3.11

This works correctly, here is the output showing the export completed ok and my /var/lib/pulp showing it mounted over NFS

[root@katello ~]# hammer content-export complete library --organization-id 1
[...................................................................................................................] [100%]
Generated /var/lib/pulp/exports/Default_Organization/Export-Library/1.0/2021-06-16T17-40-47-00-00/metadata.json

[root@katello ~]# findmnt
TARGET SOURCE FSTYPE OPTIONS
└─/var/lib/pulp nfs.toledo.satellite.lab.eng.rdu2.redhat.com:/pulp
nfs4 rw,relatime,vers=4.1,rsize=262144,wsize=262144,namlen=255,hard,p
Closing this as works.

#10 Updated by Chris Roberts 11 months ago

Partha,

It looks like 6.10 GA will be based on Katello 4.1, so we are fine to close it out.

#11 Updated by Chris Roberts 11 months ago

  • Fixed in Releases Katello 4.1.0 added

#12 Updated by Justin Sherrill 9 months ago

  • Status changed from Closed to Resolved

#13 Updated by Chris Roberts 6 months ago

  • Target version changed from Katello 4.2.0 to Katello Recycle Bin

Also available in: Atom PDF