Bug #20854: Katello repository export requires a shared filesystem with Pulp - Katello - Foreman

Custom queries

All issues closed in the last 3 weeks
Assigned to me
Closed Issues without Release
Easy and Trivial Issues
Good first issues - Foreman core
Katello 4.15.1 TODO
Mine
My issues closed in the last 3 weeks
open docker issues 2.1
Open Features
Open Katello issues by target version
Open Pull Requests
Stale
Trackers
Untriaged
Untriaged and opened in the past two weeks

Actions

Copy link

Bug #20854

closed

Katello repository export requires a shared filesystem with Pulp

Added by Ewoud Kohl van Wijngaarden over 7 years ago. Updated about 3 years ago.

Status:

Resolved

Priority:

Normal

Assignee:

Chris Roberts

Category:

Repositories

Target version:

Katello Recycle Bin

Difficulty:

Triaged:

Yes

Bugzilla link:

Pull request:

Fixed in Releases:

Katello 4.1.0

Found in Releases:

Red Hat JIRA:

Description

Copied from https://groups.google.com/forum/#!topic/foreman-dev/sofiVxnbLGQ

While working on splitting the Katello installation from its services I ran into the pulp_export_destination setting. That made me look at the Repository Export which assumes the pulp server and application server share the same filesystem.

That means I'm stuck in attempting to split services to different machines where I don't want to assume a shared filesystem. Looking at the containers it's not handled there either. I don't see a way to support this feature without a major rewrite of the code but I'm not familiar with the Katello code base.

Two suggestions:

This should be fixed in Pulp
The workaround should be in the Smart Proxy like is done in vanilla Foreman

Related issues 1 (1 open — 0 closed)

Related to Installer - Tracker #20850: Allow split deployments using the installer

New

Actions

Issue # Delay: days Cancel

History
Notes
Property changes

Actions

Copy link

Updated by Ewoud Kohl van Wijngaarden over 7 years ago

Related to Tracker #20850: Allow split deployments using the installer added

Actions

Copy link

Updated by Justin Sherrill over 7 years ago

Translation missing: en.field_release set to 114

Actions

Copy link

Updated by Ewoud Kohl van Wijngaarden over 3 years ago

Triaged set to No

This was correct for Pulp 2. It may no longer be an issue with Pulp 3. That requires verification.

Actions

Copy link

Updated by Jonathon Turel over 3 years ago

Status changed from New to Need more information

Ewoud, do you have a strategy to verify this? Although it was designed to work this way the setup doesn't seem trivial by any means.

Actions

Copy link

Updated by Ewoud Kohl van Wijngaarden over 3 years ago

What should be verified is that /var/lib/pulp is not readable by a user other than pulp. If an export can be performed correctly without the foreman and foreman-proxy user being able to read the files directly. I think that's good enough.

Actions

Copy link

Updated by Chris Roberts over 3 years ago

Category set to Repositories
Status changed from Need more information to New
Target version changed from Katello Backlog to Katello 4.2.0
Triaged changed from No to Yes

Actions

Copy link

Updated by Chris Roberts over 3 years ago

This is still not working, here are the results of my testing:

[root@katello ~]# hammer content-export complete library --organization-id 1
Could not export the library:
ERF42-3817 [Foreman::Exception]: Unable to export. 'pulpcore_export_destination' setting is not set to a valid directory.

[root@katello ~]# ll /var/lib/pulp/
total 24
drwxr-xr-x. 6 pulp pulp 4096 Jun 11 18:42 assets
drwxrwx---. 2 pulp pulp 4096 Jun 11 18:40 exports
drwxrwx---. 2 pulp pulp 4096 Jun 11 18:40 imports
drwxr-x---. 3 pulp pulp 4096 Jun 14 13:15 media
drwxrwx---. 2 pulp pulp 4096 Jun 11 18:40 sync_imports
drwxr-x---. 3 pulp pulp 4096 Jun 14 13:21 tmp

ALLOWED_IMPORT_PATHS = ["/var/lib/pulp/sync_imports", "/var/lib/pulp/imports"]
ALLOWED_EXPORT_PATHS = ["/var/lib/pulp/exports"]

2021-06-14T13:56:39 [E|app|5717a3e4] <FalseClass> false
2021-06-14T13:56:39 [I|bac|5717a3e4] Task {label: Actions::Pulp3::Orchestration::ContentViewVersion::ExportLibrary, id: e4304359-1832-47cb-80f7-892abf2f993a, execution_plan_id: 44cf2e55-38e5-4e8b-aae3-ddb49d393069} state changed: stopped result: error
2021-06-14T13:56:40 [I|bac|5717a3e4] Task {label: Actions::Pulp3::Orchestration::ContentViewVersion::ExportLibrary, id: e4304359-1832-47cb-80f7-892abf2f993a, execution_plan_id: 44cf2e55-38e5-4e8b-aae3-ddb49d393069} state changed: stopped result: error
2021-06-14T13:56:40 [E|app|5717a3e4] Foreman::Exception: ERF42-3817 [Foreman::Exception]: Unable to export. 'pulpcore_export_destination' setting is not set to a valid directory.
5717a3e4 | /opt/theforeman/tfm/root/usr/share/gems/gems/katello-4.0.1.1/app/lib/actions/pulp3/orchestration/content_view_version/export.rb:28:in `plan'
5717a3e4 | /opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.7/lib/dynflow/action.rb:514:in `block (3 levels) in execute_plan'

Actions

Copy link

Updated by Partha Aji over 3 years ago

@croberts
So master has this line removed. I am guessing we can remove https://github.com/Katello/katello/blob/KATELLO-4.0/app/lib/actions/pulp3/orchestration/content_view_version/export.rb#L26-L28 if we want this in 4.0

Actions

Copy link

Updated by Chris Roberts over 3 years ago

Status changed from New to Closed
Assignee set to Chris Roberts

Tested on Katello 4.1 with the new DRYRUN options from Pulp 3.11

This works correctly, here is the output showing the export completed ok and my /var/lib/pulp showing it mounted over NFS

[root@katello ~]# hammer content-export complete library --organization-id 1
[...................................................................................................................] [100%]
Generated /var/lib/pulp/exports/Default_Organization/Export-Library/1.0/2021-06-16T17-40-47-00-00/metadata.json

[root@katello ~]# findmnt
TARGET SOURCE FSTYPE OPTIONS
└─/var/lib/pulp nfs.toledo.satellite.lab.eng.rdu2.redhat.com:/pulp
nfs4 rw,relatime,vers=4.1,rsize=262144,wsize=262144,namlen=255,hard,p
Closing this as works.

Actions

Copy link

#10