Project

General

Profile

Bug #24249

hammer repository upload-content fails when uploading larger files

Added by Ivan Necas about 3 years ago. Updated about 3 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Hammer
Target version:
Difficulty:
Triaged:
Yes
Bugzilla link:
Fixed in Releases:
Found in Releases:

Description

Cloned from https://bugzilla.redhat.com/show_bug.cgi?id=1600906

Description of problem:

hammer repository upload-content fails when uploading larger files

Version-Release number of selected component (if applicable):
Sat 6.4

How reproducible:
always

Steps to Reproduce:
1. have a larger file (10MB+)
2. hammer repository upload-content --name image --product RHVH --organization "Default Organization" --path myfile.img

Actual results:
warning: Overriding "Content-Type" header "multipart/form-data" with "multipart/form-data; boundary=----RubyFormBoundaryueJ8T7XEKh2BLjan" due to payload
Failed to upload file 'squashfs-1.img' to repository. Please check the file and try again.

Expected results:
Successfully uploaded file 'myfile.img'

Additional info:

Pulp produced this error message:

Jul 12 16:20:03 sat-6-4-qa-rhel7 pulp: pulp.server.webservices.middleware.exception:ERROR: Unhandled Exception
Jul 12 16:20:03 sat-6-4-qa-rhel7 pulp: pulp.server.webservices.middleware.exception:ERROR: (1125-57920) Request body exceeded settings.DATA_UPLOAD_MAX_MEMORY_SIZE.
Jul 12 16:20:03 sat-6-4-qa-rhel7 pulp: pulp.server.webservices.middleware.exception:ERROR: (1125-57920) Traceback (most recent call last):
Jul 12 16:20:03 sat-6-4-qa-rhel7 pulp: pulp.server.webservices.middleware.exception:ERROR: (1125-57920) File "/usr/lib/python2.7/site-packages/django/core/handlers/base.py", line 185, in _get_response
Jul 12 16:20:03 sat-6-4-qa-rhel7 pulp: pulp.server.webservices.middleware.exception:ERROR: (1125-57920) response = wrapped_callback(request, *callback_args, **callback_kwargs)

Seems like the limit on the maximum file size changed between django releases, currently limiting the size to something around 2.5 MB:

cat /usr/lib/python2.7/site-packages/django/conf/global_settings.py | grep DATA_UPLOAD_MAX_MEMORY_SIZE
DATA_UPLOAD_MAX_MEMORY_SIZE = 2621440 # i.e. 2.5 MB

We need to update hammer to conform to this size (the chunk size there is 4 000 000 bytes at the moment.

Additional issue is the warning: Overriding "Content-Type" header "multipart/form-data" with "multipart/form-data; boundary=----RubyFormBoundaryueJ8T7XEKh2BLjan" due to payload - this seems to be an issue in rest-client: until it's fixed properly in there, we should suppress the warning, which is very confusing to the user.

Associated revisions

Revision 54e9f0b3 (diff)
Added by Ivan Necas about 3 years ago

Fixes #24249 - fix content-upload functionality (#570)

1. decrease the chunk size to be compatible with Pulp's expectations
2. suppress false-positive warnings from rest-client
3. log exceptions properly

History

#1 Updated by Ivan Necas about 3 years ago

  • Category changed from Repositories to Hammer
  • Subject changed from hammer repository upload-content fails when uploading larger files to hammer repository upload-content fails when uploading larger files

#2 Updated by Ivan Necas about 3 years ago

  • Target version set to Katello 3.7.1
  • Assignee set to Ivan Necas
  • Status changed from New to Assigned

Proposing for katello 3.7.1, as it prevents using the upload-content functionality

#3 Updated by The Foreman Bot about 3 years ago

  • Status changed from Assigned to Ready For Testing
  • Pull request https://github.com/Katello/hammer-cli-katello/pull/570 added

#4 Updated by Ivan Necas about 3 years ago

  • % Done changed from 0 to 100
  • Status changed from Ready For Testing to Closed

#5 Updated by Jonathon Turel about 3 years ago

  • Target version changed from Katello 3.7.1 to Katello 3.7.0

Also available in: Atom PDF