Bug #11324
closed
Pulp: endless stream of Resetting dropped connection in Logfiles when syncing repo
Added by Sebastian @ over 9 years ago.
Updated about 5 years ago.
Description
Pulp throws an endless stream of localhost pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: when a repo is synced currently.
Usual rate is 5-10 log events per second.
I believe this behavior slow the overall sync process + it kinda kills the origin server.
I had a test case on a local server where the load went up to 20 instantly with a huge mass of httpd processes spawning.
Changing the pulp worker count does not fix this issue entirely, even with 1 worker it still hammers the target repo server.
- Translation missing: en.field_release set to 77
- Triaged changed from No to Yes
- Translation missing: en.field_release changed from 77 to 78
- Translation missing: en.field_release changed from 78 to 113
- Translation missing: en.field_release changed from 113 to 86
We are also seeing this issue. Our capsule is connected remotely via a VPN over the internet, which accounts for some latency and delay, but I'm not sure it accounts for all the reset connections. The sync seems to continue fine, but due to the speed of the VPN connection, the sync timesout after 12 hours.
Apr 7 07:29:53 capsule pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: katello.example.com
Apr 7 07:31:10 capsule pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: katello.example.com
Apr 7 07:31:25 capsule pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: katello.example.com
Apr 7 07:31:25 capsule pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: katello.example.com
Apr 7 07:31:29 capsule pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: katello.example.com
Apr 7 07:33:14 capsule pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: katello.example.com
Apr 7 07:33:17 capsule pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: katello.example.com
Apr 7 07:33:41 capsule pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: katello.example.com
Apr 7 07:33:57 capsule pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: katello.example.com
Apr 7 07:34:09 capsule pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: katello.example.com
Apr 7 07:34:27 capsule pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: katello.example.com
Apr 7 07:35:37 capsule pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: katello.example.com
Apr 7 07:35:56 capsule pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: katello.example.com
Apr 7 07:36:45 capsule pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: katello.example.com
Apr 7 07:36:47 capsule pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: katello.example.com
Apr 7 07:37:22 capsule pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: katello.example.com
Apr 7 07:37:59 capsule pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: katello.example.com
Apr 7 07:38:12 capsule pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: katello.example.com
Apr 7 07:38:42 capsule pulp: requests.packages.urllib3.connectionpool:INFO: Resetting dropped connection: katello.example.com
Greg,
The 12 hour timeout can be adjusted under administer > settings > Katello > pulp_sync_node_action_finish_timeout
I think it is unrelated to these dropped connections.
-Justin
- Translation missing: en.field_release changed from 86 to 144
- Translation missing: en.field_release changed from 144 to 168
- Translation missing: en.field_release changed from 168 to 114
- Status changed from New to Rejected
- Target version deleted (
Katello Backlog)
Thanks for reporting this issue. This issue was created over 4 years ago and hasn't seen an update in 1 year. We are closing this in an effort to keep a realistic backlog. Please open up a new issue that includes a link to this issue if you feel this still needs to be addressed. We can then triage the new issue and reassess.
Also available in: Atom
PDF
Updated by 
Updated by 
Updated by 
Updated by 
Updated by