Project

General

Profile

Refactor #10324

Performance issues with Foreman Proxy Puppet CA plugin and large CA inventories

Added by Jon McKenzie over 7 years ago. Updated about 4 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
Puppet
Target version:
Difficulty:
Triaged:
Bugzilla link:
Fixed in Releases:
Found in Releases:

Description

Proxy::PuppetCa.list (https://github.com/theforeman/smart-proxy/blob/develop/modules/puppetca/puppetca_main.rb#L69) takes an extremely long period of time to complete when the CA inventory list is relatively large and there have been a relatively large number of revoked certificates.

The source of this issue is in Proxy::PuppetCa#ca_inventory, which performs a very slow O(n^2) operation to build up a hash of certificate data and whether each certificate has been revoked. In particular, the Proxy::PuppetCa#ca_inventory iterates through every certificate in the inventory and then looks up that certificate's serial in the array returned by Proxy::PuppetCa#revoked_serials. When both the inventory and the list of revoked clients is large, this operation is extremely slow.

The downstream effect of this issue is that trying to interact with the Puppet CA via Foreman simply times out (the read_timeout on the RestClient request is only 60 seconds, the Proxy::PuppetCa.list operation in our environment takes approximately 80s).

Perhaps a way to make this faster this would be to just return a valueless Hash from Proxy::PuppetCa#revoked_serials so that the serial lookup is ~O(1).

Just to give you some numbers to work with, we have approximately 10900 certificates in our inventory and about 8900 of them are revoked.

Associated revisions

Revision 0a83088c (diff)
Added by Jon McKenzie over 7 years ago

Fixes #10324 - Improves certificate serial lookup in Puppet CA module

Use a set instead of an array to improve lookup performance when the number of revoked certificates is large.

History

#1 Updated by Dominic Cleal over 7 years ago

  • Category set to Puppet

#2 Updated by The Foreman Bot over 7 years ago

  • Status changed from New to Ready For Testing
  • Pull request https://github.com/theforeman/smart-proxy/pull/283 added
  • Pull request deleted ()

#3 Updated by Jon McKenzie over 7 years ago

  • Status changed from Ready For Testing to Closed
  • % Done changed from 0 to 100

#4 Updated by Dominic Cleal over 7 years ago

  • Assignee set to Jon McKenzie
  • Legacy Backlogs Release (now unused) set to 50

Also available in: Atom PDF