Plugins » OpenSCAP

Upload ARF reports is causing Foreman Proxy to consume large amount of memory.
Cloned from 15483391

Description of problem:
Foreman Proxy can consume large amount of memory after uploading certain amount of ARF reports to Foreman.

I can Foreman proxy to use about 2.8GB of memory by simply using 2 clients to upload the ARF reports.
~~~

1. ps -aux | grep foreman-proxy
   foreman+ 1548024 10.4 13.9 3094828 2812948 ? Ssl 02:33 1:26 /usr/bin/ruby /usr/share/foreman-proxy/bin/smart-proxy
   ~~~

How reproducible:
Easy

Steps to Reproduce:
1. Prepare 2 or more Foreman openscap clients and run the below command on them multiple times.
~_{~
foreman_scap_client <the policy id>}
~~

2. Observe the memory usage of the Foreman proxy service in the Satellite

Actual results:
Memory usage of the Foreman Proxy keeps increasing

Expected results:
Should use reasonable amount of memory or free up unused memory.

Additional info:
After some investigations. I found that it is the nokogiri that used about 450MB to parse the XML report and never free up the memory. See below:

~_{~
MALLOC_ARENA_MAX=2 irb}
~~

~_{~
root 1549155 21.6 0.1 111156 28904 pts/0 S+ 02:55 0:01 irb}
~~

~_{~
irb(main):001:0> conf.echo = false
irb(main):002:0> require 'nokogiri'
irb(main):003:0> data = `bunzip2 -dc /root/results.xml.bz2`
irb(main):004:0> parsed_xml = ::Nokogiri::XML.parse(data, nil, nil, Nokogiri::XML::ParseOptions.new.norecover)}
~~

~_{~
root 1549155 11.4 2.4 606832 495808 pts/0 S+ 02:55 0:05 irb}
~~

I think it is not a memory leak because if I remove the references and force GC immediately, it won't free the memory up, but will reuse it.
~_{~
irb(main):006:0> parsed_xml = nil
irb(main):007:0> data = nil
irb(main):008:0> GC.start
irb(main):009:0> data = `bunzip2 -dc /root/results.xml.bz2`
irb(main):010:0> parsed_xml = ::Nokogiri::XML.parse(data, nil, nil, Nokogiri::XML::ParseOptions.new.norecover)}
~~

~_{~
root 1549155 3.2 2.7 740104 562444 pts/0 S+ 02:55 0:09 irb}
~~