Project

General

Profile

Actions

Feature #31274

open

Reports generation is memory heavy

Added by Lukas Zapletal over 3 years ago. Updated over 3 years ago.

Status:
Ready For Testing
Priority:
Normal
Category:
Templates
Target version:
-
Difficulty:
Triaged:
No
Fixed in Releases:
Found in Releases:

Description

The way we generate reports is memory unfriendly. All rows are stored in an array as hashes and then dumped into string which is returned to the client. In background job case, string is stored in the db or emailed.

Refactoring: Rewrite report_headers, report_row and report_render helpers so they immediately stream data into an IO object. Do not use any intermediate data structure, all formats we currently support (JSON, YAML. CSV, plaintext) do support this. When called via API, data can be streamed directly over TCP. In case of background processing, data can be streamed via SQL LOAD postgresql directly into blob/text.

This could be implemented without any user-facing changes - the three helpers will remain the same, but they will immediately stream content instead of storing anything in memory. The last call (report_render) will output optional footer and close the stream.


Related issues 1 (1 open0 closed)

Related to Foreman - Bug #31200: Puma memory leakNeed more informationActions
Actions #1

Updated by Lukas Zapletal over 3 years ago

Actions #2

Updated by The Foreman Bot over 3 years ago

  • Status changed from New to Ready For Testing
  • Assignee set to Lukas Zapletal
  • Pull request https://github.com/theforeman/foreman/pull/8136 added
Actions

Also available in: Atom PDF