Project

General

Profile

Bug #25778

Assets compilation failure on Jenkins+Remote Execution/Ansible due to "Allocation failed - JavaScript heap out of memory"

Added by Ivan Necas 3 months ago. Updated 3 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
JavaScript stack
Target version:
-
Difficulty:
Triaged:
No
Bugzilla link:
Team Backlog:
Fixed in Releases:
Found in Releases:

Description

We started seeing test failures in Remote Execution and Ansible PR testing job when trying to precompile the assets. Seems like we started crossing limits of the memory available for the job. There has been no changes in the assets for the remote execution itself for a while.

The last build without this issue I was able to find was from [Dec 21](http://ci.theforeman.org/job/foreman_ansible-pull-request/205/) and first occurrence of [Jan 2](http://ci.theforeman.org/job/foreman_ansible-pull-request/206/)


** Execute webpack:compile
/var/lib/workspace/workspace/foreman_remote_execution-pull-request/database/postgresql/label/fast/ruby/2.5/foreman/node_modules/.bin/webpack --config /var/lib/workspace/workspace/foreman_remote_execution-pull-request/database/postgresql/label/fast/ruby/2.5/foreman/config/webpack.config.js --bail
The Apipie cache is turned off. Enable it and run apipie:cache rake task to speed up API calls.

<--- Last few GCs --->

  195960 ms: Mark-sweep 1261.7 (1439.5) -> 1261.7 (1439.5) MB, 1109.4 / 0.0 ms [allocation failure] [GC in old space requested].
  197169 ms: Mark-sweep 1261.7 (1439.5) -> 1261.7 (1439.5) MB, 1209.1 / 0.0 ms [allocation failure] [GC in old space requested].
  198660 ms: Mark-sweep 1261.7 (1439.5) -> 1265.1 (1423.5) MB, 1491.0 / 0.0 ms [last resort gc].
  200234 ms: Mark-sweep 1265.1 (1423.5) -> 1268.8 (1423.5) MB, 1573.5 / 0.0 ms [last resort gc].

<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x12cdac3cf781 <JS Object>
    1: InnerArrayForEach(aka InnerArrayForEach) [native array.js:~935] [pc=0x2b91ba124e0f] (this=0x12cdac304381 <undefined>,bq=0x23b6a4ec2971 <JS Function (SharedFunctionInfo 0x3371ba6ffda9)>,br=0x25cfbb9c57e1 <JS Function SourceNode (SharedFunctionInfo 0x2793deb41371)>,w=0x2ec39bd3f701 <JS Array[528671]>,x=528671)
    2: forEach [native array.js:~954] [pc=0x2b91ba17b136] (this=0x2ec39bd3f701 <...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [node]

Seems like we're hitting limits of node --max_old_space_size in jenkins. The workarounds for this that I've found is to increase the limit
so that we still fit to it. Unfortunately webpack-rails doesn't provide a way how to change this and as it's not maintained anymore,
the proper fix means switching to webpacker (or something else).

As a short-term fix, we can probably provide our own webpack:compile task and set the required value there.


Related issues

Related to Foreman - Refactor #21203: Move from webpack-rails to webackerReady For Testing2017-10-04
Related to Foreman - Bug #26380: webpack:compile rake task is executed twiceClosed

Associated revisions

Revision 56640d93 (diff)
Added by Ivan Necas 3 months ago

Fixes #25778 - increase max_old_space_size for webpack:compile

The default value started causing:

Allocation failed - JavaScript heap out of memory

Increasing max_old_space_size should fix the issue. Unfortunately,
webpack-rails doesn't allow to set params for node. Therefore we to
override the rake task by our own. Take directly from (with extension of
the node parameter)

https://github.com/mipearson/webpack-rails/blob/v0.9.11/
lib/tasks/webpack.rake

History

#2 Updated by The Foreman Bot 3 months ago

  • Status changed from New to Ready For Testing
  • Pull request https://github.com/theforeman/foreman/pull/6380 added

#3 Updated by Ivan Necas 3 months ago

#4 Updated by Tomer Brisker 3 months ago

  • Fixed in Releases 1.21.0 added

#5 Updated by Ivan Necas 3 months ago

  • Status changed from Ready For Testing to Closed

#6 Updated by Tomer Brisker 5 days ago

  • Related to Bug #26380: webpack:compile rake task is executed twice added

Also available in: Atom PDF