savapage:rfc:linux-ulimit-java-exec [SavaPage

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== RFC: Limiting Java exec process resources =====

The Java [[https://devdocs.io/openjdk~8/java/lang/processbuilder|ProcessBuilder]] class provides an interface for starting and communicating with a new process that runs outside the Java process that started it. Also see [[https://devdocs.io/openjdk~8/java/lang/runtime#exec-java.lang.String-|Runtime.exec]] method.

> This new process is managed by the operating system, as part of process management. It has a self-contained execution environment, generally with a complete, private set of basic run-time resources: in particular, with its own memory space.

===== Why? =====

[[https://unix.stackexchange.com/questions/44985/limit-memory-usage-for-a-single-linux-process|This post]] describes a problem with ''pdftoppm'', one of the Poppler utilities also used by SavaPage:

> "I'm running pdftoppm to convert a user-provided PDF into a 300DPI image. This works great, except if the user provides an PDF with a very large page size.  pdftoppm will allocate enough memory to hold a 300DPI image of that size in memory, which for a 100 inch square page is 100*300 * 100*300 * 4 bytes per pixel = 3.5GB. A malicious user could just give me a silly-large PDF and cause all kinds of problems"

Indeed, what will happen when an A0 sized PDF is offered for ''pdftoppm -png'' conversion, or perhaps an invalid PDF that will trigger a program bug that leads to excessive memory consumption?

> We have to protect the SavaPage server process from excessive memory consumption of forked processes.

===== How? =====

==== ulimit ====

With ''ulimit'' we can check and limit various process resources:

<code bash>
# Check resources of user 'savapage'
# -S (soft limits) of -a (all) resources
sudo su - savapage -c "ulimit -Sa"
</code>

> Soft limits are the current setting for a particular limit. They can be increased only to the current hard limit setting.\\ Display all current soft settings with: ''ulimit -Sa''

> Hard limits are the maximum limit that can be configured. Any changes to these require root access.\\ Display all current hard settings with: ''ulimit -Ha''

We focus on the ''virtual memory (kbytes, -v)'' resource,
since the virtual address space is a good approximation for the memory used.  Things get more complicated when a process spawns other processes: see [[http://coldattic.info/post/40/|Limiting time and memory consumption of a program in Linux]]. But we assume this is not the case for the simple programs used by SavaPage.

Let's do some tests to get a feel of what ''ulimit'' does. Open a terminal session ...

<code bash>
# Check
ulimit -Sv
# this will probably show "unlimited"

# Set soft limit of virtual memory to 500MB 
# (the setting lives in this session only)
ulimit -Sv 500000

# Check again
ulimit -Sv

</code>

> A simple tactic is to use ''ulimit'' at the invocation of the new child process, as shown below:

<code bash>
# The 'sh' command simulates the Java exec.
sh -c "ulimit -Sv 500000;pdftoppm -png -f 1 -l 1 test.pdf > test.png"
# You can also throw cpu time (seconds, -t) in the mix
sh -c "ulimit -Sv 500000;ulimit -St 2;pdftoppm -png -f 1 -l 1 test.pdf > test.png"
</code>

==== limits.conf ====

''/etc/security/limits.conf'' holds the system wide limits for various users. 
The following lines could be added at the end of this file to set global virtual memory limits (the ''as'' keyword):

<code>
savapage  hard as 500000
savapage  soft as 500000 
</code>



===== Possible solutions =====

==== SavaPage Config ====

Configure ''ulimit'' properties for various exec commands in ''server.properties'' file. For example:

<code properties>
# This property will resolve as: 
#   ulimit -Sv 500000;ulimit -St 4;pdftoppm
java.runtime.exec.ulimit.pdftoppm = -Sv 500000;-St 4
</code>

==== System Config ====

Use ''/etc/security/limits.conf'' to limit virtual memory for user ''savapage''.

=== SysVinit ===

This will also limit the ''java'' process of the SavaPage server. Besides, ''java'' has its own command line options to allocate JVM memory. Therefore, set ''ulimit -Sv unlimited;'' when starting the ''java'' server process.

=== Systemd ===

systemd has its own way to limit memory to the ''java'' process:

<code bash>
systemctl show savapage.service | grep MemoryLimit
</code>

However, when java exec forks a child process, that process gets its ulimit values from ''/etc/security/limits.conf''

===== Recommended solution =====

> Relying on one "System Config" ''limits.conf'' value for all types of forked child processes is too crude a mechanism. Therefore, //a more tailored approach with "SavaPage Config" is recommended//.