[HELP]Reduce the memory consuption of a process

[Danielsan]

Hi there,

I have a XU4 (with Ubuntu 20.04) that I use a web media center to run some streaming services. It works pretty well but with Disneyplus it hangs dramatically, the way Disneyplus works produce a big chunk of memory that unavoidably fills all the memory making the Browser, Vivaldi, crashing because out of memory.

Hence I tried to use ZRAM but even tough I setup everything to write over the swap as soon as possible it starts to use the swap too late making the browser crashing. A potential solution I came out is to limit the access to memory for the browser, Vivaldi, however looking at tools for this:

https://wiki.archlinux.org/index.php/Cgroups
https://www.thegeekdiary.com/understand ... et-ulimit/

It looks like that those work for users and not for applications, I can surely put a limitation to a 1.5GB of RAM for the user and force then to use the swap since I run just one main application, however I wonder if I can add Vivaldi to a group and then limiting that group. Is it possible?

Thanks in advance,

D.

[Danielsan]

As far as I know the vivaldi browser is not an open source and it seems to be hard to change it memory occupying behaviors.
Anyway, I hope expert users can give you any help to configure the group effectively.

[Danielsan]

Vivaldi is based on Chromium, based on what I remember the closed part is mostly related to the UI. However only Vivaldi provide the widevine plugin to watch DRM content, the funny part is that for instance if you use the Android image for the UX4 the Disney app won't install because it say it doesn't support the hardware while it works with Ubuntu and Vivaldi albeit some issues...

In particular I used limits.conf to assign 1.5GB of maximum ram available it worked up to an hour then the ram suddenly increased from 1.5GB to 1.9GB crashing the browser as usual, basically for the amount of ram available the UX4 can handle only 1 hours of Disneyplus stream in HD. Netflix and Youtube can handle hours of streaming without issues...

Unfortunately even if put to use the swap (zram) as soon as possible it won't fill out...

[rooted]

Try dropping cache

Code: Select all

echo 1 | sudo tee /proc/sys/vm/drop_caches

[odroid]

I didn't know that Vavaldi supports a DRM plug-in.

Have you tried this 18.04 image based on Kernel 4.14?
https://wiki.odroid.com/odroid-xu4/os_i ... 4/20190929
One user reported there could be a memory leak issue on Kernel 5.4 armhf at a very edge condition point.

[mad_ady]

Here's an article demonstrating the use of cgroups for cpu: https://magazine.odroid.com/article/set ... rpose-nas/
You basically add your process pid to the desired cgroup.
But I don't think it will help you, since it will limit total memory for a process (ram + swap) and will cause it to die faster when it can't allocate more than 1.5G of RAM.
You may want to try to disable oom-killer, but most likely your system wil land in swap hell...

[Danielsan]

Try dropping cache

Code: Select all

echo 1 | sudo tee /proc/sys/vm/drop_caches

I think I already tried this without success...

I didn't know that Vavaldi supports a DRM plug-in.

Have you tried this 18.04 image based on Kernel 4.14?
https://wiki.odroid.com/odroid-xu4/os_i ... 4/20190929
One user reported there could be a memory leak issue on Kernel 5.4 armhf at a very edge condition point.

I had the same issue with 18.04, it looks like the stream, rather than sending small chunks of the video like Netflix or YT, downloads the entire stream on ram ending up to run out of memory.

Here's an article demonstrating the use of cgroups for cpu: https://magazine.odroid.com/article/set ... rpose-nas/
You basically add your process pid to the desired cgroup.
But I don't think it will help you, since it will limit total memory for a process (ram + swap) and will cause it to die faster when it can't allocate more than 1.5G of RAM.
You may want to try to disable oom-killer, but most likely your system wil land in swap hell...

Checking again at Arch Wiki I found an example on how to limit a memory access to a program: https://wiki.archlinux.org/index.php/Cgroups#Matlab
The problem for me is just systemd that looks to myself so clunky and counter-intuitive. Anyway the idea is to force Vivaldi to use the zram swap rather than the actual ram, but I haven't found yet the right method to achieve it!

Anyway thanks to all for your replies!

[Danielsan]

I tried also the cgroup feature, with the instructions provided by the Arch Wiki:

Code: Select all

~/.config/systemd/user/matlab.slice

Code: Select all

[Slice]
AllowedCPUs=0-5
MemoryHigh=6G

Code: Select all

$ systemd-run --user --slice=matlab.slice /opt/MATLAB/2012b/bin/matlab -desktop

And another set-up found here:

Code: Select all

# /etc/systemd/system/limit-512M.slice
[Unit]
Description=Slice with MemoryLimit=512M
Before=slices.target

[Slice]
MemoryAccounting=true
MemoryLimit=512M

Code: Select all

sudo systemd-run --slice limit-512M.slice --scope /usr/bin/sudo -u $username google-chrome-stable

None of them worked out, however for the latter I used the same per-user set up of the Arch Wiki, hence I'll try with a global seeting to see if invoking sudo may change its behavior and avoiding Disneyplus to eat all the ram available.

[mad_ady]

I think limiting RAM might not be the best approach. What about networking? Is the stream buffering with constant speed, or is it filling the buffers as fast as possible? I'd try to turn off network access when memory is almost full, to cut off communication. Maybe it would cause the browser to reconsider its buffers and free used buffers. After 20-30s you can turn back network to see what happens.
Another approach is to force the browser to run garbage collection periodically. Firefox has about:memory that let's you see memory usage. Maybe vivaldi has inherited something similar.

[Danielsan]

Eventually I tested it out all the options available... Even the cgroups one which provoked me a lot of headache since the related service unit was not available on Ubuntu and I had to copy and fix the one available for Arch Linux... However the memory always ran out...

But eventually I understood the reason but did not find a solution. Basically to enable all these cgroups feature you have to launch the application with a specific syntax, however to run Vivaldi you don't run the actual binary but a wrapper that checks something regarding plugins, drivers, drm etc.. and creates some specific environments for it. Afterwards is vivaldi-bin the binary that is actually working this is the reason why none of the tests worked out. However if you switch vivaldi for vivaldi-bin in the command line the drm and hw video acceleration do not work hence I couldn't test anything.

This is a true dilemma...

[mad_ady]

You can change cgroups of a running process. You create/configure your memory cgroup, then launch the process normally, then identify your process pids (ps -ef | grep name), then you pipe the pids to /sys/fs/cgroup/yourcgroup/pids (or similar)

[Danielsan]

You can change cgroups of a running process. You create/configure your memory cgroup, then launch the process normally, then identify your process pids (ps -ef | grep name), then you pipe the pids to /sys/fs/cgroup/yourcgroup/pids (or similar)

Yes I read that, but Vivaldi creates a lot of processes and that makes it really unfeasible. Anyway I was able to understand the wrapper has a command where I can append the systemd-run command I need to use:

Code: Select all

#!/bin/bash
#
# Copyright (c) 2011 The Chromium Authors. All rights reserved.
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.

# Let the wrapped binary know that it has been run through the wrapper.
export CHROME_WRAPPER="$(readlink -f "$0")"

HERE="$(dirname "$CHROME_WRAPPER")"
SLICE="/usr/bin/systemd-run --user --slice=vivaldi.slice"  <--- THIS WAD ADDED BY ME ###


# Proprietary media check
VIVALDI_VERSION='3.5.2115.81'
CODECS_VERSION='86.0.4240.75'
if [ -e "/var/opt/vivaldi/media-codecs-$CODECS_VERSION/libffmpeg.so" ]; then
  if [ -n "$LD_PRELOAD" ]; then
    export LD_PRELOAD="$LD_PRELOAD:/var/opt/vivaldi/media-codecs-$CODECS_VERSION/libffmpeg.so"
  else
    export LD_PRELOAD="/var/opt/vivaldi/media-codecs-$CODECS_VERSION/libffmpeg.so"
  fi
  export VIVALDI_FFMPEG_FOUND=YES
  # Allow a way for third party maintainers to provide a suitable file
elif [ -e "$HERE/libffmpeg.so.${VIVALDI_VERSION%\.*\.*}" ]; then
  if [ -n "$LD_PRELOAD" ]; then
    export LD_PRELOAD="$LD_PRELOAD:$HERE/libffmpeg.so.${VIVALDI_VERSION%\.*\.*}"
  else
    export LD_PRELOAD="$HERE/libffmpeg.so.${VIVALDI_VERSION%\.*\.*}"
  fi
  export VIVALDI_FFMPEG_FOUND=YES
elif [ -e "$HOME/.local/lib/vivaldi/media-codecs-$CODECS_VERSION/libffmpeg.so" ]; then
  if [ -n "$LD_PRELOAD" ]; then
    export LD_PRELOAD="$LD_PRELOAD:$HOME/.local/lib/vivaldi/media-codecs-$CODECS_VERSION/libffmpeg.so"
  else
    export LD_PRELOAD="$HOME/.local/lib/vivaldi/media-codecs-$CODECS_VERSION/libffmpeg.so"
  fi
  export VIVALDI_FFMPEG_FOUND=YES
else
  export VIVALDI_FFMPEG_FOUND=NO
  # Fix up Proprietary media if not present
  if [ "x${VIVALDI_FFMPEG_AUTO:-1}" = "x1" ]; then
    echo "'Proprietary media' support is not installed. Attempting to fix this for the next restart." >&2
    nohup "$HERE/update-ffmpeg" --user > /dev/null 2>&1 &
  fi
fi

# Fix up Widevine if not present for 32 and 64bit PC architecture
find_user_data () {
  eval set -- "$(getopt -q -o '' -l user-data-dir: -- "$@")"
  if [ "x${1:-}" = "x--user-data-dir" -a "x${2:0:1}" != "x-" ]; then
    USER_DATA_DIR="${2:-}"
  else
    USER_DATA_DIR="${XDG_CONFIG_HOME:-$HOME/.config}/vivaldi"
  fi
}
if [ "x${VIVALDI_WIDEVINE_AUTO:-1}" = "x1" ]; then
  if [ "armhf" = "i386" ]; then
    if [ ! -e "$HERE/WidevineCdm" -a ! -e "$HOME/.local/lib/vivaldi/WidevineCdm" ]; then
      echo "The Widevine CDM is not installed. Attempting to fix this for the next restart." >&2
      nohup "$HERE/update-widevine" --user > /dev/null 2>&1 &
    fi
    find_user_data "$@"
    if [ -d "$HOME/.local/lib/vivaldi/WidevineCdm" -a ! -e "$USER_DATA_DIR/WidevineCdm/latest-component-updated-widevine-cdm" ]; then
      mkdir -p "$USER_DATA_DIR/WidevineCdm"
      echo "{\"Path\":\"$HOME/.local/lib/vivaldi/WidevineCdm\"}" > "$USER_DATA_DIR/WidevineCdm/latest-component-updated-widevine-cdm"
    fi
  fi
fi

export CHROME_VERSION_EXTRA="stable"

# We don't want bug-buddy intercepting our crashes. http://crbug.com/24120
export GNOME_DISABLE_CRASH_DIALOG=SET_BY_GOOGLE_CHROME

# Sanitize std{in,out,err} because they'll be shared with untrusted child
# processes (http://crbug.com/376567).
exec < /dev/null
exec > >(exec cat)
exec 2> >(exec cat >&2)

# Note: exec -a below is a bashism.
exec -a "$0" "$HERE/vivaldi-bin" "$@"

Exactly is the last line!
So far, since I am a little dumb, I was unable to make it working but I want try with a global setup and see if it is going to work.

EDIT

It launches properly the browser within the SLICE only if I use this syntax:

Code: Select all

exec -a "$0" $SLICE "$HERE/vivaldi-bin" "$@"

However it doesn't work the widevine plugin... Geez I am pretty close but I have not could find a solution yet...

[va!n]

@Danielsan:
I am not sure about Linux/Android. But on Windows i use Firefox, which sucks memory like hell and doesnt works great for some special sites.
So i use more and more often "Brave", a very fast browser with tiny memory usage. Maybe you take a look, if Brave gives you what you are looking for.
Good luck.

Website of Brave Browser:
https://brave.com

[Danielsan]

Any chance to get an hint?

Thanks in advance...