Zram on Centos 7 / Nethserver 7

v7

(Thorsten) #1

Hi,

here is a tiny little howto on installing zram as an alternative to swap:

Background:
I run Nethserver as a guest on Proxmox VE. Nethserver installs a swap disk within the virtual disk provided. In my case, I do have quite a lot of processors and overall CPUs load is still low. Additionally, CPUs are not dedicated to a guest within Proxmox but a shared resource. In contrast to this, a minimum amount of RAM has to be dedicated to each Proxmox guest. I am quite limited with ram and Proxmox uses a lot of ram for ZFS.

Steps:

> yum install bc

> nano /etc/init.d/zram
#!/bin/bash
### BEGIN INIT INFO
# Provides: zram
# Required-Start:
# Required-Stop:
# Default-Start: 2 3 4 5
# Default-Stop: 0 1 6
# Short-Description: Virtual Swap Compressed in RAM
# Description: Virtual Swap Compressed in RAM
### END INIT INFO
  
start() {
    # get the number of CPUs
    num_cpus=$(grep -c processor /proc/cpuinfo)
    # if something goes wrong, assume we have 1
    [ "$num_cpus" != 0 ] || num_cpus=1
  
    # set decremented number of CPUs
    decr_num_cpus=$((num_cpus - 1))
  
    # get the amount of memory in the machine
    mem_total_kb=$(grep MemTotal /proc/meminfo | grep -E --only-matching '[[:digit:]]+')
     
    #we will only assign 50% of system memory to zram
    mem_total_kb=$((mem_total_kb / 2))
  
    mem_total=$((mem_total_kb * 1024))
  
    # load dependency modules
    modprobe zram num_devices=$num_cpus
  
    # initialize the devices
    for i in $(seq 0 $decr_num_cpus); do
    echo $((mem_total / num_cpus)) > /sys/block/zram$i/disksize
    done
  
    # Creating swap filesystems
    for i in $(seq 0 $decr_num_cpus); do
    mkswap /dev/zram$i
    done
  
    # Switch the swaps on
    for i in $(seq 0 $decr_num_cpus); do
    swapon -p 100 /dev/zram$i
    done
}
  
stop() {
    for i in $(grep '^/dev/zram' /proc/swaps | awk '{ print $1 }'); do
        swapoff "$i"
    done
  
    if grep -q "^zram " /proc/modules; then
        sleep 1
        rmmod zram
    fi
}
  
status() {
        ls /sys/block/zram* > /dev/null 2>&1 || exit 0
        echo -e "-------\nzram Compression Stats:\n-------"
        for i in /sys/block/zram*; do
            compr=$(< $i/compr_data_size)
        orig=$(< $i/orig_data_size)
        ratio=0
        if [ $compr -gt 0 ]; then
            ratio=$(echo "scale=2; $orig*100/$compr" | bc -q)
        fi
        echo -e "/dev/${i/*\/}:\t$ratio% ($orig -> $compr)"
        done
        echo -e "-------\nSWAP Stats:\n-------"
        swapon -s | grep zram
        echo -e "-------\nMemory Stats:\n-------"
        free -m -l -t
}
  
case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        stop
        sleep 3
        start
        ;;
    status)
        status
        ;;
    *)
        echo "Usage: $0 {start|stop|restart|status}"
        RETVAL=1
esac

> chmod 755 /etc/init.d/zram

> chkconfig --add  zram  && chkconfig zram on

> service zram status

Sources:
[1] oliviertoupin/zram on https://codegists.com/code/zram-centos-7/
[2] rasyiqul on https://rasyiqul.wordpress.com/2015/08/15/enable-zram-on-centos-6-x/

To Do:

  • Define priority against swap and/or turn of swap disk
  • Define reserverd memory blocks

Testers needed nethserver-arm img
(fpausp) #2

Hi thorsten, thanks for the howto. Please give us a config-example, to get an idea for which vm-size it coud be a benefit, to use zRAM or not…

regards
frank


(Thorsten) #3

Hi Frank,

I did not track SWAP size so far continously. However, to my basic knowledge, zram is fine for computers “with limited RAM resources”. To take advantage, I assume 2, maybe 4 GB, of ram, a slow HDD and / or large utilisation of Swap.

swap can be tested by

swapon -s

Generally my system is configured as follows:
Host: Dell Precision T3610 Workstation with overall 32 GB of RAM (average usage: 80 %), 12 cores (average usage < 1%), 3 x 2 TB WD Red (ZFS), 1 x 128 GB SanDisk SSD (32 GB as ZFS Cache, 32 GB as SWAP, rest unused by now)

Virtual machines:

  • CentOS Server, Nethserver: 8 GB (average usage: 50 %) / 4 cores (average usage 3 %)
  • Ubuntu Server, Dedicated to EcoDMS: 4 GB RAM (average usage: 46 %) / 4 cores (average usage: 0.5 %)
  • Ubuntu Server, Dedicated to Calibre: 2 GB RAM (average usage: 38 %), 4 cores (average usage: 0,2 %)
  • Some spare and trail machines not running continuously, e.g. Nethserver backup, Nethserver trail & error, FreeNAS, ecodms backup)

So far, I do notice a remarkably effect for the Ubuntu / Calibre System, while I installed zram on Nethserver just 1 h ago with standard settings.

I will have a look on machine data, but by now I just got only few results after reboot. I will also need to reconfigre, e.g. remove the swap disk itself…


(fpausp) #4

Thank you Thorsten, I will give it a go…


(André Wismer) #5

Hi

Most of my NethServers (Mine & Clients) run in ProxMox. During Setup, I provide only 2 GB RAM for the Guest (NethServer). This gives me a 2 GB Swapfile (Partition).After initial Install, I upgrade the RAM during a short shutdown to 4 or 8 GB RAM.
-> I only have a few clients where RAM is plentiful…
This works rather well…

My 2 cents
Andy


(Juan Carlos Fernandez) #6

Seen @thorsten script and rpodgorny zramswap script on AUR, I decided to make a python script which allows you to customize ZRAM parameters, allowing you to create a service that fit your needs.

First lets start with requirements, I decided to do the script using python 3, so first thing first:

  1. Install python3, pip, and virtualenv:
    yum install python34 python34-devel python34-virtualenv python34-pip

  2. Create a virtual enviroment, I decided to set it on /opt/zram:
    virtualenv-3 /opt/zram/env/

  3. Activate virtual enviroment and install sh and psutil using pip
    source /opt/zram/env/bin/activate
    pip install psutil sh
    deactivate

  4. Disable partition swap:
    Run:
    swapoff -a
    Comment the swap line on /etc/fstab

  5. Create folder and script files:
    mkdir /opt/zram/src/
    Using an editor create zramctrl.py and procsmem.py inside /opt/zram/src/. Please notice that procsmem.py is mostly a copy from giampaolo and is used to know which process is been swapped. You can check if the script works by running this:
    /opt/zram/env/bin/python /opt/zram/src/zramctrl.py start -A lzo
    This will create a ZRAM swap using all CPU cores and 20% of total RAM.
    To see how to use the script to adapt configuration to your needs run:
    /opt/zram/env/bin/python /opt/zram/src/zramctrl.py -h

  6. Create, install, load, enable and start a service:
    Using an editor create zramswap.service inside /opt/zram/src/ and install the service by running:
    install -Dm644 /opt/zram/src/zramswap.service /lib/systemd/system/zramswap.service
    Load the service by running:
    systemctl daemon-reload
    Enable the service by running:
    systemctl enable zramswap.service
    Start the service by running:
    systemctl start zramswap.service

That’s it, you now have a ZRAM swap online, here is some stuff you can do:
For those who use Proxmox ballooning, it’s advisable to set ZRAM to the minimum RAM assigned, to accomplish that, change line 8 from this:

ExecStart=/opt/zram/env/bin/python /opt/zram/src/zramctrl.py start -A lzo

To this:

ExecStart=/opt/zram/env/bin/python /opt/zram/src/zramctrl.py start -A lzo -P 40 -M 2147483648

Where 2147483648 = 2 *1024^3 = 2G, so this new setup would create a ZRAM of 819M (40% of 2G). Any question or suggestions are welcomed.


Testers needed nethserver-arm img
(Thorsten) #7

What a f***ing nice job. Pretty cool. Will try that soon.

Thanks.


(Mark Verlinde) #8

Hi,

Maybe you noticed the discussion about utilizing zram for arm based dives in another post.

Hence I am collecting all the experience i can find on the subject and no better place to start as among the Nethserver community.

Note i am unfamiliar with the subject so bear with me if i do not get it right away…
Also bear in mind on arm we run quite recent kernels (4.14) which can lead to differences in setup.

  • Are there objections to use the zramctl command to start/stop (setup) zram services?

In Fedora 29 some helper scripts are used. Most notably is this statement in the start script:

  • Would this apply to the 3.10* x86_64 kernel too?

  • Are there more leads to follow, especially how to benchmark performance?


(Robert Moskowitz) #9

Fedora 29 is running zram 0.2-1 as a systemd service. I just started my F29-arm system. I am vnced into my Xfce DE and started Firefox. A quick check on swap:

# swapon -s
Filename				Type		Size	Used	Priority
/dev/zram0                             	partition	487304	10752	-2

# zramctl
NAME       ALGORITHM DISKSIZE  DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0 lz4         475.9M 11.2M  4.3M  4.9M       2 [SWAP]

There is a conf file:

cat /etc/zram.conf 
# The factor is the percentage of total system RAM to allocate to the ZRAM block device(s).
FACTOR=2

The service file has:

# cat /etc/systemd/system/swap.target.wants/zram-swap.service
[Unit]
Description=Enable compressed swap in memory using zram
DefaultDependencies=no
Before=swap.target

[Service]
Type=oneshot
RemainAfterExit=yes
TimeoutStartSec=30sec
ExecStart=/usr/sbin/zramstart
ExecStop=/usr/sbin/zramstop

[Install]
WantedBy=swap.target

Checking on the status:

# systemctl -l --no-pager status zram-swap
\u25cf zram-swap.service - Enable compressed swap in memory using zram
   Loaded: loaded (/usr/lib/systemd/system/zram-swap.service; enabled; vendor preset: disabled)
   Active: active (exited) since Fri 2018-06-22 11:12:12 EDT; 3 months 16 days ago
  Process: 437 ExecStart=/usr/sbin/zramstart (code=exited, status=0/SUCCESS)
 Main PID: 437 (code=exited, status=0/SUCCESS)

Jun 22 11:12:12 localhost zramstart[437]: Setting up swapspace version 1, size = 475.9 MiB (498999296 bytes)
Jun 22 11:12:12 localhost zramstart[437]: no label, UUID=ba3bf62f-7bb0-4295-ad8c-2d26b000c3f8
Jun 22 11:12:12 localhost zramstart[437]: Activated ZRAM swap device of 499 MB
Jun 22 11:12:12 localhost systemd[1]: Started Enable compressed swap in memory using zram.

Ignore that Jun 22 date; armv7 SOC tend NOT to have a battery clock. So Fedora29-arm starts somehow with this Jun 22nd date and then counts on Chrony to fix the date, which is will do even a pretty good job with no network connection (using datetime stamp on drift file).


Testers needed nethserver-arm img
(Mark Verlinde) #10

Thank you, afso for mentioning it on the arm-dev maillist. :grinning:

Rebuild the (FC29) zram package with just a minor adjustment in packaging the license for EL7/NS7.

[root@opi2e ~]# uname -r
4.14.71-201.el7.armv7hl

[root@opi2e ~]# rpm -q zram
zram-0.2-2.ns7.noarch

[root@opi2e ~]# zramctl
NAME       ALGORITHM DISKSIZE  DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0 lz4         960.4M  3.9M   64B    4K       4 <swap>

[root@opi2e ~]# swapon -s
Filename                                Type            Size    Used    Priority
/dev/zram0                              partition       983396  4352    -2

[root@opi2e ~]# systemctl -l --no-pager status zram-swap
● zram-swap.service - Enable compressed swap in memory using zram
   Loaded: loaded (/usr/lib/systemd/system/zram-swap.service; enabled; vendor preset: disabled)
   Active: active (exited) since Mon 2018-10-08 13:20:17 CEST; 2h 16min ago
 Main PID: 4623 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/zram-swap.service

Oct 08 13:20:17 opi2e.havak.lan systemd[1]: Starting Enable compressed swap in memory using zram...
Oct 08 13:20:17 opi2e.havak.lan zramstart[4623]: Setting up swapspace version 1, size = 983396 KiB
Oct 08 13:20:17 opi2e.havak.lan zramstart[4623]: no label, UUID=15ffcc8b-a669-455a-b70d-e571cd7d6fab
Oct 08 13:20:17 opi2e.havak.lan zramstart[4623]: Activated ZRAM swap device of 1007 MB
Oct 08 13:20:17 opi2e.havak.lan systemd[1]: Started Enable compressed swap in memory using zram.

As you can see:
It just runs for 2 1/4 hours, so cannot say much more as “it seems to work”.

Yes, my research so far tells it is not a necessity to dispatch it over all core’s of the CPU. The number of streams should be a conformation of this.


(Robert Moskowitz) #11

And now you can reply on the arm-dev list your progress!

Minimally ir we put the rpm(s) in a reasonable place, I can add to my Centos howto how to switch to zram. I would rather Centos arm-dev picks it up directly and puts the instructions on their wiki.


(Robert Moskowitz) #12

Mark, how do you want to proceed with this for general Centos7-arm availability? For now, I could put the rpm on my web server and add the instructions to my howto. And give a big nod to you for making it available.

I just thought that I should check to see if rawhide has a later version.

For Nethserver-arm, you could make an app that runs a script to convert, or just put up a simple howto.

It may take Pablo a while to get it into the Centos base image…


(Mark Verlinde) #13

It’s arch independent and can be of use for all systems which have very low resources; pushed to my copr account. This way we have a convenient place to get it (install) from. If you wish you can download it from there too and make it available on your website.

https://copr.fedorainfracloud.org/coprs/markvnl/zram-swap/

If you could write a howto i’ll drop it in the Installation Instructions

https://copr.fedorainfracloud.org/coprs/markvnl/zram-swap/repo/epel-7/markvnl-zram-swap-epel-7.repo

for installation on arm you need to “hardcode” the basearch, ie:
baseurl=https://copr-be.cloud.fedoraproject.org/results/markvnl/zram-swap/epel-7-x86_64/

https://copr-be.cloud.fedoraproject.org/results/markvnl/zram-swap/epel-7-x86_64/00807447-zram/

grtz Mark


(Robert Moskowitz) #14

Since this repo is only for zram, how about just taking replacing $basearch with noarch? This would simplify instructions for everyone.

zram , formerly called compcache, is a module for creating a compressed block device in RAM, i.e. a RAM disk, but with on-the-fly “disk” compression. The block device created with zram can then be used for swap or as general-purpose RAM disk.

It is being provided for Centos7 as a memory-resident systemd controlled swap device service that is architecture independent and can be of use for all systems which have very low resources. Particularly ones that keep the file system on SD cards.

Here is a simple set of steps to switch from a physical swap partition to a zram-swap:

  1. download the repo into /etc/yum.repo
  2. yum install zram
  3. swapoff -a
  4. sed -i -e “/swap/s/^/# /w /dev/stdout” /etc/fstab
  5. systemctl start zram-swap
  6. systemctl enable zram-swap

You can check on the zram-swap device with the zramctl command.


(Mark Verlinde) #15

Copr is an automated build-system, have no say in where the builded packages end up.

yum install https://copr-be.cloud.fedoraproject.org/results/markvnl/zram-swap/epel-7-x86_64/00807447-zram/zram-0.2-2.el7.noarch.rpm

should do the job on all arch’s too

did not do the trick for me:

# sed -i -e “/swap/s/^/# /w /dev/stdout” /etc/fstab
sed: -e expression #1, char 1: unknown command: `▒'

(Robert Moskowitz) #16

Yes, I was actually thinking that instead of all the hassle to download a repo for a single rpm.

This wonderful forum changes a simple double-quote into a special quote mark. Change your command to use the regular double-quote on both sides of the -e content. Below I enclosed the command in a code block.

sed -i -e "/swap/s/^/# /w /dev/stdout" /etc/fstab

I just tried to copy and paste the above into a terminal window and it worked. Whew.


(Robert Moskowitz) #17

I had gotten tired of all the howto documents and blogs that said to edit file x and change/add the following…

So I spent time googling on using sed and have done quite a lot with it. Though, I still have to look up things, even those I have done before.

quotes and double-quotes are important things in sed and other scripting uses. You have to be very careful with them. Look at my ECDSA and EDDSA pki Internet drafts and you will see a lot of use of quotes there with environment variables (visit my github for rgmhtt for the xml sources).


(Mark Verlinde) #18

markdown is the key word…
https://guides.github.com/features/mastering-markdown/

thanx it worked and in the Installation Instructions @copr

EDIT:

Lets disagree on this, regular expressions in sed commands are hard to read and you do not learn/understand what you are doing. Telling people to edit file x has educational benefits.


(Robert Moskowitz) #19

Much more complete installation instructions. You misspelled swapon.

Also mention that compression could be 1:4-5, so a 512MB physical zram-swap could equal a 2GB physical swap? And only as much memory as needed for the current swap size is used?

But are you going to put in the direct install of the rpm as mentioned above or leave it as is.

And now you can point the centos-dev list to this.

thanks for putting it together.