Approaching NS8 Beta 2, call for review

davidep · September 8, 2023, 7:24am

As we are approaching the NS8 Beta 2 milestone, I’m preparing a list of release notes with the main changes since Beta 1. The list does not include the bug fixes, and changes that are not visible to the sysadmin or the final user. You can preview the release notes pull request at this link

Release notes — NS8 documentation

To any of you with a snapshot of Beta 1: do you have some spare time to donate to the community? would you like to test the upgrade procedure?

Any feedback is welcome! Thank you very much

do not run it in production

stephdl · September 8, 2023, 9:41am

for testing

install debian11 or rocky9

then initiate the cluster

curl curl https://raw.githubusercontent.com/NethServer/ns8-core/1.1.1/core/install.sh > install.sh > install.sh

bash +x install.sh ghcr.io/nethserver/restic:1.1.0 ghcr.io/nethserver/redis:1.1.0 ghcr.io/nethserver/core:1.1.0 ghcr.io/nethserver/rsync:1.1.0 ghcr.io/nethserver/traefik:1.0.0 ghcr.io/nethserver/loki:1.0.0 ghcr.io/nethserver/ldapproxy:0.0.6 ghcr.io/nethserver/promtail:0.0.3

install some modules to get a real cluster

then upgrade to core 1.3.1 (use the software center)

to upgrade debian
rm -f '/etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list'
sed -i 's/bullseye/bookworm/' /etc/apt/sources.list
apt update && apt full-upgrade -y

(
    set -e -x
    core_dir=/usr/local/agent/pyenv
    mv -v ${core_dir} ${core_dir}.bak
    python3.11 -mvenv ${core_dir} --upgrade-deps --system-site-packages
    ${core_dir}/bin/pip3 install -r /etc/nethserver/pyreq3_11.txt
    echo "/usr/local/agent/pypkg" >$(${core_dir}/bin/python3 -c "import sys; print(sys.path[-1] + '/pypkg.pth')")
    rm -rf ${core_dir}.bak
)

test it

api-cli run list-actions &>/dev/null && echo 'Python 3.11 upgrade was successfull :)'

to upgrade rocky9

dnf install python3.11

(
    set -e -x
    core_dir=/usr/local/agent/pyenv
    mv -v ${core_dir} ${core_dir}.bak
    python3.11 -mvenv ${core_dir} --upgrade-deps --system-site-packages
    ${core_dir}/bin/pip3 install -r /etc/nethserver/pyreq3_11.txt
    echo "/usr/local/agent/pypkg" >$(${core_dir}/bin/python3 -c "import sys; print(sys.path[-1] + '/pypkg.pth')")
    rm -rf ${core_dir}.bak
)

verify

api-cli run list-actions &>/dev/null && echo 'Python 3.11 upgrade was successfull :)'

transocean · September 8, 2023, 10:33am

Hi Friends,

Can i do the same procedure with Alma-Linux?

Regards

Uwe

stephdl · September 8, 2023, 11:50am

it should if it is version 9, do a snapshot before

GG_jr · September 8, 2023, 7:22pm

Status:

Host: Rocky Linux 9.2
NethServer 8 Software repositories: Status = Enabled; Testing = Enabled
Installed modules: CrowdSec, Mail, MariaDB, Nextcloud, WebServer, WebTop, Collabora.

I did the update from Software Center → Updates, without errors.

After update, SCP/PuTTY to RL 9.2 and I tried:

[root@ns8 ~]# dnf install python3.11
crowdsec_crowdsec 905 B/s | 1.8 kB 00:02
crowdsec_crowdsec-source 928 B/s | 1.8 kB 00:02
Package python3.11-3.11.2-2.el9_2.1.x86_64 is already installed.
Dependencies resolved.
Nothing to do.
Complete!
[root@ns8 ~]#
[root@ns8 ~]# api-cli run list-actions &>/dev/null && echo 'Python 3.11 upgrade was successful :)'
Python 3.11 upgrade was successful :)
[root@ns8 ~]#

Are the additional procedures as described here still needed?

Congratulations to everyone developing NS8!

davidep · September 8, 2023, 8:02pm

Yes you must run this one too

stephdl:

(
    set -e -x
    core_dir=/usr/local/agent/pyenv
    mv -v ${core_dir} ${core_dir}.bak
    python3.11 -mvenv ${core_dir} --upgrade-deps --system-site-packages
    ${core_dir}/bin/pip3 install -r /etc/nethserver/pyreq3_11.txt
    echo "/usr/local/agent/pypkg" >$(${core_dir}/bin/python3 -c "import sys; print(sys.path[-1] + '/pypkg.pth')")
    rm -rf ${core_dir}.bak
)

It’s a lie

The check command is now

runagent python3 --version # output should be 3.11

transocean · September 9, 2023, 8:15am

Ok,

done… But core

is now version 2.0.0-dev.107.g6dabb5df and not 1.3
This is ok?

Regrads

Uwe

Edit:

One thing I noticed. There is no error log in Nextcloud anymore. Does this have something to do with the changes?

davidep · September 9, 2023, 11:53am

Yes it’s ok, however we found a bug with traefik beta.2 affecting multi node clusters. If you have more than one node do not proceed.

transocean · September 9, 2023, 12:05pm

I have only one node. And it looks like it’s all going well.
If you continue to perform so well, maybe we can expect a Christmas present of a special kind?

mrmarkuz · September 10, 2023, 9:46am

Upgrade to Core 1.3.1 worked on Alma cluster, Debian cluster and single Rocky node. Great work!
Python is at version 3.11.2 on all nodes.
Debian dist upgrade worked without issues.

GG_jr · September 10, 2023, 1:21pm

Only the last part of the result:

[root@ns8 ~]# (
    set -e -x
    core_dir=/usr/local/agent/pyenv
    mv -v ${core_dir} ${core_dir}.bak
    python3.11 -mvenv ${core_dir} --upgrade-deps --system-site-packages
    ${core_dir}/bin/pip3 install -r /etc/nethserver/pyreq3_11.txt
    echo "/usr/local/agent/pypkg" >$(${core_dir}/bin/python3 -c "import sys; print(sys.path[-1] + '/pypkg.pth')")
    rm -rf ${core_dir}.bak
)
................................................................
Requirement already satisfied: setuptools>=62.4.0 in /usr/local/agent/pyenv/lib64/python3.11/site-packages (from python-daemon==3.0.1->-r /etc/nethserver/pyreq3_11.txt (line 35)) (68.2.0)
Collecting charset-normalizer<4.0,>=2.0 (from aiohttp==3.8.4->-r /etc/nethserver/pyreq3_11.txt (line 7))
Using cached charset_normalizer-2.1.1-py3-none-any.whl (39 kB)
Using cached ansible_core-2.15.1-py3-none-any.whl (2.2 MB)
Using cached cryptography-41.0.1-cp37-abi3-manylinux_2_28_x86_64.whl (4.3 MB)
Using cached MarkupSafe-2.1.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (28 kB)
Using cached semver-3.0.1-py3-none-any.whl (17 kB)
Using cached typing_extensions-4.6.3-py3-none-any.whl (31 kB)
Using cached frozenlist-1.4.0-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (250 kB)
Installing collected packages: resolvelib, pyasn1, ptyprocess, lockfile, urllib3, typing-extensions, six, semver, regex-engine, PyYAML, pyrsistent, pycparser, psutil, pexpect, packaging, multidict, MarkupSafe, ldap3, idna, hiredis, frozenlist, docutils, dnspython, charset-normalizer, chardet, certifi, attrs, async-timeout, yarl, requests, redis, python-daemon, jsonschema, Jinja2, cffi, aiosignal, aioredis, pycares, cryptography, brotlipy, ansible-runner, aiohttp, ansible-core, aiodns
Successfully installed Jinja2-3.1.2 MarkupSafe-2.1.3 PyYAML-6.0 aiodns-3.0.0 aiohttp-3.8.4 aioredis-2.0.1 aiosignal-1.3.1 ansible-core-2.15.1 ansible-runner-2.3.1 async-timeout-4.0.2 attrs-22.2.0 brotlipy-0.7.0 certifi-2022.9.24 cffi-1.15.1 chardet-5.1.0 charset-normalizer-2.1.1 cryptography-41.0.1 dnspython-2.3.0 docutils-0.20 frozenlist-1.4.0 hiredis-2.2.3 idna-3.3 jsonschema-4.10.3 ldap3-2.9.1 lockfile-0.12.2 multidict-6.0.4 packaging-23.0 pexpect-4.8.0 psutil-5.9.4 ptyprocess-0.7.0 pyasn1-0.4.8 pycares-4.3.0 pycparser-2.21 pyrsistent-0.18.1 python-daemon-3.0.1 redis-4.5.5 regex-engine-1.1.0 requests-2.28.1 resolvelib-1.0.1 semver-3.0.1 six-1.16.0 typing-extensions-4.6.3 urllib3-1.26.12 yarl-1.9.2
+ echo /usr/local/agent/pypkg
++ /usr/local/agent/pyenv/bin/python3 -c 'import sys; print(sys.path[-1] + '\''/pypkg.pth'\'')'
+ rm -i -rf /usr/local/agent/pyenv.bak
[root@ns8 ~]#
[root@ns8 ~]#
[root@ns8 ~]# runagent python3 --version # output should be 3.11
Python 3.11.2
[root@ns8 ~]#

GG_jr · September 11, 2023, 10:57am

However, I think something went wrong after I did the last update as I described above.
In the Webserver - Status module, SFTPGo appears as not configured: see “1_Status after shutdown_start NS8.png”.
(BTW: can we have a button for Reboot / Shutdown NS8/Host+NS8? TIA!)
If I press the button Save in Settings, the following error appears: see “2_Configure SFTPGo error.png”.
After the error message, Webserver - Status appears like this: see “3_Status after Configure SFTPGo error.png”.

Does this error occur only to me?

PS:
4_ Status after “walking” through various windows in NS8.png

stephdl · September 11, 2023, 12:56pm

Nginx is failed, could you check in your log

journalctl _UID=$(id -u webserver1)

and check what is making falling nginx, you could restart the service from the sftpgo page and check logs with

journalctl -f _UID=$(id -u webserver1)

GG_jr · September 13, 2023, 7:20am

Hi @stephdl and sorry for the late response!
After many errors (remove virtual host(s), SFTPGo configuration, uninstall webserver module, …) and wasted time, I decided to rollback to the moment before the WebServer module was installed.
With both repositories enabled (Status & Test) I updated NS8, then installed the WebServer module.
So far, the module seems to be working correctly.
I will try to add 2 virtual hosts (one for WordPress) and see what happens.

alefattorini · September 13, 2023, 7:50am

Why not create a NS8_testing group?
People that are willing to do some tests when new releases come out.
Who wanna join? I take @transocean @GG_jr @mrmarkuz for granted
I would add @rasi @capote @Schnabeltierchen @djx @Mastermind @EddieA @xwu @alex1 @michelandre @maja2020
Anyone else?

stephdl · September 13, 2023, 8:23am

reinstall is good and not good, find the root cause in log is often better to help the development, we could understand the exception that has raised errors… next time

GG_jr · September 13, 2023, 8:34am

I saved some logs from my tests in Notepad++.
I can send them to you, if you want.
I can publish them here, but they are a bit big.
As you wish.

GG_jr · September 13, 2023, 8:39am

A very good idea!
I’m having some problems with Nextcloud and Collabora, with the installation of WordPress and OSTicket as Virtual Hosts and I don’t know if it’s just me or from NS8, being still in the development/testing period.

davidep · September 13, 2023, 8:43am

It would be wonderful! With the feedback gathered so far, we already fixed many bugs before the release

stephdl · September 13, 2023, 8:45am

relevant to ns8 & wordpress NS8 & WordPress

I think the major issue to understand is that the mysql user must be allowed to connect from any host use user@% instead of user@localhost and point to the ip:20009 (it is an external random port, check it) to connect the database and not localhost obviously