NS8-App Mayhem: A Modules Dev thread on multiple modules issues faced

oneitonitram · February 2, 2024, 4:33pm

I have embarked on a Journey that started well, but the going keeps getting tougher by the day.

Here is a List of Modules I have been working on, both as a way to sharpen my module development skills, and also as a way to contribute to NS8 modules.

My hope was that by the time NS8 gets to release, I would have these modules up and running, but mahn, have i entered a rabbit hole.

Paperless NGX = Completed and production ready(Published)
JoplinServer = Completed-released
n8n = Completed-Released
MAtomo = Completed-Released

StirlingPDF = Completed-Released
GLPi = Completed-Released
Dolibarr = Completed-Released
IT-Tools = Completed-Released
Calibre-Web = Completed-testing
Calibre = Completed-testing
Homarr = Completed-testing
goauthentik = Completed-testing

NTFY = Completed-WIP(more to be done)
Linkwarden = WIP
VAultwarden - WIP
Coder = WIP
SurrealDB = WIP
SocFortress Copilot(not AI related)= WIP
TActicalRMM = not yet began
2FAUTH = not yet began
MAtrix = Not sure where to begin
Zammad = not yet began @mrmarkuz take it up

Would love to have teleport as well, https://goteleport.com/ but whoosh…

So i have created this, to simplify and collect into one, some of challenges i am facing, and potential resolution to some of the issues being faced (Not sure if its the right course of Action) I could probably Open individual threads for each module, I am open to sugegstion.

At the moment been trying to implement a single service module, and i seem to be doing something wrong.

Could a Dev kindly nudge me in the right direction.

What could be wrong with this service file here:

ns8-vaultwarden/imageroot/systemd/user/vaultwarden.service at main · compgeniuses/ns8-vaultwarden (github.com)

oneitonitram · February 4, 2024, 2:08pm

removed: not necessary for this thread

stephdl · February 5, 2024, 5:58pm

one task at a time

oneitonitram · February 5, 2024, 6:21pm

hello @stephdl true to the point, unfortunately my brain just doesnt know how to compute complex things one at a time.

Now, doing multiple modules at the same time has actually helped alot, each of the different modules had different caveats to solve and handle, and each was a learning point for the other or next modules being built.

Also, when one modules starts to misbehave, instead of wasting time trying to figure the problem, i jump to the next module in the queue, which might or might not have the same or different set of challenges.

Overall i am abit confident in the noamrl modules building, its the complex ones, i am looking forward to figuring out,

the ones that combine UDP and TCP ports, that require compiling own docker image from source, and other advance stuff, though learning one module at a time…

Welcome back how was fosdem?

stephdl · February 5, 2024, 6:30pm

amazing times with mates

most of time the error is specific to the module, I need time to fix something broken and jump to break other things has never helped me.

oneitonitram · February 7, 2024, 8:28am

I have since made significant Milestone on the VAultwarden as well as the Go authentic module.

for Authentik, i am getting a weird error i am unsure the origins from

<7>dump_env() is deprecated and implemented as a no-op
<7>dump_env() is deprecated and implemented as a no-op
Traceback (most recent call last):
  File "/home/goauthentik4/.config/actions/configure-module/40request-LE-certificate.py", line 19, in <module>
    agent.tasks.run_nowait(
  File "/usr/local/agent/pypkg/agent/tasks/run.py", line 45, in run_nowait
    return run(agent_id, action, data, nowait=True, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/agent/pypkg/agent/tasks/run.py", line 39, in run
    results = runp([taskrq], **kwargs)
              ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/agent/pypkg/agent/tasks/run.py", line 50, in runp
    return asyncio.run(_runp(tasks, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/usr/local/agent/pypkg/agent/tasks/run.py", line 120, in _runp
    return await asyncio.gather(*runners, return_exceptions=(len(tasks) > 1))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/agent/pypkg/agent/tasks/run.py", line 136, in _run_with_protocol_nowait
    return await run_apiclient_nowait(taskrq, **pconn)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/agent/pypkg/agent/tasks/apiclient.py", line 35, in run_apiclient_nowait
    return await _retry_request(_apost_task, taskrq, client=client, theaders=theaders, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/agent/pypkg/agent/tasks/apiclient.py", line 191, in _retry_request
    raise exhttp
  File "/usr/local/agent/pypkg/agent/tasks/apiclient.py", line 166, in _retry_request
    retval = await request_procedure(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/agent/pypkg/agent/tasks/apiclient.py", line 246, in _apost_task
    async with client.post(
  File "/usr/local/agent/pyenv/lib64/python3.11/site-packages/aiohttp/client.py", line 1141, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File "/usr/local/agent/pyenv/lib64/python3.11/site-packages/aiohttp/client.py", line 643, in _request
    resp.raise_for_status()
  File "/usr/local/agent/pyenv/lib64/python3.11/site-packages/aiohttp/client_reqrep.py", line 1005, in raise_for_status
    raise ClientResponseError(
aiohttp.client_exceptions.ClientResponseError: 403, message='Forbidden', url=URL('http://cluster-leader:9311/api/module/traefik1/tasks')

Before this error, All containrs in the pod were running, but i could not reach them from the browser.

to test the authentik issue:

add-module ghcr.io/compgeniuses/goauthentik:certgen 1

All other commits before trying to implement certificate generation are here:

add-module ghcr.io/compgeniuses/goauthentik:latest1

stephdl · February 8, 2024, 7:01am

you are not allowed to use traefik look your buildscript

oneitonitram · February 8, 2024, 7:09am

I am not allowed to use traefik? WHat does that mean?

by the build script do you mean build-images.sh file?

stephdl · February 8, 2024, 7:12am

yes exactly

oneitonitram · February 8, 2024, 7:20am

from the looks of things, i see that everything is fine, Unless there is something i am no seeing.
IF you’ve spotted something could you point me to where the problem is and possibly how to resolve it?

EDIT, seen the issue

Would modifyign to this resolve the issue:
--label="org.nethserver.authorizations=traefik@node:certadm node:fwadm cluster:accountconsumer" \

or should it be:
--label="org.nethserver.authorizations=traefik@node:fulladm node:fwadm cluster:accountconsumer" \

stephdl · February 8, 2024, 10:11am

i don’t know what you want to do, so no clue

cluster:accountconsumer is for ldap
node:fwadm is for to open port in firewall
traefik@node:fulladm open route, full admin
traefik@node:certadm admin only to create certificate inside traefik

oneitonitram · February 8, 2024, 10:19am

Note yet added ldap integration yet, it will be added Eventually.
I need to create certificate of course.

WOuld fulladm also have certadm functions as well?

stephdl · February 8, 2024, 10:26am

I think so

stephdl · February 8, 2024, 11:45am

All this authorizations part is not well documented but good developers never read documentation, they always read the code of others developers who have read the documentation

oneitonitram · February 12, 2024, 6:25am

Happy to Announce Currently Testing Dolibarr 18.0.4 and Glpi modules

oneitonitram · March 8, 2024, 12:13pm

@chrkli i was not aware you had attempted to work on a vaultwarden App, here NS8 module template fails on creating repository - Bug - NethServer Community

if youre open to a collab, I would be happy to engage

oneitonitram · March 8, 2024, 1:10pm

@alefattorini could you kindly edit the first post to a wiki post?

EDIT: Thank you, now i can update the content

chrkli · March 8, 2024, 10:50pm

Hi Martin, sure I am.
Glad to see that there really is any interest in Vaultwarden as NS8-module going further.

I’ve managed to recover my broken VM I’ve used last year and pushed some minor changes I do not understand myself anymore to my GitHub repository.
Edit: Workflow is broken, too. That was not my intention

As I’m neither a programmer nor had anything else than the ns8-kickstart I had quite a hard time to get to where I’ve left off last year. Thus it might be better to take your work as the base rather than my try - feel free to have a look.

Kind regards
Christoph

oneitonitram · March 9, 2024, 4:21am

Update your yarn lock file, its out of date thats why.

oneitonitram · March 9, 2024, 4:58am

could someone help me figure out why i am getting the following error here

2024-03-08T23:29:05+03:00 [1:2fauth2:systemd] 2fauth-app.service: Ignoring invalid environment assignment '2FAUTH_IMAGE=docker.io/2fauth/2fauth:5.0.4': /home/2fauth2/.config/state/environment
2024-03-08T23:29:05+03:00 [1:2fauth2:systemd] 2fauth.service: Ignoring invalid environment assignment '2FAUTH_IMAGE=docker.io/2fauth/2fauth:5.0.4': /home/2fauth2/.config/state/environment
2024-03-08T23:29:05+03:00 [1:2fauth2:systemd] Stopping Podman 2fauth.service...
2024-03-08T23:29:05+03:00 [1:2fauth2:systemd] 2fauth.service: Ignoring invalid environment assignment '2FAUTH_IMAGE=docker.io/2fauth/2fauth:5.0.4': /home/2fauth2/.config/state/environment

In relation to this App geniusdynamics/ns8-2fa: NS8 App for 2FAuth https://docs.2fauth.app/ (github.com)

is it that its not permisible to have something like this

${2FAUTH_IMAGE}