Fetchmail first draft

Inspired from this thread I started a first draft of ns8-fetchmail which could be the pop connector of NS8 to get mails from a mailserver and send them to the local one.

Documentation: GitHub - mrmarkuz/ns8-fetchmail
Install via Software Center from repo: ns8:mrmarkuz_repository [NethServer & NethSecurity]

It’s just an alpine container with fetchmail and cron so it should be possible to use cronjobs to run fetchmail tasks. There’s no UI, it needs to be configured manually. Your configurations and logs are included in backup.

Thanks to @NLS and @Tiago for testing and providing configuration examples.
Maybe you want to test the app as you seem to have working configs.

TODO

  • Find a good configuration together
  • User Interface

BTW, there’s also a roundcube fetchmail plugin that could be interesting: Roundcube fetchmail plugin | fetchmail

10 Likes

Definitely going to try this. Will revert when I do.

3 Likes

Oh wow that’s veeery good

Episode 1 Applause GIF by Friends

1 Like

So, after way too much delay (but I am also without a server for days, only today probably having solved it’s issues - probably)… I will start using this today.

Before this, the solution I found (since NS8 is a VM inside my UNRAID), was to use a fetchmail container in UNRAID that “pushed” mail to my NS8. Was working fine.

But:

  1. That container has no support (the author hasn’t even visited UNRAID forum for months).
  2. In the meantime, since it last worked, many “cogs” of my system changed (NIC, router) - supposedly these are configured OK, but you never know, better eliminate “cogs” I cannot control.
  3. I now get some problem in that container getting email from my mail provider. I will write what here, because we might hit the same with this here.
    I have a dynamic IP at home (and always worked this way with the help of ddns, after all my mail was received by my domain host’s cpanel -and pulled by NS8- and send relayed through my ISP).
    The problem is that now for some reasons it fails like that:
550-[<current dynamic IP>]:40126 is in an RBL: Listed by PBL, see
550 https://check.spamhaus.org/query/ip/<current dynamic IP>"
Feb 28 08:57:52 fetchmail: Connection errors for this poll:
name 0: connection to localhost:smtp [127.0.0.1/25] failed: Connection refused.
Feb 28 08:57:51 fetchmail: reading message <email address>:1 of 1Feb 28 08:57:51 fetchmail:  (6855 header octets) not flushed

…but the blacklisting I suspect happens right there. It is not that by chance I use a dynamic IP that was already marked. Something is done trying to pull the messages, that black lists the IP (I tried re-connecting my router, so quite a few IP). I hope this won’t happen with this, any idea what could cause it?

Anyway, will get back to you when I configure it.

1 Like

mrmarkuz I want to try this, but have questions… I prefer to “stupid proof” things (even things I know), than mess something.

  1. What is the use of the host name I define? Is it my mail domain? Is it my NS8 (internal) domain? Is it another new host name the container uses for whatever? (what is the use?) In the last case, I need to set it in my internal DNS also?
    I currently set it to fetchmail.local.mydomain (where “local.mydomain” is used in all my internal hosts), but haven’t set it in DNS (after all, I don’t even know what IP to set).

  2. Noob question, in NS8 console (and specifically within fetchmail1 agent context, how do I create/edit the files you mention in the config? OR how to I bring the files from some other location in the network?

  3. Seeing that fetchmailrc, is well… fetchmailrc (I already have one from my UNRAID container), I suspect I will see the same errors (see post above).
    Entries for each user are like that:

poll mail.<my domain> proto IMAP
  user '<an email address>' password '<the password>' is <local user> here options ssl
  smtphost <my nethserver hostname in the form nethserver.local.mydomain>
  smtpname <same email address>

So I wonder what is the problem is.
Remember those entries used to work fine.

But before we reach this point I need to somehow create the files needed. So please help with #1 and #2 first.

Thanks.

And…

  1. I would love if you would add some GUI config. Maybe just the cron config initially and/or editing fetchmailrc (even just pasting the text is useful enough and auto-resetting the permissions of the file, no need for buttons and selectors).
2 Likes

It seems the local MTA in the container isn’t working. Is sendmail available?

Sorry, the documentation isn’t clear. The hostname doesn’t matter, I should remove it from the UI too.

Best way is to get instance user:

runagent -m fetchmail1

and create/edit the files like

nano cron/cronjob1

or

nano fetchmail/fetchmailrc

You can copy the files to the following locations via SCP for example.

Cron: /home/fetchmail1/.config/state/cron
Fetchmail: /home/fetchmail1/.config/state/fetchmail

I’m going to add an adapted UI from imapsync. More complex configs are still possible via CLI.

2 Likes

Thank you for the reply.
I figured out the issue with UNRAID container (which would probably be an issue with your plugin too) - it was a communication issue with my local DNS.

The problem with editing/creating the files is that the pre-made NS8 image, doesn’t have nano installed (I have actually not touched the pre-made image OS side, except to set networking initially). I will figure it out.
It does have scp (after all, I have a working fetchmailrc already).

My UNRAID fetchmail container now works, but I will try this anyway as I said I would (waiting for my parity rebuild etc. as I had huge different issues with my server - which for a period seems it just hated me).

Thanks again.

1 Like

20 days after my last post, were (almost) 20 days with my server off line!

I got in to an Odyssey (!) of issues which led to me replacing (gradually) everything on my server except my storage. Motherboard actually got replaced twice.

Anyway, I cannot promise anything after all. :smiley: I hope to try this eventually (for now fetchmail container works). Strangely enough, I haven’t seen much interest from others though.

1 Like

Hello, I tried to start with the fetchmail module (mainly to get mail spam-checked when retrieving it), but I’m not bringing it up, maybe also due to my lack of knowledge.

I installed fetchmail container via GUI/Software Center
When I try to start setup according to here:
runagent -m fetchmail1

  • I do not find the config files in dir /home/fetchmail1/.config/state as stated in the doc

created then these directories

/home/fetchmail1/.config/state/cron
/home/fetchmail1/.config/state/fetchmail

with the respective files as you suggested:

[fetchmail1@ns8 cron]$ more cronjob1 
*/5 * * * * root /usr/bin/fetchmail -d0 -v -f /etc/fetchmail/fetchmailrc -L /var/log/fetchmail.log

[root@ns8 fetchmail]# sudo more fetchmailrc 
set no bouncemail
set no spambounce
set properties ""

#poll pop.server tracepolls proto pop3 uidl auth password port 995 timeout 60
#    user "<USER_NAME>" password "<PASSWORD>" ssl keep is <localmailuser@localmailserver.com> smtphost <Neth_IP>

#   IMAP
poll securemail.>a1.net proto imap port 993 
    user "USER" password "PASSWORD" keep is "NS8USERMAIL" here smtphost <<NS8IP>>

What is unclear:

  • which ownership do the files need? (currrently root/root). Do they need to be owned by fetchmail1/fetchmail1

In addition:
when configuring at the NS8 GUI The fetchmail hostname, the following error occurs:

{"context":{"action":"configure-module","data":{"host":"HOSTNAMR"},"extra":{"description":"Configuring","eventId":"a4ce4372-6f29-49eb-b7a0-d0873b4fcc21","title":"Configure fetchmail"},"id":"3e6f4ad7-0807-438c-bea3-a18b90b8b394","parent":"","queue":"module/fetchmail1/tasks","timestamp":"2025-06-14T05:27:10.909580501Z","user":"admin"},"status":"aborted","progress":50,"subTasks":[],"validated":true,"result":{"error":"<7>dump_env() is deprecated and implemented as a no-op\nCreated symlink /home/fetchmail1/.config/systemd/user/default.target.wants/fetchmail.service → /home/fetchmail1/.config/systemd/user/fetchmail.service.\nJob for fetchmail.service failed because the control process exited with error code.\nSee \"systemctl --user status fetchmail.service\" and \"journalctl --user -xeu fetchmail.service\" for details.\n","exit_code":1,"file":"task/module/fetchmail1/3e6f4ad7-0807-438c-bea3-a18b90b8b394","output":""}}
<7>dump_env() is deprecated and implemented as a no-op
Created symlink /home/fetchmail1/.config/systemd/user/default.target.wants/fetchmail.service → /home/fetchmail1/.config/systemd/user/fetchmail.service.
Job for fetchmail.service failed because the control process exited with error code.
See "systemctl --user status fetchmail.service" and "journalctl --user -xeu fetchmail.service" for details.

systemctl and journalctl does reply the following:

[root@ns8 fetchmail]# systemctl --user status fetchmail.service
Failed to connect to bus: No medium found
[root@ns8 fetchmail]# systemctl status fetchmail.service
Unit fetchmail.service could not be found.

In the logs for fetchmail1 node I see the following:

2025-06-14T07:27:14+02:00 [1:fetchmail1:systemd] Stopped fetchmail server.
2025-06-14T07:27:14+02:00 [1:fetchmail1:systemd] Starting fetchmail server...
2025-06-14T07:27:15+02:00 [1:fetchmail1:podman] Error: lsetxattr /home/fetchmail1/.config/state/fetchmail: operation not permitted
2025-06-14T07:27:15+02:00 [1:fetchmail1:systemd] fetchmail.service: Control process exited, code=exited, status=126/n/a
2025-06-14T07:27:15+02:00 [1:fetchmail1:podman] 59e29f77c0457e2665b951068031006d9bb18c94c3ad0b33e12d19907daf697f
2025-06-14T07:27:15+02:00 [1:fetchmail1:systemd] fetchmail.service: Failed with result 'exit-code'.
2025-06-14T07:27:15+02:00 [1:fetchmail1:systemd] Failed to start fetchmail server.
2025-06-14T07:27:15+02:00 [1:fetchmail1:systemd] fetchmail.service: Scheduled restart job, restart counter is at 4.
2025-06-14T07:27:15+02:00 [1:fetchmail1:systemd] Stopped fetchmail server.
2025-06-14T07:27:15+02:00 [1:fetchmail1:systemd] Starting fetchmail server...
2025-06-14T07:27:15+02:00 [1:fetchmail1:podman] Error: lsetxattr /home/fetchmail1/.config/state/fetchmail: operation not permitted
2025-06-14T07:27:15+02:00 [1:fetchmail1:systemd] fetchmail.service: Control process exited, code=exited, status=126/n/a
2025-06-14T07:27:15+02:00 [1:fetchmail1:podman] f35bb0b36599b9cefc70bce91c39c58b226a65945fb0c3fff019c6e90ae6daa1
2025-06-14T07:27:15+02:00 [1:fetchmail1:systemd] fetchmail.service: Failed with result 'exit-code'.
2025-06-14T07:27:15+02:00 [1:fetchmail1:systemd] Failed to start fetchmail server.
2025-06-14T07:27:16+02:00 [1:fetchmail1:systemd] fetchmail.service: Scheduled restart job, restart counter is at 5.
2025-06-14T07:27:16+02:00 [1:fetchmail1:systemd] Stopped fetchmail server.
2025-06-14T07:27:16+02:00 [1:fetchmail1:systemd] fetchmail.service: Start request repeated too quickly.
2025-06-14T07:27:16+02:00 [1:fetchmail1:systemd] fetchmail.service: Failed with result 'exit-code'.
2025-06-14T07:27:16+02:00 [1:fetchmail1:systemd] Failed to start fetchmail server.
2025-06-14T18:15:15+02:00 [1:fetchmail1:systemd] Starting Cleanup of User's Temporary Files and Directories...
2025-06-14T18:15:15+02:00 [1:fetchmail1:systemd] Finished Cleanup of User's Temporary Files and Directories.
2025-06-15T18:16:15+02:00 [1:fetchmail1:systemd] Starting Cleanup of User's Temporary Files and Directories...
2025-06-15T18:16:15+02:00 [1:fetchmail1:systemd] Finished Cleanup of User's Temporary Files and Directories.

Any hints what I’m doing wrong here (as a tried to follow the documentation)?
Many thanks!

Yes, in the local filesystem they need to be owned by fetchmail1:fetchmail1 which maps to root in the fetchmail container.
And the fetchmailrc file needs 700 permission.

I’m trying to reproduce the issue…

EDIT:

It’s working here.
Please check the owner and permissions of the cronfile and the fetchmailrc. I guess this is the cause the service isn’t starting.

The systemctl commands need to be executed in the fetchmail instance user environment:

Enter fetchmail1 instance environment:

runagent -m fetchmail1

Check service:

systemctl --user status fetchmail.service

To restart the service:

systemctl --user restart fetchmail.service

EDIT2:

There’s a wrong character > before a1.

Thanks a lot for the fast response:
I updated the ownership of the files, and also of the directories cron and fetchmail to fetchmail1/fetchmail1

Thanks also for spotting the typo in the fetchmailrc file → corrected.

restarted the fetchmail.service with your systemctl command → and it’s running!

Now I need to see what the error is during mail polling:

[root@ns8 log]# tail -f  fetchmail.log
fetchmail: 6.4.38 querying <MAILSERVER> (protocol IMAP) at Sun, 15 Jun 2025 19:55:00 +0000 (UTC): poll started
fetchmail: Trying to connect to <ip>/993...connected.
fetchmail: socket error while fetching from <USER@MAIL.DOMAIN>
fetchmail: 6.4.38 querying <MAILSERVER> (protocol IMAP) at Sun, 15 Jun 2025 19:58:01 +0000 (UTC): poll completed
fetchmail: Query status=2 (SOCKET)
fetchmail: normal termination, status 2

Probably I need to check security settings in fetchmailrc

To connect via IMAPS, the ssl option needs to be added like

#   IMAPS
poll mrmarkuz.domain.tld proto imap port 993
    user "<USER_NAME>" password "<PASSWORD>" ssl keep is "markus@ns8test.com" here smtphost <Neth_IP>

I updated the README.

Thanks again for the hint. Updated the fetchmailrc accordingly.

After moving away another typo it worked - and most important, including spam check.
Many thanks again for helping and providing this module.

A follow-up question: Do I need to care about logrotate of the fetchmail.log or is it included? If yes, what is the best way in the container?

never include log inside a container, normally we print logs to stderr and it goes to journald
look after stderr in the dockerfile : ns8-lamp/container/Containerfile at d2d161fcbc948ee827bfe9f9994b21856e7d2aa2 · stephdl/ns8-lamp · GitHub

also in the supervisord

1 Like

I’m going to provide an update to either log to stderr or just log the last sync.

1 Like

As fetchmail doesn’t run in daemon mode, it seems not possible to send logs to stderr so I decided to change the cronjob to just keep the last sync in fetchmail.log so no logrotate is needed, see also Fetchmail Manual

The cronjob should look like this, I also changed it in the README:

*/5 * * * * root /usr/bin/fetchmail -d0 -v -f /etc/fetchmail/fetchmailrc > /var/log/fetchmail.log

1 Like

Thanks for the update. May I ask, just out of curiosity:

As I understood in the systemd service the crond is started via “podman run” which is itself then invoking the fetchmail service as per cronjob.
Would it be an alternative to start directly the fetchmail service (in deamon mode providing the time interval for performing the mailchecks) in the systemd file of the module (in the podman run command)?