Overcoming Imapsync Limitations: Working with Sieve Filters

I see the config file (reading the official man), should be able to accept multiple “pulls”, so I can implement all accounts I need to.
I will try that.

Also has the option to not touch the headers, which can at least make it work properly to look as the server itself received the mail.

1 Like

@NLS

Hi Nick

Looking forward to feedback if you can get this idea working.
It might even be interesting for our devs here, either as a internal workaround or a workflow concept how it “could” work internally.

As said: Good luck!

My 2 cents
Andy

Please do not take the “good luck” personally or as any form of reflection of your capabilities…
No matter how good any of us are in their professions or hobbies, no one needs a power outage, big forest fire or other huge nature catastrophe starting up next door when you’re in the middle of planing, implementing, or testing stuff.
Sh*t happens, luck can help!

:slight_smile:

Anyone in a situation like the following understands immediately:

You are cloning a physical, defective disk to a newer one, the old disk is giving off scraping noises.
You are at 95% complete and things look good.
Power and lights go out…
When the lights do come back on, the disk doesn’t make ANY noises any more, it’s dead…

Don’t worry, with decades in IT, I don’t wait for merit or criticism of my capabilities online in a forum. :smiley:
I first tested Linux pre-1.0 kernel (and before distros), yet I cannot claim to be anywhere near “real” Linux experts.

Yes, indeed, it is tough fighting obstacles and when you think everything is finally ok and HAVE moved to “production” (ok still only the home install, but was scheduling the other install also), something unexpected doesn’t work.

Don’t tell me about disks, because after many months I seem to have (few) read errors in one of my array disks at home - and while I am not afraid of losing data (disk is “ok-ish” still), I cannot also make a swap any time soon, as I just recently swapped two disks with 18TB ones and Ι prefer to have a wife, than feed my home lab more.


Back to topic, I will probably test it within the weekend.

1 Like

…btw I just realized, there is no way to disable an app instance (or the tasks, if talking about imapsync specifically). Would be nice for a future update.

Now I have to make the tasks error out for now…

It should be possible to disable it, just edit the imapsync task and set the frequency to disabled.

Another way could be to stop the service:

runagent -m imapsync1 systemctl --user stop imapsync

2 Likes

OK people, reporting back on fetchmail container.
WORKS GREAT!
Of course I had to check the original documentation of fetchmail to make a proper for my case fetchmailrc, I made it get ALL my mailboxes and send them to proper “back-end” NS8 mailboxes and works fine AND rules work!

The only issue is that I cannot make it have a different schedule for each mailbox (in imapsync I had less important mailboxes update every 5 minutes and more important update every 3 or 1 minute), now I am forced to use a single schedule for all - but this is minor right now.

A good thing is that it locks fetchmailrc file so that passwords are not exposed (although not encoded either).
A bad thing is that I cannot make it log to a file properly (possibly my fault) - but I can see the container’s live log and I’ve seen it working.

So, great WORKAROUND and food for thought for dev team. Maybe make something of it within NS8 own GUI. It would even fix the scheduling issue, because if it was embedded in NS8, it can call fetchmail with different parameters and scheduling be done by NS8 itself (ok cron separate fetchmail tasks).

Thank’s people for giving ideas.

3 Likes

:star_struck: this looks very promising

The Sieve filtering normally applied at delivery is applied to the
matching messages. This allows e.g. re-filtering messages that
were handled wrong at actual delivery. – pigeonhole/doc/rfc/draft-bosch-imap-filter-sieve-00.txt at main · dovecot/pigeonhole · GitHub

For example, in “Synchronize only INBOX” mode we could trigger the DELIVERY filters re-execution, so the messages added by imapsync are still subject to the same Sieve rules defined by Webtop or other MUA.

3 Likes

Depends on how it is triggered AND the load it produces.
If it can really be triggered transparently, right after message retrieval and ONLY on new mail, then why not - because imagine if filters re-run every time on full inbox (which could be GBs large).

Else just giving the option to people to implement imapsync OR getmail as a module, is good enough.

1 Like

just a blind shoot

We could trigger a cronjob to reorder email in inbox
sieve-filter -e -W -C -u "administrator" ./administrator/.dovecot.sieve INBOX
however this command must be triggered in the dovecot container itself, no way to run it with imapsync

still looking to imap_filter_sieve, there is something I did not catch on how to make it workable

1 Like

My suggestion:

  • /home/fetchmailuser/fetchmail1
#
set no syslog
set logfile /home/fetchmailuser/fetchmail1.log
set postmaster "Postmaster"
set no bouncemail
set no spambounce
set properties ""
#
#   GMAIL --> APP PASSWORD
poll pop.gmail.com tracepolls proto pop3 uidl auth password port 995 timeout 60
    user "username@gmail.com" password "APP_PASSWORD" keep ssl is user@my_domain here smtphost Nethserver_IP
#
#   POP3
poll pop.server tracepolls proto pop3 uidl auth password port 995 timeout 60
    user "USER_NAME" password "PASSWORD" ssl keep is user@my_domain here smtphost Nethserver_IP
#
#   POP3
poll pop.server tracepolls proto pop3 uidl auth password port 995 timeout 60
    user "USER_NAME" password "PASSWORD" ssl keep is user@my_domain here smtphost Nethserver_IP
#
#   POP3
poll pop.server tracepolls proto pop3 uidl auth password port 995 timeout 60
    user "USER_NAME" password "PASSWORD" ssl keep is user@my_domain here smtphost Nethserver_IP
#
  • /home/fetchmailuser/fetchmail2
#
set no syslog
set logfile /home/fetchmailuser/fetchmail2.log
set postmaster "Postmaster"
set no bouncemail
set no spambounce
set properties ""
#
#   POP3
poll pop.server tracepolls proto pop3 uidl auth password port 995 timeout 60
    user "USER_NAME" password "PASSWORD" ssl keep is user@my_domain here smtphost Nethserver_IP
#
#   POP3
poll pop.server tracepolls proto pop3 uidl auth password port 995 timeout 60
    user "USER_NAME" password "PASSWORD" ssl keep is user@my_domain here smtphost Nethserver_IP
#
  • fetchmail_all.sh
#
#!/bin/bash
#
fetchmail -d0 -v -f /home/fetchmailuser/fetchmail1;
fetchmail -d0 -v -f /home/fetchmailuser/fetchmail2;
#

Instead of using the fetchmail daemon, I’ve set two cron jobs:

1 - “fetchmail -d0 -v -f /home/fetchmailuser/fetchmail1” → runs at 0, 20, 40 minutes.
2 - “sh /home/fetchmailuser/fetchmail_all.sh” → runs at 5, 10, 15, 25, 30, 35, 45, 50, 55 minutes.

I am using a fetchmail container on my host that works nicely.
Would be even nicer if it was embedded in NS8 (to rectify for lost functionality over NS7 - because in the end, this is what it is), but its fine like that too.

1 Like