Email 2 (Beta) released

EddieA · March 10, 2018, 1:43am

Looks like no:

[root@Nethserver ~]# systemctl status redis
● redis.service - Redis persistent key-value database
   Loaded: loaded (/usr/lib/systemd/system/redis.service; disabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/redis.service.d
           └─limit.conf
   Active: inactive (dead)
[root@Nethserver ~]#

Looking in the log, I see this:

Mar  9 16:36:21 Nethserver redis-server: /usr/bin/redis-server: error while loading shared libraries: libjemalloc.so.1: cannot open shared object file: No such file or directory
Mar  9 16:36:21 Nethserver systemd: redis-rspamd.service: main process exited, code=exited, status=127/n/a

Looks like a pre-req was possibly missed.

Cheers.

EddieA · March 10, 2018, 2:10am

OK, installing “jemalloc” allows redis to start, but I’m not sure what other services now need to be restarted to pick this up.

All I’ve done so far is “rspamd” via the services UI. Is that sufficient.

Cheers.

mrmarkuz · March 10, 2018, 2:21am

It should be enough, rspamd uses redis.
If there are no errors anymore it should be OK.
Do you have nethserver-redis installed? If not it may be the reason for the missing dependency.

EddieA · March 10, 2018, 2:35am

Nope, not there. Which NS feature is that part of.

Update It’s Bandwidth Monitor (ntopng), which I don’t have installed.

Cheers.

stephdl · March 10, 2018, 9:16am

we simply use redis, because we bring our customisations

jemalloc is a dependency of redis

Installing:
 redis                                              x86_64                                           3.2.10-2.el7                                             epel                                           545 k
Installing for dependencies:
 jemalloc                                           x86_64                                           3.6.0-1.el7                                              epel                                           105 k

I recall sometime with certain (not predictable) servers some problem of dependency for transmission, we could call ourself jemalloc to be sure it never happen more. @davidep ???

to check if redis work

redis-cli -s /var/run/redis-rspamd/rspamd

then do at the prompt, and wait one minute

monitor

davidep · March 10, 2018, 11:43am

Could you share the contents of /var/log/messages* around that time? Maybe it’s a race condition on service (re)startup…

As redis already pulls in jemalloc it shouldn’t be required

stephdl · March 10, 2018, 12:06pm

yes but it seems the jemalloc was not installed

davidep · March 10, 2018, 1:19pm

If so, where’s the RPM transaction error? Why it occurred for certain installs and not for others?

stephdl · March 10, 2018, 2:17pm

Could you check your log to understand why jemalloc was not installed.

I solved my transmission issue by adding all depencies to nethserver-transmission @davidep

EddieA · March 10, 2018, 7:56pm

As luck would have it, the yum install was still in the PuTTY scrollback buffer:

[root@Nethserver ~]# yum swap \
>     -- remove nethserver-mail-{common,filter} nethserver-p3scan nethserver-spamd \
>     -- install nethserver-mail2-{common,filter,p3scan}
Loaded plugins: changelog, fastestmirror, nethserver_events
Loading mirror speeds from cached hostfile
 * base: mirror.riverfrontnetworks.com
 * epel: mirrors.develooper.com
 * extras: mirrors.ocf.berkeley.edu
 * nethforge: mirror.framassa.org
 * nethserver-base: mirror.framassa.org
 * nethserver-updates: mirror.framassa.org
 * updates: repo1.dal.innoscale.net
Resolving Dependencies
--> Running transaction check
---> Package nethserver-mail-common.noarch 0:1.6.6-1.ns7 will be erased
---> Package nethserver-mail-filter.noarch 0:1.4.5-1.ns7 will be erased
---> Package nethserver-mail2-common.noarch 0:2.0.0-1.ns7 will be installed
---> Package nethserver-mail2-filter.noarch 0:2.0.0-1.ns7 will be installed
--> Processing Dependency: zstd for package: nethserver-mail2-filter-2.0.0-1.ns7.noarch
--> Processing Dependency: rspamd for package: nethserver-mail2-filter-2.0.0-1.ns7.noarch
--> Processing Dependency: redis for package: nethserver-mail2-filter-2.0.0-1.ns7.noarch
--> Processing Dependency: mod_authnz_pam for package: nethserver-mail2-filter-2.0.0-1.ns7.noarch
---> Package nethserver-mail2-p3scan.noarch 0:2.0.0-1.ns7 will be installed
---> Package nethserver-p3scan.noarch 0:1.1.3-1.ns7 will be erased
---> Package nethserver-spamd.noarch 0:1.0.1-1.ns7 will be erased
--> Running transaction check
---> Package mod_authnz_pam.x86_64 0:1.1.0-1.el7 will be installed
---> Package redis.x86_64 0:3.2.10-2.el7 will be installed
---> Package rspamd.x86_64 0:1.6.6-4 will be installed
--> Processing Dependency: libfann.so.2()(64bit) for package: rspamd-1.6.6-4.x86_64
---> Package zstd.x86_64 0:1.3.3-1.el7 will be installed
--> Running transaction check
---> Package fann.x86_64 0:2.2.0-6.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

================================================================================
 Package                   Arch     Version         Repository             Size
================================================================================
Installing:
 nethserver-mail2-common   noarch   2.0.0-1.ns7     nethserver-updates     56 k
 nethserver-mail2-filter   noarch   2.0.0-1.ns7     nethserver-updates     64 k
 nethserver-mail2-p3scan   noarch   2.0.0-1.ns7     nethserver-updates     36 k
Removing:
 nethserver-mail-common    noarch   1.6.6-1.ns7     @nethserver-updates   103 k
 nethserver-mail-filter    noarch   1.4.5-1.ns7     @nethserver-updates   399 k
 nethserver-p3scan         noarch   1.1.3-1.ns7     @nethserver-updates    47 k
 nethserver-spamd          noarch   1.0.1-1.ns7     @nethserver-updates    35 k
Installing for dependencies:
 fann                      x86_64   2.2.0-6.el7     epel                   81 k
 mod_authnz_pam            x86_64   1.1.0-1.el7     base                   15 k
 redis                     x86_64   3.2.10-2.el7    epel                  545 k
 rspamd                    x86_64   1.6.6-4         nethserver-updates     10 M
 zstd                      x86_64   1.3.3-1.el7     epel                  287 k

Transaction Summary
================================================================================
Install  3 Packages (+5 Dependent packages)
Remove   4 Packages

Total download size: 11 M
Is this ok [y/d/N]: y
Downloading packages:
(1/8): fann-2.2.0-6.el7.x86_64.rpm                         |  81 kB   00:00
(2/8): redis-3.2.10-2.el7.x86_64.rpm                       | 545 kB   00:00
(3/8): mod_authnz_pam-1.1.0-1.el7.x86_64.rpm               |  15 kB   00:00
(4/8): zstd-1.3.3-1.el7.x86_64.rpm                         | 287 kB   00:00
(5/8): nethserver-mail2-common-2.0.0-1.ns7.noarch.rpm      |  56 kB   00:01
(6/8): nethserver-mail2-filter-2.0.0-1.ns7.noarch.rpm      |  64 kB   00:01
(7/8): nethserver-mail2-p3scan-2.0.0-1.ns7.noarch.rpm      |  36 kB   00:01
(8/8): rspamd-1.6.6-4.x86_64.rpm                           |  10 MB   00:10
--------------------------------------------------------------------------------
Total                                              1.1 MB/s |  11 MB  00:10
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : zstd-1.3.3-1.el7.x86_64                                     1/12
  Installing : redis-3.2.10-2.el7.x86_64                                   2/12
  Installing : nethserver-mail2-common-2.0.0-1.ns7.noarch                  3/12
  Installing : fann-2.2.0-6.el7.x86_64                                     4/12
  Installing : rspamd-1.6.6-4.x86_64                                       5/12
  Installing : mod_authnz_pam-1.1.0-1.el7.x86_64                           6/12
  Installing : nethserver-mail2-filter-2.0.0-1.ns7.noarch                  7/12
  Installing : nethserver-mail2-p3scan-2.0.0-1.ns7.noarch                  8/12
  Erasing    : nethserver-p3scan-1.1.3-1.ns7.noarch                        9/12
  Erasing    : nethserver-mail-filter-1.4.5-1.ns7.noarch                  10/12
  Erasing    : nethserver-mail-common-1.6.6-1.ns7.noarch                  11/12
  Erasing    : nethserver-spamd-1.0.1-1.ns7.noarch                        12/12
  Verifying  : nethserver-mail2-p3scan-2.0.0-1.ns7.noarch                  1/12
  Verifying  : mod_authnz_pam-1.1.0-1.el7.x86_64                           2/12
  Verifying  : fann-2.2.0-6.el7.x86_64                                     3/12
  Verifying  : nethserver-mail2-common-2.0.0-1.ns7.noarch                  4/12
  Verifying  : redis-3.2.10-2.el7.x86_64                                   5/12
  Verifying  : rspamd-1.6.6-4.x86_64                                       6/12
  Verifying  : nethserver-mail2-filter-2.0.0-1.ns7.noarch                  7/12
  Verifying  : zstd-1.3.3-1.el7.x86_64                                     8/12
  Verifying  : nethserver-p3scan-1.1.3-1.ns7.noarch                        9/12
  Verifying  : nethserver-spamd-1.0.1-1.ns7.noarch                        10/12
  Verifying  : nethserver-mail-common-1.6.6-1.ns7.noarch                  11/12
  Verifying  : nethserver-mail-filter-1.4.5-1.ns7.noarch                  12/12

Broadcast message from nut@Nethserver.BogoLinux.net (Fri Mar  9 16:35:57 2018):

Communications with UPS UPS@127.0.0.1 lost

Broadcast message from nut@Nethserver.BogoLinux.net (Fri Mar  9 16:36:02 2018):

Communications with UPS UPS@127.0.0.1 established

Removed:
  nethserver-mail-common.noarch 0:1.6.6-1.ns7
  nethserver-mail-filter.noarch 0:1.4.5-1.ns7
  nethserver-p3scan.noarch 0:1.1.3-1.ns7
  nethserver-spamd.noarch 0:1.0.1-1.ns7

Installed:
  nethserver-mail2-common.noarch 0:2.0.0-1.ns7
  nethserver-mail2-filter.noarch 0:2.0.0-1.ns7
  nethserver-mail2-p3scan.noarch 0:2.0.0-1.ns7

Dependency Installed:
  fann.x86_64 0:2.2.0-6.el7          mod_authnz_pam.x86_64 0:1.1.0-1.el7
  redis.x86_64 0:3.2.10-2.el7        rspamd.x86_64 0:1.6.6-4
  zstd.x86_64 0:1.3.3-1.el7

Complete!
[root@Nethserver ~]#

No jemalloc.

Cheers.

davidep · March 10, 2018, 8:01pm

Could you attach the output of

yum history pkg-info jemalloc

EddieA · March 10, 2018, 8:09pm

[root@Nethserver ~]# yum history pkg-info jemalloc
Loaded plugins: changelog, fastestmirror, nethserver_events
Transaction ID : 127
Begin time     : Fri Mar  9 17:52:52 2018
Package        : jemalloc-3.6.0-1.el7.x86_64
State          : Install
Size           : 324,797
Build host     : buildvm-24.phx2.fedoraproject.org
Build time     : Mon Mar 31 15:33:02 2014
Packager       : Fedora Project
Vendor         : Fedora Project
License        : BSD
URL            : http://www.canonware.com/jemalloc/
Source RPM     : jemalloc-3.6.0-1.el7.src.rpm
Commit Time    : Mon Mar 31 05:00:00 2014
Committer      : Ingvar Hagelund <ingvar@redpill-linpro.com>
Reason         : user
Command Line   : install jemalloc
From repo      : epel
Installed by   : eddie <xxxx@xxxxxxxx.xxx>
history pkg-info
[root@Nethserver ~]#

Which is when I installed it manually, over an hour after the upgrade to email.

Cheers.

davidep · March 10, 2018, 8:21pm

This is mine, spot the difference

Begin time     : Thu Feb  8 18:01:16 2018
Package        : jemalloc-3.6.0-1.el7.x86_64
State          : Dep-Install
Size           : 324,797
Build host     : buildvm-24.phx2.fedoraproject.org
Build time     : Tue Apr  1 00:33:02 2014
Packager       : Fedora Project
Vendor         : Fedora Project
License        : BSD
URL            : http://www.canonware.com/jemalloc/
Source RPM     : jemalloc-3.6.0-1.el7.src.rpm
Commit Time    : Mon Mar 31 14:00:00 2014
Committer      : Ingvar Hagelund <ingvar@redpill-linpro.com>
Reason         : dep
Command Line   : --enablerepo=nethserver-testing swap -- remove nethserver-mail-co
mon nethserver-mail-filter nethserver-mail-server -- install nethserver-mail2-comm
n nethserver-mail2-filter nethserver-mail2-server
From repo      : nh-epel
Installed by   : davidep <davidep@nethesis.it>

EddieA · March 10, 2018, 8:31pm

Yep, yours installed as part of the email upgrade, but from a different repo. Obviously mine didn’t, based on what I pasted.

Should I have activated the testing repo. If so, that wasn’t mentioned either in this thread or the doc.

Any further information I can provide ?

Cheers.

davidep · March 10, 2018, 9:09pm

It’s an epel mirror, it does not differ from yours.

At that time rspamd was still from testing.

In the end I still don’t understand what happened neither know how to reproduce the service startup issue. Let’s see if someone else is hit.

EddieA · March 10, 2018, 9:17pm

That’s easy. Don’t install jemalloc and then start redis.

The issue appears to be entirely that jemalloc wasn’t picked up as a pre-requisite during the yum swap command, despite the fact that it is:

[root@Nethserver ~]# repoquery --requires --resolve redis
shadow-utils-2:4.1.5.1-24.el7.x86_64
bash-0:4.2.46-29.el7_4.x86_64
jemalloc-0:3.6.0-1.el7.x86_64
systemd-0:219-42.el7_4.10.x86_64
glibc-0:2.17-196.el7_4.2.i686
gawk-0:4.0.2-4.el7_3.1.x86_64
glibc-0:2.17-196.el7.x86_64
logrotate-0:3.8.6-14.el7.x86_64
[root@Nethserver ~]#

Cheers.

davidep · March 10, 2018, 9:23pm

As you can see from the yum history output jemalloc was pulled in by a yum swap command.

We are comparing two cases one of success the other of failure. We need a tie-breaker thanks for the info provided so far!

Edit: in another time and another place there had been a similar discussion! It’s a mistery!

https://forum.remirepo.net/viewtopic.php?id=2863

EddieA · March 11, 2018, 7:58am

Not part of the issue I reported above, but for the documented POP3 Proxy update, should nethserver-mail-server have also been included.

Looking at the Software Center following the update, I see both versions of nethserver-mail-server as not installed:

Where for the other modules involved, I see the EMail 2 versions under the installed tab:

Checking what I have installed shows:

[root@Nethserver ~]# yum list installed | grep mail
mailcap.noarch                      2.1.41-2.el7             @nethserver
mailx.x86_64                        12.5-16.el7              @base
nethserver-mail-server.noarch       1.11.0-1.ns7             @nethserver-updates
nethserver-mail-smarthost.noarch    1.0.0-1.ns7              @nethserver-updates
nethserver-mail2-common.noarch      2.0.0-1.ns7              @nethserver-updates
nethserver-mail2-filter.noarch      2.0.0-1.ns7              @nethserver-updates
nethserver-mail2-p3scan.noarch      2.0.0-1.ns7              @nethserver-updates
procmail.x86_64                     3.22-36.el7_4.1          @updates
[root@Nethserver ~]#

So, I’m wondering if the nethserver-mail-server should also have been included as part of the update.

Cheers.

davidep · March 11, 2018, 9:51am

Yes, should be upgraded too. The only “-mail-” that survives is smarthost

I see there’s also another thing to fix: the optional nethserver-getmail package should be actually nethserver-mail2-getmail

davidep · March 12, 2018, 10:47am

Closing the announcement topic: please open new topics for any issue, bug, support or feature request