Cleanup httpd configuration files at removal nethserver-module

davidep · September 9, 2018, 8:01pm

I back the use of RPM features, like config(nore place) to implement the wanted behavior. The %postun hook could be redundant if the config file is renamed .rpmsave

IIRC the template-expand command attempts to remove that suffix along with .rpmnew, to keep the dir tidy.

github.com

NethServer/nethserver-lib/blob/81e4c1d5db6143584f0f60f485d435413613ab59/lib/perl/esmith/templates.pm#L727-731


          && warn(
            "Updating $outputfile - MD5 was $oldMD5sum, now $newMD5sum\n");


        rename( "$tempfile", "$outputfile" )
          or carp(
            "ERROR: Could not rename $tempfile " . "to $outputfile: $!\n" );
    }


    # copy any additional files


    # A side effect of this routine is that it removes any old copies or
    # proposed new copies that RPM leaves lying around. (i.e. F<.rpmsave>
    # and F<.rpmnew> files.
    -e "$outputfile.rpmsave" and unlink "$outputfile.rpmsave";
    -e "$outputfile.rpmnew"  and unlink "$outputfile.rpmnew";
}
elsif ( $p{'OUTPUT_TYPE'} eq 'string' ) {
    if ( defined $p{FILTER} ) {
        $text = _filter_text( $text, $p{FILTER} );
    }
    return $text;

mark_nl · September 11, 2018, 6:48am

Worked on this implementation a bit and not able to sort one thing out:

If an application installs a apache configuration by its self (ie /etc/httpd/conf.d/package.conf), which will be overwritten with our own templated version, it turns out to be difficult to get rid of it.

(AFAIK now) It is impossible to add a placeholder in the nethserver-module package; at install this throws a transition error because you try to install one file 2 times. And removing the nethserver-module does not remove the underling package.

Any ideas how to solve this?

davidep · September 11, 2018, 8:53am

I’m not sure I understand well …Could you provide an example with a real module?

mark_nl · September 11, 2018, 9:32am

As examples sogo and roundcubemail packages install a configuration file in /etc/hhtpd/conf.d/

Examine (at the Files section) the roundcubemail-1.1.12-2.el7.noarch.rpm package

Files
Path
/etc/httpd/conf.d/roundcubemail.conf

The nethserver-roundcubemail package overwrites this default configuration installed by the “underlying” roundcubemail package

Adding a placeholder for roundcubemail.conf in nethserver-roundcubemail would (1) throw:

Transaction check error:
  file /etc/httpd/conf.d/roundcubemail.conf conflicts between attempted installs of .....

removing nethserserver-roundcubemail does not remove roundcubemail -…rpm , hence /etc/httpd/conf.d/roundcubemail.conf stays present / will not be renamed until you remove this “underlying” package manually.

(1) translation of a test case done with nethserver-gitea

Transaction check error:
  file /etc/gitea/app.ini conflicts between attempted installs of nethserver-gitea-0.1.0-1.7.g953d0b5.ns7.noarch and gitea-1.5.1-2.ns7.x86_64

davidep · September 11, 2018, 9:37am

Instead of a “placeholder” (like an empty file), please try with a file with the same contents (same md5), owner and permissions. RPM could not raise the conflict if all of them are the same across the conflicting packages.

mrmarkuz · September 11, 2018, 9:37am

yum autoremove nethserver-roundcubemail removes the underlying package roundcubemail.

mark_nl · September 11, 2018, 9:40am

I can try this, but is this not an maintaining nightmare?

(I mean always needing to check it this is this true?)

mark_nl · September 11, 2018, 9:42am

would love this!, but it failed in the past to do this by default in the service manager

davidep · September 11, 2018, 9:44am

Yes: every time the upstream package changes the config file we must change it too. Luckily it should happen rarely, and if it happens it means that our templates are likely to be affected by a new feature.

It can be considered a safety guard

If we can’t rely on RPM to do the job, we have to implement and maintain something else by ourselves…

Yes, package removal is a really complex matter and we avoided to deal with it: is it worth the effort?

mark_nl · September 11, 2018, 9:48am

Just out of curiosity, what are your concerns with: ?

%postun
if [ $1 == 0 ]; then
  /usr/bin/rm -f ...
fi

dnutan · September 11, 2018, 9:49am

Should not be the upstream package (or the package owning the file) responsible of the conf file removal? nethserver-* should not remove the conf file if it doesn’t own it.

davidep · September 11, 2018, 9:51am

No concerns at all. If RPM can’t do the cleanup for us that seems the only available solution…

Generally I prefer a “declarative” approach, instead of adding runtime code. But here the code is trivial so I think it is acceptable too.

mark_nl · September 11, 2018, 9:51am

We do overwrite them with templated content (some expeditions…) , so “who” owns the file?

dnutan · September 11, 2018, 9:54am

the package creating the file in first place. If the file is created by nethserver-*, then remove it. Otherwise, leave it even if modified from templates. But I could be missing something on the conversation.

davidep · September 11, 2018, 9:56am

I agree with @dnutan, if the .conf was installed by sogo we have to consider the sogo removal!

Anyway we have also some cases like zzz_whatever.conf that we have to clean up by ourselves

mark_nl · September 11, 2018, 10:01am

roundcubemail is a nice example:
removing nethserver-rouncubemail basically does not do much:
the dashboard widget and the rewrite rules in default-virtualhost.inc are removed .

because /etc/httpd/conf.d/roundcubemail.conf “stays behind” apache keeps reloading this config and rounscubemail actually stays functional.

mark_nl · September 11, 2018, 10:04am

thats true, but dislike to introduce a extra reason to keep this sub-optimal solution.

This is the real solution, but as said quite complex

mark_nl · September 11, 2018, 4:01pm

Did an implementation on SOGo for conversation:

Did you have this in mind?

After removal of nethserver-sogo zzz_SOGo.conf.rpmsave and a by a template overwritten (harmless) SOGo.conf are kept in /etc/httpd/conf.d/. Effectively disabling all apache configuration for the Application.

mark_nl · September 19, 2018, 8:29pm

Worked on this a bit more and I can not Agree. It makes no sense to keep a apache configuration file after a nethserver module is removed.

Dug in to two packages who install their own configuration, webtop-dav and rouncubemail.
webtop5-webdav.conf is completely static and roundcubemail.conf gets overwritten by templates.

If nethserver-webtop is removed and then SOGo is installed it breaks card / caldav for support SOGo due to the apache configuration staying behind.

As said before removing nethserver-roundcubemail has no effect, the service is still available after removal. Situation actually gets worse as we lose control over this configuration because it does not get updated (as example) if you change the trussed networks.

IMHO nethserver / e-smith shall be in control over the (content of) the loaded apache configuration files for nethserver modules.
This leeds me to a more radical idea:
After removal of a nethserver module no apache configuration file for this module shall be (re)loaded.
Until package removal is improved: It should be considered bad practice to keep apache configuration files installed by a package it configures (ie Roundcubemail) in the first place. They must be removed at install with metadata template.

stephdl · September 19, 2018, 8:45pm

e-smith API was made around 1999, updated and maintained since ages…it is something that NS has cleaned, maybe it is time to make something new and better.

Yes IMHO, when a template is no longer present in the rpm or of course if the RPM is no longer installed, the template should not be present anymore.

Obviously, saying that is easy, to make it possible is another game. FYI I do not like the %post way, we should try to do by another way (the esmith API)

Actually to remove a template, the official way is to expand it with a content DELETE=1 inside.