CentOS 7.4 - Proposal for repository policy change (temporary)

Two and half years ago we decided together to always release upstream updates as soon as possible (Change of NethServer upgrade policy).

But it seems we had to much faith on RH guys this time! :slight_smile:
SSSD and samba where released many months ago in Fedora, and RH did internal and external tests for about 6 months on all packages, but they didn’t catch the bug we hit on Samba authentication ([SOLVED] CentOS 7.4 (1708) - Shared folder access).

I’d like to propose a temporary policy change for YUM repository and let all installations access frozen repositories containing only packages from CentOS 7.3.

For those of you who don’t know, Nethesis hosts a mirror of all upstream repository for their customers, we are happy to share this service with the Community until the upstream bug is fixed.

You can already do it following these instructions.

Create a file named /etc/yum.repos.d/00vault.repo and cut & paste the following inside:

# Temporary fixed repositories
# to avoid upgrade to CentOS 7.4
#
# See https://github.com/NethServer/dev/issues/5350

[base]
name=CentOS-7.3.1611 - Base - Vault
mirrorlist=http://packages.nethserver.org/temporary-mirror.php?release=$releasever&arch=$basearch&repo=base
# baseurl=http://update.nethesis.it/nethserver/7.3.1611/nh-base/$basearch
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

#released updates 
[updates]
name=CentOS-7.3.1611 - Updates - Vault
mirrorlist=http://packages.nethserver.org/temporary-mirror.php?release=$releasever&arch=$basearch&repo=updates
# baseurl=http://update.nethesis.it/nethserver/7.3.1611/nh-updates/$basearch
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

#additional packages that may be useful
[extras]
name=CentOS-7.3.1611 - Extras - Vault
mirrorlist=http://packages.nethserver.org/temporary-mirror.php?release=$releasever&arch=$basearch&repo=extras
# baseurl=http://update.nethesis.it/nethserver/7.3.1611/nh-extras/$basearch
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7

At the end, execute:

yum clean all

You can ignore any warning message like these ones:

Repository base is listed more than once in the configuration
Repository updates is listed more than once in the configuration
Repository extras is listed more than once in the configuration

Now the real question: do you think we should release a NethServer package which includes this fix until everything is resolved? Or do you feel it’s better to leave freedom to users and document this solution?

/cc @dev_team @quality_team

1 Like

Probably these guys are interested to discuss it: @EddieA @Jim @sharpec @indra @compsos @gerald_FS @greavette @bwdjames @des @Andy_Wismer

I would say that this is something to strongly consider doing as a temporary measure

@giacomo

Hi

I’d personally welcome the option of a “frozen” repo.
I need to install 1-2 servers this weekend, so far I’ve put it off because of the 7.4 issue.
These servers aren’t going productive right now, but I’d like to set up the Hardware / VMs in advance.

Having such a repo would give those inclined the option of choice…

My 2 cents…

Regards from Switzerland
and keep up the great work!

2 Likes

For admins who wants to have full AD implementation I think it is a great proposal!

BTW. i have looked on on ClearOS repos - base repos don’t point on CentOS repos but only on ClearOS repos, of course epel repo etc works like on CentOS… maybe this could also be an option

2 Likes

I’m basing the entire corporate network on nethserver. I have other 3 companies where the implementation of almost all services, AD, mail, vpn is anticipated. I would like to sleep at night :sleeping:

5 Likes

@giacomo, I agree that a frozen repo would greatly benefit the nethserver community. Although it’s always best to have a development server to test updates and have a good backup strategy in place, people that don’t use VM’s or are small and perhaps can’t afford a second dev box would be very frustrated if the update broke something critical. I would welcome this temporary fix to ensure the nethserver community is protected from major issues…it definitely boosts nethserver as a caring product who looks after it’s community. :slight_smile:

6 Likes

I think this is best practice. As soon updates are rolled out, it’s testing time. If you run into a problem you do not pursue adopting the update since it will break features you rely on.
Is there more testing necessary from us, the community and project, to avoid adopting broken packages in the (near) future? Or maybe even a (small) change in testing procedure?
Maybe it would be a good thing to have another go on the policy and process of implementing upstream updates. (or link to the procedure, if it already is documented)

4 Likes

Hi,

The “Rolling release” feature for a server is always a risky deal!

Is the reason the Debian project has a so good reputation: Repositories arr “frozen”, and well tested, to offer a rock solid based.

I think a Centos duplication and a time retention to test if all is okay before spread the update is a way to go! :blush:

A bad update from the upstream is rare, but unfortunally, can apply :confused:

2 Likes

Hi All,

If I recall correctly, there was a similiar issue with SME Server a few years back.
The issue had to do with migrating from ISO-8859-1 to UTF8 codepage inside the server itself.
Well, the update changed all referenced passwords.
Even the ones binding a Windows-DomainAccount to the LDAP Server used.
No PC could log in anymore.
A rollback of RPMs didn’t help as the passwords were already changed.
Those with a working backup were thankful, the others had to remove Windows Desktops from the Domain, reboot, re add the PC to the Domain and reboot…
Upstream also had pushed out those unreflected “updates”…

Well, it shouldn’t happen. Then again, as the saying goes: Sh*t happens!
But with a frozen repo, that wouldn’t be a BIG issue anymore!

Thanks to all involved!

Best Regards
Andy

1 Like

As a sysadmin who has all core services on Nethserver, I would like to ALWAYS have to deliberately decide to upgrade to versions that might not have been fully tested or that involve a distribution upgrade.

I’d like a Debian like frozen repo and the regular one, and decide at installation which one to use for scheduled updates, and be able to change it of course :stuck_out_tongue:

Right now I do not dare schedule automatic updates, and the bug that spawned this topic would have hit us hard, so I trade security for peace of mind atm … which is never good.

1 Like

Hi everybody,

If I understand well, for a brand new installation of the NS 7.3.1611, to avoid to be upgraded to CentOS 7.4, we shall do (till the next announcement):

  • install NS 7.3.1611 (ISO)

  • first login from CLI and follow @giacomo procedure from here (first post).

  • proceed to the NS 7.3.1611 configuration and update as is needed

Is this correct?
This procedure will block and avoid the update to CentOS 7.4?
In this mode, we can configure also samba AD without issues (without applied also the other workarounds posted regarding CentOS 7.4 issues)?

TIA,
Gabriel

1 Like

Yes to all questions.

BUT I may found a working fix which doesn’t requires this repository hack, take a look:

2 Likes

Hmm… (sorry about bad english).
I have this problem with CentOS 7.4 after the yesterday update (Solved by now, with forced downgrade). It is strange because in any moment I see a message like “This is a test version” or something like this. Just run yum update (I do regular updates, every week) and the SMB service stop working. To be clear… In any moment I recall to make a explicit choice to use non tested packages. In others linux flavors, there is a HUGE message - Do not use in production machines…
I see that this update broke the smb server and blocked some root filesystems. The version of the packages was beyound below the limit of acceptable to be in any update repository. So, this is a major problem to me. I don´t want to review every package before doing a update in a production server. So… I think, maybe using a strict tested repository or some message before running updates be a good practice.

Take a look at the first post on this topic:

We’re not using “testing” repositories from CentOS, and CentOS guys are very careful about being always stable.
They just didn’t catch a bug :slight_smile:
The proposal is about adding an additional layer between CentOS and us.
@giacomo do you think that it is still on?

No one should need this workaround anymore :wink:

3 Likes