Again with the hoarding of kernels

fasttech · July 10, 2018, 12:41am

Release
System version
NethServer release 7.5.1804 (final)
Kernel release
3.10.0-862.3.3.el7.x86_64

---> Package sssd-proxy.x86_64 0:1.16.0-19.el7_5.5 will be an update
---> Package sudo.x86_64 0:1.8.19p2-13.el7 will be updated
---> Package sudo.x86_64 0:1.8.19p2-14.el7_5 will be an update
--> Finished Dependency Resolution
--> Running transaction check
---> Package kernel.x86_64 0:3.10.0-327.el7 will be erased
---> Package kernel.x86_64 0:3.10.0-514.6.1.el7 will be erased
---> Package kernel.x86_64 0:3.10.0-514.6.2.el7 will be erased
---> Package kernel.x86_64 0:3.10.0-514.10.2.el7 will be erased
---> Package kernel.x86_64 0:3.10.0-514.16.1.el7 will be erased
---> Package kernel.x86_64 0:3.10.0-514.21.1.el7 will be erased
---> Package kernel.x86_64 0:3.10.0-514.21.2.el7 will be erased
---> Package kernel.x86_64 0:3.10.0-514.26.1.el7 will be erased
---> Package kernel.x86_64 0:3.10.0-514.26.2.el7 will be erased
---> Package kernel.x86_64 0:3.10.0-693.2.2.el7 will be erased
---> Package kernel.x86_64 0:3.10.0-693.5.2.el7 will be erased
--> Finished Dependency Resolution

(36/37): kernel-tools-libs-3.10.0-862.6.3.el7.x86_64.rpm                                                                       | 6.2 MB  00:00:22
(37/37): kernel-3.10.0-862.6.3.el7.x86_64.rpm                                                                                  |  46 MB  00:00:19
------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                 2.6 MB/s |  95 MB  00:00:36
Running transaction check
Running transaction test


Transaction check error:
  installing package kernel-3.10.0-862.6.3.el7.x86_64 needs 13MB on the /boot filesystem

Error Summary
-------------
Disk Requirements:
  At least 13MB more space needed on the /boot filesystem.

The gui error is one of yum cache and metadata, so it’s misleading.

m.traeumner · July 10, 2018, 7:14am

You can clean it with:

package-cleanup --oldkernels --count=2

Oh, I’ve seen you’ve posted at this thread, which is linked in mine:

The problem always occurs if updates are installed over softwarecenter. Perhaps
@giacomo has an idea what to do, that also by updating at softwarecenter old kernels will be removed?

giacomo · July 10, 2018, 7:26am

If so, the problem is here nethserver-base/root/usr/libexec/nethserver/pkgaction at master · NethServer/nethserver-base · GitHub

But we need an expert on yum internals to find the issue.
Anyone want to join?

davidep · July 10, 2018, 7:40am

We usually type the shell command

  yum update

But pkgaction actually does

  yum update \*

If somebody can confirm the problem is reproducible by the latter, we’ve found a fix…

EddieA · July 10, 2018, 8:16am

Looking at my yum logs, it looks like the issue with ever increasing kernels was resolved before the update I applied on Mar 9th, where 10 old kernels were removed.

But it looks like the OP is in a catch-22 situation where yum wants to remove 11 kernels, but will only do so after it installs the latest one, which it can’t because of space issues.

So perhaps following the @m.traeumner advice will allow the update to continue and then the issue will be resolved moving forward.

Cheers.

dnutan · July 10, 2018, 8:16am

By command line, both show the same output (packages to install, and to remove).
@EddieA,the updates were handled from the server-manger GUI or by yum-cron? Edit: Said nothing, yum-cron was officially released at a later date.

dnutan · July 10, 2018, 11:39am

# yum update \*    #same result as `yum update`
Loaded plugins: changelog, fastestmirror, nethserver_events
(...)
---> Package kernel.x86_64 0:3.10.0-862.6.3.el7 will be installed
(...)
--> Running transaction check
---> Package kernel.x86_64 0:3.10.0-693.11.6.el7 will be erased
--> Finished Dependency Resolution
(...)
================================================================================
 Package                  Arch   Version               Repository          Size
================================================================================
Installing:
 kernel                   x86_64 3.10.0-862.6.3.el7    updates             46 M
Updating:
 kernel-tools             x86_64 3.10.0-862.6.3.el7    updates            6.3 M
 kernel-tools-libs        x86_64 3.10.0-862.6.3.el7    updates            6.2 M
 ...
Removing:
 kernel                   x86_64 3.10.0-693.11.6.el7   @updates            59 M
Installing for dependencies:
 python-requests-toolbelt noarch 0.8.0-1.el7           epel                77 k

Transaction Summary
================================================================================
Install   1 Package  (+1 Dependent package)
Upgrade  44 Packages
Remove    1 Package

github.com

NethServer/nethserver-base/blob/master/root/usr/share/nethesis/NethServer/Module/PackageManager/Modules/Update.php#L38


      
          {
          
              public $notifications;
          
              public function process()
              {
                  parent::process();
                  if ($this->getRequest()->isMutation()) {
                      $db = $this->getPlatform()->getDatabase('configuration');
                      $this->getPlatform()->exec('/usr/bin/sudo /usr/libexec/nethserver/pkgaction ${1} \*', array('--update'), TRUE);
                  }
              }
          
              private function extractMajorMinor($versionString)
              {
                  $parts = array_slice(explode('.', $versionString, 3), 0, -1);
                  return implode('.', $parts);
              }
          
              private function findDistroUpgrade($updates)
              {

# /usr/libexec/nethserver/pkgaction --update \*

/var/log/yum.log (actions done by pkgaction --update \*)

Jul 10 13:22:49 Updated: libsss_certmap-1.16.0-19.el7_5.5.x86_64
Jul 10 13:22:50 Updated: libsss_idmap-1.16.0-19.el7_5.5.x86_64
Jul 10 13:22:50 Updated: libsss_nss_idmap-1.16.0-19.el7_5.5.x86_64
Jul 10 13:22:55 Updated: 1:NetworkManager-libnm-1.10.2-16.el7_5.x86_64
Jul 10 13:22:56 Updated: 1:NetworkManager-1.10.2-16.el7_5.x86_64
Jul 10 13:22:57 Updated: sudo-1.8.19p2-14.el7_5.x86_64
Jul 10 13:22:57 Updated: nethserver-base-3.4.0-1.ns7.noarch
Jul 10 13:22:57 Updated: sssd-client-1.16.0-19.el7_5.5.x86_64
Jul 10 13:22:57 Updated: sssd-libwbclient-1.16.0-19.el7_5.5.x86_64
Jul 10 13:22:57 Updated: httpd-tools-2.4.6-80.el7.centos.1.x86_64
Jul 10 13:22:59 Updated: httpd-2.4.6-80.el7.centos.1.x86_64
Jul 10 13:22:59 Updated: 1:mod_ssl-2.4.6-80.el7.centos.1.x86_64
Jul 10 13:22:59 Updated: python-sssdconfig-1.16.0-19.el7_5.5.noarch
Jul 10 13:22:59 Updated: libsss_autofs-1.16.0-19.el7_5.5.x86_64
Jul 10 13:22:59 Updated: nethserver-lang-en-1.2.12-1.ns7.noarch
Jul 10 13:23:05 Updated: kernel-tools-libs-3.10.0-862.6.3.el7.x86_64
Jul 10 13:23:05 Updated: libipa_hbac-1.16.0-19.el7_5.5.x86_64
Jul 10 13:23:05 Installed: python-requests-toolbelt-0.8.0-1.el7.noarch
Jul 10 13:23:05 Updated: python2-acme-0.25.1-1.el7.noarch
Jul 10 13:23:05 Updated: python2-certbot-0.25.1-1.el7.noarch
Jul 10 13:23:06 Updated: selinux-policy-3.13.1-192.el7_5.4.noarch
Jul 10 13:23:06 Updated: libsss_sudo-1.16.0-19.el7_5.5.x86_64
Jul 10 13:23:09 Updated: sssd-common-1.16.0-19.el7_5.5.x86_64
Jul 10 13:23:09 Updated: sssd-krb5-common-1.16.0-19.el7_5.5.x86_64
Jul 10 13:23:09 Updated: sssd-common-pac-1.16.0-19.el7_5.5.x86_64
Jul 10 13:23:09 Updated: sssd-ipa-1.16.0-19.el7_5.5.x86_64
Jul 10 13:23:10 Updated: sssd-ad-1.16.0-19.el7_5.5.x86_64
Jul 10 13:23:10 Updated: sssd-ldap-1.16.0-19.el7_5.5.x86_64
Jul 10 13:23:10 Updated: sssd-krb5-1.16.0-19.el7_5.5.x86_64
Jul 10 13:23:10 Updated: sssd-proxy-1.16.0-19.el7_5.5.x86_64
Jul 10 13:23:10 Updated: sssd-1.16.0-19.el7_5.5.x86_64
Jul 10 13:23:13 Updated: nethserver-sssd-1.4.0-1.ns7.noarch
Jul 10 13:23:13 Updated: nethserver-directory-3.3.0-1.ns7.noarch
Jul 10 13:23:21 Updated: selinux-policy-targeted-3.13.1-192.el7_5.4.noarch
Jul 10 13:23:22 Updated: certbot-0.25.1-1.el7.noarch
Jul 10 13:23:23 Updated: kernel-tools-3.10.0-862.6.3.el7.x86_64
Jul 10 13:23:24 Updated: nethserver-httpd-admin-2.3.2-1.ns7.noarch
Jul 10 13:23:24 Updated: nethserver-httpd-3.2.5-1.ns7.noarch
Jul 10 13:23:24 Updated: 1:NetworkManager-tui-1.10.2-16.el7_5.x86_64
Jul 10 13:23:24 Updated: 1:NetworkManager-ppp-1.10.2-16.el7_5.x86_64
Jul 10 13:23:24 Updated: 1:NetworkManager-wifi-1.10.2-16.el7_5.x86_64
Jul 10 13:23:24 Updated: nethserver-lang-es-1.2.12-1.ns7.noarch
Jul 10 13:23:24 Updated: python-perf-3.10.0-862.6.3.el7.x86_64
Jul 10 13:23:25 Updated: binutils-2.27-28.base.el7_5.1.x86_64
Jul 10 13:23:25 Updated: gsettings-desktop-schemas-3.24.1-2.el7_5.x86_64
Jul 10 13:23:45 Installed: kernel-3.10.0-862.6.3.el7.x86_64

# rpm -q kernel
kernel-3.10.0-693.11.6.el7.x86_64
kernel-3.10.0-693.17.1.el7.x86_64
kernel-3.10.0-693.21.1.el7.x86_64
kernel-3.10.0-862.3.2.el7.x86_64
kernel-3.10.0-862.3.3.el7.x86_64
kernel-3.10.0-862.6.3.el7.x86_64

m.traeumner · July 10, 2018, 12:38pm

My systems also shows 5 kernels now.

@fasttech Which version of Nethserver have you installed? Is it your machine with 7.5 beta?

Edit: It was accidentilly that I had 5 kernels. Now I have installed the latest updates and I have 6 kernels again. My System is 7.5 final

EddieA · July 10, 2018, 5:16pm

Mea culpa. I misread the times on the kernel deletes. They were actually done manually, via package-cleanup, immediately following the updates.

Cheers.

fasttech · July 10, 2018, 5:19pm

@m.traeumner the machine version is at the top of the post.

Apologies, I posted this as a heads up because I thought this issue resolved long ago with v6, I had already solved this machine’s issue by removing the kernels after installing yum-utils and then applying the updates via the Software center instead of at terminal.

m.traeumner · July 11, 2018, 7:59am

Sorry, you are right. In Germany we say:

who can read is clearly in the advantage

I think I can’t.

I did this too and only updated with softwarecenter. Now I have 6 kernels again.

giacomo · July 11, 2018, 8:47am

So it seems something really related to the UI … I hate the UI

m.traeumner · July 12, 2018, 7:13am

I think I can’t help finding the cause, but later I can help to test.

EddieA · July 18, 2018, 4:11pm

A little more information here, as today I received the email saying that there are updates ready to apply. At the bottom of that email I noticed this:

Removing:
 kernel                  x86_64 3.10.0-693.11.1.el7 @updates               59 M
 kernel                  x86_64 3.10.0-693.11.6.el7 @updates               59 M

This got me looking back at the previous update, and here’s what I found:

The email stated it had a single kernel to remove:

Removing:
 kernel              x86_64 3.10.0-693.11.1.el7     @updates               59 M

However looking at the yum.log (and the contents of /boot), this action did NOT take place. The updates were applied via the Software Center UI.

As I haven’t applied the latest updates, so far, is there any particular way you would like me to try in an attempt to narrow this down.

Cheers.

dnutan · July 18, 2018, 4:31pm

Thoughts and a recap:

yum and yum-cron are able to remove old kernels.
server-manger GUI does not remove old kernels due to either pkgaction update not implementing a “packages to remove” action, or some quirks in yum’s python-API version.

EddieA · July 19, 2018, 12:36am

I can confirm this after applying the latest set of updates from the command line.

Cheers.

mark_nl · August 15, 2018, 6:35pm

I not a software developer most definitely not a python developer nor a yum specialist;
so the next can be not accurate or even completely wrong, still i think it valuable to try to describe my findings.

Yum update from the command line and yum update from the server-manger (GUI) are completely different beasts.

The yum update from the command line populates a transaction-object which includes the whole dependency resolve result. It does this thought cli.YumBaseCli() by building a transaction object which is run / done
(doc YumBaseCLI)

The server-manager update (amusing we did not select packages to be installed/removed) makes a transaction object solely for the update action.

as i see it the “cli method” in not easy to implement , it can only run one command (install update remove) at once.

Hope it helps a real software-developer

m.traeumner · August 28, 2018, 9:16am

The discussion goes on at

Please have a look there.

giacomo · August 28, 2018, 12:43pm

A post was merged into an existing topic: To much kernels installed, but config gives a limit of 5