YUM update repository problems (from Russia)

yum
updates

(fpausp) #1

good morning…

yesterday I did the updates on a few servers, everything went through and looked good but today I get something like this:

[root@infra01 ~]# yum clean all && rm -rf /var/cache/yum && yum -y update

Loaded plugins: changelog, fastestmirror, nethserver_events
Cleaning repos: base centos-sclo-rh centos-sclo-sclo epel extras nethforge nethserver-base nethserver-updates updates
Cleaning up everything
Maybe you want: rm -rf /var/cache/yum, to also free up space taken by orphaned data from disabled or removed repos
Cleaning up list of fastest mirrors
Loaded plugins: changelog, fastestmirror, nethserver_events
Determining fastest mirrors
epel/x86_64/metalink                                                                                                       |  14 kB  00:00:00
 * base: ftp.tugraz.at
 * epel: mirror.miletic.net
 * extras: ftp.tugraz.at
 * nethforge: mrmarkuz.goip.de
 * nethserver-base: mrmarkuz.goip.de
 * nethserver-updates: mrmarkuz.goip.de
 * updates: ftp.tugraz.at
base                                                                                                                       | 3.6 kB  00:00:00
centos-sclo-rh                                                                                                             | 3.0 kB  00:00:00
centos-sclo-sclo                                                                                                           | 2.9 kB  00:00:00
epel                                                                                                                       | 4.7 kB  00:00:00
extras                                                                                                                     | 3.4 kB  00:00:00
nethforge                                                                                                                  | 4.1 kB  00:00:00
nethserver-base                                                                                                            | 2.9 kB  00:00:00
nethserver-updates                                                                                                         | 4.1 kB  00:00:00
updates                                                                                                                    | 3.4 kB  00:00:00
(1/14): base/7/x86_64/group_gz                                                                                             | 166 kB  00:00:00
(2/14): epel/x86_64/group_gz                                                                                               |  84 kB  00:00:00
epel/x86_64/updateinfo         FAILED                                                                           ]  0.0 B/s | 669 kB  --:--:-- ETA
http://mirror.23media.de/epel/7/x86_64/repodata/3ada5bcbccaa9772074a773b15bd01dd48e753a184ebecfed590de6c389ada03-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found
Trying other mirror.
To address this issue please refer to the below wiki article

https://wiki.centos.org/yum-errors

If above article doesn't help to resolve this issue please use https://bugs.centos.org/.

(3/14): centos-sclo-sclo/x86_64/primary_db                                                                                 | 202 kB  00:00:00
(4/14): extras/7/x86_64/primary_db                                                                                         | 104 kB  00:00:00
(5/14): nethforge/7/x86_64/group_gz                                                                                        | 1.7 kB  00:00:00
(6/14): nethserver-base/7/x86_64/primary_db                                                                                | 241 kB  00:00:00
(7/14): nethserver-updates/7/x86_64/group_gz                                                                               |  20 kB  00:00:00
(8/14): epel/x86_64/primary_db                                                                                             | 1.6 MB  00:00:01
(9/14): nethserver-updates/7/x86_64/primary_db                                                                             | 382 kB  00:00:00
(10/14): base/7/x86_64/primary_db                                                                                          | 5.9 MB  00:00:02
(11/14): nethforge/7/x86_64/primary_db                                                                                     |  32 kB  00:00:01
(12/14): updates/7/x86_64/primary_db                                                                                       | 1.0 MB  00:00:01
(13/14): epel/x86_64/updateinfo                                                                                            | 921 kB  00:00:00
(14/14): centos-sclo-rh/x86_64/primary_db                                                                                  | 3.2 MB  00:00:03


 One of the configured repositories failed (Unknown),
 and yum doesn't have enough cached data to continue. At this point the only
 safe thing yum can do is fail. There are a few ways to work "fix" this:

     1. Contact the upstream for the repository and get them to fix the problem.

     2. Reconfigure the baseurl/etc. for the repository, to point to a working
        upstream. This is most often useful if you are using a newer
        distribution release than is supported by the repository (and the
        packages for the previous distribution release still work).

     3. Run the command with the repository temporarily disabled
            yum --disablerepo=<repoid> ...

     4. Disable the repository permanently, so yum won't use it by default. Yum
        will then just ignore the repository until you permanently enable it
        again or use --enablerepo for temporary usage:

            yum-config-manager --disable <repoid>
        or
            subscription-manager repos --disable=<repoid>

     5. Configure the failing repository to be skipped, if it is unavailable.
        Note that yum will try to contact the repo. when it runs most commands,
        so will have to try and fail each time (and thus. yum will be be much
        slower). If it is a very temporary problem though, this is often a nice
        compromise:

            yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true

file is encrypted or is not a database

How can I be sure that everything will be ok with the server?

Is there a command to make a self-test?


The REMOVE button is not active (Nextcloud)
Update to 7.5 problem(s)?
Server does not open in browser
(Евгений Копьев) #2

Yesterday I upgraded 2 production servers on bare metal. Now I think what to do now, usually the updates ns does not lead to such sad consequences. :disappointed_relieved:

# yum clean all && rm -rf /var/cache/yum && yum -y update
Loaded plugins: changelog, fastestmirror, nethserver_events
Cleaning repos: base centos-sclo-rh centos-sclo-sclo collaboraoffice.com_repos_CollaboraOnline_CODE-centos7 epel extras
              : nethforge nethserver-base nethserver-updates remi-php56 remi-safe updates veeam
Cleaning up everything
Maybe you want: rm -rf /var/cache/yum, to also free up space taken by orphaned data from disabled or removed repos
Cleaning up list of fastest mirrors
Loaded plugins: changelog, fastestmirror, nethserver_events
Determining fastest mirrors
epel/x86_64/metalink                                                                              |  16 kB  00:00:00
Could not retrieve mirrorlist http://mirrorlist.nethserver.org/7/nethserver?release=7&repo=nethforge&arch=x86_64 error was
12: Timeout on http://mirrorlist.nethserver.org/7/nethserver?release=7&repo=nethforge&arch=x86_64: (28, 'Connection timed out after 30001 milliseconds')


 One of the configured repositories failed (Unknown),
 and yum doesn't have enough cached data to continue. At this point the only
 safe thing yum can do is fail. There are a few ways to work "fix" this:

     1. Contact the upstream for the repository and get them to fix the problem.

     2. Reconfigure the baseurl/etc. for the repository, to point to a working
        upstream. This is most often useful if you are using a newer
        distribution release than is supported by the repository (and the
        packages for the previous distribution release still work).

     3. Run the command with the repository temporarily disabled
            yum --disablerepo=<repoid> ...

     4. Disable the repository permanently, so yum won't use it by default. Yum
        will then just ignore the repository until you permanently enable it
        again or use --enablerepo for temporary usage:

            yum-config-manager --disable <repoid>
        or
            subscription-manager repos --disable=<repoid>

     5. Configure the failing repository to be skipped, if it is unavailable.
        Note that yum will try to contact the repo. when it runs most commands,
        so will have to try and fail each time (and thus. yum will be be much
        slower). If it is a very temporary problem though, this is often a nice
        compromise:

            yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true

Cannot find a valid baseurl for repo: nethforge/7/x86_64
You have new mail in /var/spool/mail/root

Any idea how to fix this now? I have 1 server on distance 900 km. Jet in great need of repair…


(Davide Principi) #3

I cannot reproduce it here:

[root@vm5 ~]# curl 'http://mirrorlist.nethserver.org/7/nethserver?release=7&repo=nethforge&arch=x86_64'
http://mirror.nethserver.org/nethserver/7/nethforge/x86_64/
http://mirror1.nethserver.org/nethserver/7/nethforge/x86_64/
http://mirror2.nethserver.org/nethserver/7/nethforge/x86_64/
https://mirror.nethesis.it/nethserver/7/nethforge/x86_64/
http://nethserver.de-labrusse.fr/7/nethforge/x86_64/
http://mirror.framassa.org/nethserver/7/nethforge/x86_64/
http://mirror.nordest.systems/nethserver/7/nethforge/x86_64/
https://mirror.alpix.eu/nethserver/7/nethforge/x86_64/
http://mrmarkuz.goip.de/mirror/nethserver/7/nethforge/x86_64/
http://mirror.pcxlan.es/nethserver/7/nethforge/x86_64/
https://markusneuberger.at/mirror/nethserver/7/nethforge/x86_64/
http://buck.goip.de/nethserver/7/nethforge/x86_64/
http://nethserver.globalcortex.net/mirror/nethserver/7/nethforge/x86_64/
[root@vm5 ~]# 

Do you have any firewall / http proxy / network settings that can block the HTTP request?


(Davide Principi) #4

EPEL failures are quite common and are due to mirror synchronizations. The YUM client should fall back to another mirror and solve the problem. I can’t figure out why it failed. Did you try again?


(Евгений Копьев) #5

All this immediately after upgrading to the new version. Tried several times. On both servers the same problem.


(Davide Principi) #6

Please attach the output of

curl -v 'http://mirrorlist.nethserver.org/7/nethserver?release=7&repo=nethforge&arch=x86_64'

(Евгений Копьев) #7
# curl -v 'http://mirrorlist.nethserver.org/7/nethserver?release=7&repo=nethforge&arch=x86_64'
* About to connect() to mirrorlist.nethserver.org port 80 (#0)
*   Trying 159.89.233.254...

(fpausp) #8
[root@infra01 ~]# curl -v 'http://mirrorlist.nethserver.org/7/nethserver?release=7&repo=nethforge&arch=x86_64'

* About to connect() to mirrorlist.nethserver.org port 80 (#0)
*   Trying 159.89.233.254...
* Connected to mirrorlist.nethserver.org (159.89.233.254) port 80 (#0)
> GET /7/nethserver?release=7&repo=nethforge&arch=x86_64 HTTP/1.1
> User-Agent: curl/7.29.0
> Host: mirrorlist.nethserver.org
> Accept: */*
>
< HTTP/1.1 200 OK
< Date: Fri, 11 May 2018 09:50:31 GMT
< Server: Apache/2.4.6 (CentOS) PHP/5.4.16
< X-Powered-By: PHP/5.4.16
< Content-Length: 773
< Content-Type: text/plain; charset=UTF-8
<
http://mirror.nethserver.org/nethserver/7/nethforge/x86_64/
http://mirror1.nethserver.org/nethserver/7/nethforge/x86_64/
http://mirror2.nethserver.org/nethserver/7/nethforge/x86_64/
https://mirror.nethesis.it/nethserver/7/nethforge/x86_64/
http://nethserver.de-labrusse.fr/7/nethforge/x86_64/
http://mirror.framassa.org/nethserver/7/nethforge/x86_64/
http://mirror.nordest.systems/nethserver/7/nethforge/x86_64/
https://mirror.alpix.eu/nethserver/7/nethforge/x86_64/
http://mrmarkuz.goip.de/mirror/nethserver/7/nethforge/x86_64/
http://mirror.pcxlan.es/nethserver/7/nethforge/x86_64/
https://markusneuberger.at/mirror/nethserver/7/nethforge/x86_64/
http://buck.goip.de/nethserver/7/nethforge/x86_64/
http://nethserver.globalcortex.net/mirror/nethserver/7/nethforge/x86_64/
* Connection #0 to host mirrorlist.nethserver.org left intact

(Евгений Копьев) #9

curl -v ’ http://mirrorlist.nethserver.org/7/nethserver?release=7&repo=nethforge&arch=x86_64

  • Protocol http not supported or disabled in libcurl
  • Closing connection -1
    curl: (1) Protocol http not supported or disabled in libcurl

(Davide Principi) #10

Thank you @fausp, it seems @kelevra’s issue is on his side!

Did you install DPI module? Can you reach/ping other sites?

 ping 8.8.8.8
 curl google.com

(Евгений Копьев) #11

PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.

64 bytes from 8.8.8.8: icmp_seq=7 ttl=56 time=1.93 ms

64 bytes from 8.8.8.8: icmp_seq=8 ttl=56 time=1.83 ms
64 bytes from 8.8.8.8: icmp_seq=9 ttl=56 time=2.60 ms
64 bytes from 8.8.8.8: icmp_seq=10 ttl=56 time=1.99 ms
64 bytes from 8.8.8.8: icmp_seq=11 ttl=56 time=2.63 ms
64 bytes from 8.8.8.8: icmp_seq=12 ttl=56 time=2.89 ms
64 bytes from 8.8.8.8: icmp_seq=13 ttl=56 time=2.70 ms
64 bytes from 8.8.8.8: icmp_seq=14 ttl=56 time=2.65 ms
64 bytes from 8.8.8.8: icmp_seq=15 ttl=56 time=2.80 ms
^C
— 8.8.8.8 ping statistics —
15 packets transmitted, 9 received, 40% packet loss, time 14013ms
rtt min/avg/max/mdev = 1.833/2.449/2.893/0.390 ms
12:54:04 root@es jobs:0 cmd:891 /root

curl google.com

301 Moved

301 Moved

The document has moved here. 12:54:15 root@es jobs:0 cmd:892 /root

Deep Packet Inspection (DPI) module is not available
Restart the system and select a Linux kernel with DPI module support


(Saito Benkei) #12

Try:

traceroute 159.89.233.254

I have a suspect…


(Евгений Копьев) #13

1 server

traceroute 159.89.233.254

traceroute to 159.89.233.254 (159.89.233.254), 30 hops max, 60 byte packets
1 89-106-172-125.in-addr.mastertelecom.ru (89.106.172.125) 10.269 ms 10.888 ms 11.521 ms
2 89-106-169-230.in-addr.mastertelecom.ru (89.106.169.230) 4.032 ms 4.373 ms 4.709 ms
3 217-67-176-250.in-addr.mastertelecom.ru (217.67.176.250) 8.111 ms 8.535 ms 8.961 ms
4 * * *
5 109-73-43-254.in-addr.mastertelecom.ru (109.73.43.254) 0.573 ms 0.571 ms 0.563 ms

yum update

Loaded plugins: changelog, fastestmirror, nethserver_events
Loading mirror speeds from cached hostfile
Could not retrieve mirrorlist http://mirrorlist.nethserver.org/7/nethserver?rele ase=7&repo=nethforge&arch=x86_64 error was
14: curl#7 - "Failed connect to mirrorlist.nethserver.org:80; Connection refused "
Could not retrieve mirrorlist http://mirrorlist.nethserver.org/?release=7&repo=b ase&arch=x86_64&nsrelease=$nsrelease error was
14: curl#7 - "Failed connect to mirrorlist.nethserver.org:80; Connection refused "
Could not retrieve mirrorlist http://mirrorlist.nethserver.org/?release=7&repo=u pdates&arch=x86_64&nsrelease=$nsrelease error was
14: curl#7 - "Failed connect to mirrorlist.nethserver.org:80; Connection refused "

One of the configured repositories failed (Unknown),
and yum doesn’t have enough cached data to continue. At this point the only
safe thing yum can do is fail. There are a few ways to work “fix” this:

 1. Contact the upstream for the repository and get them to fix the problem.

 2. Reconfigure the baseurl/etc. for the repository, to point to a working
    upstream. This is most often useful if you are using a newer
    distribution release than is supported by the repository (and the
    packages for the previous distribution release still work).

 3. Run the command with the repository temporarily disabled
        yum --disablerepo=<repoid> ...

 4. Disable the repository permanently, so yum won't use it by default. Yum
    will then just ignore the repository until you permanently enable it
    again or use --enablerepo for temporary usage:

        yum-config-manager --disable <repoid>
    or
        subscription-manager repos --disable=<repoid>

 5. Configure the failing repository to be skipped, if it is unavailable.
    Note that yum will try to contact the repo. when it runs most commands,
    so will have to try and fail each time (and thus. yum will be be much
    slower). If it is a very temporary problem though, this is often a nice
    compromise:

        yum-config-manager --save --setopt=<repoid>.skip_if_unavailable=true

file is encrypted or is not a database

2 server
traceroute 159.89.233.254
traceroute to 159.89.233.254 (159.89.233.254), 30 hops max, 60 byte packets
1 * * *
2 * * *
3 * * *
4 * * *
5 * * *
6 * * *
7 * * *
8 * * *
9 * * *
10 * * *
11 * * *
12 * * *
13 * * *
14 * * *
15 * * *
16 * * *
17 * * *
18 * * *
19 * * *
20 * * *
21 * * *
22 * * *
23 * * *
24 * * *
25 * * *
26 * * *
27 * * *
28 * * *
29 * * *
30 * * *


(Saito Benkei) #14

Maybe this?

https://www.google.com/search?q=digitalocean+blocked+in+russia

https://www.google.com/search?q=telegram+ip+blocked+russia


(Евгений Копьев) #15

Both servers are in Russia, I’m in Russia. Until yesterday’s update, everything worked fine.


(fpausp) #16

It looks like they just have been syncronized now?

[root@neth13 ~]# yum clean all && rm -rf /var/cache/yum && yum -y update

Loaded plugins: changelog, fastestmirror, nethserver_events
Cleaning repos: base centos-sclo-rh centos-sclo-sclo epel extras nethforge nethserver-base nethserver-updates updates
Cleaning up everything
Maybe you want: rm -rf /var/cache/yum, to also free up space taken by orphaned data from disabled or removed repos
Cleaning up list of fastest mirrors
Loaded plugins: changelog, fastestmirror, nethserver_events
Determining fastest mirrors
epel/x86_64/metalink                                                                                           | 9.6 kB  00:00:00
 * base: mirror.nextlayer.at
 * epel: mirror.karneval.cz
 * extras: mirror.nextlayer.at
 * nethforge: mirror.nordest.systems
 * nethserver-base: mirror.nordest.systems
 * nethserver-updates: mirror.nordest.systems
 * updates: mirror.nextlayer.at
base                                                                                                           | 3.6 kB  00:00:00
centos-sclo-rh                                                                                                 | 3.0 kB  00:00:00
centos-sclo-sclo                                                                                               | 2.9 kB  00:00:00
epel                                                                                                           | 4.7 kB  00:00:00
extras                                                                                                         | 3.4 kB  00:00:00
nethforge                                                                                                      | 4.1 kB  00:00:00
nethserver-base                                                                                                | 2.9 kB  00:00:00
nethserver-updates                                                                                             | 4.1 kB  00:00:00
updates                                                                                                        | 3.4 kB  00:00:00
(1/14): base/7/x86_64/group_gz                                                                                 | 166 kB  00:00:00
(2/14): epel/x86_64/group_gz                                                                                   |  84 kB  00:00:00
(3/14): centos-sclo-sclo/x86_64/primary_db                                                                     | 202 kB  00:00:00
(4/14): extras/7/x86_64/primary_db                                                                             | 104 kB  00:00:00
(5/14): nethforge/7/x86_64/group_gz                                                                            | 1.7 kB  00:00:00
(6/14): nethforge/7/x86_64/primary_db                                                                          |  32 kB  00:00:00
(7/14): nethserver-base/7/x86_64/primary_db                                                                    | 241 kB  00:00:00
(8/14): nethserver-updates/7/x86_64/group_gz                                                                   |  20 kB  00:00:00
(9/14): epel/x86_64/updateinfo                                                                                 | 921 kB  00:00:02
(10/14): nethserver-updates/7/x86_64/primary_db                                                                | 382 kB  00:00:00
(11/14): updates/7/x86_64/primary_db                                                                           | 1.0 MB  00:00:01
(12/14): centos-sclo-rh/x86_64/primary_db                                                                      | 3.2 MB  00:00:06
(13/14): epel/x86_64/primary_db                                                                                | 6.4 MB  00:00:07
(14/14): base/7/x86_64/primary_db                                                                              | 5.9 MB  00:00:13
No packages marked for update

(Davide Principi) #17

@saitobenkei is probably right! Yesterday we moved the hosting of mirrorlist.nethserver.org to another IP…

See Relocated packages.nethserver.org (2)

[root@vm5 ~]# host mirrorlist.nethserver.org
mirrorlist.nethserver.org is an alias for packages.nethserver.org.
packages.nethserver.org has address 159.89.233.254

Please try this workaround:

  • Check if IP 46.101.114.111 is reachable

  • See if this command gives back the mirror list:

    curl -v -H 'Host: mirrorlist.nethserver.org' 'http://packages.nethesis.it/7/nethserver?release=7&repo=nethforge&arch=x86_64'
    
  • Go to DNS page and set an override for mirrorlist.nethserver.org pointing to 46.101.114.111

You get the list of mirrors, but you can’t access nethserver-testing any more :frowning:


(fpausp) #18

OK, can we say everything is green now? - :thinking:


(Davide Principi) #19

Not for our Russian friends :roll_eyes:


(fpausp) #20

OK, :zipper_mouth_face: