Restore Data Backup from 6.9F to 7F

migrate
restore
backup
migration

(Jonathan Colby) #1

I did a data backup last night to my NAS from 6.9 Final, and wiped and installed the current V7 Release.

My previous hostname for nethserver did not have .com or .net at the end and the new V7 requres it, and I need help restoring the data backups. I have many individual tar.gz files for each data backup that was created… maybe 18 to 20 for the most recent backup.

Is anyone able to help me ? I am willing to pay some $ for help if needed… My nethserver has been a mail server for about 2 years so i can sync my local devices to one IMAP account as my email providers inbox size constraints are very small… I only had one email account and it was fetching externally my email from my ISP (Cox.net)

Direct Message or continue here if you would like to help me.

Greatest of thanks.
-Jonathan


(Markus Neuberger) #2

Hi @joncolby,

so it’s about reconfiguring the new V7 server and importing the old mails.

There’s an upgrade path using disaster recovery and V6 backup:

http://docs.nethserver.org/en/v7/upgrade.html#upgrade-from-product-6

New install and import method:

You may think about changing your setup away from smarthost and getmail via ISP to direct mail server but generally configuring an imap account and a pop3connector shouldn’t be a problem:

http://docs.nethserver.org/en/v7/mail.html
http://docs.nethserver.org/en/v7/pop3_connector.html

For importing the mails you have to extract your backup somewhere. You may use doveadm to import:

https://wiki.dovecot.org/Tools/Doveadm/Import

Feel free to ask if you need more help…


(Jonathan Colby) #3

the only data i need from the backup is my old email data… which i guess leads me to my next question… since I am only using nethserver to fetch mail for one email account and store it locally so my IMAP devices can stay synchronized, is there any good argument to set up the new v7 nethserver in any particular way ? OpenLDAP vs Active Directory ?

I was going to set it up with the hostname as LocalMail.cox.net so that I can create my cox.net email address on there and have fetchmail drop new messages into that mailbox… basically like I had before.

I tried openLDAP setup last night but it made strange stuff show up on the domain accounts tab :slight_smile:
image


(Markus Neuberger) #4

One point may be to don’t use smarthost and fetchmail (or getmail) anymore and use direct mailing instead to be independent from the ISP but I don’t know if your ISP supports to enter DNS entries. For your case with one account it’s fully ok to keep everything as it was. Here are just some thoughts about smarthosts and what you could do on your own mail server.

If you don’t need to share files I’d recommend LDAP:

http://docs.nethserver.org/en/v7/accounts.html#choosing-the-right-account-provider


(Jonathan Colby) #5

im trying to find the fetchmail log in the new V7 and havent been able to find it yet. any clue where i can find it to diagnose its behavior ?


(Jonathan Colby) #6

…Still fighting getmail…

getmail: getmailOperationError error (POP error (-ERR [AUTH] Password: ))

Tried all 4 choices, pop, pop+ssl, imap, imap+ssl… password is good. is there a way to specify to getmail if it should use secure login or a specific port number ?


(Markus Neuberger) #7

You’ll find log entries in /var/log/maillog.

Example config:

Only via protocol but that should be enough.

I assume your ISP uses standard ports:

http://docs.nethserver.org/en/v7/mail.html#client-configuration


(Jonathan Colby) #8

I think I am narrowing in on my problem… I named my nethserver LM.cox.net

but when i use nethserver diagnostic tab to ping imap.cox.net it pings its self…
so i think that is breaking the function of the getmail… maybe getmail is trying to log onto nethserver to get mail instead of my ISP actual mail server…

What is the right way to make a local someone@cox.net email address on my netsherver that getmail can drop email into without having nethserver route all *.cox.net traffic to its self ?

is there a way to tell nethserver to not point to its self for other .cox.net domains ?
Should I re-install neth and give it a different fqdn ?


(Markus Neuberger) #9

You could disable the imap alias which is responsible for pinging your own server. After that change getmail should fetch mail from the right server.

To disable local MX and aliases, access the root’s console and type:

config setprop postfix MxRecordStatus disabled
signal-event nethserver-hosts-update

http://docs.nethserver.org/en/v7/mail.html#client-configuration

No, you can change FQDN when you unbind your LDAP, change the server name and rebind LDAP. As you just have one user it should be no problem.


(Jonathan Colby) #10

having trouble getting access to the backup files… i cant seem to get the hostname to be the same as it was with 6.9

it requires a .com or .net and i never had that before… before i only hand one dot… superserver.blablabla

any advice to getting the backup extracted so i can begin to import the emails ?

thank you…


(Jonathan Colby) #11

i found how to use duplicity but have no idea what the gnupg passphrase would have been. is there a way to recover this data ?


(Markus Neuberger) #12

Sorry, you have to change the hostname/domainname to have at least two dots now, but it shouldn’t be a problem. You seem to use your providers mailserver so to avoid naming conflicts you should name your Nethserver like LocalMail.internal.cox.net. This way you avoid pinging the own server on imap.cox.net. There’s some discussion about naming as one shouldn’t use “invented” tlds like .local.

I’ll fire up a Nethserver 6.9 VM and reproduce your scenario to find an easy way to migrate your mails.

EDIT:

In my case Nethesis,1234 as passphrase worked. I could also open the backup files with 7-zip under Windows. I couldn’t extract the files with command line`, this is what I tried, I got only folders:

for t in duplicity-full.20180215T115508Z.*.difftar.gz; do tar xvf $t; done

What I did to upgrade from old NS6.9 backup files to a new installed NS7.4:

http://docs.nethserver.org/en/v7/upgrade.html#upgrade-from-backup

  • Install backup from software center
  • Upload and restore from the backup config file - you’ll find the backup-config.tar.xz in the extracted backup under multivol_snapshot/var/lib/nethserver/backup/
  • Now you see some warning todos, we solve them step by step:
  • Set network role and ip of new interface, in my case I gave an IP to ens33
  • Release role of old interface and delete it, in my case eth0
  • Run restore-data on root shell
  • I recognized pop3connector was set to POP3 after restore but on NS6 it was set to IMAP with SSL checked so you may change this to fit your needs (I recommend IMAP with SSL)
  • In my case POP3 without SSL worked too because ISP still supports it.
  • I found my mails restored in /var/lib/nethserver/vmail/markus@domain.local and in roundcube webmail.

(Jonathan Colby) #13

so if that command only gave you folders from your duplicity-full.20180215T115508Z.*.difftar.gz file, what is the right command to use to extract it and all the files ?

on windows with 7 zip, i can only extract each of the .gz files into a seperate folder, and each file has its own difftar file… should i put all the difftar files in one directory and then try to use 7zip to extract them ?

from neth shell i tried 'duplicity restore file:///mnt/restore27/servername /home/username/restorefolder
but keep getting prompted for gnuPG key, and i tried Netheses,1234 as well as all the other admin / root pw’s ive used but i only get a 7.6 mb result… and im starting with 196 GB backup files


(Markus Neuberger) #14

Sorry, my fault, I found it in the first difftar in snapshot dir:

\\backupserver\backupshare\neth69\duplicity-full.20180215T115508Z.vol1.difftar.gz\duplicity-full.20180215T115508Z.vol1.difftar\snapshot\var\lib\nethserver\backup\


(Markus Neuberger) #15

Does somebody have an idea how to restore as much as possible from duplicity backup when a file is missing in the chain?

I extract it:

for t in duplicity-full.20180215T115508Z.*.difftar.gz; do tar xvf $t; done

Then I put the merged deltas to a file called content

find multivol_snapshot/ -type f -printf '%h\0' | sort -uz | xargs -0 -n 1 sh -c 'cd "$1" ; cat $(ls | sort -n) > content' spacer

Then I copy the content files to a new location:

#!/bin/bash
while IFS='' read -r line || [[ -n "$line" ]]; do
    mkdir -p ~/restore1/$(dirname $(dirname $line))
    cp $line ~/restore1/$(dirname $line)
done < "$1"

Now I can copy the files back to where they were:

cp -Rf ~/restore1 /

Is there an easier way???

My sources:

https://wiki.gnome.org/Apps/DejaDup/Help/Restore/WorstCase#Restoring_by_Hand