DUC reports wrong

v7
(Ralf Jeckel) #1

NethServer Version: 7.4.1804 final
Module: DUC

Hi @edoardo_spadoni , I beg your pardon. I know it’s alway me who’s moaning about DUC, but I’ll report it again, because I think it’s not a good thing to have a feature, that doesn’t do what it should do.
AFAIK DUC is your module.

DUC reports a diskusage of 365 GB:

image
image.jpg561x646 152 KB

du -sh : 215 GB Difference 150 GB

Dashboard reports correctly:
image

The difference comes from the urbackup directory:

image

/var/lib/urbackup/du -sh : 153 GB Difference again 150 GB

It seems. that DUC counts the backups twice and tripple. Maybe because of the way urbackup saves the incremental backups. If the urbackup grows enough, the reported diskusage will even exeed the real size of the disk.

I’m sorry to bother you again with this, but is it possble to solve this behaviour of DUC?

TIA Ralf

(Mark Verlinde) #2

Hi Ralf,

Just out of curiosity which file sizes does the file system report for the the backups? eg ls -l

EDIT: I think i am barking on the wrong tree; this does not seem to be the issue.

Still fun to check: :grinning:
duc ui / -d /var/cache/duc/duc.db
and with apparent size option:
duc ui / -d /var/cache/duc/duc.db -a

(Ralf Jeckel) #3

It’s fun, yes. :smile:
But between -a and without there is only 1 GB in /var/lib/urbackup diffrenece in my case. :thinking:

(Mark Verlinde) #4

duc is a fun utility, maybe we should put it in de doc or readme about the command line usage…

Does the “duc ui” report something strange if you navigate to the /var/lib/urbackup directory with arrow keys?
It does show the size of files too, maybe we can figure out why it reports a wrong value here…

(Ralf Jeckel) #5

I didn’t notice something strange. But I’m at home on my tablet. Will play a bit tomorrow and check if I can figure out something.
Thanks for answers.
Till tomorrow.

(Ralf Jeckel) #6

I think I found it.

I had a look at the code of duc at github an found the -H option for indexing. The manual states that this option tells duc to count hardlinked files only once.

When I use the -H option in nethserver-duc-index the reported disksize shrinks to the value of the one reported by du. (from 386,6 to 216,5 => du and dashboard report the same)

image

What do you think @giacomo ? Can we change this in source-file?

(Mark Verlinde) #7

Great Find! and +1

AFAIK only nethserver-restore-data depends on duc too. It builds it’s own (duc) database/index on another location and would not be effected by this change.

EDIT: some further background info:

(Mark Verlinde) #8

Hi Ralf,

It works for me although used the more descriptive option --check-hard-links to make it more clear what it is about. According to the documentation (cc in the commit) it is the same.

IMHO all con’s and pro’s in mind, it is a usefull enhancement.
Or would it be better to add and db-prop for it and you have the possibility to choose? IF SO which default behavior would we favor?

(Markus Neuberger) #9

As we have community modules like urbackup or backuppc which make use of hardlinks I think this enhancement really makes sense. :+1:

I don’t think that duc will make big problems so I’d prefer to just make the new option to default for keeping it simple.

(Ralf Jeckel) #10

Since it seems to make no diffreence if you have installed no hardlinking software, but it make a bif difference if you have, I’ll vote for making -H default.
It does matter on very large systems, because it makes indexing slower, but I don’t think, that those system are the main porpose for NS.
So, yes I’d make it default.

(Giacomo Sanchietti) #11

I agree, we could add it as default.

@stephdl do you mind to add it to the board? (I will be on vacation next week).

Ralph, if you don’t have news in two weeks, feel free to ping me again! :smile:

(Mark Verlinde) #12

Have a nice week off!

I’ll summit a PR and we can test the auto-build ;
and you have an easy Job to start with after your vacation.

grtz Mark

(Mark Verlinde) #13

@stephdl created a project card which could be converted into an issue

(Ralf Jeckel) #14

Thanks Giacomo. No need to hurry for me, I changed it already manually in my installations. :wink:

Have a nice and recreative week. :beach_umbrella: :champagne:

(Davide Principi) #15

Please do it :wink:

(Mark Verlinde) #16
(Mark Verlinde) #17

Hi Ralf, could you test the package and if it works as expected report this in the raised issue for verification?

yum --enablerepo=nethserver-testing update nethserver-duc

(Ralf Jeckel) #18

Done. Everything seems to work as expected.
Thanks to all.