Nextcloud: "Cannot sync due to invalid modification time"

danb35 · January 11, 2025, 2:07pm

NethServer Version: 8.current (core 3.4.0)
Module: Nextcloud 1.2.1 (28.0.14)

Since migrating from NS7 to NS8, I’m seeing clients failing to sync many, but not all, of my files (the ones I’ve noticed being mainly photos) with a message of “Cannot sync due to invalid modification time.” A representative sample of the files with issues shows that they do indeed have incorrect modification times:

As they’re all dated May 2006, the modification time should presumably be about 18 years ago, not 24 (for the files that are failing to sync) or 4.

Some searching suggests that this has been a known issue with Nextcloud for at least the last three years, but apparently they figure it’s sufficient to document how to repair the problem rather than to actually fix the bug:

In principle, seems simple enough: download solvable_files.sh. Run it, specifying the path to the data folder, database type, database host, user, password, and name, and the action to peform. Profit!

On NS7, I would have been comfortable doing this. But in the containerized environment (and, specifically, the podman-containerized environment) of NS8, not so much. How would I go about this fix here?

Andy_Wismer · January 11, 2025, 5:24pm

Hi @danb35

I’m not a container/docker Guru, but this should help:

Make a backup first!

Log in with SSH or locally to your NS8 base host as root.

Enter the environment of the container used (Check the number at the end!)

runagent -m nextcloud1

Use curl or wget to download the script. Make excutable when downloaded, or run with bash first.

Then follow the instructions from the script.

You can restart the container, but simpler would be to restart your host!

Good Luck!

Hope this helps!

My 2 cents
Andy

danb35 · January 11, 2025, 5:43pm

“The remaining steps are left as an exercise for the student.”

It seems straightforward enough, except that the script needs access to the data directory, and the nextcloud1 user doesn’t have that–the data directory is owned by 755441:755441 and has permissions of 750. As a result, the script fails with:

[nextcloud1@ns8 .config]$ ./solvable_files.sh /home/nextcloud1/.local/share/containers/storage/volumes/nextcloud-app-data/_data/data mysql localhost nextcloud nextcloud nextcloud list
find: ‘/home/nextcloud1/.local/share/containers/storage/volumes/nextcloud-app-data/_data/data’: Permission denied

(and yes, nextcloud 3x is correct–the first is the database username, the second is the database password, and the third is the database name).

Andy_Wismer · January 11, 2025, 5:49pm

@danb35

chown & chmod (with -R ) become best friends.
Change, run script, change back as before.
Make a note of whatever directories are used by the script - and if they need additional, special permissions to run there.

Should be readable from the script.

Good Luck!

My 2 cents
Andy

danb35 · January 11, 2025, 6:21pm

I’m not so sure about that:

But based on a later post in that topic, I was able to put together runagent -m nextcloud1 podman exec -ti nextcloud-app sh to get me into a shell in the container. wget the script, chmod +x it, and I’m set, right?

Nope, because the script depends on /bin/bash, which doesn’t exist inside the container. And the script depends on bash-specific syntax, so I can’t just edit the #! line to #!/bin/sh.

Andy_Wismer · January 11, 2025, 6:27pm

Maybe @mrmarkuz or @davidep has an idea…

mrmarkuz · January 11, 2025, 7:01pm

I tested installing bash and findutils in the container and at least the script ran without errors.

Enter nextcloud env:

runagent -m nextcloud1

Enter nextcloud-app container:

podman exec -ti nextcloud-app sh

Install bash and findutils which is needed because the busybox find doesn’t support the nextermt option used in the script, see also Not able to run newermt find flag on 13.1.1 (#12639) · Issues · alpine / aports · GitLab

apk add bash findutils

Exit container:

exit

Enter again using bash:

podman exec -ti nextcloud-app bash

Download script:

wget https://raw.githubusercontent.com/nextcloud-gmbh/mtime_fixer_tool_kit/master/solvable_files.sh

File needs to be executable:

chmod +x solvable_files.sh

I needed to use the absolute path of data in the container:

./solvable_files.sh /var/www/html/data mysql localhost nextcloud nextcloud nextcloud list

danb35 · January 11, 2025, 7:12pm

Following these instructions, I can confirm the script runs without errors. Unfortunately, it doesn’t appear to actually work. The list command returns nothing, when in fact there are over 900 files with reported “invalid modification time.”

…and as I look at the script, I think I see why–in order to clean things up, the script needs to access the database using the mysql command, which also isn’t present in the container. I guess I could try installing that as well (apk add mariadb-client?), but I’m kind of hesitant about installing random other software inside the container. Or will that all be gone when the next update comes out?

mrmarkuz · January 11, 2025, 7:29pm

It will be gone with the next restart of the service.

Yes, that could work, see also MySQL - Alpine Linux

danb35 · January 11, 2025, 7:55pm

OK, with that added, I can run the query confirming I’m affected:

nextcloud:/var/www/html# mysql nextcloud -u nextcloud -p
Enter password:
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A

Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 14306
Server version: 10.6.20-MariaDB-ubu2004 mariadb.org binary distribution

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [nextcloud]> SELECT COUNT(*) FROM oc_filecache WHERE mtime < 86400;
+----------+
| COUNT(*) |
+----------+
|      904 |
+----------+
1 row in set (0.003 sec)

But the script still returns nothing:

nextcloud:/var/www/html# ./solvable_files.sh /var/www/html/data mysql localhost nextcloud nextcloud nextcloud list scan use_birthday verbose
nextcloud:/var/www/html#

Let’s see if adding set +x to the script tells us anything. Nope, not a thing.

OK, digging further into the script, it starts with a find command. Let’s try that from the shell:

nextcloud:/var/www/html/data# find .  -type f ! -newermt "@86400"

…and it returns nothing. Curious. If I understand the find manpage correctly, that should find any files with a modification time not newer than (i.e., at or older than) 2 Jan 1970 at 0000Z. And I know there are 904 files for which the database thinks that’s the case.

Well, let’s take a look at some of the files. A bit of poking around to figure out which UUID corresponds to my account leads to this:

nextcloud:/var/www/html/data/9ea222d8-a9c4-1037-923c-01a044b2ff54/files/Photos/2006/05# ls -lh
total 15M
-rw-r--r--    1 www-data www-data    1.2M Jan 19  2038 06-05-16 03-09-40 E0F4.jpg
-rw-r--r--    1 www-data www-data    1.2M Jan 19  2038 06-05-16 03-09-46 60AB.jpg
-rw-r--r--    1 www-data www-data    1.2M Jan 19  2038 06-05-16 03-52-31 93C1.jpg
-rw-r--r--    1 www-data www-data    1.1M Jan 19  2038 06-05-16 03-53-24 948C.jpg
-rw-r--r--    1 www-data www-data    1.1M Jan 19  2038 06-05-16 03-54-18 9487.jpg
-rw-r--r--    1 www-data www-data    1.2M Mar 28  2021 06-05-16 03-54-54 2F48.jpg
-rw-r--r--    1 www-data www-data    1.1M Jan 19  2038 06-05-16 03-57-34 D66E.jpg
-rw-r--r--    1 www-data www-data    1.1M Aug 24  2020 06-05-16 03-57-53 E513.jpg
-rw-r--r--    1 www-data www-data    1.1M Aug 23  2020 06-05-16 03-58-18 816A.jpg
-rw-r--r--    1 www-data www-data    1.1M Mar 28  2021 06-05-16 03-58-31 CF77.jpg
-rw-r--r--    1 www-data www-data    1.1M Aug 23  2020 06-05-16 04-01-19 7C37.jpg
-rw-r--r--    1 www-data www-data    1.1M Aug 23  2020 06-05-16 04-01-30 5650.jpg
-rw-r--r--    1 www-data www-data    1.2M Aug 23  2020 06-05-16 04-01-38 A441.jpg

Well, the mod date on those files isn’t 1970; it’s 2038. And since I don’t have a time machine, that isn’t very plausible. But unless this is some kind of rollover, it doesn’t represent “before 86400” and thus isn’t found by the find command.

mrmarkuz · January 11, 2025, 8:40pm

That’s strange, maybe it’s another issue.

To test the script I set a file to an old date:

nextcloud:/var/www/html# touch -mt 197001011000 data/admin/files/Readme.md

I needed to use 127.0.0.1 instead of localhost to make the script run without errors:

nextcloud:/var/www/html# ./solvable_files.sh /var/www/html/data mysql 127.0.0.1 nextcloud nextcloud nextcloud list scan use_birthday verbose
mtime in database do not match fs mtime (fs: 36000, db: 1733407628). Skipping /var/www/html/data/admin/files/Readme.md

danb35 · January 11, 2025, 9:17pm

Maybe, but it has to be one that’s closely related. The database lists 904 files with mtime in the first 24 hours of 1970. And if I look for files with mtime after 2030, I get the same number:

nextcloud:/var/www/html/data# find .  -type f -newermt "@1895082552" | wc -l
904

I don’t think that can be a coincidence.

I suppose I could do something like find . -type f -newermt "@1895082552" -exec touch -mt 197001011000 to force the script to pick up those files, but they’d still fail to match mtime in the database and it looks like the script would then skip them.

So if what’s in the database doesn’t match what’s on disk, what to do? How about a files scan? runagent -m nextcloud1 occ files:scan --all runs without apparent issues. Once it does, SELECT COUNT(*) FROM oc_filecache WHERE mtime < 86400; returns 0. And as if by magic, the 904 troublesome files now sync without complaint. They do still have implausible dates, though:

I don’t think I’m thrilled with that result, but all files are now syncing to all devices.

And now I’m even more certain that that date reflects some kind of rollover. That time (in EST, my local time) translates to 2147483640, or 0x7FFFFFF8, or binary 1111111111111111111111111111000. Which seems awfully close to what -1 would work out to.

mrmarkuz · January 11, 2025, 10:53pm

Maybe just cosmetic but you could change the wrong dates to a more normal date like 2024-01-01

find .  -type f -newermt "2038-01-01" -exec touch -mt "202401010000" {} +

and rerun the files scan which should apply the changed dates to the database.

Maybe try with just one file first.

danb35 · January 12, 2025, 9:23pm

I think what the script does if you don’t use the use_birthday flag is just touch with the current time, so I don’t suppose this would be all that different.

mrmarkuz · January 12, 2025, 9:49pm

That’s right.

Relevant code part of use_birthday

                if [ "$use_birthday" == "use_birthday" ]                                                                                                                                                                     
                then                                                                                                                                                                                                         
                        newdate=$(stat -c "%w" "$filepath")                                                                                                                                                                  
                                                                                                                                                                                                                             
                        if [ "$newdate" == "-" ]                                                                                                                                                                             
                        then                                                                                                                                                                                                 
                                if [ "$verbose" == "verbose" ]                                                                                                                                                               
                                then                                                                                                                                                                                         
                                        echo "$filepath has no birthday. Using change date."                                                                                                                                 
                                fi                                                                                                                                                                                           
                                                                                                                                                                                                                             
                                newdate=$(stat -c "%z" "$filepath")                                                                                                                                                          
                        fi                                                                                                                                                                                                   
                                                                                                                                                                                                                             
                        touch -c -d "$newdate" "$filepath"                                                                                                                                                                   
                else                                                                                                                                                                                                         
                        touch -c "$filepath"                                                                                                                                                                                 
                fi

The script won’t change anything if the mtime is not the same for file and db.

There’s an explanation of what the script does at line 26 of the script:

# 1. Return if fs mtime <= 86400                                                                                                                                                                                             
# 2. Compute username from filepath                                                                                                                                                                                          
# 3. Query mtime from the database with the filename and the username                                                                                                                                                        
# 4. Return if mtime_on_fs != mtime_in_db                                                                                                                                                                                    
# 5. Correct the fs mtime with touch (optionally using the files change date/timestamp)

EDIT:

By using the following command in the nextcloud1 environment, you can edit the script easily:

podman unshare nano $(podman volume inspect nextcloud-app-data --format={{.Mountpoint}})/solvable_files.sh