I think we need to open a github issue 
1 Like
good idea
1 Like
seems better
/var/lib/sftpgo # ls -la /srv/sftpgo/data/
total 12
drwxr-xr-x 5 root root 76 Jan 15 21:35 .
drwxr-xr-x 1 sftpgo sftpgo 33 Nov 28 06:10 ..
-rw-r--r-- 1 root root 497 Nov 26 17:22 50x.html
drwxrwsr-x 4 root 992 145 Jan 15 20:58 9001
drwxr-xr-x 18 bin 9002 4096 Jan 15 21:10 9002
drwxr-xr-x 2 bin 9003 6 Jan 15 21:35 9003
-rw-r--r-- 1 root root 615 Nov 26 17:22 index.html
1 Like
oneliner powa
this brings me back few years ago when I could play with perl
1 Like
I tested it using
api-cli run update-module --data '{"module_url":"ghcr.io/nethserver/webserver:sdl-chown-sftpgo-user","instances":["webserver1"],"force":true}'
and it works as expected. On new vhosts
- Joomla installation is working
- Rsync over SSH is working
1 Like
I did nothing gor this. Sis you apllied your fix manually ?
1 Like
It’s a nice side effect. The owner bin:root also works for rsync over SSH.
See also Rsync to virtual host? - #14 by mrmarkuz
EDIT:
Oops, one still needs to adapt the systemd service to make rsync work as explained here: Rsync to virtual host? - #3 by mrmarkuz
1 Like
I tested it and it works as expected:
- cron is started
- root crontab is available at
./crontabs - a test cronjob is working
Nice, asking myself does rsync is installed inside sftpgo, maybe we should rebuild over the container and add rsync, wget or any tools we need
1 Like
yes they are
/var/lib/sftpgo # rsync
rsync version 3.2.7 protocol version 31
Copyright (C) 1996-2022 by Andrew Tridgell, Wayne Davison, and others.
Web site: https://rsync.samba.org/
Capabilities:
64-bit files, 64-bit inums, 64-bit timestamps, 64-bit long ints,
socketpairs, symlinks, symtimes, hardlinks, hardlink-specials,
hardlink-symlinks, IPv6, atimes, batchfiles, inplace, append, ACLs,
xattrs, optional secluded-args, iconv, prealloc, stop-at, no crtimes
Optimizations:
no SIMD-roll, no asm-roll, no openssl-crypto, asm-MD5
Checksum list:
xxh128 xxh3 xxh64 (xxhash) md5 md4 none
Compress list:
zstd lz4 zlibx zlib none
Daemon auth list:
md5 md4
rsync comes with ABSOLUTELY NO WARRANTY. This is free software, and you
are welcome to redistribute it under certain conditions. See the GNU
General Public Licence for details.
rsync is a file transfer program capable of efficient remote update
via a fast differencing algorithm.
Usage: rsync [OPTION]... SRC [SRC]... DEST
or rsync [OPTION]... SRC [SRC]... [USER@]HOST:DEST
or rsync [OPTION]... SRC [SRC]... [USER@]HOST::DEST
or rsync [OPTION]... SRC [SRC]... rsync://[USER@]HOST[:PORT]/DEST
or rsync [OPTION]... [USER@]HOST:SRC [DEST]
or rsync [OPTION]... [USER@]HOST::SRC [DEST]
or rsync [OPTION]... rsync://[USER@]HOST[:PORT]/SRC [DEST]
The ':' usages connect via remote shell, while '::' & 'rsync://' usages connect
to an rsync daemon, and require SRC or DEST to start with a module name.
Options
--verbose, -v increase verbosity
--info=FLAGS fine-grained informational verbosity
--debug=FLAGS fine-grained debug verbosity
--stderr=e|a|c change stderr output mode (default: errors)
--quiet, -q suppress non-error messages
--no-motd suppress daemon-mode MOTD
--checksum, -c skip based on checksum, not mod-time & size
--archive, -a archive mode is -rlptgoD (no -A,-X,-U,-N,-H)
--no-OPTION turn off an implied OPTION (e.g. --no-D)
--recursive, -r recurse into directories
--relative, -R use relative path names
--no-implied-dirs don't send implied dirs with --relative
--backup, -b make backups (see --suffix & --backup-dir)
--backup-dir=DIR make backups into hierarchy based in DIR
--suffix=SUFFIX backup suffix (default ~ w/o --backup-dir)
--update, -u skip files that are newer on the receiver
--inplace update destination files in-place
--append append data onto shorter files
--append-verify --append w/old data in file checksum
--dirs, -d transfer directories without recursing
--old-dirs, --old-d works like --dirs when talking to old rsync
--mkpath create destination's missing path components
--links, -l copy symlinks as symlinks
--copy-links, -L transform symlink into referent file/dir
--copy-unsafe-links only "unsafe" symlinks are transformed
--safe-links ignore symlinks that point outside the tree
--munge-links munge symlinks to make them safe & unusable
--copy-dirlinks, -k transform symlink to dir into referent dir
--keep-dirlinks, -K treat symlinked dir on receiver as dir
--hard-links, -H preserve hard links
--perms, -p preserve permissions
--executability, -E preserve executability
--chmod=CHMOD affect file and/or directory permissions
--acls, -A preserve ACLs (implies --perms)
--xattrs, -X preserve extended attributes
--owner, -o preserve owner (super-user only)
--group, -g preserve group
--devices preserve device files (super-user only)
--copy-devices copy device contents as a regular file
--write-devices write to devices as files (implies --inplace)
--specials preserve special files
-D same as --devices --specials
--times, -t preserve modification times
--atimes, -U preserve access (use) times
--open-noatime avoid changing the atime on opened files
--crtimes, -N preserve create times (newness)
--omit-dir-times, -O omit directories from --times
--omit-link-times, -J omit symlinks from --times
--super receiver attempts super-user activities
--fake-super store/recover privileged attrs using xattrs
--sparse, -S turn sequences of nulls into sparse blocks
--preallocate allocate dest files before writing them
--dry-run, -n perform a trial run with no changes made
--whole-file, -W copy files whole (w/o delta-xfer algorithm)
--checksum-choice=STR choose the checksum algorithm (aka --cc)
--one-file-system, -x don't cross filesystem boundaries
--block-size=SIZE, -B force a fixed checksum block-size
--rsh=COMMAND, -e specify the remote shell to use
--rsync-path=PROGRAM specify the rsync to run on remote machine
--existing skip creating new files on receiver
--ignore-existing skip updating files that exist on receiver
--remove-source-files sender removes synchronized files (non-dir)
--del an alias for --delete-during
--delete delete extraneous files from dest dirs
--delete-before receiver deletes before xfer, not during
--delete-during receiver deletes during the transfer
--delete-delay find deletions during, delete after
--delete-after receiver deletes after transfer, not during
--delete-excluded also delete excluded files from dest dirs
--ignore-missing-args ignore missing source args without error
--delete-missing-args delete missing source args from destination
--ignore-errors delete even if there are I/O errors
--force force deletion of dirs even if not empty
--max-delete=NUM don't delete more than NUM files
--max-size=SIZE don't transfer any file larger than SIZE
--min-size=SIZE don't transfer any file smaller than SIZE
--max-alloc=SIZE change a limit relating to memory alloc
--partial keep partially transferred files
--partial-dir=DIR put a partially transferred file into DIR
--delay-updates put all updated files into place at end
--prune-empty-dirs, -m prune empty directory chains from file-list
--numeric-ids don't map uid/gid values by user/group name
--usermap=STRING custom username mapping
--groupmap=STRING custom groupname mapping
--chown=USER:GROUP simple username/groupname mapping
--timeout=SECONDS set I/O timeout in seconds
--contimeout=SECONDS set daemon connection timeout in seconds
--ignore-times, -I don't skip files that match size and time
--size-only skip files that match in size
--modify-window=NUM, -@ set the accuracy for mod-time comparisons
--temp-dir=DIR, -T create temporary files in directory DIR
--fuzzy, -y find similar file for basis if no dest file
--compare-dest=DIR also compare destination files relative to DIR
--copy-dest=DIR ... and include copies of unchanged files
--link-dest=DIR hardlink to files in DIR when unchanged
--compress, -z compress file data during the transfer
--compress-choice=STR choose the compression algorithm (aka --zc)
--compress-level=NUM explicitly set compression level (aka --zl)
--skip-compress=LIST skip compressing files with suffix in LIST
--cvs-exclude, -C auto-ignore files in the same way CVS does
--filter=RULE, -f add a file-filtering RULE
-F same as --filter='dir-merge /.rsync-filter'
repeated: --filter='- .rsync-filter'
--exclude=PATTERN exclude files matching PATTERN
--exclude-from=FILE read exclude patterns from FILE
--include=PATTERN don't exclude files matching PATTERN
--include-from=FILE read include patterns from FILE
--files-from=FILE read list of source-file names from FILE
--from0, -0 all *-from/filter files are delimited by 0s
--old-args disable the modern arg-protection idiom
--secluded-args, -s use the protocol to safely send the args
--trust-sender trust the remote sender's file list
--copy-as=USER[:GROUP] specify user & optional group for the copy
--address=ADDRESS bind address for outgoing socket to daemon
--port=PORT specify double-colon alternate port number
--sockopts=OPTIONS specify custom TCP options
--blocking-io use blocking I/O for the remote shell
--outbuf=N|L|B set out buffering to None, Line, or Block
--stats give some file-transfer stats
--8-bit-output, -8 leave high-bit chars unescaped in output
--human-readable, -h output numbers in a human-readable format
--progress show progress during transfer
-P same as --partial --progress
--itemize-changes, -i output a change-summary for all updates
--remote-option=OPT, -M send OPTION to the remote side only
--out-format=FORMAT output updates using the specified FORMAT
--log-file=FILE log what we're doing to the specified FILE
--log-file-format=FMT log updates using the specified FMT
--password-file=FILE read daemon-access password from FILE
--early-input=FILE use FILE for daemon's early exec input
--list-only list the files instead of copying them
--bwlimit=RATE limit socket I/O bandwidth
--stop-after=MINS Stop rsync after MINS minutes have elapsed
--stop-at=y-m-dTh:m Stop rsync at the specified point in time
--fsync fsync every written file
--write-batch=FILE write a batched update to FILE
--only-write-batch=FILE like --write-batch but w/o updating dest
--read-batch=FILE read a batched update from FILE
--protocol=NUM force an older protocol version to be used
--iconv=CONVERT_SPEC request charset conversion of filenames
--checksum-seed=NUM set block/file checksum seed (advanced)
--ipv4, -4 prefer IPv4
--ipv6, -6 prefer IPv6
--version, -V print the version + other info and exit
--help, -h (*) show this help (* -h is help only on its own)
Use "rsync --daemon --help" to see the daemon-mode command-line options.
Please see the rsync(1) and rsyncd.conf(5) manpages for full documentation.
See https://rsync.samba.org/ for updates, bug reports, and answers
rsync error: syntax or usage error (code 1) at main.c(1776) [client=3.2.7]
/var/lib/sftpgo # wget
BusyBox v1.36.1 (2024-06-12 06:28:12 UTC) multi-call binary.
Usage: wget [-cqS] [--spider] [-O FILE] [-o LOGFILE] [--header STR]
[--post-data STR | --post-file FILE] [-Y on/off]
[-P DIR] [-U AGENT] [-T SEC] URL...
Retrieve files via HTTP or FTP
--spider Only check URL existence: $? is 0 if exists
--header STR Add STR (of form 'header: value') to headers
--post-data STR Send STR using POST method
--post-file FILE Send FILE using POST method
-c Continue retrieval of aborted transfer
-q Quiet
-P DIR Save to DIR (default .)
-S Show server response
-T SEC Network read timeout is SEC seconds
-O FILE Save to FILE ('-' for stdout)
-o LOGFILE Log messages to FILE
-U STR Use STR for User-Agent header
-Y on/off Use proxy
/var/lib/sftpgo #
1 Like
coming soon, thank to all
cron documentation at GitHub - NethServer/ns8-webserver: A webserver (nginx based) with php-fpm stack
2 Likes
Could you solve the sync issue @danb35 ?
1 Like
I’m not sure I can. Whatever changes have been made to the webserver module now seem to break scp entirely:
[root@neth interociter]# scp -r -P 3092 * 9004@ns8.familybrown.org:/
exec request failed on channel 0
lost connection
Adding a -v flag gives much more output, but I don’t think it’s especially helpful:
[root@neth interociter]# scp -rv -P 3092 * 9004@ns8.familybrown.org:/
Executing: program /usr/bin/ssh host ns8.familybrown.org, user 9004, command scp -v -r -d -t /
OpenSSH_7.4p1, OpenSSL 1.0.2k-fips 26 Jan 2017
debug1: Reading configuration data /root/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 12: Applying options for *
debug1: Connecting to ns8.familybrown.org [66.94.117.228] port 3092.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file /root/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.4
debug1: Remote protocol version 2.0, remote software version SFTPGo_2.6.4
debug1: no match: SFTPGo_2.6.4
debug1: Authenticating to ns8.familybrown.org:3092 as '9004'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: curve25519-sha256 need=64 dh_need=64
debug1: kex: curve25519-sha256 need=64 dh_need=64
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:bYY4+wGo1xkRQHlNwBGlt5VUq1eFXZDLqWrVAwLcm6U
debug1: Host '[ns8.familybrown.org]:3092' is known and matches the ECDSA host key.
debug1: Found key in /root/.ssh/known_hosts:16
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,rsa-sha2-256,rsa-sha2-512>
debug1: kex_input_ext_info: ping@openssh.com=<0>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: password,publickey,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /root/.ssh/id_rsa
debug1: Server accepts key: pkalg rsa-sha2-512 blen 277
debug1: Authentication succeeded (publickey).
Authenticated to ns8.familybrown.org ([66.94.117.228]:3092).
debug1: channel 0: new [client-session]
debug1: Entering interactive session.
debug1: pledge: network
debug1: Sending environment.
debug1: Sending env LANG = en_US.UTF-8
debug1: Sending command: scp -v -r -d -t /
exec request failed on channel 0
lost connection
1 Like
Thanks, I can reproduce it on NethServer 7, it works on newer systems.
I’m trying to find a fix.
Did you try rsync? It worked on Neth 7.
rsync -av -e "ssh -p 3092" * 9004@ns8.domain.tld:/
In the meanwhile you could go back to the previous webserver version to make scp work:
api-cli run update-module --data '{"module_url":"ghcr.io/nethserver/webserver:1.0.21","instances":["webserver1"],"force":true}'
1 Like
rsync does in fact work.
1 Like
Adding scp to the enabled ssh commands seems to solve it. Weird that it works without on newer systems…
SFTPGO_SFTPD__ENABLED_SSH_COMMANDS=rsync,scp
To edit the service file:
runagent -m webserver1 systemctl --user edit --full sftpgo
Restart service:
runagent -m webserver1 systemctl --user restart sftpgo
From SSH - SFTPGo documentation it should be enabled by default:
The following SSH commands are enabled by default:
scp
2 Likes
So to return to (one of the) reason(s) I opened this topic, the mirror of the Stephdl repo. Created the virtual host, set the DNS records, got the Let’s Encrypt cert, used rsync to copy over everything on my NS7 server to the NS8 server. All easy enough.
Then runagent -m webserver1, then podman exec -ti sftpgo sh. Then crontab -e and put the recurring command in there. Is that it?
1 Like
Yes, that should do the trick.
EDIT:
I think you also need to enable “Root directory file listings” in the advanced settings in the NS8 webserver virtual host settings.
1 Like
new version is coming with sftpgo, nginx updates and this fix
2 Likes