Hello people.
I have the latest version of NetshServer installed. My data mass has millions of small files where searching is slow.
I would like to have an index file for faster searching when users try to browse some file on samba server.
It used to be “netatalk AFP” for linux rpm, and it has its own index, but AFP falls out of favor.
When I installed the AFP version for Windows Server, AFP used the native Windows feature “Windows Search”, there was always some problem reindexing, but ok, there was something.
I haven’t found anything on Linux, I know XFS works with journaling, but it doesn’t even come close to having an index file.
I don’t know if NetshServer already has some complementary support feature for what I need in order to integrate with Samba, or if there is any syntax I can enable in smb.conf.
I have an SSD for the operating system where I intend to store this index, in order to speed up the search in the file itself, besides 32 GB RAM and only use terminal mode.
I read the documentation on https://www.samba.org/ but found no parameter to specify an index file.
Does anyone in the community know anything about this and could help me?
Thank you in advance.
Paulo
EDIT: other than proprietary Windows Server solution, a project to index samba shares compatible with Windows Search Protocol (samba WSP) was started by SUSE but it is not finished and no estimated timeline that I’m aware of:
Hello people
A few days ago I got in touch to check out a search solution by indexing on Linux / SMB. Both on Netshserver and in other communities.
After some frustrating research, reading smb.org, communities, expert friends, no one had ever come across this need.
There is a reason, the Linux file system (EXT 3/4, XSF etc) and its find-based search method is very fast.
But I am not in the console performing search, but my end user on a Windows and / or Apple station.
When we are talking up to 500 thousand files we do not even notice, but as it passes the 5 million files begins to change the scenario.
I used to use AFP (netatalk) in CentOS, and it has its proprietary indexing mechanism.
But as AFP is discontinued and SMB is doing well on Microsoft, Linux and Apple platforms, and the audit helps a lot … I started the journey on how to solve the problem.
These 5 million files are small, at most 10 Mega, usually 5 Mega. Audio files. Few have Gibabyte, exceptions.
however from what I understand the same only works with GUI interface, and I haven’t seen how I can integrate it with SMB.
I am going to research to see if it is possible to integrate the feature without GUI, without attacking the concept of Netshserver, and invite everyone in the community to the journey.
It would be amazing to have this package bundled with NetshServer version 8 as an addon.
Hello @dnutan
I’m sorry for the late reply.
I searched so much on the Web that I arrived at the same as indicated.
The information in the article: “Directories with a Large Number of Files”, this one I hadn’t seen, thanks.
I added the parameters below, it improved, but it still hasn’t solved.
In Mac OSx I opened the terminal and made OSx create the index on the machine
Enabling Network Volume for Index
mdutil / Volumes / DEPARTMENTS -i on
mdutil / Volumes / JOBS_OLD -i on
mdutil / Volumes / WORKS -i on
If samba needs compiling, does it informs of something missing?
Sorry, I don’t know what is required. tracker (and tracker-devel) is in default CentOS repos (with libtracker-sparql being part of it if I’m not mistaken).
P.S.:If it’s of any use, here’s an OMV (debian based) forum thread with similar goal:
Could you please share a video capture of the problem?
I’m not sure about the issue, I can say that we never had performance problems with millions of files.
When searching with a Windows or Apple client, it doesn’t bring any results.
we have already reinstalled NethServer and tested it on other networks and servers.
Note: it is a DELL T-440 Server
VOL System SSD
VOL Data RAID 5
is there any procedure I can do to send a test?
Basically I go to Find on Mac or Windows and it doesn’t work.
We tested on Synology’s DSM System and it works very fast.
@pbasque, I spent a lot of time investigating your request. I guess you need the Spotlight search feature on shared samba folders.
It has been introduced in a samba version newer than what is provided by Red Hat / CentOS, so we can’t support it in NethServer 7.
But you mention audio files, where spotlight is ineffective.
Maybe I completely misunderstood your request.
I was looking to solve the problem too and ended up migrating my user base to… nextcloud.
Users subscribe to the folders they need and sync them on their computers. For searching, they can use their local filesystem indexer or go to nextcloud’s UI and search there.
Users are happy. They can even access their file on the go. I ditched SMB.
But now everyone needs a local copy of their data. That’s kind of a big benefit to network storage that Nextcloud doesn’t give you (yes, they’ve promised this for a couple of years; like a lot of their other headline features, it seems relegated to wallow in limited-release, pre-alpha status).
