the problem is the self-signed certificate or that Synology doesn’t trust it. In LDAPAdmin you can ignore it, so maybe there’s a possibity to setup Synology to trust/ignore the self-signed cert. I didn’t find one in my research but I don’t use a Synology NAS.
As regards security I really recommend to have a valid cert but if you just need to make it work you may follow these steps and connect without SSL/TLS (INSECURE!):