I tested it and it just worked. Your configuration seems to be correct.
Did you create/edit the file
/etc/backup-data.d/custom.exclude with a Windows text editor?
How did you check if the excluded directory is backuped?
You may start the backup data process on command line with
backup-data and afterwards check the output of
backup-data-list that shows you all files in the last backup.
My exclude file:
[root@testserver ~]# cat /etc/backup-data.d/custom.exclude