Account provider connection refused

compsos · April 16, 2017, 6:32am

NethServer Version: CentOS Linux release 7.3.1611 (Core) NethServer release 7.3.1611 (Final)
Module: nethserver-dc-1.1.3-1.ns7.x86_64
If I sound a little confused I am. Basically can not get any Users or Groups up in the Manager page. Tests on the AD pass. The only consistent proof of a problem is the Manger page throwing up a banner “Account provider connection refused” The messages log confirms with
"Apr 16 16:07:36 srv httpd: [ERROR] NethServer\Tool\UserProvider: Account provider connection refused
Apr 16 16:07:36 srv httpd: [ERROR] Connection refused"
net ads info is good but the command
"systemctl -M nsdc status samba-provision.service" returns
"● samba-provision.service - Domain controller provisioning
Loaded: loaded (/etc/systemd/system/samba-provision.service; enabled; vendor preset: disabled)
Active: inactive (dead)
Condition: start condition failed at Sun 2017-04-16 15:16:13 AEST; 59min ago
ConditionPathExists=!/var/lib/samba/private/krb5.conf was not met"
restart “systemctl -M nsdc restart samba-provision.service” shows no error but the status does.
the contents of the krb5 file looks OK
[libdefaults]
default_realm = DOMAIN.COM.AU
dns_lookup_realm = false
dns_lookup_kdc = true

net ads info
LDAP server: 192.168.35.2
LDAP server name: nsdc-srv.domain.com.au
Realm: DOMAIN.COM.AU
Bind Path: dc=DOMAIN,dc=COM,dc=AU
LDAP port: 389
Server time: Sun, 16 Apr 2017 16:20:50 AEST
KDC server: 192.168.35.2
Server time offset: 0
Last machine account password change: Wed, 22 Feb 2017 18:07:50 AEST

OK What am I missing? Thanks

compsos · April 17, 2017, 8:31am

If I test the sssd status I get that it is running but lots of
"Apr 17 18:22:20 srv.domain.com.au sssd[be[legacy]][23182]: Warning: user would have been denied GPO-based logon access if the ad_gpo_access_control option were set to enforcing mode."
Did find an error stopping the loading of host0 /var/lib/machines/nsdc/etc/systemd/network/green.network had an incorrect gateway setting. Pointed to gateway used during setting up. But has not fixed the main issue.

compsos · April 18, 2017, 11:51pm

Could this be an issue with how the Manager web interface calls the data?
If I log into the NSDC and use samba-tool, I can get a full list of users and groups. but rejected in the web page.

httpd-admin error log
[Wed Apr 19 09:48:27.659034 2017] [:error] [pid 2347] [client ] [ERROR] NethServer\Tool\UserProvider: Account provider connection refused, referer: https://…/en-US/Account
[Wed Apr 19 09:48:27.659158 2017] [:error] [pid 2347] [client ]… [ERROR] Connection refused\n, referer: https://…/en-US/Account

Messages log
Apr 19 13:46:25 srv httpd: [ERROR] NethServer\Tool\GroupProvider: Account provider connection refused
Apr 19 13:46:25 srv httpd: [ERROR] Connection refused
Apr 19 13:46:26 srv admin-todos: Connection refused
Apr 19 13:46:38 srv httpd: [ERROR] NethServer\Tool\UserProvider: Account provider connection refused
Apr 19 13:46:38 srv httpd: [ERROR] Connection refused
Apr 19 13:47:01 srv httpd: [ERROR] NethServer\Tool\GroupProvider: Account provider connection refused
Apr 19 13:47:01 srv httpd: [ERROR] Connection refused
Apr 19 13:47:02 srv admin-todos: Connection refused
Apr 19 13:47:19 srv httpd: [ERROR] NethServer\Tool\GroupProvider: Account provider connection refused
Apr 19 13:47:19 srv httpd: [ERROR] Connection refused
Apr 19 13:47:19 srv admin-todos: Connection refused

davidep · April 19, 2017, 7:11am

Hi @compsos thanks for your report!

Do you see some relevant log lines in the output of

journalctl -M nsdc

And

journalctl -M nsdc -u systemd-networkd

compsos · April 20, 2017, 2:38am

Hi @davidep
We have setup a another “control” system to be able to compare a working unit against the problem one.
In answer to your questions, we can see some entries that look suspect but they are the same on the control unit. So nothing there. Access to the DC from RSAT is fine and inside nsdc returns answer just not the web page. We do have 2 systems displaying this issue and I “suspect” have arisen after adding some acls via the webpage. Is there a method to remove them say within the nsdc?

systemd-networkd
Apr 19 13:07:03 nsdc-srv.domain.com.au systemd-networkd[49]: host0 : Cannot configure IPv4 forwarding for interface host0: Read-only file system
Apr 19 13:07:03 nsdc-srv.domain.com.au systemd-networkd[49]: host0 : Cannot configure IPv6 forwarding for interface: Read-only file system
Apr 19 13:07:03 nsdc-srv.domain.com.au systemd-networkd[49]: Enumeration completed
Apr 19 13:07:03 nsdc-srv.domain.com.au systemd-networkd[49]: host0 : link configured
Apr 19 13:07:03 nsdc-srv.domain.com.au systemd[1]: Started Network Service.

and these are in Red
Apr 19 13:06:32 nsdc-srv.domain.com.au winbindd[38]: [2017/04/19 13:06:32.495606, 0] …/source3/winbindd/winbindd_cache.c:3245(initialize_winbindd_cache)
Apr 19 13:06:32 nsdc-srv.domain.com.au winbindd[38]: initialize_winbindd_cache: clearing cache and re-creating with version number 2
Apr 19 13:06:32 nsdc-srv.domain.com.au winbindd[38]: [2017/04/19 13:06:32.627655, 0] …/lib/util/become_daemon.c:124(daemon_ready)
Apr 19 13:06:32 nsdc-srv.domain.com.au winbindd[38]: STATUS=daemon ‘winbindd’ finished starting up and ready to serve connections
Apr 19 13:06:32 nsdc-srv.domain.com.au smbd[29]: [2017/04/19 13:06:32.640109, 0] …/lib/util/become_daemon.c:124(daemon_ready)
Apr 19 13:06:32 nsdc-srv.domain.com.au smbd[29]: STATUS=daemon ‘smbd’ finished starting up and ready to serve connections
Apr 19 13:07:03 nsdc-srv.domain.com.au systemd[1]: Starting Network Service…

fix this error but did not solve the problem
bash-4.2# samba-tool dbcheck --cross-ncs --reset-well-known-acls --fix
Checking 3449 objects
Reset nTSecurityDescriptor on DC=domain,DC=com,DC=au back to provision default?
Part dacl is different between reference and current here is the detail:
(D;;DC;;;WD) ACE is not present in the reference
[y/N/all/none] y
Fixed attribute ‘nTSecurityDescriptor’ of ‘DC=domain,DC=com,DC=au’

Checked 3449 objects (1 errors)

davidep · April 20, 2017, 8:08am

Do you think it the problem is reproducible?

Could you write down some steps to reproduce it?

compsos · April 20, 2017, 8:19am

@davidep
Hi Davide
I think if I knew how we got there then maybe the reversal would be easier to find. Did find how to remove the facls.
In the server and not the nsdc issue the command
getfacl -d /var/lib/nethserver/ibay/[ibay_name]
will list acls applied and then
setfacl -b /var/lib/nethserver/ibay/[ibay_name]
Will remove them.
But only slightly better off as in the “Shared folders” acl tab Domain Users is selectable but nothing else shows.
So now really need to find how to reset the “Provider” access rights. Any clues:slight_smile:

davidep · April 20, 2017, 8:27am

Please attach the output of

 account-provider-test dump
 config show sssd
 config show nsdc

compsos · April 20, 2017, 8:40am

@davidep
Thank you for looking at this. I am sure if a “repair” routine comes out of the pain then it will be useful to a lot of people in the future.

account-provider-test dump
{
“startTls” : “”,
“bindUser” : “SRV$”,
“userDN” : “dc=compsos,dc=com,dc=au”,
“port” : 389,
“isAD” : “1”,
“host” : “compsos.com.au”,
“groupDN” : “dc=compsos,dc=com,dc=au”,
“isLdap” : “”,
“ldapURI” : “ldap://compsos.com.au”,
“baseDN” : “dc=compsos,dc=com,dc=au”,
“bindPassword” : “9&f+4~v91pJO8f”,
“bindDN” : “COMPSOS\SRV$”
}

sssd=service
AdDns=192.168.35.2
LdapURI=ldap://compsos.com.au
Provider=ad
status=enabled

nsdc=service
IpAddress=192.168.35.2
bridge=br0
status=enabled

Was also looking at the log on the “clean” machine at the parts related to the migration-import. In there are routines for leaving , cleaning up conf and keytab files and then resetting the DC. Would that fix the issue or is that too big of a hammer?

compsos · April 20, 2017, 8:47am

just check those commands with the clean machine and the differences are
port : 636

ldapURI : ldaps://domain.org.au

and in the sssd output
LdapURI= <-- empty

netstat -an | grep 636 on the test platform returns
tcp 0 0 192.168.50.1:44169 192.168.50.2:636 ESTABILISHED
netstat -an |} grep 389
tcp 0 0 192.168.50.1:36275 192.168.50.2:389 ESTABILISHED
tcp 0 0 192.168.50.1:36280 192.168.50.2:389 ESTABILISHED

on the production nothing on 636 but 389
tcp 0 0 192.168.35.1:51034 192.168.35.2:389 ESTABLISHED

davidep · April 20, 2017, 9:05am

Is it a sme8 migration?

Attach the output of this command that queries the DC rootDSE:

 ldapsearch -x -D '' -w '' -h $(config getprop nsdc IpAddress) -s base -b ''

Then

ldapsearch -x -D '' -w '' -H ldaps://$(config getprop nsdc IpAddress) -s base -b ''

Finally

/usr/libexec/nethserver/list-users

And

host compsos.com.au
host $(hostname -d)

compsos · April 20, 2017, 9:13am

On the test platform it was SME9.1 configuration

# extended LDIF
#
# LDAPv3
# base <> with scope baseObject
# filter: (objectclass=*)
# requesting: ALL
#

#
dn:
configurationNamingContext: CN=Configuration,DC=compsos,DC=com,DC=au
defaultNamingContext: DC=compsos,DC=com,DC=au
rootDomainNamingContext: DC=compsos,DC=com,DC=au
schemaNamingContext: CN=Schema,CN=Configuration,DC=compsos,DC=com,DC=au
subschemaSubentry: CN=Aggregate,CN=Schema,CN=Configuration,DC=compsos,DC=com,D
 C=au
supportedCapabilities: 1.2.840.113556.1.4.800
supportedCapabilities: 1.2.840.113556.1.4.1670
supportedCapabilities: 1.2.840.113556.1.4.1791
supportedCapabilities: 1.2.840.113556.1.4.1935
supportedCapabilities: 1.2.840.113556.1.4.2080
supportedLDAPVersion: 2
supportedLDAPVersion: 3
vendorName: Samba Team (http://samba.org)
isSynchronized: TRUE
dsServiceName: CN=NTDS Settings,CN=NSDC-SRV,CN=Servers,CN=Default-First-Site-N
 ame,CN=Sites,CN=Configuration,DC=compsos,DC=com,DC=au
serverName: CN=NSDC-SRV,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Conf
 iguration,DC=compsos,DC=com,DC=au
dnsHostName: nsdc-srv.compsos.com.au
ldapServiceName: compsos.com.au:nsdc-srv$@COMPSOS.COM.AU
currentTime: 20170420090740.0Z
supportedControl: 1.2.840.113556.1.4.841
supportedControl: 1.2.840.113556.1.4.319
supportedControl: 1.2.840.113556.1.4.473
supportedControl: 1.2.840.113556.1.4.1504
supportedControl: 1.2.840.113556.1.4.801
supportedControl: 1.2.840.113556.1.4.801
supportedControl: 1.2.840.113556.1.4.805
supportedControl: 1.2.840.113556.1.4.1338
supportedControl: 1.2.840.113556.1.4.529
supportedControl: 1.2.840.113556.1.4.417
supportedControl: 1.2.840.113556.1.4.2064
supportedControl: 1.2.840.113556.1.4.1413
supportedControl: 1.2.840.113556.1.4.1413
supportedControl: 1.2.840.113556.1.4.1413
supportedControl: 1.2.840.113556.1.4.1413
supportedControl: 1.2.840.113556.1.4.1413
supportedControl: 1.2.840.113556.1.4.1339
supportedControl: 1.2.840.113556.1.4.1340
supportedControl: 1.2.840.113556.1.4.1413
supportedControl: 1.2.840.113556.1.4.1341
namingContexts: DC=compsos,DC=com,DC=au
namingContexts: CN=Configuration,DC=compsos,DC=com,DC=au
namingContexts: CN=Schema,CN=Configuration,DC=compsos,DC=com,DC=au
namingContexts: DC=DomainDnsZones,DC=compsos,DC=com,DC=au
namingContexts: DC=ForestDnsZones,DC=compsos,DC=com,DC=au
supportedSASLMechanisms: GSS-SPNEGO
supportedSASLMechanisms: GSSAPI
supportedSASLMechanisms: NTLM
highestCommittedUSN: 3997
domainFunctionality: 4
forestFunctionality: 4
domainControllerFunctionality: 4
isGlobalCatalogReady: TRUE

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1


# extended LDIF
#
# LDAPv3
# base <> with scope baseObject
# filter: (objectclass=*)
# requesting: ALL
#

#
dn:
configurationNamingContext: CN=Configuration,DC=compsos,DC=com,DC=au
defaultNamingContext: DC=compsos,DC=com,DC=au
rootDomainNamingContext: DC=compsos,DC=com,DC=au
schemaNamingContext: CN=Schema,CN=Configuration,DC=compsos,DC=com,DC=au
subschemaSubentry: CN=Aggregate,CN=Schema,CN=Configuration,DC=compsos,DC=com,D
 C=au
supportedCapabilities: 1.2.840.113556.1.4.800
supportedCapabilities: 1.2.840.113556.1.4.1670
supportedCapabilities: 1.2.840.113556.1.4.1791
supportedCapabilities: 1.2.840.113556.1.4.1935
supportedCapabilities: 1.2.840.113556.1.4.2080
supportedLDAPVersion: 2
supportedLDAPVersion: 3
vendorName: Samba Team (http://samba.org)
isSynchronized: TRUE
dsServiceName: CN=NTDS Settings,CN=NSDC-SRV,CN=Servers,CN=Default-First-Site-N
 ame,CN=Sites,CN=Configuration,DC=compsos,DC=com,DC=au
serverName: CN=NSDC-SRV,CN=Servers,CN=Default-First-Site-Name,CN=Sites,CN=Conf
 iguration,DC=compsos,DC=com,DC=au
dnsHostName: nsdc-srv.compsos.com.au
ldapServiceName: compsos.com.au:nsdc-srv$@COMPSOS.COM.AU
currentTime: 20170420091015.0Z
supportedControl: 1.2.840.113556.1.4.841
supportedControl: 1.2.840.113556.1.4.319
supportedControl: 1.2.840.113556.1.4.473
supportedControl: 1.2.840.113556.1.4.1504
supportedControl: 1.2.840.113556.1.4.801
supportedControl: 1.2.840.113556.1.4.801
supportedControl: 1.2.840.113556.1.4.805
supportedControl: 1.2.840.113556.1.4.1338
supportedControl: 1.2.840.113556.1.4.529
supportedControl: 1.2.840.113556.1.4.417
supportedControl: 1.2.840.113556.1.4.2064
supportedControl: 1.2.840.113556.1.4.1413
supportedControl: 1.2.840.113556.1.4.1413
supportedControl: 1.2.840.113556.1.4.1413
supportedControl: 1.2.840.113556.1.4.1413
supportedControl: 1.2.840.113556.1.4.1413
supportedControl: 1.2.840.113556.1.4.1339
supportedControl: 1.2.840.113556.1.4.1340
supportedControl: 1.2.840.113556.1.4.1413
supportedControl: 1.2.840.113556.1.4.1341
namingContexts: DC=compsos,DC=com,DC=au
namingContexts: CN=Configuration,DC=compsos,DC=com,DC=au
namingContexts: CN=Schema,CN=Configuration,DC=compsos,DC=com,DC=au
namingContexts: DC=DomainDnsZones,DC=compsos,DC=com,DC=au
namingContexts: DC=ForestDnsZones,DC=compsos,DC=com,DC=au
supportedSASLMechanisms: GSS-SPNEGO
supportedSASLMechanisms: GSSAPI
supportedSASLMechanisms: NTLM
highestCommittedUSN: 3997
domainFunctionality: 4
forestFunctionality: 4
domainControllerFunctionality: 4
isGlobalCatalogReady: TRUE

# search result
search: 2
result: 0 Success

# numResponses: 2
# numEntries: 1

**********************************************
/usr/libexec/nethserver/list-users
Connection refused
*****************************************
host compsos.com.au
compsos.com.au has address 192.168.35.1
host $(hostname -d)
compsos.com.au has address 192.168.35.1

davidep · April 20, 2017, 9:18am

Those IPs don’t match!

What’s the dnsmasq.conf?

grep -F server= /etc/dnsmasq.conf

compsos · April 20, 2017, 9:19am

server=192.168.35.1
server=192.168.35.2
server=8.8.8.8
server=/uribl.com/127.0.0.1#10053
server=/dnswl.org/127.0.0.1#10053
server=/spamhaus.org/127.0.0.1#10053
server=/compsos.com.au/192.168.35.2

davidep · April 20, 2017, 9:35am

Looks weird!

I’d expect

server=192.168.35.1
server=/compsos.com.au/192.168.35.2

~~Your dnsmasq.conf is not consistent with sssd and nsdc records above… Do you have any template-custom?~~

Sorry, I didn’t notice it…

Is it a secondary DNS? Please, remove it!

Must not be set as upstream DNS! It’s the nsdc IP!

Then see if this fixes your problem:

config setprop sssd LdapURI ldaps://$(config getprop nsdc IpAddress)
signal-event nethserver-sssd-save

compsos · April 20, 2017, 9:46am

@davidep
The only weird thing at this end will be me if I don’t get my head around this “provider” !!
That has been one of the troubles in not enough in the logs to pin it down. And of coarse that 90% is working.
Thank you.

OK that last set of commands fixed it. Was it just a poor choice in the DNS box under network?

davidep · April 20, 2017, 9:51am

I don’t know! …because that box sets a primary and a secondary DNS server only, whilst your dnsmasq.conf seemed to have three DNS servers!

I suggest set one DNS server only. It must not be the nsdc IP address.

I think we should add a validator to avoid it! /cc @dev_team

compsos · April 20, 2017, 9:53am

Ok I am about to apply the same to another affected system. But its S|DNS box only has 2 entries the 1st pointing to the server and the 2nd 8.8.8.8. So will just try the command line arguments and report back

compsos · April 20, 2017, 10:02am

On the other affected Server

OK before the commands
sssd=service
AdDns=192.168.100.2
LdapURI=
Provider=ad
status=enabled
After the commands
sssd=service
AdDns=192.168.100.2
LdapURI=ldaps://192.168.100.2
Provider=ad
status=enabled

no change to the DNS box and YES the “Provider” works. So it was all about the LdapURI being empty

And again Thank you Davide. that other Howto I have been working on can now be completed as nothing done on that side was playing with these settings.