Same result here.
Let’s check the config file:
sudo -u mattermost /opt/mattermost/bin/mattermost config validate
Please also check permissions of the config file:
[root@server2 ~]# ls -l /opt/mattermost/config/config.json
-rw-r--r-- 1 mattermost mattermost 19128 Jan 21 21:58 /opt/mattermost/config/config.json
Same:
Error: failed to initialize config store: failed to create store: unable to load on store creation: parsing error at line 1, character 13860: invalid character ‘a’ after top-level value
and
ls -l /opt/mattermost/config/config.json
-rw-r–r-- 1 mattermost mattermost 18894 Jan 21 07:19 /opt/mattermost/config/config.json
[root@teamserver ~]# sudo -u mattermost /opt/mattermost/bin/mattermost
Error: failed to load configuration: failed to create store: unable to load on s tore creation: parsing error at line 1, character 13870: invalid character ‘,’ a fter top-level value
Usage:
mattermost [flags]
mattermost [command]
Available Commands:
channel Management of channels
command Management of slash commands
config Configuration
export Export data from Mattermost
group Management of groups
help Help about any command
import Import data.
integrity Check database data integrity
jobserver Start the Mattermost job server
ldap LDAP related utilities
license Licensing commands
logs Display logs in a human-readable format
permissions Management of the Permissions system
plugin Management of plugins
reset Reset the database to initial state
roles Management of user roles
sampledata Generate sample data
server Run the Mattermost server
team Management of teams
user Management of users
version Display version information
webhook Management of webhooks
Flags:
-c, --config string Configuration file to use.
–disableconfigwatch When set config.json will not be loaded from disk w hen the file is changed.
-h, --help help for mattermost
Use “mattermost [command] --help” for more information about a command.
[root@teamserver ~]# sudo -u mattermost /opt/mattermost/bin/mattermost config va lidate
Error: failed to initialize config store: failed to create store: unable to load on store creation: parsing error at line 1, character 13870: invalid character ‘,’ after top-level value
Usage:
mattermost config validate [flags]
Flags:
-h, --help help for validate
Global Flags:
-c, --config string Configuration file to use.
–disableconfigwatch When set config.json will not be loaded from disk w hen the file is changed.
[root@teamserver ~]# ls -l /opt/mattermost/config/config.json
-rw-r–r-- 1 mattermost mattermost 18820 Jan 21 06:51 /opt/mattermost/config/config.json
OK, so let’s check the config file syntax with another method:
cat /opt/mattermost/config/config.json | jq
You should get an error or it prints the complete file content.
[root@teamserver ~]# cat /opt/mattermost/config/config.json | jq
{
"DisplaySettings": {
"ExperimentalTimezone": false,
"CustomUrlSchemes": []
},
"SqlSettings": {
"MaxOpenConns": 300,
"DriverName": "postgres",
"QueryTimeout": 30,
"ConnMaxLifetimeMilliseconds": 3600000,
"DataSourceSearchReplicas": [],
"DisableDatabaseSearch": false,
"DataSource": "postgres://mattuser:71WlRcdJ0IVnY_IW@localhost:55434/mattermost?sslmode=disable&connect_timeout=10",
"DataSourceReplicas": [],
"MaxIdleConns": 20,
"Trace": false,
"AtRestEncryptKey": "m3egdd4t7ba4srghzc4rhkjonprw1qa4"
},
"ExperimentalAuditSettings": {
"AdvancedLoggingConfig": "",
"FileCompress": false,
"FileEnabled": false,
"FileMaxAgeDays": 0,
"FileMaxBackups": 0,
"FileName": "",
"FileMaxQueueSize": 1000,
"FileMaxSizeMB": 100
},
"JobSettings": {
"RunScheduler": true,
"RunJobs": true
},
"BleveSettings": {
"EnableAutocomplete": false,
"IndexDir": "",
"EnableIndexing": false,
"BulkIndexingTimeWindowSeconds": 3600,
"EnableSearching": false
},
"CloudSettings": {
"CWSUrl": "https://customers.mattermost.com"
},
"MessageExportSettings": {
"DownloadExportResults": false,
"ExportFromTimestamp": 0,
"ExportFormat": "actiance",
"EnableExport": false,
"BatchSize": 10000,
"DailyRunTime": "01:00",
"GlobalRelaySettings": {
"SMTPServerTimeout": 1800,
"SmtpPassword": "",
"EmailAddress": "",
"SmtpUsername": "",
"CustomerType": "A9"
}
},
"ComplianceSettings": {
"EnableDaily": false,
"Directory": "./data/",
"Enable": false
},
"AnalyticsSettings": {
"MaxUsersForStatistics": 2500
},
"NativeAppSettings": {
"IosAppDownloadLink": "https://about.mattermost.com/mattermost-ios-app/",
"AppDownloadLink": "https://mattermost.com/download/#mattermostApps",
"AndroidAppDownloadLink": "https://about.mattermost.com/mattermost-android-app/"
},
"PrivacySettings": {
"ShowFullName": true,
"ShowEmailAddress": true
},
"ElasticsearchSettings": {
"IndexPrefix": "",
"Sniff": true,
"PostsAggregatorJobStartTime": "03:00",
"ChannelIndexReplicas": 1,
"SkipTLSVerification": false,
"UserIndexReplicas": 1,
"Password": "changeme",
"PostIndexReplicas": 1,
"Trace": "",
"Username": "elastic",
"BulkIndexingTimeWindowSeconds": 3600,
"LiveIndexingBatchSize": 1,
"EnableAutocomplete": false,
"EnableSearching": false,
"ChannelIndexShards": 1,
"ConnectionUrl": "http://localhost:9200",
"PostIndexShards": 1,
"UserIndexShards": 1,
"AggregatePostsAfterDays": 365,
"RequestTimeoutSeconds": 30,
"EnableIndexing": false
},
"ThemeSettings": {
"DefaultTheme": "default",
"EnableThemeSelection": true,
"AllowCustomThemes": true,
"AllowedThemes": []
},
"LogSettings": {
"AdvancedLoggingConfig": "",
"EnableDiagnostics": false,
"EnableConsole": false,
"EnableWebhookDebugging": true,
"FileLocation": "",
"ConsoleLevel": "INFO",
"FileJson": true,
"EnableSentry": false,
"ConsoleJson": false,
"FileLevel": "INFO",
"EnableFile": true
},
"MetricsSettings": {
"ListenAddress": ":8067",
"BlockProfileRate": 0,
"Enable": false
},
"GitLabSettings": {
"Scope": "",
"Id": "",
"Enable": false,
"UserApiEndpoint": "",
"AuthEndpoint": "",
"TokenEndpoint": "",
"Secret": ""
},
"LocalizationSettings": {
"AvailableLocales": "de,en",
"DefaultServerLocale": "de",
"DefaultClientLocale": "de"
},
"ClusterSettings": {
"IdleConnTimeoutMilliseconds": 90000,
"MaxIdleConnsPerHost": 128,
"GossipPort": 8074,
"AdvertiseAddress": "",
"NetworkInterface": "",
"OverrideHostname": "",
"ReadOnlyConfig": true,
"BindAddress": "",
"MaxIdleConns": 100,
"EnableExperimentalGossipEncryption": false,
"Enable": false,
"ClusterName": "",
"StreamingPort": 8075,
"UseIpAddress": true,
"UseExperimentalGossip": false
},
"PasswordSettings": {
"Uppercase": true,
"Number": true,
"MinimumLength": 10,
"Symbol": true,
"Lowercase": true
},
"SamlSettings": {
"IdpDescriptorUrl": "",
"FirstNameAttribute": "",
"PositionAttribute": "",
"PublicCertificateFile": "",
"CanonicalAlgorithm": "Canonical1.0",
"PrivateKeyFile": "",
"ScopingIDPName": "",
"EnableAdminAttribute": false,
"LastNameAttribute": "",
"IdpUrl": "",
"UsernameAttribute": "",
"IdpMetadataUrl": "",
"AssertionConsumerServiceURL": "",
"GuestAttribute": "",
"IgnoreGuestsLdapSync": false,
"Enable": false,
"SignRequest": false,
"NicknameAttribute": "",
"IdpCertificateFile": "",
"EmailAttribute": "",
"EnableSyncWithLdapIncludeAuth": false,
"LoginButtonText": "SAML",
"LoginButtonBorderColor": "#2389D7",
"LoginButtonTextColor": "#ffffff",
"Verify": true,
"ScopingIDPProviderId": "",
"ServiceProviderIdentifier": "",
"LocaleAttribute": "",
"Encrypt": true,
"AdminAttribute": "",
"SignatureAlgorithm": "RSAwithSHA1",
"EnableSyncWithLdap": false,
"IdAttribute": "",
"LoginButtonColor": "#34a28b"
},
"LdapSettings": {
"MaxPageSize": 0,
"FirstNameAttribute": "",
"PublicCertificateFile": "",
"PositionAttribute": "",
"PrivateKeyFile": "",
"SkipCertificateVerification": false,
"SyncIntervalMinutes": 60,
"BaseDN": "",
"BindPassword": "",
"GuestFilter": "",
"GroupIdAttribute": "",
"LastNameAttribute": "",
"QueryTimeout": 60,
"UsernameAttribute": "",
"GroupDisplayNameAttribute": "",
"Enable": false,
"LdapPort": 389,
"NicknameAttribute": "",
"BindUsername": "",
"EnableAdminFilter": false,
"EmailAttribute": "",
"AdminFilter": "",
"LoginFieldName": "",
"LoginButtonBorderColor": "#2389D7",
"LoginIdAttribute": "",
"Trace": false,
"LoginButtonTextColor": "#2389D7",
"ConnectionSecurity": "",
"GroupFilter": "",
"EnableSync": false,
"PictureAttribute": "",
"LdapServer": "",
"UserFilter": "",
"IdAttribute": "",
"LoginButtonColor": "#0000"
},
"AnnouncementSettings": {
"NoticesSkipCache": false,
"EnableBanner": false,
"BannerTextColor": "#333333",
"BannerColor": "#f2a93b",
"BannerText": "",
"NoticesFetchFrequency": 3600,
"AllowBannerDismissal": true,
"NoticesURL": "https://notices.mattermost.com/",
"UserNoticesEnabled": true,
"AdminNoticesEnabled": true
},
"Office365Settings": {
"Id": "",
"UserApiEndpoint": "https://graph.microsoft.com/v1.0/me",
"Scope": "User.Read",
"DirectoryId": "",
"Enable": false,
"AuthEndpoint": "https://login.microsoftonline.com/common/oauth2/v2.0/authorize",
"Secret": "",
"TokenEndpoint": "https://login.microsoftonline.com/common/oauth2/v2.0/token"
},
"DataRetentionSettings": {
"DeletionJobStartTime": "02:00",
"MessageRetentionDays": 365,
"EnableFileDeletion": false,
"FileRetentionDays": 365,
"EnableMessageDeletion": false
},
"ExperimentalSettings": {
"ClientSideCertEnable": false,
"EnableSharedChannels": false,
"CloudUserLimit": 0,
"LinkMetadataTimeoutMilliseconds": 5000,
"ClientSideCertCheck": "secondary",
"RestrictSystemAdmin": false,
"UseNewSAMLLibrary": false,
"CloudBilling": false,
"EnableClickToReply": false
},
"SupportSettings": {
"PrivacyPolicyLink": "https://about.mattermost.com/default-privacy-policy/",
"ReportAProblemLink": "https://about.mattermost.com/default-report-a-problem/",
"SupportEmail": "feedback@mattermost.com",
"EnableAskCommunityLink": true,
"CustomTermsOfServiceReAcceptancePeriod": 365,
"TermsOfServiceLink": "https://about.mattermost.com/default-terms/",
"CustomTermsOfServiceEnabled": false,
"AboutLink": "https://about.mattermost.com/default-about/",
"HelpLink": "https://about.mattermost.com/default-help/"
},
"PluginSettings": {
"AutomaticPrepackagedPlugins": true,
"PluginStates": {
"com.mattermost.plugin-channel-export": {
"Enable": true
},
"com.mattermost.plugin-incident-management": {
"Enable": true
},
"com.mattermost.nps": {
"Enable": false
}
},
"SignaturePublicKeyFiles": [],
"RequirePluginSignature": false,
"Plugins": {
"com.mattermost.nps": {
"enablesurvey": false
}
},
"EnableUploads": false,
"EnableRemoteMarketplace": true,
"Directory": "./plugins",
"MarketplaceUrl": "https://api.integrations.mattermost.com",
"EnableMarketplace": true,
"ClientDirectory": "./client/plugins",
"Enable": true,
"EnableHealthCheck": true,
"AllowInsecureDownloadUrl": false
},
"ClientRequirements": {
"AndroidMinVersion": "",
"AndroidLatestVersion": "",
"DesktopMinVersion": "",
"DesktopLatestVersion": "",
"IosMinVersion": "",
"IosLatestVersion": ""
},
"GoogleSettings": {
"Scope": "profile email",
"Id": "",
"Enable": false,
"UserApiEndpoint": "https://people.googleapis.com/v1/people/me?personFields=names,emailAddresses,nicknames,metadata",
"AuthEndpoint": "https://accounts.google.com/o/oauth2/v2/auth",
"TokenEndpoint": "https://www.googleapis.com/oauth2/v4/token",
"Secret": ""
},
"ImageProxySettings": {
"ImageProxyType": "local",
"RemoteImageProxyURL": "",
"RemoteImageProxyOptions": "",
"Enable": false
},
"GuestAccountsSettings": {
"AllowEmailAccounts": true,
"RestrictCreationToDomains": "",
"EnforceMultifactorAuthentication": false,
"Enable": false
},
"FileSettings": {
"MaxFileSize": 52428800,
"EnableFileAttachments": true,
"AmazonS3PathPrefix": "",
"InitialFont": "nunito-bold.ttf",
"AmazonS3Region": "",
"PublicLinkSalt": "kzj3gpmxo1auw37d5uoiw6i1gs9c6hkx",
"AmazonS3Endpoint": "s3.amazonaws.com",
"AmazonS3SSE": false,
"DriverName": "local",
"EnablePublicLink": false,
"AmazonS3SSL": true,
"AmazonS3Bucket": "",
"Directory": "/var/lib/nethserver/mattermost/data",
"EnableMobileDownload": true,
"AmazonS3SignV2": false,
"AmazonS3Trace": false,
"EnableMobileUpload": true,
"AmazonS3AccessKeyId": "",
"AmazonS3SecretAccessKey": ""
},
"TeamSettings": {
"RestrictPrivateChannelDeletion": "all",
"RestrictPublicChannelDeletion": "all",
"ExperimentalEnableAutomaticReplies": false,
"CustomBrandText": "",
"RestrictDirectMessage": "any",
"MaxChannelsPerTeam": 2000,
"CustomDescriptionText": "",
"EnableUserCreation": true,
"RestrictPrivateChannelManageMembers": "all",
"MaxUsersPerTeam": 50,
"SiteName": "Teamwork",
"MaxNotificationsPerChannel": 1000000,
"RestrictCreationToDomains": "",
"RestrictPrivateChannelManagement": "all",
"EnableTeamCreation": true,
"RestrictPublicChannelManagement": "all",
"ExperimentalPrimaryTeam": "",
"LockTeammateNameDisplay": false,
"ExperimentalTownSquareIsReadOnly": false,
"EnableUserDeactivation": false,
"ExperimentalHideTownSquareinLHS": false,
"EnableCustomBrand": true,
"EnableConfirmNotificationsToChannel": true,
"TeammateNameDisplay": "username",
"RestrictPrivateChannelCreation": "all",
"RestrictPublicChannelCreation": "all",
"ExperimentalViewArchivedChannels": false,
"RestrictTeamInvite": "all",
"EnableXToLeaveChannelsFromLHS": false,
"UserStatusAwayTimeout": 300,
"EnableOpenServer": false,
"ExperimentalDefaultChannels": []
},
"NotificationLogSettings": {
"AdvancedLoggingConfig": "",
"EnableConsole": true,
"FileLocation": "",
"ConsoleLevel": "INFO",
"FileJson": true,
"ConsoleJson": true,
"FileLevel": "INFO",
"EnableFile": true
},
"EmailSettings": {
"SMTPServerTimeout": 10,
"SMTPPort": "465",
"InviteSalt": "qfEuE9P35oGl_KtNM2Eco5RyeFRGCuKO",
"EnableEmailBatching": false,
"SMTPPassword": "",
"FeedbackEmail": "teamwork@tiedt-online.com",
"EnableSignInWithUsername": true,
"LoginButtonBorderColor": "#2389D7",
"RequireEmailVerification": true,
"EnableSignUpWithEmail": true,
"SendEmailNotifications": false,
"LoginButtonTextColor": "#2389D7",
"EnablePreviewModeBanner": false,
"PushNotificationBuffer": 1000,
"PushNotificationContents": "generic_no_channel",
"ConnectionSecurity": "TLS",
"EnableSignInWithEmail": true,
"EnableSMTPAuth": false,
"SkipServerCertificateVerification": false,
"UseChannelInEmailNotifications": false,
"ReplyToAddress": "teamwork@tiedt-online.com",
"EmailBatchingBufferSize": 256,
"PushNotificationServer": "https://push-test.mattermost.com",
"SendPushNotifications": true,
"EmailNotificationContentsType": "full",
"FeedbackOrganization": "Tiedt",
"EmailBatchingInterval": 30,
"SMTPServer": "localhost",
"SMTPUsername": "",
"LoginButtonColor": "#0000",
"FeedbackName": "Tiedt Teamwork"
},
"ServiceSettings": {
"FeatureFlagSyncIntervalSeconds": 30,
"ExperimentalEnableDefaultChannelLeaveJoinMessages": true,
"EnableGifPicker": false,
"ImageProxyOptions": "",
"GfycatApiKey": "2_KtH_W5",
"ImageProxyURL": "",
"TimeBetweenUserTypingUpdatesMilliseconds": 5000,
"ListenAddress": "127.0.0.1:8065",
"EnableLatex": false,
"GfycatApiSecret": "3wLVZPiswc3DnaiaFoLkDvB4X0IV6CpMkj4tf2inJRsBY6-FnkT08zGmppWFgeof",
"ThreadAutoFollow": true,
"AllowCookiesForSubdomains": false,
"LicenseFileLocation": "",
"RestrictPostDelete": "all",
"EnablePostSearch": true,
"EnableTesting": false,
"WriteTimeout": 300,
"EnableSVGs": false,
"ExperimentalChannelSidebarOrganization": "disabled",
"SessionLengthWebInDays": 180,
"EnableDeveloper": false,
"EnableIncomingWebhooks": true,
"EnableTutorial": true,
"DisableLegacyMFA": true,
"SessionLengthSSOInDays": 30,
"TLSOverwriteCiphers": [],
"AllowedUntrustedInternalConnections": "",
"UseLetsEncrypt": false,
"EnablePostIconOverride": false,
"ManagedResourcePaths": "",
"CorsExposedHeaders": "",
"WebsocketURL": "",
"DebugSplit": false,
"EnableEmojiPicker": true,
"ExtendSessionLengthWithActivity": false,
"ConnectionSecurity": "",
"EnableSecurityFixAlert": true,
"RestrictCustomEmojiCreation": "all",
"EnableOnlyAdminIntegrations": true,
"CorsAllowCredentials": false,
"GoogleDeveloperKey": "",
"DisableBotsWhenOwnerIsDeactivated": true,
"LetsEncryptCertificateCacheFile": "./config/letsencrypt.cache",
"ExperimentalDataPrefetch": true,
"SplitKey": "",
"PostEditTimeLimit": -1,
"EnableMultifactorAuthentication": true,
"TLSStrictTransportMaxAge": 63072000,
"EnableChannelViewedMessages": true,
"TLSKeyFile": "",
"EnableUserStatuses": true,
"TrustedProxyIPHeader": [
"X-Forwarded-For",
"X-Real-IP"
],
"MaximumLoginAttempts": 5,
"EnablePreviewFeatures": true,
"EnableAPIChannelDeletion": false,
"EnableAPIUserDeletion": false,
"EnableCommands": true,
"EnableOAuthServiceProvider": false,
"AllowCorsFrom": "",
"CorsDebug": false,
"ExperimentalGroupUnreadChannels": "disabled",
"AllowEditPost": "always",
"EnableBotAccountCreation": false,
"Forward80To443": false,
"ImageProxyType": "",
"ReadTimeout": 300,
"EnforceMultifactorAuthentication": false,
"EnableOutgoingWebhooks": true,
"LocalModeSocketLocation": "/var/tmp/mattermost_local.socket",
"SiteURL": "https://teamwork.tiedt-online.com",
"WebsocketPort": 80,
"SessionLengthMobileInDays": 90,
"EnableCustomEmoji": false,
"ClusterLogTimeoutMilliseconds": 2000,
"EnableOpenTracing": false,
"EnableEmailInvitations": false,
"MinimumHashtagLength": 3,
"EnableAPITeamDeletion": false,
"EnablePostUsernameOverride": false,
"EnableUserAccessTokens": false,
"ExperimentalStrictCSRFEnforcement": false,
"EnableUserTypingMessages": true,
"EnableInsecureOutgoingConnections": false,
"SessionCacheInMinutes": 10,
"TLSCertFile": "",
"WebsocketSecurePort": 443,
"ExperimentalChannelOrganization": false,
"CloseUnusedDirectMessages": false,
"ExperimentalEnableAuthenticationTransfer": true,
"EnableLinkPreviews": false,
"TLSMinVer": "1.2",
"SessionIdleTimeoutInMinutes": 43200,
"EnableAWSMetering": false,
"EnableLocalMode": false,
"GoroutineHealthThreshold": -1,
"TLSStrictTransport": false,
"WebserverMode": "gzip",
"ExperimentalEnableHardenedMode": false,
"IdleTimeout": 60
},
"RateLimitSettings": {
"VaryByRemoteAddr": true,
"MaxBurst": 100,
"PerSec": 10,
"Enable": false,
"VaryByUser": false,
"VaryByHeader": "",
"MemoryStoreSize": 10000
}
}
parse error: Expected value before ',' at line 1, column 13869
I must have a break, because i must sleep now.
Thanks so far…
When I copy this file on my computer and open it, I don’t like the way it looks. I have only one line up to the invalid character (column 13871) then every argument has its own line. LF seems to be missing before “MemoryStoreSize”: 10000.
And there is no coma after the one } before the last
don’t ask me why; around line 56 ( just under “CustomerType”: “A9”
) json seems to be corrupt
“GlobalRelaySettings”: {
“SMTPServerTimeout”: 1800,
“SmtpPassword”: “”,
“EmailAddress”: “”,
“SmtpUsername”: “”,
“CustomerType”: “A9”
}
},
“ComplianceSettings”: {
EDIT:
After further investigation this is not the issue
Thanks for the help, I need to go for the time being. See you tomorrow!
I can reproduce the errors when I move away my /opt/mattermost/config/config.json
and do a signal-event nethserver-mattermost-update
.
Then I get a config.json with just 1 line and similar error and the service does not start anymore.
With following steps it worked again:
Backup config.json:
mv /opt/mattermost/config/config.json ~
Get a working config.json from config.json.rpmnew
cp /opt/mattermost/config/config.json.rpmnew /opt/mattermost/config/config.json
Reconfigure mattermost:
signal-event nethserver-mattermost-update
Hello Markus
Hope you don’t mind me chiming in here. I too was having the same issues with Mattermost today. Upgraded earlier on and it wouldn’t start back up. Same issue lines in the config.json seemed to be corrupted.
Your instructions worked for me
Backup config.json:
mv /opt/mattermost/config/config.json ~
Get a working config.json from config.json.rpmnew
cp /opt/mattermost/config/config.json.rpmnew /opt/mattermost/config/config.json
Reconfigure mattermost:
signal-event nethserver-mattermost-update
All is backup and running.
Many Thanks
And
You’re welcome, many thanks for your feedback.
Same from me, many thanks for the help!!
Thank you for reporting but I can’t reproduce.
This is not the correct way to reproduce: config.json
file is never empty on a new installation nor during and update.
But seems at least 3 servers faced the same issue, I suspect a concurrence problem while the event is changing the config file.
Could you please report what was the nethserver-mattermost
version before the update?
Hello all
This is going to have to be quick as I’m just on my way out of the door.
I think this maybe permissions related as thinking back it happened before and I found the solution on Github (I think)
I think it had to do with running the initial Mattermost web interfaced config/setup under a non-admin account?
I’ll try and track down the info
Kind Regards
And
Sorry, I don’t know how to formulate better. It was a way for me to get a similar error and to get a starting point for finding a solution.
I assume it was version 1.6.0-1 because it happened after a recent update on the same day so I assume the people having the issues are updating regularly.
Than you, I suspected the same thing.
I’ve just update 4 more production server without issue
This seems an Heisenbug …
Hi @ All,
i solved the problem with the solution of mrmarkuz. Many thanks for this.
Regards…
Uwe