* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w`
* added goimports lint check to .golangci.yml
* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w` for a corner case
* make app-layers, *-mocks and store-layers for ci check
Co-authored-by: Mahmudul Haque <mahmudulhaque@protonmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-31370: Fix racy TestClusterShutdownRace
This race wasn't specific to this test, but somehow got triggered by it.
This was a critical race because the PluginsLock is widely used
throughout the codebase to gate access to the PluginsEnvironment.
However, a config listener can often run in its own goroutine leading to
a wide variety of races.
https://mattermost.atlassian.net/browse/MM-31370
```release-note
NONE
```
* incorporate review comments
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-31063: Change constants to use CamelCase
* store package
* change allcaps to camel case (#16615)
* New tools.mod
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* Added log to print golang version before running Server
* Minor fix
* Review Fixes
* Review Fixes - 2
* Update app/server.go
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
* MM-31356: Add a minimum required version check for Postgres
To keep conformance with our failing fast and obvious philosophy,
we add a check to prevent Mattermost server from starting
if the postgres version is below 10.0.
This gives customers a chance to upgrade their database before upgrading
their Mattermost version, than to run into weird compatibility issues
after they have finished the upgrade.
https://mattermost.atlassian.net/browse/MM-31356
```release-note
NONE
```
* fix lint errors
* Use a function to pretty-print version string
* rectify comment
* Remove usages of AppError on filesstore service
* Fixing a golint error
* Fixing shadowed variable
* Adding err.Error() to the NewAppError calls
* Fixing tests
* Adding missed translations
* Fix error handling and updating the translation that affects it
* Fixing two typos
* Removing supplier concept from the sql store
* Removing other metions to supplier
* Fixing gofmt
* Fixing gofmt
* Renaming NewSqlStore to New
* Fixing tests
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-30988 - Fix racy test ServerSystemdNotification
The translateFunc is a global variable which was unguarded.
So we convert that into an atomic variable to prevent against races.
https://mattermost.atlassian.net/browse/MM-30988
```release-note
NONE
```
* fix tests
* move call upwards
* use race
* after translations init
* guard with a sync.Once
* missed import
* revert race
* update timeout
A cluster changed listener always runs in a separate goroutine.
This causes a race condition when a new member gets added in the cluster.
We fix it by accessing the value atomically.
I tried to write a test for this, but it wasn't triggering at all.
The best way is to just start the server with cluster enabled in race mode,
and verify that the race doesn't happen.
https://mattermost.atlassian.net/browse/MM-30319
```release-notes
NONE
```
MM-27744 disable Zap for unit tests.
Zap has no concept of shutdown or close. Zap is only shutdown when the app exits. Not a problem for console logging, but when creating a new Zap logger that outputs to files on every unit test, that leaves no easy way to clean up until process exit. Depending on what else is running this can exhaust all file handles and cause unit tests to fail.
Zap is now disabled unit tests and uses Logr instead, regardless of config settings. `make test-server` peak file handle usage dropped from ~5K to less than 100.
* Moving diagnostics into a service
* Fixing golint checks
* Fixing tests
* Renaming from diagnostics to telemetry
* Adding missing files
* Initializing telemetry earlier in the server startup
* Fixing tests
* Adding a log for the telemetryID initialization error
* Addressing PR review comments
* Fixing merge problem
* Removing some extra Diagnostics mentions
* Making tests pass
* Adding Upgrade to Enterprise version feature
* Addressing PR review comments, and adding some minor improvements
* Add tests file
* Addressing PR comments
* fix linter checks
* Storing and exposing the upgraded from TE info
* Fix showing errors on mac
* A more appropiate status code for not-supported upgrade
* Fixing tests
* Handling permissions errors
* More server logging around upgrade failures
* Apply text changes suggested from code review
Co-authored-by: Eric Sadur <57730300+esadur@users.noreply.github.com>
* Address PR review comments
* Only allow to restart the system after an upgrade
* Verify file signature before upgrade
* Adding limit to the downloaded file
* Simplifying the upgrade binary process with backup in memory
* Fixing backup/restore mechanism for the binary file
* Improve file permissions handling
* Askin the permissions for the right place (the parent directory)
* Fixing tests
* Addressing PR review comments
* Fix license headers
* Fixing retry layer
* Making it work on windows builds
* Adding license header
* Fixing 2 tests
* Fixing tests that need UpgradeFromTE System key mock
* Extracting i18n translation
* Apply suggestions from code review
Co-authored-by: Eric Sadur <57730300+esadur@users.noreply.github.com>
* Improving how the errors are written
* Fixing another error text
* Removing unneeded translation
* Fixing upgrade status strings
* Update i18n/en.json
Co-authored-by: Eric Sadur <57730300+esadur@users.noreply.github.com>
* Fixing tests
Co-authored-by: Eric Sadur <57730300+esadur@users.noreply.github.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* To allow for ease of testing telemetry changes, we should make it so that the rudder key and dataplane URL can be customized through the config or environment.
* Instead of using a real config element that would be exposed to end users, we'll just use 'secret' environment variables to inject Rudder config data.
Co-authored-by: Alex Dovenmuehle <alex.dovenmuehle@mattermost.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-26575: Move app initialization inside Websocket router out of handler
The websocket router struct is shared amongst multiple handlers. Therefore,
there is a race condition while setting the app field and reading from it.
We move the initialization of the app field when the router struct is initialized.
And we also remove the initializations of some app fields which caused race
conditions by being written from multiple goroutines. They are already being
written once inside the AppInitializedOnce.Do method and it's redundant
to set them again to the same value.
* Add missing fields in server connector
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-27116: Drain push notification channel before closing
Without this, pending push notifications will not be sent when shutting down
or restarting the server.
* Fix race
* Re-arrange server shutdown
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Adds the advanced logging config for audit. Existing support for auditing to a single file remains for E0 and E10 licenses instances, and a new config item ExperimentalAuditSettings.AdvancedLoggingConfig is added that behaves like LogSettings.AdvancedLoggingConfig.
Supported destinations:
- file
- syslog (with out without TLS)
- raw TCP socket (with out without TLS)
ExperimentalAuditSettings.AdvancedLoggingConfig can contain a filespec to a config file, a database DSN, or JSON.
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
Adds Advanced Logging to server. Advanced Logging is an optional logging capability that allows customers to send log records to any number of destinations.
Supported destinations:
- file
- syslog (with out without TLS)
- raw TCP socket (with out without TLS)
Allows developers to specify discrete log levels as well as the standard trace, debug, info, ... panic. Existing code and logging API usage is unchanged.
Log records are emitted asynchronously to reduce latency to the caller. Supports hot-reloading of logger config, including adding removing targets.
Advanced Logging is configured within config.json via "LogSettings.AdvancedLoggingConfig" which can contain a filespec to another config file, a database DSN, or JSON.
* Extracting email functions into a service
* Fixing two shadowing errors
* Address PR review comments
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-24972: make GetHubForUserId lock-less and zero-alloc
The hubs aren't meant to be modified after the server starts up.
But the key problem was the SetHubs method which would zero out
the underlying hubs slice.
This wasn't ideally necessary because an hub would only be stopped
once during server shutdown. However, this was required by a test
which would shutdown the hub twice. And since the hub shutdown isn't
idempotent, zeroing the slice would just skip over shutting down the hub again.
To improve the overall situation, we apply several optimizations.
- We use the new hash/maphash package which exposes Go runtime's
internal hash algorithms to be used as a package. This is much faster
than hash/fnv.
- We move around the initialization of the hub to happen before
the metrics server starts. This allows us to initialize the hub
before any of the hub elements are being accessed.
- To make the test run successfully, we do not call th.TearDown.
This is fine for a test, because anyways the test process would eventually stop
and relinquish the resources to the OS.
This allows us to completely remove any mutexes and thereby
we can remove all the methods and any edge-case checks related to
index being out of bounds. As a result, the fast path becomes
very straightforward and zero-alloc.
name old time/op new time/op delta
GetHubForUserId-8 116ns ± 1% 38ns ± 7% -67.22% (p=0.000 n=10+10)
name old alloc/op new alloc/op delta
GetHubForUserId-8 36.0B ± 0% 0.0B -100.00% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
GetHubForUserId-8 2.00 ± 0% 0.00 -100.00% (p=0.000 n=10+10)
Manually tested with some load testing and running Hub tests in -race mode.
* remove mutex
* incorporate review comments
* MM-25404: Add GetHealthScore metric for the cluster
This PR adds the necessary function to the cluster interface
so that they can be called from App code.
* Add mocks
* Remove fakeapp
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Removing some other fake apps
* More FakeApp removed
* Removing entirely FakeApp
* Fixing some tests
* Fixing get Cluster id from get plugin status
* Fixing failing tests
* Fixing tests
* Fixing test initialization for web
* Fixing InitServer for server tests
* Fixing InitServer for server tests
* Reverting go.sum and go.mod
* Removing unneded HTMLTemplates function in App layer
* Moving back some functions to its old place to easy the review
* Moving back some functions to its old place to easy the review
* Using the last struct2interface version
* Generating store layers
* Fixing merge problems
* Addressing PR comments
* Small fix
* Fixing app tests build
* Fixing tests
* fixing tests
* Fix tests
* Fixing tests
* Fixing tests
* Fixing tests
* Moving license to server struct
* Adding some fixes to the test compilation
* Fixing cluster and some jobs initialization
* Fixing some license tests compilation problems
* Fixing recursive cache invalidation
* Regenerating app layers
* Fix test compilation
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
