* Extracting html templates into a library
* Moving tests to the right place
* Fixing tests
* Addressing PR review comments
* Addressing PR review comments
* Replacing attomic with RWMutex
* Returning errors as channel for Templates watcher
* Address PR review comments
* Other small fixes
* Simplifying NewWithWatcher
* Addressing PR review comments
* Making error handling on rendering templates more robust
* Fixing tests
* Changing how we return errors
* Fixing shadow variables
* Addressing PR review comments
* Logging errors from the outside of sendNotificationEmail
* Fixing lock in shutdown
* Fixing the resource copy for commands tests temporary directories
* Removing unused import
* A couple of tiny fixes
* app/server: add pprof endpoint
* reflect review comments
* make metrics link conditional
* app/server: add metrics server setup to licence listeners
* refactor a bit
* trigger CI
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Include user data in the cloud endpoints
Those headers will include the user ID and Email so we can use them
in CWS
* Removed AppError from enterprise/cloud
We're removing the AppError from all the places that don't belong
to the app or api4 packages.
* Remove unused i18n strings
* Move it to the server init of enterprise
Also moved the initialization of the enterprise part in the server after the store is initialized
* Initialize after the store is set in NewServer
The ideal way to do it should be to move the initEnterprise call after
the store is set but that would lead to undesired side-effects so we
initialize the cloud part alone.
Signed-off-by: Mario de Frutos <mario@defrutos.org>
* Reduce the coupling of the mailservice with the rest of the application
* Fixing tests in CI
* Simplifiying mailservice config
* Addressing PR review comments
* Fixing tests
* Removing unnecesary type definition
* Fixing ServerName usage
* Reducing the filestore dependencies from the rest of the source code
* Making more generic config conversion to FileBackendSettings
* Fixing usage of the NewFileBackend function
* Fixing more usages of the NewFileBackend function
* Fix some linter errors
* Fix more linter errors
* Fixing some unit tests
* Fixing linter problem
* Addressing PR review comments
* Simplifing the CopyFile for tests
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-21012: Revamp websocket implementation
We replace the old gorilla/websocket implementation with the
gobwas/ws library. The gorilla library was in maintenance mode
and had a high level API due to which we cannot use that for
situations where a large number of concurrent connections needs
to be supported.
The ws library is a very low-level library that allows us
to work with raw net.Conns. We make several improvements:
- We completely remove the reader goroutines, and instead
replace them with a manual epoll implementation which sends off
messages to be read when it receives any data on the connection.
This lets us scale to a much larger number of connections.
- The reader buffer is eliminated, because we directly read
from the connection now.
https://mattermost.atlassian.net/browse/MM-21012
```release-notes
Improved the websocket implementation by using epoll manually
to read from a websocket. As a result, the number of goroutines
is expected to go down by half.
```
* fix tests
* fix shadowing errors
* final changes
* windows support!
* Remove pointer to waitgroup
* Fix edge case
* Trigger CI
* Trigger CI
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* init commit
* clean up the code
* make mocks
* fix translations
* mocks and lint fixes
* add tests
* little fixes
* Update i18n/en.json
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
* Update i18n/en.json
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
* Update i18n/en.json
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
* Update i18n/en.json
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
* Update i18n/en.json
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
* Address Comments
* fix i18n
* update api endpoint
* add enable file and file level for conditional show of banner
* Address Comments
* Make it more clear about returns
* Create zip file utility function
* update en.json
* address comments
* write tests
* check for data in test
* remove warning string
* Correct expected and actual
* set database through environment variables
* reset environment variable at end of test
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
* docker: prometheus and grafana for development
Add prometheus and grafana docker configuration when developing with access to the enterprise repository. This simplifies the setup for developers adding new metrics or even examining events from their development server. The services are disabled by default when no enterprise source is detected.
Grafana is provisioned automatically with the Prometheus datasource, as well as the canonical dashboards used with Mattermost. Furthermore, no authentication is required to access Grafana to simplify access from a development environment. The default home dashboard is customized to automatically show links to the provisioned dashboards. Dashboards can be saved, and login remains available via the default `admin/admin` credentials (which initiates a password reset), but no dashboard or system configuration is (currently) persisted if the container is destroyed.
Linux requires slightly special handling (well, really it's Docker for MacOS/Windows), in that `host.docker.internal` is the canonical way to resolve the host on MacOS/Windows, but on Linux it's usually sufficient to just use `localhost`. Until https://github.com/docker/for-linux/issues/264 is resolved, this PR includes code to customize the Prometheus configuration to point at the required address for the running platform.
* metrics: track active jobs by type
* metrics: active jobs chart
* metrics: server start and job annotations
* Update build/docker-compose.common.yml
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
* prometheus: resolve docker host via 172.17.0.1 instead
* skip enabling prometheus and grafana by default
* handle JOB_STATUS_ERROR and JOB_STATUS_CANCELED end states
* handle nil srv.metrics
* lookup job to determine type for metrics
* mocked unit tests for jobs
* goimports lint fixes
* missing license
* add instance to server start tag
* filter annotations by selected instance
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w`
* added goimports lint check to .golangci.yml
* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w` for a corner case
* make app-layers, *-mocks and store-layers for ci check
Co-authored-by: Mahmudul Haque <mahmudulhaque@protonmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-31370: Fix racy TestClusterShutdownRace
This race wasn't specific to this test, but somehow got triggered by it.
This was a critical race because the PluginsLock is widely used
throughout the codebase to gate access to the PluginsEnvironment.
However, a config listener can often run in its own goroutine leading to
a wide variety of races.
https://mattermost.atlassian.net/browse/MM-31370
```release-note
NONE
```
* incorporate review comments
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-31063: Change constants to use CamelCase
* store package
* change allcaps to camel case (#16615)
* New tools.mod
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* Added log to print golang version before running Server
* Minor fix
* Review Fixes
* Review Fixes - 2
* Update app/server.go
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
* MM-31356: Add a minimum required version check for Postgres
To keep conformance with our failing fast and obvious philosophy,
we add a check to prevent Mattermost server from starting
if the postgres version is below 10.0.
This gives customers a chance to upgrade their database before upgrading
their Mattermost version, than to run into weird compatibility issues
after they have finished the upgrade.
https://mattermost.atlassian.net/browse/MM-31356
```release-note
NONE
```
* fix lint errors
* Use a function to pretty-print version string
* rectify comment
* Remove usages of AppError on filesstore service
* Fixing a golint error
* Fixing shadowed variable
* Adding err.Error() to the NewAppError calls
* Fixing tests
* Adding missed translations
* Fix error handling and updating the translation that affects it
* Fixing two typos
* Removing supplier concept from the sql store
* Removing other metions to supplier
* Fixing gofmt
* Fixing gofmt
* Renaming NewSqlStore to New
* Fixing tests
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-30988 - Fix racy test ServerSystemdNotification
The translateFunc is a global variable which was unguarded.
So we convert that into an atomic variable to prevent against races.
https://mattermost.atlassian.net/browse/MM-30988
```release-note
NONE
```
* fix tests
* move call upwards
* use race
* after translations init
* guard with a sync.Once
* missed import
* revert race
* update timeout
A cluster changed listener always runs in a separate goroutine.
This causes a race condition when a new member gets added in the cluster.
We fix it by accessing the value atomically.
I tried to write a test for this, but it wasn't triggering at all.
The best way is to just start the server with cluster enabled in race mode,
and verify that the race doesn't happen.
https://mattermost.atlassian.net/browse/MM-30319
```release-notes
NONE
```
MM-27744 disable Zap for unit tests.
Zap has no concept of shutdown or close. Zap is only shutdown when the app exits. Not a problem for console logging, but when creating a new Zap logger that outputs to files on every unit test, that leaves no easy way to clean up until process exit. Depending on what else is running this can exhaust all file handles and cause unit tests to fail.
Zap is now disabled unit tests and uses Logr instead, regardless of config settings. `make test-server` peak file handle usage dropped from ~5K to less than 100.
* Moving diagnostics into a service
* Fixing golint checks
* Fixing tests
* Renaming from diagnostics to telemetry
* Adding missing files
* Initializing telemetry earlier in the server startup
* Fixing tests
* Adding a log for the telemetryID initialization error
* Addressing PR review comments
* Fixing merge problem
* Removing some extra Diagnostics mentions
* Making tests pass
