* diagnostics_test.go: fix spacing
* diagnostics_test.go: explicitly assert payload
This fails, since the package is currently receiving struct pointers and won't set the MessageId or Timestamp on the corresponding Message.
* MM-18115: fix segment v3 usage
In v5.14, we updated [github.com/segmentio/analytics-go](https://github.com/segmentio/analytics-go) to v3 as part of https://mattermost.atlassian.net/browse/MM-12389. As noted in the [migration guide](https://segment.com/docs/sources/server/go/#migrating-from-v2), the API subtly changed to expect a struct value and not a struct pointer:
```go
// in v2, you would call the `Track` method with a `Track` struct.
client.Track(&track)
// in v3, you would call the `Enqueue` method with a `Track` struct.
// Note that a pointer is not used here.
client.Enqueue(track)
```
Unfortunately, we kept passing a pointer, and the package didn't complain since it only required an interface -- which the pointer to these structs still implemented. Internally, it only checked for the value types, and failed to annotate our payloads with the requisite metadata. Upstream, segment.io accepted the payload, but then discarded it silently.
This has since been reported and fixed in https://github.com/segmentio/analytics-go/pull/146, but isn't yet part of a tagged release of the package.
Fix our code to pass struct values instead.
Fixes: MM-18115
* revert 4e5f6fcfbc
This reverts the configuration setting to disable plugin health checks, preferring instead to retain this functionality for advanced cases. In part, this was driven by the discovery that the health checks were failing on Windows, though that will be addressed separately.
Fixes: MM-16378
* simplify interval handling/logging
Also shutdown the job when plugins are altogether disabled.
The job workers and cluster discovery components require access to the
database during shutdown. To avoid issues, we leave the database open
until the last step of the shutdown process.
Properly handle analytics client life cycle
This change improves the life cycle management of the analytics
client. It does so in two primary ways:
- Move the client from being a shared pointer to belonging to the
Server struct.
- Expose the client.Close() method for proper shutdown of the
client.
* Updated ResetAll function to return object
* Modified the defined interface
* Changed the data type where ResetAll was referrenced
* Refactored code
Changed user's email
* Fixed indentation
* Add config setting to explicitly define which IP headers are trusted
* fix variable shadowing
* Optimize code flow; Add Ratelimit test for header set
* Extend Ratelimit tests
* Add additional unit tests
* Structured logging
* MM-15494 Adding warning log when someone has configured a Mattermost push proxy with HTTP endpoint
* MM-15494 go fmt check
* MM-15494 Making function private
* MM-15494 Updating warning message
* Remove NotificationRegistry table and use structured logging
* Fix ackId for notification sent
* Notification logger at server level
* Remove unused i18n strings
* introduce plugin health check
* implement plugin health check job
* add support for checking pid of plugin process and RPC ping, to determine a plugin's health
* implement restart policy with back-offs
* support "EnableHealthCheck" boolean from config file.
* add tests for supervisor.PerformHealthCheck() and shouldDeactivatePlugin()
* improve error handling. clean up if blocks to be more concise
* [MM-10199] Printing stack traces from http handlers panics in mattermost logs
Previously recovery handlers from gorilla mux was used, which was printing stack traces from panics in stderr
Removing it should print stack traces into logger passed to http.Server, which as of now is created from mlog
Refer #10351
* Logging errors from http server at error level instead of info level
Added test for verifying whether panic gets logged in server logger
* Fixed failing test due to tls error
* Closing temp file for panic log before removing and checking errors for closure and removal
* Added regular log line in panic handler for TestPanicLog to check whether it gets logged
* Added support for PostActions in ephemeral posts
The general approach is that we take all the metadata that DoPostAction
needs to process client DoPostActionRequests, and store it in a
serialized, encrypted Cookie field, in the PostAction struct.
The client then must send it back, and it is then used to process
PostActions as a fallback top the metadata in the database.
This PR adds a new config setting, `ServiceSettings.ActionCookieSecret`.
In a cluster environment it must be the same for all instances.
- Added type PostActionCookie, and a Cookie string to PostAction.
- Added App.AddActionCookiesToPost.
- Use App.AddActionCookiesToPost in api4.createEphemeralPost,
App.SendEphemeralPost, App.UpdateEphemeralPost.
- Added App.DoPostActionWithCookie to process incoming requests with
cookies. For backward compatibility, it prefers the metadata in the
database; falls back to cookie.
- Added plugin.API.UpdateEphemeralPost and plugin.API.DeleteEphemeralPost.
- Added App.encryptActionCookie/App.decryptActionCookie.
* Style
* Fixed an unfortunate typo, tested with matterpoll
* minor PR feedback
* Fixed uninitialized Context
* Fixed another test failure
* Fixed permission check
* Added api test for DoPostActionWithCookie
* Replaced config.ActionCookieSecret with Server.PostActionCookieSecret
Modeled after AsymetricSigningKey
* style
* Set DeleteAt in DeleteEphemeralPost
* PR feedback
* Removed deadwood comment
* Added EXPERIMENTAL comment to the 2 APIs in question
* Rerouted the unit test logs through t.Log
* resolving merge confilvts
* Update testing.go
* Update helper_test.go
* Added godocs for NewTestingLogger
* Added go docs for NewTestingLogger
* Resolving conflicts
* config file store
Introduce an interface and concrete implementation for accessing the config.
This mostly maps 1:1 with the exiting usage in `App`, except for internalizing the watcher. A future change will likely eliminate `App.PersistConfig()` and make this implicit on `Set` or `Patch`
* experimental file test changes
* emoji: move file driver checks from api4 to app
It is no longer possible to app.UpdateConfig and provide an invalid configuration, making it hard to test this case. This check doesn't really belong in the api anyway, since it's a configuration validity check and not a permissions check. Either way, the check now occurs at the App level.
* api4: generate valid public link salts for test
* TestStartServerRateLimiterCriticalError: use mock store to test invalid config
* remove config_test.go
* remove needsSave, and have Load() save to the backing store as necessary
* restore README.md
* move ldap UserFilter check to model isValid checks
* remove databaseStore until ready
* remove unimplemented Patch
* simplify unlockOnce implementation
* revert forgetting to set s.Ldap
* config/file.go: rename ReadOnlyConfigurationError to ErrReadOnlyConfiguration
* config: export FileStore
* add TestFileStoreSave
* improved config/utils test coverage
* restore config/README.md copy
* tweaks
* file store: acquire a write lock on Save/Close to safely close watcher
* fix unmarshal_test.go
* refactor utils/config* to config/
* pull validateLdapFilter into app
* clean up Config/GetConfig/GetSanitizedConfig usage
Eliminate app.GetConfig() in favour of just using app.Config() directly,
but expose app.GetSanitizedConfig() for when the old behaviour was
required.
* web: isolate config setup
* TestInvitePeopleProvider: make config explicit
* regenerateClientConfig: avoid racey map access
* integrate watch flag into app.ConfigFile option
* make app.Option return an error
* release.mk: only cp static files from config/
* release.mk: fix cp static files from config/
* api4: TestPlugin cleanup
* s/c/cfg/ for clarity
* fix merge conflict
* testlib: allow customization of testlib driver name
* MM-10417 Add local image proxy and enable by default
* Remove unused function
* Add dependencies for willnorris/imageproxy
* Fixed compilation errors
* Lock to the master version of willnorris/imageproxy
* Fix atmos/camo proxy when no SiteURL is specified
* Re-add default values for deprecated settings
* Fix unit tests added by merge
* Pass imageproxy to App struct
* Remove unneeded locking when creating the image proxy
* Remove empty test file
* MM-10856: deduplicate posts with the same pending post id
Leverage a fixed size cache with a window of 30 seconds to deduplicate posts received by a single app server. Clients that duplicate the same pending post id will see one request potentially delayed until the first finishes, after which the same payload should be returned by both. Duplicate posts outside the 30 second window will not be de-duplicated.
Note that the cache is not synchronized between app servers. In an HA cluster consisting of more than one app server, sticky load balancing (e.g. hashing by IP or Session ID) is required to route the users to the same app instance for multiple requests.
Other options considered for this feature:
* adding a column to the `Posts` table: rejected as being too heavyweight
* maintaining a `PendingPostIds` table: similarly rejected for the database impact
* using the pending post id as the post id and relying on the unique constraints on the Post table: rejected for being difficult to show that it's safe to use a client-provided value as the row identifier
* utils/lru: simplify to ttl internally and for new methods
* move seenPendingPostIdsCache to App.Server
* just fail concurrent post requests, vs. trying to wait
* add debug log when create post is deduplicated
* guard app plugins with mutex
Shutting down the app could race with a goroutine that uses the plugins environment, since we shut down the plugins first before cleaning up goroutines.
* fix go vet issues
* Moving goroutine pool
* Auto refactor
* Moving plugins.
* Auto refactor
* Moving fields to server
* Auto refactor
* Removing siteurl duplication.
* Moving reset of app fields
* Auto refactor
* Formatting
* Moving niling of Server to after last use
* Fixing unit tests.
Config Checks at StartUp Part1
Config Checks; Tests for TLS Server
HSTS header implementation + tests
make gofmt happy with new go version...
make gofmt happy with new go version #2...
fix logic bug
fix typo
Fix unnecessary code block
* factor out GetSubpathFromConfig
* mv web/subpath.go to utils/subpath.go
* serve up web, api and ws on /subpath if configured
* pass config to utils.RenderWeb(App)?Error
This allows the methods to extract the configured subpath and redirect
to the appropriate `/subpath/error` handler.
* ensure GetSubpathFromConfig returns trailing slashes deterministically
* fix error 404 handling
* redirect /subpath to /subpath/
This is necessary for the static handler to match, otherwise none of the
registered routes find anything. This also makes it no longer necessary
to add trailing slashes in the root router.
* Implementing structured logging
* Changes to en.json to allow refactor to run.
* Fixing global logger
* Structured logger initalization.
* Add caller.
* Do some log redirection.
* Auto refactor
* Cleaning up l4g reference and removing dependancy.
* Removing junk.
* Copyright headers.
* Fixing tests
* Revert "Changes to en.json to allow refactor to run."
This reverts commit fd8249e99bcad0231e6ea65cd77c32aae9a54026.
* Fixing some auto refactor strangeness and typo.
* Making keys more human readable.
* If Forward80To443 is true, but not configured to listen on 443, fail to start the server with an error message.
* If Forward80To443 is false and LetsEncrypt is true, fail to start the server with an error message.
Only a handful of critical errors are present in the codebase.
They all occur during server startup (in `app.StartServer()`).
Currently, when one of these critical error occurs, it is simpled
mentionned in the logs – then the error is discarded, and the app
attempts to continue the execution (and probably fails pretty quickly in
a weird way).
Rather than continuing operations in an unknow state, these errors should
trigger a clean exit.
This commit rewrites critical startup errors to be correctly
propagated, logged, and then terminate the command execution.
Additionnaly, it makes the server return a proper error code to the
shell.
