* MM-30882: Fix read-after-write issue for demoting user
In (*App).DemoteUserToGuest, we would demote a user, and then immediately
read it back to do future operations from the user. This reading back
of the user had the effect of sticking the old value into the cache
after which it would never be updated.
There was another issue along with this, which was when the invalidation
message would broadcast across the cluster, it would hit the cache invalidation
problem where an unrelated store call would miss the cache because
it was invalidated, and then again read from replica and stick the old value.
To fix all these, we return the new value directly from the store method
to avoid having the app to read it again.
And we add a map in the localcache layer which tracks invalidations made,
and then switch to use master if it's true.
The core change is fairly limited, but due to changing the store method signatures,
a lot of code needed to be updated to pass "context.Background". Therefore the PR
just "appears" to be big, but the main changes are limited to app/user.go,
sqlstore/user_store.go and user_layer.go
https://mattermost.atlassian.net/browse/MM-30882
```release-note
Fix an issue where demoting a user to guest would not take effect in
an environment with read replicas.
```
* Fix concurrent map access
* Fixing mistakes
* fix tests
During user creation via CLI, we would create the user,
but pass the user ID instead when updating the roles.
This falls into the category of read-after-write within a single request.
We fix this by passing the already created user and
directly update the roles.
https://mattermost.atlassian.net/browse/MM-32471
```release-note
NONE
```
* Include filepaths for post attachments
* Cleanup
* Enable exporting file attachments
* Fix file import
* Enable zip export
* Support creating missing directories when unzipping
* Add test
* Add translations
* Export direct channel posts attachments
* Fix returned values order
Remove pointer to slice in return
* [MM-31597] Implement export process job (#16626)
* Implement export process job
* Add translations
* Remove unused value
* [MM-31249] Add /exports API endpoint (#16633)
* Implement API endpoints to list, download and delete export files
* Add endpoint for single resource
* Update i18n/en.json
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* Update i18n/en.json
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* Fix var name
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* MM-5639: Write panic to log on the way out
We introduce a panic pass-through layer which logs
the panic at a critical level before crashing.
This helps customers to easily get the panic output
in their logging infrastructure, rather than having to
go through journaltctl logs.
https://mattermost.atlassian.net/browse/MM-5639
```release-note
Server crashes due to runtime panics are now captured
as a log line.
```
* re-arrange panic order
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w`
* added goimports lint check to .golangci.yml
* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w` for a corner case
* make app-layers, *-mocks and store-layers for ci check
Co-authored-by: Mahmudul Haque <mahmudulhaque@protonmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-31063: Change constants to use CamelCase
* store package
* change allcaps to camel case (#16615)
* New tools.mod
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
This change addresses a few issues where config stores were not
properly closed when an error was encountered on server startup.
This could result in leaked database connections when dealing with
a database config store.
* call InitServer() to initialize Enterprise for cli
* only call initEnterprise
* back to InitServer()
* move below license load
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
MM-27744 disable Zap for unit tests.
Zap has no concept of shutdown or close. Zap is only shutdown when the app exits. Not a problem for console logging, but when creating a new Zap logger that outputs to files on every unit test, that leaves no easy way to clean up until process exit. Depending on what else is running this can exhaust all file handles and cause unit tests to fail.
Zap is now disabled unit tests and uses Logr instead, regardless of config settings. `make test-server` peak file handle usage dropped from ~5K to less than 100.
* add warning as success result
* revert config changes
* add unit test, update unit test
* add a couple more tests
* update store layers
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-27507: Propagate rate limit errors to client
We return an error from SendInviteEmails instead of just logging it
to let the client know that a rate limit error has happened.
The status code is chosen as 413 (entity too large) instead of 429 (too many requests)
because it's not the request which is rate limited, but the payload inside it which is.
Ideally, the email sending should have been implemented by a queue which would just return
an error to the client when full. That is also why we are not returning an X-Retry-After
and X-Reset-After in the headers because that would mix with the actual rate limiting.
A separate header X-Email-Invite-Reset-After might do the job, but it comes at an extra cost
of additional API surface and a clunky API. Instead, that information is contained in the error
response. The web client needs to just surface the error. An API client will have to do
a bit more work to parse the error if it needs to automatically know when to retry. Given that
an email sending client is not a very common use case, we decide to keep the API clean.
This decision can be revisited if it becomes problematic in the future.
https://mattermost.atlassian.net/browse/MM-27507
* Fixing translations
* Added retry_after and reset_after in API response.
* MM-27178: use FileSettings.Directory in export
* Set empty FileSettings.Directory to default value
* Validate that FileSettings.Directory is non-empty
* Don't set FileSettings.Directory to / in fixConfig
* Run make i18n-extract
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Fix direct/group channel false positives
* Move public structures to model package
* Expose CheckIntegrity as a local API method
* Remove extra file
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Extracting email functions into a service
* Fixing two shadowing errors
* Address PR review comments
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Removing some other fake apps
* More FakeApp removed
* Removing entirely FakeApp
* Fixing some tests
* Fixing get Cluster id from get plugin status
* Fixing failing tests
* Fixing tests
* Fixing test initialization for web
* Fixing InitServer for server tests
* Fixing InitServer for server tests
* Reverting go.sum and go.mod
* Removing unneded HTMLTemplates function in App layer
* Moving back some functions to its old place to easy the review
* Moving back some functions to its old place to easy the review
* Using the last struct2interface version
* Generating store layers
* Fixing merge problems
* Addressing PR comments
* Small fix
* Fixing app tests build
* Fixing tests
* fixing tests
* Fix tests
* Fixing tests
* Fixing tests
* Fixing tests
* Moving license to server struct
* Adding some fixes to the test compilation
* Fixing cluster and some jobs initialization
* Fixing some license tests compilation problems
* Fixing recursive cache invalidation
* Regenerating app layers
* Fix test compilation
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* make Group.Name a pointer to allow null
* fix unit tests
* fix build error
* fix unit test
* ensure Name field not nil
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* [MM-20684] Initial implementation of the Command Autocomplete (#13602)
* Implement Autocomplete Data
* Change CommandName to Trigger
* Fix Autocomplete test
* Make stylistic changes
* Rename a bunch of fields and methods
* Fix variable names, safer type assertions
* [MM-20684] plugin autocomplete implementation (#14259)
* Add an endpoint for command autocomplete suggestions
* Add full Suggestion to the AutocompleteSugestion struct
* Add Dynamic Argument support
* Tidy up things
* Fix missed test case
* Add support of the named arguments
* Update autocomplete API
Fix review issues
Implement dynamic args as a local request
* Fix ineffassign
* Add support of the uppercase letters in arguments
* Add support of the optional arguments
* Remove ineffectual assignment
* Add support for icons (#14489)
* Address couple of nits
* Add comment to IconData
* Add types to all consts
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* dont allow message export for non E20 if export to csv acadiance or global relay
* add tests
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* [MM-24146] Add unix socket listener for mmctl local mode (#14296)
* add unix socket listener for mmctl local mode
* add a constant for local-mode socket path
* reflect review comments
* [MM-24401] Base approach for Local Mode (#14333)
* add unix socket listener for mmctl local mode
* First working PoC
* Adds the channel list endpoint
* Add team list endpoint
* Add a LocalClient to the api test helper and start local mode
* Add helper to test with both SystemAdmin and Local clients
* Add some docs
* Adds TestForAllClients test helper
* Incorporating @ashishbhate's proposal for adding test names to the helpers
* Fix init errors after merge
* Adds create channel tests
* Always init local mode to allow for enabling-disabling it via config
* Check the RemoteAddr of the request before marking session as local
* Mark the request as errored if it's local and the origin is remote
* Set the socket permissions to read/write when initialising
* Fix linter
* Replace RemoteAddr check to ditch connections with the IP:PORT shape
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* Fix translations order
* [MM-24832] Migrate plugin endpoints to local mode (#14543)
* [MM-24832] Migrate plugin endpoints to local mode
* Fix client reference in helper
* [MM-24776] Migrate config endpoints to local mode (#14544)
* [MM-24776] Migrate get config endpoint to local mode
* [MM-24777] Migrate update config endpoint to local mode
* Fix update config to bypass RestrictSystemAdmin flag
* Add patchConfig endpoint
* MM-24774/MM-24755: local mode for addLicense and removeLicense (#14491)
Automatic Merge
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Ashish Bhate <bhate.ashish@gmail.com>
* MM-24135: Migrate AppError from SaveChannel/channel_store.go
This is the first POC of migration of store app errors to plain error.
We create a few basic error types in the store package and use
them to return the errors from store methods. In the app layer,
we inspect the error and re-create the exact app errors. This lets
us preserve the same error content, but yet move to plain errors.
Since this is a gradual migration, this means that the error inspection
code will be duplicated across the app layer whenever a store method
is invoked. But all of that should go away once we start propagating
the errors higher up the hierarchy.
There have been a significant amount of changes in the storetest and searchtest
layer, primarily because we have to rename the err variable now that it is of
a different type.
* Addressed review comments
* MM-24132: Migrate AppError from SaveDirectChannel/channel_store.go
This PR migrates 2 new methods SaveDirectChannel and CreateDirectChannel
to return error instead of AppError.
We also need to handle the error internally in SaveMultipleMember for now
until that is migrated too.
* MM-24133: Migrate AppError from bot_store.go
* Fix errors
* Fix err
* Fix bad return
* Fix vet errors
* Fix incorrect error check
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* add warning count as return value
* add warning count as return value
* fix file name
* update mock
* add setting warning to db
* replace wrongly removed string
* add dummy function to see if it will build
* remove dummy function
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
Add auditing to server CLI.
Also:
- simplify auditing in API layer
- reduce number of AddMeta calls
- have models serialize themselves
- more consistent field naming
- replace all instances of `fmt.Sprintf("whatever")` with simply "whatever".
- replace all instances of `fmt.Println(fmt.Sprintf("whatever %d", i))` with `fmt.Printf("whatever %d\n", i)`
* MM-22212: Read non-moderated permissions from higher-scoped scheme.
* MM-2212: Corrects test count in comment.
* MM-22212: Adds godoc comment.
* MM-2212: Switches to the channel roles check in a few more places.
* MM-22212: Refactors and fixes.
* MM-22212: Reverts change, no longer required.
* MM-22212: Removes translation.
* MM-22212: Un-comments merged new permission.
* MM-22212: Un-comments merged new permission.
* MM-22212: Performance tweak.
* MM-22212: Fixes some fmting.
* MM-22212: Add unit test for newly-added store methods.
* MM-22212: Renames app method.
* MM-22212: Re-uses existing function to find string in slice.
* MM-22212: Keeps 'higher-scoped' terminology for consistency.
* MM-22212: Refactors based on PR feedback.
* MM-22212: Fix for some bad merging.
* MM-22212: Renamed some things.
* MM-22212: Use an 'else' instead of a 'continue' for readability.
* MM-22212: Caches (*SqlRoleStore).ChannelRolesUnderTeamRole.
* MM-22212: Adds mock to new cache store.
* MM-22212: Adds missing open tracing app layer methods.
* MM-22212: Adds migration to add moderated permissions to channel_admin if present on channel_user.
* MM-22212: Migrates team schemes. Removes unused AppError.
* MM-22212: Fix for for if.
* MM-22212: Fixes iterator.
* MM-22212: Updates open tracing generated methods.
* MM-22212: Fix mocks.
* MM-22212: Change migration key name.
* MM-22212: Switched to data structure from other branch.
* MM-22212: Fixes tests after adding 'use_channel_mentions' to the channel_admin role.
* MM-22212: Adds tracking of channel moderation.
* Revert "MM-22212: Adds tracking of channel moderation."
This reverts commit 23689efa22c112e4ba37f6a212535dd7ebfb63db.
* MM-22212: Switch some functions to methods and vice versa.
* MM-22212: Fix for refactor bug not notifiying websocket about changed role.
* MM-22212: Adds test for public/private 'manage_members' handling.
* MM-22122 Fix manage channel members edge case for public and private channels (#14049)
* MM-22212: Adds moderated permission to team_admin.
* MM-22212: Updates migration.
* MM-22212: Revert unnecessary update to default roles.
* Add channel scheme updated event when channel scheme is deleted or created (#14057)
* MM-22212: Adds newline.
* MM-22212: Migration fix.
* MM-22212: Fix for migration.
* MM-22212: Test fix.
Co-authored-by: Farhan Munshi <3207297+fm2munsh@users.noreply.github.com>
