* [MM-20684] Initial implementation of the Command Autocomplete (#13602)
* Implement Autocomplete Data
* Change CommandName to Trigger
* Fix Autocomplete test
* Make stylistic changes
* Rename a bunch of fields and methods
* Fix variable names, safer type assertions
* [MM-20684] plugin autocomplete implementation (#14259)
* Add an endpoint for command autocomplete suggestions
* Add full Suggestion to the AutocompleteSugestion struct
* Add Dynamic Argument support
* Tidy up things
* Fix missed test case
* Add support of the named arguments
* Update autocomplete API
Fix review issues
Implement dynamic args as a local request
* Fix ineffassign
* Add support of the uppercase letters in arguments
* Add support of the optional arguments
* Remove ineffectual assignment
* Add support for icons (#14489)
* Address couple of nits
* Add comment to IconData
* Add types to all consts
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-25006: Fix occurences of unclosed http bodies
And while here, we also use ioutil.Discard to read the remaining
body instead of reading with ioutil.ReadAll which allocates a separate
byte buffer.
* Fix mistake
* Address review comments
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* [MM-20979] Add first implementation of the Bleve search engine
* Fix i18n
* Migrate searchengine utils tests
* Fix linter
* Don't add allTermsQ if both termQueries and notTermQueries are empty
* Fix test that should work if user is system admin
* Modify naming according to review comments
* Abstract getIndexDir function
* Extracting bleve engine name as a constant
* Merge both Indexer interfaces into one
* Add worker stopped message
* Allow worker to be started/stopped with config change
* Use constants for index names
* Modify test order
* Fix linter
* Trying to unlock the CI
* [MM-24146] Add unix socket listener for mmctl local mode (#14296)
* add unix socket listener for mmctl local mode
* add a constant for local-mode socket path
* reflect review comments
* [MM-24401] Base approach for Local Mode (#14333)
* add unix socket listener for mmctl local mode
* First working PoC
* Adds the channel list endpoint
* Add team list endpoint
* Add a LocalClient to the api test helper and start local mode
* Add helper to test with both SystemAdmin and Local clients
* Add some docs
* Adds TestForAllClients test helper
* Incorporating @ashishbhate's proposal for adding test names to the helpers
* Fix init errors after merge
* Adds create channel tests
* Always init local mode to allow for enabling-disabling it via config
* Check the RemoteAddr of the request before marking session as local
* Mark the request as errored if it's local and the origin is remote
* Set the socket permissions to read/write when initialising
* Fix linter
* Replace RemoteAddr check to ditch connections with the IP:PORT shape
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* Fix translations order
* [MM-24832] Migrate plugin endpoints to local mode (#14543)
* [MM-24832] Migrate plugin endpoints to local mode
* Fix client reference in helper
* [MM-24776] Migrate config endpoints to local mode (#14544)
* [MM-24776] Migrate get config endpoint to local mode
* [MM-24777] Migrate update config endpoint to local mode
* Fix update config to bypass RestrictSystemAdmin flag
* Add patchConfig endpoint
* MM-24774/MM-24755: local mode for addLicense and removeLicense (#14491)
Automatic Merge
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Ashish Bhate <bhate.ashish@gmail.com>
* MM-24176: Regenerate invite ID on team privacy change.
* Fix the other place this setting can be changed.
* Fix tests.
* Satisfy the tyrannical golangci-lint.
* MM-24135: Migrate AppError from SaveChannel/channel_store.go
This is the first POC of migration of store app errors to plain error.
We create a few basic error types in the store package and use
them to return the errors from store methods. In the app layer,
we inspect the error and re-create the exact app errors. This lets
us preserve the same error content, but yet move to plain errors.
Since this is a gradual migration, this means that the error inspection
code will be duplicated across the app layer whenever a store method
is invoked. But all of that should go away once we start propagating
the errors higher up the hierarchy.
There have been a significant amount of changes in the storetest and searchtest
layer, primarily because we have to rename the err variable now that it is of
a different type.
* Addressed review comments
* MM-24132: Migrate AppError from SaveDirectChannel/channel_store.go
This PR migrates 2 new methods SaveDirectChannel and CreateDirectChannel
to return error instead of AppError.
We also need to handle the error internally in SaveMultipleMember for now
until that is migrated too.
* MM-24133: Migrate AppError from bot_store.go
* Fix errors
* Fix err
* Fix bad return
* Fix vet errors
* Fix incorrect error check
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-24135: Migrate AppError from SaveChannel/channel_store.go
This is the first POC of migration of store app errors to plain error.
We create a few basic error types in the store package and use
them to return the errors from store methods. In the app layer,
we inspect the error and re-create the exact app errors. This lets
us preserve the same error content, but yet move to plain errors.
Since this is a gradual migration, this means that the error inspection
code will be duplicated across the app layer whenever a store method
is invoked. But all of that should go away once we start propagating
the errors higher up the hierarchy.
There have been a significant amount of changes in the storetest and searchtest
layer, primarily because we have to rename the err variable now that it is of
a different type.
* Addressed review comments
* MM-24132: Migrate AppError from SaveDirectChannel/channel_store.go
This PR migrates 2 new methods SaveDirectChannel and CreateDirectChannel
to return error instead of AppError.
We also need to handle the error internally in SaveMultipleMember for now
until that is migrated too.
* Fix layers
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* [MM-23535] Add disable database search flag and return empty posts results if set
* Add UpdateConfig function for the SearchStore and hook it into the app lifecycle
* Add the config listener in the server instance instead of using FakeApp
* Instantiate searchlayer as a pointer to avoid passing around copies of it
* Retry on failed download of plugins from marketplace.
* minor polishing.
* Handling http errors during downloading
Adding unit test for DownloadFromURL
* Adding suggested changes from the PR
Close response body before returning error during progressive retry
Remove separate struct for the downloadURL parameter in download_test
Adding comment to clarify the share retries variable
Changing to NoError and Error in the test assertion.
* Added license header to download_test.go
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-24135: Migrate AppError from SaveChannel/channel_store.go
This is the first POC of migration of store app errors to plain error.
We create a few basic error types in the store package and use
them to return the errors from store methods. In the app layer,
we inspect the error and re-create the exact app errors. This lets
us preserve the same error content, but yet move to plain errors.
Since this is a gradual migration, this means that the error inspection
code will be duplicated across the app layer whenever a store method
is invoked. But all of that should go away once we start propagating
the errors higher up the hierarchy.
There have been a significant amount of changes in the storetest and searchtest
layer, primarily because we have to rename the err variable now that it is of
a different type.
* Addressed review comments
* Made all appError origins to be CreateChannel
* Remove typed internal error
* Fix translations
* fix layer generation
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
We would call SetStatusOnline in a goroutine before actually calling HubRegister.
This could cause the message not to be sent after all, because there's no guarantee
that Register would actually happen before it.
To fix it, we just change the order of things.
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
After registering the conn in the hub, we proceeded to send a
direct message to the user. We had changed it to send the direct message
in the same hub goroutine that handles the registration. This was the correct
behavior and fixes chances of having panics due to sending to closed channels.
However, often fixing something unearths some deeper underlying bug. This was
such a case :)
The issue was that register channel had a buffer size of 1. And we were sending
a direct message after registration. In the code to send direct message, we
were checking if the user has been registered or not, and if not, then skip it.
Therefore, since the register channel buffer was 1, it could very well be that
the select case would pick up the direct message send case first - in which
case it would not have been registered, and therefore no hello message would be sent.
The fix is to unbuffer the register and unregister channels. There does not seem
to be a valid reason to make these buffered channels. They are meant to be
synchronous operations, because the code following them assumes that the user
has been registered.
While here, we also remove all the time.Sleeps before waiting on the Response channel
because they are not required at all. Waiting on a channel is already blocking.
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
* MM-24987 Dont call a.GetSchemeRolesForChannel and instead load the scheme using the channel directly
* Trigger CI
* MM-24987 Add a test for concurrent patch to channel moderation
* MM-24987 Cleanup
* MM-24828 Take into account group synced state of team and channel when getting groups for mentions
* Update app/notification_test.go
* i18n-extract
* Revert "i18n-extract"
This reverts commit dcb0426b98afa4646c26870c0e3a1236f99fda17.
* Trigger CI
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* Disable read/search db replicas in TE/E0
* fixing tests
* Removing unnecesary text.
* Updating without-license read-replicas config before store initialization
* Reconnecting to database after remove read replicas
* MM-23935 extend session expiry on user activity
- if user types anything before a session expires the session will be extended to now + session length
- ensures new session expiries are not written to DB too frequently
- new session store func for updating session ExpiresAt
- session length defaults for mobile and web/ldap changed from 180 days to 30 days
* Fixing system messages about non-visible users
* Adding unit tests to verify the new behavior
* Regenerating app layers
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
We invert the skipSend condition and outdent the remaining block
to make the code a bit more idiomatic.
While here, we also change the dropping message level from info to
warn because that's what it should be.
On user activity, we were clearing the job.pendingNotifications map.
But we had already created a copy of the notifications slice while
iterating the map. Therefore, if we pass the copied slice, it would
still have the old notifications which were originally deleted.
The unit tests would not catch this because it was testing the
job.pendingNotifications map and not actually checking if the email
handler was being called or not. We fix that now.
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-23800: remove goroutineID and stack printing
Each hub has a goroutineID which is calculated with a known hack.
The FAQ clearly explains why goroutines don't have an id:
https://golang.org/doc/faq#no_goroutine_id.
We only added that because sometimes the hub would be deadlocked and
having the goroutineID would be useful when getting the stack trace.
This is also problematic in stress tests because the hubs would
frequently get overloaded and the logs would unnecessarily have stack traces.
But that was in the past, and we have done extensive testing with
load tests and fuzz testing to smooth any rough edges remaining.
Including adding additional metrics for hub buffer size.
Monitoring the metrics is a better way to approach this problem.
Therefore, we remove these kludges from the code.
* Also remove deadlock checking code
There is no need for that anymore since
we are getting rid of the stack printing anyways.
Let's do a wholesale refactor and clean up the codebase.
* MM-23805: Refactor web_hub
This is a beginning of the refactoring of the websocket code.
To start off with, we unexport some methods and constants which did not
need to be exported. There are more remaining but some are out of scope for this PR.
The main chunk of refactor is to unexport the webconn send channel
which was the main cause of panics. Since we were directly sending
to the connection from various parts of the codebase, it would be possible
that the send channel would be closed and we could still send a message.
This would crash the server.
To fix this, we refactor the code to centralize all sending from the main
hub goroutine. This means we can leverage the connections map to check
if the connection exists or not, and only then send the message.
We also move the cluster calls to cluster.go.
* bring back cluster code inside hub
* Incorporate review comments
* Address review comments
* rename index
* MM-23807: Refactor web_conn
- Unexport some struct fields and constants which are not necessary
to be accessed from outside the package. This will help us moving
the entire websocket handling code to a separate package later.
- Change some empty string checks to check for empty string rather
than doing a len check which is more idiomatic. Both of them compile
to the same code. So it doesn't make a difference performance-wise.
- Remove redundant ToJson calls to get the length.
- Incorporate review comments
- Unexport some more methods
* Fix field name
* Run make app-layers
* Add note on hub check
The current code path for `CreatePostAsUser` tries to update the `LastViewedAt` for bots, which logs a warning message if the bot isn't actually in the channel.
This pull request changes the semantics of bot posting to not update the bot's `LastViewedAt` timestamp, avoiding this log altogether. It matches the semantics of `from_webhook`, but notably makes bots slightly less like users in that they no longer "read" channels when they post. This seems reasonable, but I'm both looking for validation of this semantic change in addition to the code review.
Fixes: https://mattermost.atlassian.net/browse/MM-23926
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-23800: remove goroutineID and stack printing
Each hub has a goroutineID which is calculated with a known hack.
The FAQ clearly explains why goroutines don't have an id:
https://golang.org/doc/faq#no_goroutine_id.
We only added that because sometimes the hub would be deadlocked and
having the goroutineID would be useful when getting the stack trace.
This is also problematic in stress tests because the hubs would
frequently get overloaded and the logs would unnecessarily have stack traces.
But that was in the past, and we have done extensive testing with
load tests and fuzz testing to smooth any rough edges remaining.
Including adding additional metrics for hub buffer size.
Monitoring the metrics is a better way to approach this problem.
Therefore, we remove these kludges from the code.
* Also remove deadlock checking code
There is no need for that anymore since
we are getting rid of the stack printing anyways.
Let's do a wholesale refactor and clean up the codebase.
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-23017 Check group mentions as part of notification logic
* Add nil groups to existing test cases
* MM-23017 Add tests for insertGroupMention and addGroupMention
* MM-23017 Add tests for getExplicitMentions that have groups
* Add tests for group store GetMemberUsersNotInChannel
* MM-23017 Add tests for AllowGroupMentions
* MM-23017 Fix error message name
* MM-23017 Swap Checks to Name
* MM-23017 Code review fixes
* Rename var and fix allowGroupMentions test
* MM-23017 Use GetMemberUsersInTeam inside of insertGroupMentions
* MM-23017 use group mentions permission
* Actually call GetMemberUsersInTeam
* Remove unnecessary new line
* Uncomment filter allow reference
* MM-23017 Fix group channel notifications
* Update store layer
* MM-23017 Improve test coverage for group channels
* Trigger CI
* Trigger CI
