Load tests show that channelstore.GetMember and
channelstore.GetMembersForUser are among the chief
queries that take up CPU in the DB.
In this PR, we attempt some strategic optimizations to
reduce/optimize calls to channelstore.GetMember
1. Optimize `(a *App) HasPermissionToChannel`
We replace GetChannelMember with GetAllChannelMembersForUser
because it's cache backed. So although it gets more data,
it does not hit the DB and saves some latency.
2. Optimize getPostsForChannelAroundLastUnread
We repace getChannelMember with getChannelMemberOnly
which is a lite version of the store call which queries
just the ChannelMembers table. This is because
in the app layer, we just use the LastViewedAt attribute.
Therefore, there is no reason to join with 5 tables when
a single table can do the job.
3. Optimize publishWebsocketEventForPermalinkPost
We use GetAllChannelMembersById instead of GetChannelMembersPage
which again joins with a lot of other tables.
4. Optimize countMentionsFromPost
Again, we replace GetChannelMember which is a costly call joining
multiple tables, with GetAllChannelMembersNotifyPropsForChannel
which is cache-backed and gives us just what we need in the app
layer - notify props.
```release-note
Make small optimizations in several DB calls:
- App.HasPermissionToChannel
- getPostsForChannelAroundLastUnread
- publishWebsocketEventForPermalinkPost
- countMentionsFromPost
```
https://mattermost.atlassian.net/browse/MM-55295
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* [MM-52445] Fix double url encoding of oauth redirect URI params
* Additional test based on code review
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* MM-54238 Initial implementation
* MM-54238 Move websocket hook into app package
* MM-54238 Add tests for mentions in posted websocket messages
* Fix styling
* Fix other styling
* Idiomatic ID naming for new code
* Fix more styles
* Separate hooks to add mentions and followers
* Improved error handling for invalid types in hooks
* Rename HasChanges to ShouldProcess
* Pass broadcast hooks through hubStart
* Add test helper for asserting json unmarshaling
* Fix missing arguments in tests
* Ensure broadcast hooks are sent across the cluster and not to users
* Ensure tests actually cover following a post
* Fix code broken by merge
* Go vet again...
* Deep copy event before processing it with hooks
* Replace RemoveBroadcastHooks with WithoutBroadcastHooks
* Address feedback
* Add helper to fix type information for hook args
* Wrap WebSocketEvent and simplify BroadcastHook
* Address feedback
* Address feedback
* update session roles from user roles.
* update so user is not retrieved again
* return error, rather than log warning
* Update session.go
Fix bad merge
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Marshalling a json.RawMessage is not zero overhead. Instead,
it compacts the raw message which starts to have an overhead
at scale.
https://github.com/golang/go/issues/33422
Since we have full control over the message constructed, we
can simply write the byte slice into the network stream.
This gives considerable performance boost.
```
goos: linux
goarch: amd64
pkg: github.com/mattermost/mattermost/server/public/model
cpu: Intel(R) Core(TM) i5-8265U CPU @ 1.60GHz
│ old.txt │ new_2.txt │
│ sec/op │ sec/op vs base │
EncodeJSON-8 1640.5n ± 2% 289.6n ± 1% -82.35% (p=0.000 n=10)
│ old.txt │ new_2.txt │
│ B/op │ B/op vs base │
EncodeJSON-8 528.0 ± 0% 503.0 ± 0% -4.73% (p=0.000 n=10)
│ old.txt │ new_2.txt │
│ allocs/op │ allocs/op vs base │
EncodeJSON-8 5.000 ± 0% 4.000 ± 0% -20.00% (p=0.000 n=10)
```
P.S. No concerns over changing the model API because we are
still using 0.x
https://mattermost.atlassian.net/browse/MM-54998
```release-note
Improve websocket event marshalling performance
```
* Make use of GenericStoreResult in channels/app/channel.go
* Make use of GenericStoreResult in channels/app/channel.go
* Make use of GenericStoreResult in channels/app/channel.go
* Make use of GenericStoreResult in channels/app/channel.go
* Make use of GenericStoreResult in channels/app/channel.go
* Make use of GenericStoreResult in channels/app/channel.go
* Make use of GenericStoreResult in channels/app/channel.go
---------
Co-authored-by: fazil-syed <fazil@Gojo>
Co-authored-by: Mattermost Build <build@mattermost.com>
* Fix mark as unread on GMs
* Don't count own messages in gms when marking as unread
* Change argument name
* Rename userId
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* MM-54201 Move ExplicitMentions to its own file and rename it (#24932)
* MM-54201 Move ExplicitMentions to its own file and rename it
* Fix vet
* MM-54201 Refactor current mention parsing into MentionParserStandard (#24936)
* MM-54201 Refactor current mention parsing into MentionParserStandard
* Fix vet
* MM-54201 Unify user and group mention parsing logic (#24937)
* MM-54201 Add MentionKeywords type
* MM-54201 Move group mentions into MentionKeywords
* Fix flaky test caused by random iteration order
* Update server/channels/app/mention_results.go
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
* Address feedback
---------
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
---------
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
* feature: implemented basic MessageWillBeConsumed hook and applied on GetSinglePost method.
* use hook in get methods
* chore: refactored hook usage and created utils functions to apply hook
* bugfix: single post not updating
* chore: adjusted hook to return post
* chore: reverted some uneeded changes
* chore: updated hook to accept slice of posts
* bugfix: slice filled with niil values
* chore: MessageWillBeConsumed ranamed to MessagesWillBeConsumed
* Update plugin/hooks.go
Co-authored-by: Jesse Hallam <jesse@thehallams.ca>
* Add feature flag
* Update min version
* update tests to account for feature flag
* fix linting issues
---------
Co-authored-by: Matej Topolovac <>
Co-authored-by: mtopolovac <43346061+mtopolovac@users.noreply.github.com>
Co-authored-by: Jesse Hallam <jesse@thehallams.ca>
Co-authored-by: Kevin Hsieh <kevinh@qrypt.com>
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
I was looking at the high number of user_updated ws events
and ran across this piece of code. Had a hard time digesting it,
thought of adding a comment for people to understand it more
clearly.
```release-note
NONE
```
* MM-54640 Add API to get multiple emojis by name at once
* Fix status code when too many names are requested
* Address feedback
* Update unit tests
* Fix styling
* Fix more styling
* Fix mismatched i18n id
* fix openApi vetting
The underlying mattermost-govet tool effectively hasn't been called for some time, as we weren't checking out and building the spec files to pass. Now that hte API is in the monorepo, build it locally and check against it.
Unfortunately, our API documentation isn't up-to-date, and this PR isn't fixing that. For now, add a discrete `make vet-api` and workflow that won't block the build until the API documentation is back in sync and can be merged into the existing `make vet` directive.
* mattermost-govet: use upstream@new
* fix missing /api/v4 prefix for commands autocomplete suggestion
* document /api/v4/ldap/users/{user_id}/group_sync_memberships
* document /api/v4/groups/{group_id}/restore
* fix /files/{file_id}/public actually at root
* document /api/v4/users/invalid_emails
* fix SetThreadUnreadByPostId
* Revert "fix SetThreadUnreadByPostId"
This reverts commit b16bcc80442b4391c3e7472ce47b9da65809697b.
* Revert "Revert "fix SetThreadUnreadByPostId""
This reverts commit 8bda05dc8aa8c57b20e2fd525431b43e21474859.
* workaround undocumented API endpoints
* remove unnecessary whiteline
* ignore go tool output
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* POC page_load_context header
* refactor to make this.pageLoadContext a private field, and relocate setTimeout code
* add pageLoadContext to `ObserveAPIEndpointDuration` method
* move to telemetry_actions
* move PageContext constant in webapp
* give server control of possible page_load_context values
* variable name change
---------
Co-authored-by: mickmister <mickmister>
Co-authored-by: Mattermost Build <build@mattermost.com>
* Add origin device to ObserveAPIEndpointDuration
* Fix generation of einterfaces mocks
* make einterfaces-mocks
* Use request's query and headers to get origin
* Add desktop to the origin device identification
* Test originDevice function
* Rename origin device to origin client
* [MM-53428] Delete empty drafts on upsert
* Add migrations to fix existing drafts
* Fix CI
* Delete empty drafts entirely from the DB
* Fix lint
* Implement batch migration for deleting drafts
* Missing store layers
* Add updated mock
* Remove unnecessary test
* PR feedback
* Add check for cluster migration
* Fix MySQL
* Don't check for len<2
* Bit of PR feedback
* Use query builder for parameters
* PR feedback
* More PR feedback
* Merge'd
* unit test GetLastCreateAtAndUserIdValuesForEmptyDraftsMigration
* simplified builder interface
* fix DeleteEmptyDraftsByCreateAtAndUserId for MySQL
* rework as batch migration worker
* fix typo
* log ip address on version mismatches too
* simplify reset semantics
* remove trace log in favour of low spam
* document parameters for clarity
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Jesse Hallam <jesse.hallam@gmail.com>
