During v6 indexing changes, we replaced
idx_posts_root_id with idx_posts_root_id_delete_at.
This causes MySQL to trigger the index_merge path
again with PRIMARY and idx_posts_root_id_delete_at
as shown below:
```
mysql> UPDATE Posts SET DeleteAt = 1637998911685, UpdateAt = 1637998911685, Props = JSON_SET(Props, '$.deleteBy', 'buqskqrwmjnhfuqskqrwmjn4ca') Where Id = 'q38uaydtpink5f4wkmcsn8h47o' OR RootId = 'q38uaydtpink5f4wkmcsn8h47o';
Query OK, 9 rows affected (17.29 sec)
Rows matched: 10 Changed: 9 Warnings: 0
mysql> EXPLAIN UPDATE Posts SET DeleteAt = 1637998911685, UpdateAt = 1637998911685, Props = JSON_SET(Props, '$.deleteBy', 'buqskqrwmjnhfuqskqrwmjn4ca') Where Id = 'q38uaydtpink5f4wkmcsn8h47o' OR RootId = 'q38uaydtpink5f4wkmcsn8h47o'\G
*************************** 1. row ***************************
id: 1
select_type: UPDATE
table: Posts
partitions: NULL
type: index_merge
possible_keys: PRIMARY,idx_posts_root_id_delete_at
key: idx_posts_root_id_delete_at,PRIMARY
key_len: 107,106
ref: NULL
rows: 9
filtered: 100.00
Extra: Using sort_union(idx_posts_root_id_delete_at,PRIMARY); Using where; Using temporary
1 row in set, 1 warning (0.00 sec)
```
To fix the temporary sort, we order by Id
```
mysql> UPDATE Posts SET DeleteAt = 1637998911686, UpdateAt = 1637998911686, Props = JSON_SET(Props, '$.deleteBy', 'buqskqrwmjnhfuqskqrwmjn4ca') Where Id = 'q38uaydtpink5f4wkmcsn8h47o' OR RootId = 'q38uaydtpink5f4wkmcsn8h47o' ORDER BY Id;
Query OK, 9 rows affected (0.01 sec)
Rows matched: 9 Changed: 9 Warnings: 0
mysql> EXPLAIN UPDATE Posts SET DeleteAt = 1637998911686, UpdateAt = 1637998911686, Props = JSON_SET(Props, '$.deleteBy', 'buqskqrwmjnhfuqskqrwmjn4ca') Where Id = 'q38uaydtpink5f4wkmcsn8h47o' OR RootId = 'q38uaydtpink5f4wkmcsn8h47o' ORDER BY Id\G
*************************** 1. row ***************************
id: 1
select_type: UPDATE
table: Posts
partitions: NULL
type: index_merge
possible_keys: PRIMARY,idx_posts_root_id_delete_at
key: idx_posts_root_id_delete_at,PRIMARY
key_len: 107,106
ref: NULL
rows: 9
filtered: 100.00
Extra: Using sort_union(idx_posts_root_id_delete_at,PRIMARY); Using where; Using filesort
1 row in set, 1 warning (0.00 sec)
```
Postgres uses Bitmap Heap scan which does
a Bitmap OR of the index tuples and _then_
fetches the rows from the heap. This is a
much better and sophisticated way. Sadly, MySQL
will fetch the rows from the indexes first,
and then do an OR, which is why index_merge_intersection
is so bad.
See: https://developers.mattermost.com/blog/mysql-index-merge/
for more info.
```release-note
NONE
```
* Migrate Save from gorp to sqlx
* Migrate checkTeamsExist from gorp to sqlx
* Migrate checkChannelsExist from gorp to sqlx
* Migrate Patch from gorp to sqlx
* Migrate buildGetPoliciesQuery from gorp to sqlx
* Migrate Get from gorp to sqlx
* Change buildGetPolicyQuery return values
* Migrate GetAll from gorp to sqlx
* Migrate GetCount from gorp to sqlx
* Migrate Delete from gorp to sqlx
* Migrate GetChannels from gorp to sqlx
* Migrate GetChannelsCount from gorp to sqlx
* Migrate AddChannels from gorp to sqlx
* Migrate RemoveChannels from gorp to sqlx
* Migrate GetTeams from gorp to sqlx
* Migrate GetTeamsCount from gorp to sqlx
* Migrate AddTeams from gorp to sqlx
* Migrate RemoveTeams from gorp to sqlx
* Migrate DeleteOrphanedRows from gorp to sqlx
* Migrate GetTeamPoliciesForUser from gorp to sqlx
* Migrate GetTeamPoliciesCountForUser from gorp to sqlx
* Migrate GetChannelPoliciesForUser from gorp to sqlx
* Migrate GetChannelPoliciesCountForUser from gorp to sqlx
* Migrate Delete from gorp to sqlx
* Add mapper tag function before saving a record
Summary:
Since there is a db tag in this model for the ID column,
I set a mapper in this transaction to lowercase
all mapping columns before saving the record.
* Use quoted identifiers when selecting Id
* Address PR comments, replace variable declarations for their short hand syntax
* Enhance subquery implementation
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Add playbooks related permissions.
* Add RolesGrantPermission to pluginapi
* Fixing scopes.
* New defaults.
* Fix defaults
* Fix tests.
* Fix migration.
* More test and migration fixes.
* Need to add everything to system admin too.
* Move to 63
* Feedback fixes.
* Fix system manager editing playbook permissions.
* MM-36862: removes participant from thread
Removing a participant upon last reply deleted from thread didn't work
reliably, a suspect on this is the replica lag, since we are first
deleting the post and then counting non-deleted posts of the participant
to decide on whether to delete or not.
The findings that led to this conclusion is that the reply count gets
updated but the participant is not removed (participant removal depends
on the number of replies this participant has in the thread.)
This commit fixes that by removing first the participant and then
deleting the post.
So we delete the participant if they have 1 post in that thread, and
then we delete the post, so now they have no posts in the thread.
* Makes deleting posts transactional
This commit makes deleting a post transactional and also tries to fix
permanent deletion of posts.
Currently when we permanently delete all posts by a user we don't update
the threads reply count nor the participant's array. This commit tries
to fix that.
* Adds comments on deleting posts
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-39634: Migrated gorp to sqlx
* MM-39634: Removed redundant limit on get since only one value is returned
* MM-39485: Fixed returned errors + goimports
Co-authored-by: Alex <alex@rumandcode.io>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Summary
The addition of the TotalMsgCountRoot and MsgCountRoot columns to support CRT caused several issues with previously read threads and channels being marked as unread. Previously we attempted to fix this purely in a SQL migration [MM-35345][MM-35494] fixes for incorrect mentions and unreads for threads and channels #17803 but that turned out to be too heavy and it was decided to break up some of the fixes into async jobs.
This PR implements an async job to mark channels as read if there are no user posts since the last time the user viewed the channel.
Ticket Link
https://mattermost.atlassian.net/browse/MM-37013
* Show private channels in autocomplete
This is supported in all Engines:
MySQL, Postgres, Bleve, Elasticsearch.
https://mattermost.atlassian.net/browse/MM-18496
```release-note
Private channels will now appear in channel autocomplete.
If you are using Bleve or ElasticSearch, you will have to reindex
the channels again to populate them with the new attributes.
```
A large chunk of this work has been based on the earlier
effort at https://github.com/mattermost/mattermost-server/pull/17804.
Full credit goes to https://github.com/arvinDarmawan.
* Add comment
```release-note
NONE
```
* Adding more tests
```release-note
NONE
```
* fix more tests
```release-note
NONE
```
* tmp
```release-note
NONE
```
* more fixes
```release-note
NONE
```
* add tests
```release-note
NONE
```
* Add review comments from previous PR
```release-note
NONE
```
* Add API to return all channels from all team
```release-note
NONE
```
* Added support for bleve and ES
```release-note
NONE
```
* Streaming response for GetAllChannels
```release-note
NONE
```
* Fix tests
```release-note
NONE
```
* Trigger CI
```release-note
NONE
```
* fix tests
```release-note
NONE
```
* Addressing review comments
```release-note
NONE
```
* Fix lint
```release-note
NONE
```
* Removing flaky test
```release-note
NONE
```
* Address comments
```release-note
NONE
```
* Trigger CI
```release-note
NONE
```
* Added /users/<userid>/channel_members endpoint
```release-note
NONE
```
* Minor edit
```release-note
NONE
```
* Improve embedding
```release-note
NONE
```
* Fix lint error
```release-note
NONE
```
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Migrated from gorp to sqlx in store/sqlstore/plugin_store.go
* fixed lint
* updated QueryRowx and keys type
* used QueryRowX
* Fix query
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
* [MM-39232] Migrate from gorp to sqlx in store/sqlstore/reaction_store.go
* Fix failing mysql tests
* Fix failing postgres specs
* Use struct instead of params
