Граф коммитов

1191 Коммитов

Автор SHA1 Сообщение Дата
Ibrahim Serdar Acikgoz
ca02fabb91 allow upsert for duplicate key 2021-12-17 11:39:25 +03:00
Ujjwal Sharma
1fc8a2ea20 [MM-39640] Migrate from gorp to sqlx in store/sqlstore/role_store.go (#19022)
Automatic Merge
2021-12-15 17:40:31 +02:00
Yair Facio
3b3ab701b1 [MM-39231] Migrate license store from gorp to sqlx in store/sqlstore/license_store.go (#19146)
Automatic Merge
2021-12-12 08:10:31 +02:00
Yair Facio
62d4d8166d [MM-39636] Migrate from gorp to sqlx in store/sqlstore/retention_policy_store.go (#18994)
* Migrate Save from gorp to sqlx

* Migrate checkTeamsExist from gorp to sqlx

* Migrate checkChannelsExist from gorp to sqlx

* Migrate Patch from gorp to sqlx

* Migrate buildGetPoliciesQuery from gorp to sqlx

* Migrate Get from gorp to sqlx

* Change buildGetPolicyQuery return values

* Migrate GetAll from gorp to sqlx

* Migrate GetCount from gorp to sqlx

* Migrate Delete from gorp to sqlx

* Migrate GetChannels from gorp to sqlx

* Migrate GetChannelsCount from gorp to sqlx

* Migrate AddChannels from gorp to sqlx

* Migrate RemoveChannels from gorp to sqlx

* Migrate GetTeams from gorp to sqlx

* Migrate GetTeamsCount from gorp to sqlx

* Migrate AddTeams from gorp to sqlx

* Migrate RemoveTeams from gorp to sqlx

* Migrate DeleteOrphanedRows from gorp to sqlx

* Migrate GetTeamPoliciesForUser from gorp to sqlx

* Migrate GetTeamPoliciesCountForUser from gorp to sqlx

* Migrate GetChannelPoliciesForUser from gorp to sqlx

* Migrate GetChannelPoliciesCountForUser from gorp to sqlx

* Migrate Delete from gorp to sqlx

* Add mapper tag function before saving a record

Summary:
Since there is a db tag in this model for the ID column,
I set a mapper in this transaction to lowercase
all mapping columns before saving the record.

* Use quoted identifiers when selecting Id

* Address PR comments, replace variable declarations for their short hand syntax

* Enhance subquery implementation

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-12-10 14:57:24 +05:30
Yair Facio
fa38202d98 [MM-39636] Migrate from gorp to sqlx in store/sqlstore/product_notices_store.go (#19012)
* Migrate from gorp to sqlx in ProductNoticesStore

* Address PR comments

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-12-10 14:48:46 +05:30
Christopher Speller
5ec78546e2 MM-38150 Adding playbooks permissions (#19096)
* Add playbooks related permissions.

* Add RolesGrantPermission to pluginapi

* Fixing scopes.

* New defaults.

* Fix defaults

* Fix tests.

* Fix migration.

* More test and migration fixes.

* Need to add everything to system admin too.

* Move to 63

* Feedback fixes.

* Fix system manager editing playbook permissions.
2021-12-07 09:43:33 -08:00
Kyriakos Z
36c8d1d1a0 MM-36862: removes participant from thread (#18795)
* MM-36862: removes participant from thread

Removing a participant upon last reply deleted from thread didn't work
reliably, a suspect on this is the replica lag, since we are first
deleting the post and then counting non-deleted posts of the participant
to decide on whether to delete or not.

The findings that led to this conclusion is that the reply count gets
updated but the participant is not removed (participant removal depends
on the number of replies this participant has in the thread.)

This commit fixes that by removing first the participant and then
deleting the post.
So we delete the participant if they have 1 post in that thread, and
then we delete the post, so now they have no posts in the thread.

* Makes deleting posts transactional

This commit makes deleting a post transactional and also tries to fix
permanent deletion of posts.

Currently when we permanently delete all posts by a user we don't update
the threads reply count nor the participant's array. This commit tries
to fix that.

* Adds comments on deleting posts

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-12-06 12:01:04 +02:00
Agniva De Sarker
6a7067e4b6 Clarify comment on golang-migrate connection (#19104)
The old comment wasn't exactly clear on
why is it needed to spawn a new connection.

```release-note
NONE
```
2021-12-03 14:15:43 +01:00
Ben Cooke
27dacb82ce [MM-38239 & MM-39788] Recent files causing crash (#18942)
* wip

* adding tests for new endpoint

* tool updates

* new function for getting postsByIds

* fixing test

* adding limit of 1000 to post query

* fixing PR comments

* fixing permission logic

Co-authored-by: Collin <collineng@gmail.com>
Co-authored-by: Collin Eng <eng.engineereng@gmail.com>
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MBP.ht.home>
2021-11-26 11:51:32 -05:00
Carlos Tadeu Panato Junior
44d324a45f prepera db upgrade to 6.3 (#19086) 2021-11-26 16:51:25 +01:00
Carlos Tadeu Panato Junior
810edf34a1 upgrade db to 6.2.0 (#19085) 2021-11-26 15:50:09 +01:00
Ashish Bhate
8440a754b4 [MM-37013] Better post filtering (#19013)
Summary:
- Skip searching for deleted posts.

Ticket:
- Part of https://mattermost.atlassian.net/browse/MM-37013
2021-11-23 21:34:29 +05:30
Alex Boutin
3710aae7a1 MM-39635: Migrate gorp to sqlx #18873 (#18945)
Automatic Merge
2021-11-23 07:05:13 +01:00
Ernest Nguyen
47de1f37e6 MM-39632: Migrate from using gorp to sqlx (#18949)
Automatic Merge
2021-11-22 17:05:13 +01:00
Alex Boutin
478b3f7108 MM-39634: Migrated gorp to sqlx (#18951)
* MM-39634: Migrated gorp to sqlx

* MM-39634: Removed redundant limit on get since only one value is returned

* MM-39485: Fixed returned errors + goimports

Co-authored-by: Alex <alex@rumandcode.io>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-11-22 20:58:50 +05:30
Ashish Bhate
0da249c651 [MM-37013] Async job to fix CRT channel unreads (#18340)
Summary
The addition of the TotalMsgCountRoot and MsgCountRoot columns to support CRT caused several issues with previously read threads and channels being marked as unread. Previously we attempted to fix this purely in a SQL migration [MM-35345][MM-35494] fixes for incorrect mentions and unreads for threads and channels #17803 but that turned out to be too heavy and it was decided to break up some of the fixes into async jobs.
This PR implements an async job to mark channels as read if there are no user posts since the last time the user viewed the channel. 

Ticket Link
https://mattermost.atlassian.net/browse/MM-37013
2021-11-18 15:31:18 +05:30
Agniva De Sarker
12dc171a60 MM-39341: Sentry crash: nil dereference in *User.ClearNonProfileFields (#18966)
In some cases, an invalid participant id like an empty string
might creep into the database. It's not exactly clear
how can this happen, but if it does, then it breaks the logic
of assuming that the users slice will contain all users
in the participants slice.

To prevent this, we check if a match was found before
adding it in the slice.

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4/7sixm4t7c6rsr3d8ydxbtfgy9gr

```release-note
NONE
```
2021-11-17 20:49:40 +05:30
Penthaa Patel
397edef1d8 [MM-39637] - Migrate from gorp to sqlx in store/sqlstore/shared_channel_store.go (#18898)
Automatic Merge
2021-11-16 05:40:13 +01:00
Ujjwal Sharma
627d2b9d6c [MM-39628] Migrate from gorp to sqlx in store/sqlstore/scheme_store.go (#18895)
Automatic Merge
2021-11-15 17:40:13 +01:00
Cesar Bretana
2153dc4d6d [MM-39641] Migrate Webhook store to sqlx (#18896)
* Migrated to sqlx INSERT + UPDATE incoming_webhooks

* Finish migrating IncomingWebhook to sqlx

* wip: outgoing webhooks

* Fix sql syntax error for incoming webhooks

* Add missing fields to insert incoming webhooks query

* Migrate outgoing_webhooks store to sqlx

* Migrate incoming_webhooks analytics count to sqlx

* Migrate outgoing_webhooks analytics count to sqlx

* Fix lint goimports alert

* Migrate unmigrated gorp methods

* Add missing fields to update incoming webhooks sql + remove unnecessary guards

* Address comments from review
2021-11-15 21:13:14 +05:30
Penthaa Patel
25b727d197 [MM-39638] - Migrate from gorp to sqlx in store/sqlstore/status_store.go (#18897)
* migrate from gorp to sqlx

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

* fixes

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-11-15 21:12:30 +05:30
Yair Facio
e1eb0a0cf3 [MM-39643] Migrate from gorp to sqlx in store/sqlstore/system_store.go (#18893)
* Migrate from gorp to sqlx in SystemStore

* Add BeginXWithIsolation method to *sqlxDBWrapper

* Address PR comments
2021-11-15 21:03:55 +05:30
Penthaa Patel
bde2b540d6 [MM-39644] - Migrate from gorp to sqlx in store/sqlstore/upload_session_store.go (#18892)
* migrate from gorp to sqlx

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

* fix insert query

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

* replace NamedExec() with Exec()

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

* fix typo

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

* remove string()

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>
2021-11-15 19:40:54 +05:30
Penthaa Patel
ae897bc59a [MM-39642] - Migrate from gorp to sqlx in store/sqlstore/session_store.go (#18891)
* migrate from gorp to sqlx

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

* fix Save function

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

* fix Get function

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

* use json.Marshal() for session.Props

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>
2021-11-15 19:40:37 +05:30
Penthaa Patel
df3ed307df [MM-39639] - Migrate from gorp to sqlx in store/sqlstore/user_access_token_store.go (#18888)
Automatic Merge
2021-11-11 05:40:13 +01:00
Penthaa Patel
c88c9b6625 [MM-39630] - Migrate from gorp to sqlx in store/sqlstore/tokens_store.go (#18887)
* migrate from gorp to sqlx

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

* fixes

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>
2021-11-05 10:00:03 +05:30
Carlos Tadeu Panato Junior
69eb0ca773 db: upgrade db to 6.0 and prepare for 6.1 (#18923)
* db: upgrade db to 6.1

* db: prepare db upgrade to 6.2

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-11-03 16:16:33 +01:00
Agniva De Sarker
5f4199db4f Revert "Optimize TotalUnreadMentions (#18789)" (#18889)
This reverts commit ec56cac42d.
2021-10-29 14:54:11 +05:30
Agniva De Sarker
ec56cac42d Optimize TotalUnreadMentions (#18789)
* Optimize TotalUnreadMentions

We avoid joining with the Threads table entirely
as ChannelId is available from Posts table directly.

Also we replace left joins by inner joins.

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4?c=c9aa17f3-b918-4a78-86aa-01562e418634

Old plan

```
 Finalize Aggregate  (cost=30869.29..30869.30 rows=1 width=32) (actual time=27.216..31.674 rows=1 loops=1)
   ->  Gather  (cost=30869.18..30869.29 rows=1 width=32) (actual time=27.089..31.665 rows=2 loops=1)
         Workers Planned: 1
         Workers Launched: 1
         ->  Partial Aggregate  (cost=29869.18..29869.19 rows=1 width=32) (actual time=18.442..18.445 rows=1 loops=2)
               ->  Nested Loop  (cost=1881.68..29869.11 rows=25 width=8) (actual time=2.052..18.417 rows=72 loops=2)
                     ->  Hash Join  (cost=1881.12..29771.45 rows=25 width=62) (actual time=2.028..17.744 rows=72 loops=2)
                           Hash Cond: ((threads.channelid)::text = (channels.id)::text)
                           ->  Nested Loop  (cost=0.84..27883.40 rows=2963 width=89) (actual time=0.220..15.849 rows=163 loops=2)
                                 ->  Parallel Index Scan using threadmemberships_pkey on threadmemberships  (cost=0.41..7423.65 rows=2963 width=35) (actual time=0.031..3.937 rows=2504 loops=2)
                                       Index Cond: ((userid)::text = 'tc3p1yqw67d8idcp3g98awexqe'::text)
                                       Filter: following
                                 ->  Index Scan using threads_pkey on threads  (cost=0.42..6.91 rows=1 width=54) (actual time=0.004..0.004 rows=0 loops=5007)
                                       Index Cond: ((postid)::text = (threadmemberships.postid)::text)
                           ->  Hash  (cost=1866.60..1866.60 rows=1094 width=27) (actual time=1.763..1.764 rows=1002 loops=2)
                                 Buckets: 2048  Batches: 1  Memory Usage: 74kB
                                 ->  Index Scan using idx_channels_team_id_type on channels  (cost=0.42..1866.60 rows=1094 width=27) (actual time=0.022..1.493 rows=1002 loops=2)
                                       Index Cond: ((teamid)::text = '8ywxyw9ocp8smxrmjzrkqhrdwe'::text)
                     ->  Index Scan using posts_pkey on posts  (cost=0.56..3.91 rows=1 width=27) (actual time=0.009..0.009 rows=1 loops=145)
                           Index Cond: ((id)::text = (threads.postid)::text)
                           Filter: (deleteat = 0)
 Planning Time: 2.415 ms
 Execution Time: 31.798 ms
```

New plan

```
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Finalize Aggregate  (cost=34717.01..34717.02 rows=1 width=32) (actual time=42.375..48.143 rows=1 loops=1)
   ->  Gather  (cost=34716.89..34717.00 rows=1 width=32) (actual time=42.249..48.117 rows=2 loops=1)
         Workers Planned: 1
         Workers Launched: 1
         ->  Partial Aggregate  (cost=33716.89..33716.90 rows=1 width=32) (actual time=30.960..30.962 rows=1 loops=2)
               ->  Hash Join  (cost=1881.25..33716.83 rows=25 width=8) (actual time=2.438..30.780 rows=1058 loops=2)
                     Hash Cond: ((posts.channelid)::text = (channels.id)::text)
                     ->  Nested Loop  (cost=0.98..31828.77 rows=2963 width=35) (actual time=0.091..27.374 rows=2504 loops=2)
                           ->  Parallel Index Scan using threadmemberships_pkey on threadmemberships  (cost=0.41..6815.65 rows=2963 width=35) (actual time=0.034..4.674 rows=2504 loops=2)
                                 Index Cond: ((userid)::text = 'tc3p1yqw67d8idcp3g98awexqe'::text)
                                 Filter: following
                           ->  Index Scan using posts_pkey on posts  (cost=0.56..8.44 rows=1 width=54) (actual time=0.008..0.008 rows=1 loops=5007)
                                 Index Cond: ((id)::text = (threadmemberships.postid)::text)
                                 Filter: (deleteat = 0)
                     ->  Hash  (cost=1866.60..1866.60 rows=1094 width=27) (actual time=2.289..2.289 rows=1002 loops=2)
                           Buckets: 2048  Batches: 1  Memory Usage: 74kB
                           ->  Index Scan using idx_channels_team_id_type on channels  (cost=0.42..1866.60 rows=1094 width=27) (actual time=0.028..1.927 rows=1002 loops=2)
                                 Index Cond: ((teamid)::text = '8ywxyw9ocp8smxrmjzrkqhrdwe'::text)
```

```release-note
NONE
```

* Bring back coalesce

```release-note
NONE
```
2021-10-29 09:07:10 +05:30
Saul Pinales
b877e2a95e Migrate from gorp to sqlx in store/sqlstore/group_store.go (#18543)
Automatic Merge
2021-10-28 20:25:26 +02:00
Agniva De Sarker
685b311401 ThreadStore: Remove quadratic complexity for participants population (#18755)
* ThreadStore: Remove quadratic complexity for participants population

While populating the participants, we would traverse the global list
twice to get the users for that thread. For threads with large number
of users, this can take considerable time. We replace that with a map
to change it to O(n).

While here, we improve memory usage of this code by pre-allocating
slices and maps. And also add some comments to help readers.

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4?c=c9aa17f3-b918-4a78-86aa-01562e418634

```release-note
NONE
```

* Apply in post store as well

```release-note
NONE
```

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-10-28 09:57:54 +05:30
Syerikjan Kh
1091710592 migrate from gorp to sqlx for Preference store (#18742)
Automatic Merge
2021-10-26 16:55:27 +02:00
Ashish Bhate
097d32f409 MM-38445: Set isFollowing to false only when explicitly unfollowed (#18740)
Summary
Set post.IsFollowing to false only if the post is explicitly unfollowed. Otherwise leave it as nil.

Ticket Link
https://mattermost.atlassian.net/browse/MM-38445
2021-10-26 18:50:02 +05:30
Agniva De Sarker
c91ab6267a Revert "Optimize TotalUnreadThreads (#18788)" (#18845)
This reverts commit 971af6935c.
2021-10-26 14:02:03 +05:30
Agniva De Sarker
e24f22745e [MM-18946] [MM-26721] [MM-6842] Cross team search+private channel autocomplete (#18468)
* Show private channels in autocomplete

This is supported in all Engines:
MySQL, Postgres, Bleve, Elasticsearch.

https://mattermost.atlassian.net/browse/MM-18496

```release-note
Private channels will now appear in channel autocomplete.
If you are using Bleve or ElasticSearch, you will have to reindex
the channels again to populate them with the new attributes.
```

A large chunk of this work has been based on the earlier
effort at https://github.com/mattermost/mattermost-server/pull/17804.

Full credit goes to https://github.com/arvinDarmawan.

* Add comment

```release-note
NONE
```

* Adding more tests

```release-note
NONE
```

* fix more tests

```release-note
NONE
```

* tmp

```release-note
NONE
```

* more fixes

```release-note
NONE
```

* add tests

```release-note
NONE
```

* Add review comments from previous PR

```release-note
NONE
```

* Add API to return all channels from all team

```release-note
NONE
```

* Added support for bleve and ES

```release-note
NONE
```

* Streaming response for GetAllChannels

```release-note
NONE
```

* Fix tests

```release-note
NONE
```

* Trigger CI
```release-note
NONE
```

* fix tests

```release-note
NONE
```

* Addressing review comments

```release-note
NONE
```

* Fix lint

```release-note
NONE
```

* Removing flaky test

```release-note
NONE
```

* Address comments

```release-note
NONE
```

* Trigger CI
```release-note
NONE
```

* Added /users/<userid>/channel_members endpoint

```release-note
NONE
```

* Minor edit

```release-note
NONE
```

* Improve embedding

```release-note
NONE
```

* Fix lint error

```release-note
NONE
```

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-10-26 11:30:59 +05:30
Agniva De Sarker
971af6935c Optimize TotalUnreadThreads (#18788)
We remove the left joins with inner joins and also
remove the coalesce on posts.DeleteAt which improves
the join plan from a nested loop to a hash join,
and also allows the DeleteAt to be used as part of the
index filter.

Old plan

```
 Aggregate  (cost=666288.49..666288.50 rows=1 width=8) (actual time=5114.328..5119.991 rows=1 loops=1)
   ->  Gather  (cost=2880.69..666288.27 rows=87 width=8) (actual time=614.214..5119.575 rows=533 loops=1)
         Workers Planned: 2
         Workers Launched: 2
         ->  Nested Loop  (cost=1880.69..665279.57 rows=36 width=8) (actual time=607.821..5099.078 rows=178 loops=3)
               ->  Hash Join  (cost=1880.28..664932.93 rows=229 width=16) (actual time=224.374..1494.698 rows=1866781 loops=3)
                     Hash Cond: ((posts.channelid)::text = (channels.id)::text)
                     ->  Parallel Seq Scan on posts  (cost=0.00..662980.29 rows=27568 width=43) (actual time=201.053..1013.362 rows=4410819 loops=3)
                           Filter: (COALESCE(deleteat, '0'::bigint) = 0)
                     ->  Hash  (cost=1866.60..1866.60 rows=1094 width=27) (actual time=1.486..1.486 rows=1002 loops=3)
                           Buckets: 2048  Batches: 1  Memory Usage: 74kB
                           ->  Index Scan using idx_channels_team_id_type on channels  (cost=0.42..1866.60 rows=1094 width=27) (actual time=0.028..1.331 rows=1002 loops=3)
                                 Index Cond: ((teamid)::text = '8ywxyw9ocp8smxrmjzrkqhrdwe'::text)
               ->  Index Scan using threadmemberships_pkey on threadmemberships  (cost=0.41..1.50 rows=1 width=35) (actual time=0.002..0.002 rows=0 loops=5600344)
                     Index Cond: (((postid)::text = (posts.rootid)::text) AND ((userid)::text = 'tc3p1yqw67d8idcp3g98awexqe'::text))
                     Filter: (following AND (posts.createat > lastviewed))
                     Rows Removed by Filter: 0
 Planning Time: 2.005 ms
 JIT:
   Functions: 59
   Options: Inlining true, Optimization true, Expressions true, Deforming true
   Timing: Generation 9.905 ms, Inlining 122.509 ms, Optimization 293.764 ms, Emission 186.064 ms, Total 612.241 ms
 Execution Time: 5125.739 ms
```

New plan

```
 Aggregate  (cost=683129.17..683129.18 rows=1 width=8) (actual time=1650.746..1656.455 rows=1 loops=1)
   ->  Gather  (cost=3766.65..683085.55 rows=17450 width=27) (actual time=641.073..1656.086 rows=533 loops=1)
         Workers Planned: 2
         Workers Launched: 2
         ->  Hash Join  (cost=2766.65..680340.55 rows=7271 width=27) (actual time=617.096..1635.203 rows=178 loops=3)
               Hash Cond: ((posts.rootid)::text = (threadmemberships.postid)::text)
               Join Filter: (posts.createat > threadmemberships.lastviewed)
               Rows Removed by Join Filter: 6
               ->  Hash Join  (cost=1880.28..679334.08 rows=45742 width=16) (actual time=15.196..1242.832 rows=1866781 loops=3)
                     Hash Cond: ((posts.channelid)::text = (channels.id)::text)
                     ->  Parallel Seq Scan on posts  (cost=0.00..662980.29 rows=5513543 width=43) (actual time=0.028..775.118 rows=4410819 loops=3)
                           Filter: (deleteat = 0)
                     ->  Hash  (cost=1866.60..1866.60 rows=1094 width=27) (actual time=1.572..1.573 rows=1002 loops=3)
                           Buckets: 2048  Batches: 1  Memory Usage: 74kB
                           ->  Index Scan using idx_channels_team_id_type on channels  (cost=0.42..1866.60 rows=1094 width=27) (actual time=0.025..1.372 rows=1002 loops=3)
                                 Index Cond: ((teamid)::text = '8ywxyw9ocp8smxrmjzrkqhrdwe'::text)
               ->  Hash  (cost=823.41..823.41 rows=5037 width=35) (actual time=256.240..256.241 rows=5007 loops=3)
                     Buckets: 8192  Batches: 1  Memory Usage: 417kB
                     ->  Bitmap Heap Scan on threadmemberships  (cost=159.45..823.41 rows=5037 width=35) (actual time=253.011..255.255 rows=5007 loops=3)
                           Recheck Cond: ((userid)::text = 'tc3p1yqw67d8idcp3g98awexqe'::text)
                           Filter: following
                           Heap Blocks: exact=601
                           ->  Bitmap Index Scan on idx_thread_memberships_user_id  (cost=0.00..158.19 rows=5037 width=0) (actual time=0.627..0.627 rows=5037 loops=3)
                                 Index Cond: ((userid)::text = 'tc3p1yqw67d8idcp3g98awexqe'::text)
 Planning Time: 1.952 ms
 JIT:
   Functions: 83
   Options: Inlining true, Optimization true, Expressions true, Deforming true
   Timing: Generation 10.148 ms, Inlining 115.819 ms, Optimization 395.923 ms, Emission 244.130 ms, Total 766.021 ms
 Execution Time: 1661.791 ms
```

No special test has been done on MySQL since these are pretty
basic optimizations and should apply to both equally.

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4?c=c9aa17f3-b918-4a78-86aa-01562e418634

```release-note
NONE
```
2021-10-26 08:56:38 +05:30
Joremy Sh
ff4df47cb1 Migrated from gorp to sqlx in store/sqlstore/plugin_store.go (#18729)
* Migrated from gorp to sqlx in store/sqlstore/plugin_store.go

* fixed lint

* updated QueryRowx and keys type

* used QueryRowX

* Fix query

Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
2021-10-21 09:57:49 +05:30
Shobhit Gupta
e36d0576a6 Migrate from gorp to sqlx in store/sqlstore/command_store.go (#18728) 2021-10-21 09:57:33 +05:30
Anusha Bhat
f5bcb12aab [MM-39232] Migrate from gorp to sqlx in store/sqlstore/reaction_store.go (#18657)
* [MM-39232] Migrate from gorp to sqlx in store/sqlstore/reaction_store.go

* Fix failing mysql tests

* Fix failing postgres specs

* Use struct instead of params
2021-10-21 09:55:49 +05:30
Archit Mathur
8ac9a3a5b6 [MM-39157] Thread participant removal with just one query using DB's JSON functions (#18725)
* feat: removes thread participant with just one query using DB's native JSON functions

* fix: corrects syntax post test failure
2021-10-19 14:07:17 -04:00
Agniva De Sarker
acd08f0a48 Fix migration scripts (#18756)
* Fix migration scripts

Set the DB schema version in DB prior to running migration

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4?c=c2525dbb-23eb-4fe7-8fb4-e9a5f503dc8c

```release-note
NONE
```

* Revert schema to 6.0.0 and update CI scripts

* Fix typo

* Add missing default

* Fix extra comma

Co-authored-by: Claudio Costa <cstcld91@gmail.com>
2021-10-19 19:56:09 +05:30
Penthaa Patel
078fdd8422 [MM-39229] - Migrate from gorp to sqlx in store/sqlstore/link_metadata_store.go (#18702)
* migrate from gorp to sqlx

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

* fix INSERT query

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

* fixes

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

* fix Get function

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

* fixes - serialize metadata_Data to JSON

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

* fixes - wrap metadataBytes with string

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>
2021-10-19 19:49:45 +05:30
Shobhit Gupta
e12e0df447 Migrate from gorp to sqlx in store/sqlstore/job_store.go (#18518) 2021-10-19 19:29:24 +05:30
Igor German
93f956e478 [GH-18504] Migrate from gorp to sqlx in store/sqlstore/file_info_store.go (#18637)
* Migrate from gorp to sqlx in store/sqlstore/file_info_store.go

Signed-off-by: Igor German <germanenak@gmail.com>

* Use separate structs for FileInfo

Signed-off-by: Igor German <germanenak@gmail.com>

* Fix CR comments

Signed-off-by: Igor German <germanenak@gmail.com>

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-10-19 19:29:10 +05:30
Kyriakos Z
0f16c59b0a MM-36862: should remove user from participants (#18451)
* MM-36862: should remove user from participants

When deleting a reply in a thread we should also delete the participant
from the participants array. This should happen if they have no other
replies in that thread.

This commit fixes that.

* Adds warning logs

* Delete Post: fetches participants only when needed

* Minor refactor

* Remove participant should check for error in count

* Fixes error when binary_parameters=true

* Fixes: removes ArrayToJSON so we can handle the error

* Satisfies the linter

* Better error handling

* Satisfy govet

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-10-15 19:40:09 +03:00
Penthaa Patel
c02cdd842a [MM-38996] - Migrate from gorp to sqlx in store/sqlstore/compliance_store.go (#18633)
* migrate from gorp to sqlx in store/sqlstore/compliance_store.go

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

* fix SQL syntax

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

* fix SQL syntax

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

* fix SQL syntax

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

* fix SQL syntax error

Signed-off-by: Penthaa Patel <penthaapatel@gmail.com>

* Fixes

```release-note
NONE
```

Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
2021-10-15 20:06:57 +05:30
Agniva De Sarker
99e6039472 Add functionality to cleanup old jobs (#18646)
* Add functionality to cleanup old jobs

Historically, we never cleaned up old jobs from the DB
leading to them being accumulated forever.

This PR introduces functionality to cleanup old jobs
older than a defined threshold.

The functionality is set to false by default and has
to be enabled for it to work.

```release-note
2 new config settings were added.

JobSettings.CleanupOldJobs: This indicates whether to clean up old jobs
from the DB or not. Default is false.

JobSettings.CleanupJobsThresholdHours: This defines the time gap in hours beyond
which older jobs will be removed. This has no effect if the above config
setting is set to false. Default is -1
```

* fix copy pasta

```release-note
NONE
```

* address review comments

```release-note
NONE
```

* Fix lint

```release-note
NONE
```

* Use single config option for everything

```release-note
NONE
```

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-10-15 11:03:54 +05:30
Claudio Costa
9d8214c72a Drop existing default on Users.Timezone prior to column type change (#18669) 2021-10-14 17:18:18 +02:00
Kyriakos Z
4b85f9ee6a MM-36743: adds last_root_post_at in channels table (#18366)
* MM-36743: adds last_root_post_at in channels table

Channel recency for CRT users should not count replies,
this commit solves that issue by adding a new column to the channels
table: LastRootPostAt.
With that new info CRT users can have recent channels to work as
expected.

* Adds the 'LastRootPostAt' to the SQL scripts

* Fixes tests

* Adds LastRootPostAt migration to v6.1

* Fixes index

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-10-13 15:20:44 -04:00
Agniva De Sarker
e2c87ef950 Display tableName and columnName for jsonb schema failures (#18672)
These are incredibly helpful during failures and should be there.

```release-note
NONE
```
2021-10-13 20:30:35 +05:30