Граф коммитов

166 Коммитов

Автор SHA1 Сообщение Дата
Miguel de la Cruz
809ad4f76d Adds Remote Cluster related API endpoints (#27432)
* Adds Remote Cluster related API endpoints

New endpoints for the following routes are added:

- Get Remote Clusters at `GET /api/v4/remotecluster`
- Create Remote Cluster at `POST /api/v4/remotecluster`
- Accept Remote Cluster invite at `POST
/api/v4/remotecluster/accept_invite`
- Generate Remote Cluster invite at `POST
/api/v4/remotecluster/{remote_id}/generate_invite`
- Get Remote Cluster at `GET /api/v4/remotecluster/{remote_id}`
- Patch Remote Cluster at `PATCH /api/v4/remotecluster/{remote_id}`
- Delete Remote Cluster at `DELETE /api/v4/remotecluster/{remote_id}`

These endpoints are planned to be used from the system console, and
gated through the `manage_secure_connections` permission.

* Update server/channels/api4/remote_cluster_test.go

Co-authored-by: Doug Lauder <wiggin77@warpmail.net>

* Fix AppError names

---------

Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
Co-authored-by: Mattermost Build <build@mattermost.com>
2024-07-04 10:35:26 +02:00
Christopher Poile
cc5e87ae24 [MM-57942] Fix a panic on password is too long (#27449)
* return error from bcrypt, handle gracefully; remove dead code

* linting

* linting

* i18n

* fix test

* fill out translations
2024-07-03 17:58:26 -04:00
Caleb Roseland
90534b13cf MM-56774: Delete file along with bookmark (#27495) 2024-07-03 08:31:39 -05:00
Miguel de la Cruz
d490fdc1d9 Marks the RemoteTeamId field of RemoteClusters as deprecated (#27364)
* Marks the RemoteTeamId field of RemoteClusters as deprecated

The `RemoteTeamId` was used both in the `RemoteCluster` model and as
part of remote invites. It existed so two different remotes could have
multiple secure connections between them, and have each of those
connections scoped to a team, sharing through each only the channels
that belong to their corresponding team.

The way that we're thinking on the feature currently only contemplates
one secure connection between two servers, and shares all the
channels through that secure connection, so this field is no longer
needed.

As we don't have a system in place for the user to choose in which
team a channel should be created from an invite, this change adds a
mechanism that checks the invite for a teamId, and if it's not
present, fetches a team from the database to create the channel
into. This makes the change backwards compatible for secure
connections that already have an established behavior and allows us to
move forward with the implementation of an alternative.

* Mark invite teamId field as deprecated

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-06-22 23:08:55 +02:00
Ben Cooke
9187c772b6 [MM-56074] mmctl job commands (#26855)
* add job list and update  job status command to mmctl
2024-06-17 12:07:05 -04:00
Doug Lauder
594ba6e665 MM-58577 Check remote ownership for posts and reactions (#27317)
* - ensure that posts and reactions can only be added via sync when coming from a remote that the target channel is shared with.
- ensure that posts and reactions are only modified/deleted by the remote that owns them.

* check that reaction belongs to post that belongs to channel that is shared with remote;  check that posts belong to channel shared with remote

* check for correct error type in unit test

* tweak unit test
2024-06-11 17:51:00 +02:00
Devin Binnie
f3e760008c [MM-58355] Send invalidate cache message across the cluster so that websocket connections on other instances are invalidated correctly (#27204)
* [MM-58355] Send invalidate cache message across the cluster so that websocket connections on other instances are invalidated correctly

* Add suggestion to clear the session cache on the local node as well

* Force read from master DB when gettting channel members for websocket to avoid any DB sync issues

* PR feedback

* Missed generated files
2024-06-07 09:38:53 -04:00
Ibrahim Serdar Acikgoz
179d28ee3f [MM-58341] db/migrations: mysql-8.4 fix for 000027_create_status (#27080) 2024-06-05 18:52:53 +02:00
Agniva De Sarker
6f3327ce0f MM-58038: Use context to call master for DeletePost (#27098)
Calling app.DeletePost immediately after creating a post
is susceptible to replica lag because we were calling the
replica to check for the post.

We fix this by passing a context to always query master.

https://mattermost.atlassian.net/browse/MM-58038

```release-note
NONE
```
2024-05-24 19:35:48 +05:30
Devin Binnie
3fdf4fc63c Use GetMasterX() to ensure writeability for the RefreshPostStats job (#26986) 2024-05-13 08:55:21 -04:00
Ben Schumacher
09c39cf3ec [MM-58020] Improve error message of NotFound errors in store (#26870)
* Improve error message of NotFound errors in store

* update mmctl tests
2024-05-07 15:30:48 +02:00
Nick Misasi
f1019d076e [CLD-7567] Deprecate Self Serve: Second Pass (#26853)
* Deprecate Self Serve: First Pass

* Fix ci

* Fix more ci

* Remmove outdated server tests

* Fix a missed spot opening purchase modal in Self Hosted

* Fix i18n

* Clean up some more server code, fix webapp test

* Fix alignment of button

* Fix linter

* Fix i18n server side

* Deprecate in product true up

* Add back translation

* Remove client functions

* Put back client functions

* webapp deprecation

* Deprecate Self Serve: Second Pass

* Fix various pipeline issues

* Fix linter

* Fix pipelines

* Fix handlers_test.go

* Fix console.error around hostedCustomer in reducer

* PICKY LINTER PLEASE

* Fix webapp tests, various other fixes for the CI pipelines

* Fix i18n

* Updates to accomadate enterprise code removal

* Fix mocks

* More removal

* Fix

* Adjustments from PR

* Fixes for QA Feedback

* Update

* Add migrations to remove true up review history

* Fix migrations check

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: maria.nunez <maria.nunez@mattermost.com>
2024-05-02 09:15:15 -04:00
Ben Schumacher
5746bb8df3 Updated permanentDelete to receive user context as the first argument (#26884)
Co-authored-by: Ezekiel <ezekielchow94@gmail.com>
2024-04-29 10:44:55 +02:00
pacop
9e6da03ab1 MM-57391: improve error message (#26630)
The error reporte when moving channels and failing has been improved to show
that the problem was the repeated name on the team.
The error message has been unified with MM-53756

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-04-22 07:40:08 +00:00
Doug Lauder
6aaabfb376 MM-57786 Fix Shared Channels plugin api (#26753)
* always ping on plugin registration; SharedChannel.IsValid allow no team for GM

* wait for services to start before ping

* ping plugin remotes synchronously on startup

* remove the waitForInterClusterServices stuff

* don't set remoteid when inviting remote to channel

* Update server/public/model/remote_cluster_test.go

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>

* address review comments

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
2024-04-15 16:18:25 -04:00
Agniva De Sarker
870cee30c6 MM-57759: Added mockery support for 1.22 (#26774)
* Revert "Revert "MM-57759: Bump mockery to version 2.42.2 to support go  1.22^" (#26772)"

This reverts commit cd3b5b46e1.

* Added the hooks.go file changes as well

```release-note
NONE
```
2024-04-13 08:15:59 +05:30
Agniva De Sarker
cd3b5b46e1 Revert "MM-57759: Bump mockery to version 2.42.2 to support go 1.22^" (#26772) 2024-04-12 15:31:24 +02:00
Ezekiel
98712737e6 MM-57759: Bump mockery to version 2.42.2 to support go 1.22^ (#26761) 2024-04-12 10:38:34 +02:00
Devin Binnie
39bb8a1121 [MM-57517] Add exact user ID lookup to the search query for Users (#26661)
* [MM-57517] Add exact user ID lookup to the search query for Users

* Fix lint

* Add test for ID search
2024-04-10 07:54:02 -04:00
Ben Schumacher
1e0de8f559 [MM-57356] Make use of go1.21 features (#26620) 2024-04-04 13:44:03 +02:00
Ben Schumacher
7337c384a8 [MM-57106] Remove unused cluster settings (#26490) 2024-03-22 10:44:38 +01:00
Agniva De Sarker
9431239b2e MM-57378: Bump up golangci version (#26535)
https://mattermost.atlassian.net/browse/MM-57378
```release-note
NONE
```

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-03-22 10:23:21 +05:30
Christopher Poile
17d11db395 MM-55320 - Limit length of browser user agent version; ratelimit the /sessions endpoint (#25900)
* add ratelimit to /sessions; cap userAgent version length; tests

* add MaxSessionsLimit; remove oldest session first; tests

* can't use slices in 1.20; improve test

* nits

* add GetLRUSessions; move limiting to CreateSession; remove rate limiting

* use queryBuilder

* mysql needs a limit when using offset

* update i18n

* refactor into limitNumberOfSessions; protect createSessionForUserAccessToken

* add comment to GetLRUSessions

* add limit to oauth path; PR comments
2024-03-21 08:48:24 -04:00
Alejandro García Montoro
9a2d96073e MM-57344: Improve ElasticSearch indexing query performance (#26517)
* Use a row constructor comparison to avoid a filter

The original query had a WHERE condition that looked like:

    (x > a) OR (x = a AND y < b)

This commit changes it to a compound comparison that, given the
lexicographical order it uses, is semantically the same:

    (x, y) > (a, b)

This makes the plan in Postgres use an Index Cond instead of a Filter,
having two main performance improvements:
1. The query's time is more or less constant, not increasing every time
this query is executed. This is important because the query is executed
repeatedly until all posts are indexed.
2. The query's time is much shorter: while the original query eventually
takes 30s, hitting the timeout, the new one takes ~30ms.

Notice the difference in the plans below:
- the original query uses a `Filter` step, while the new one uses an
`IndexCond` + `Filter` step.
- the original query has a shared hit count of 40557115 (109GiB), while
the new one has a shared hit count of 13951 (109MiBA). This does not
show exactly how much data is read from disk, given it's a nested loop
node, but it gives us an idea of the amount of data processed in the
server.

Original plan:

agnivaltdb=> EXPLAIN (ANALYZE, BUFFERS) SELECT Posts.*, Channels.TeamId FROM Posts LEFT JOIN Channels ON Posts.ChannelId = Channels.Id WHERE Posts.CreateAt > '1687424888405' OR (Posts.CreateAt = '1687424888405' AND Posts.Id > 'tpomh9yu1tffmdp6dopobwuc9h') ORDER BY Posts.CreateAt ASC, Posts.Id ASC LIMIT 10000;
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=1.22..2287.94 rows=10000 width=464) (actual time=23921.236..23954.229 rows=10000 loops=1)
   Buffers: shared hit=40557115
   ->  Incremental Sort  (cost=1.22..13998881.93 rows=61217938 width=464) (actual time=23921.235..23953.205 rows=10000 loops=1)
         Sort Key: posts.createat, posts.id
         Presorted Key: posts.createat
         Full-sort Groups: 311  Sort Method: quicksort  Average Memory: 45kB  Peak Memory: 45kB
         Buffers: shared hit=40557115
         ->  Nested Loop Left Join  (cost=1.00..11421751.31 rows=61217938 width=464) (actual time=23920.970..23947.067 rows=10001 loops=1)
               Buffers: shared hit=40557115
               ->  Index Scan using idx_posts_create_at on posts  (cost=0.57..9889434.54 rows=61217938 width=461) (actual time=23920.930..23931.063 rows=10001 loops=1)
                     Filter: ((createat > '1687424888405'::bigint) OR ((createat = '1687424888405'::bigint) AND ((id)::text > 'tpomh9yu1tffmdp6dopobwuc9h'::text)))
                     Rows Removed by Filter: 40920000
                     Buffers: shared hit=40553119
               ->  Memoize  (cost=0.43..0.70 rows=1 width=30) (actual time=0.001..0.001 rows=1 loops=10001)
                     Cache Key: posts.channelid
                     Cache Mode: logical
                     Hits: 9002  Misses: 999  Evictions: 0  Overflows: 0  Memory Usage: 151kB
                     Buffers: shared hit=3996
                     ->  Index Scan using channels_pkey on channels  (cost=0.42..0.69 rows=1 width=30) (actual time=0.007..0.007 rows=1 loops=999)
                           Index Cond: ((id)::text = (posts.channelid)::text)
                           Buffers: shared hit=3996
 Planning:
   Buffers: shared hit=112
 Planning Time: 0.501 ms
 Execution Time: 23954.974 ms
(25 rows)

New plan:

agnivaltdb=> EXPLAIN (ANALYZE, BUFFERS) SELECT Posts.*, Channels.TeamId FROM Posts LEFT JOIN Channels ON Posts.ChannelId = Channels.Id WHERE (Posts.CreateAt, Posts.Id) > ('1687424888405', 'tpomh9yu1tffmdp6dopobwuc9h') ORDER BY Posts.CreateAt ASC, Posts.Id ASC LIMIT 10000;
                                                                           QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=1.15..1666.83 rows=10000 width=464) (actual time=0.366..34.053 rows=10000 loops=1)
   Buffers: shared hit=13951
   ->  Incremental Sort  (cost=1.15..10196977.36 rows=61217938 width=464) (actual time=0.365..33.031 rows=10000 loops=1)
         Sort Key: posts.createat, posts.id
         Presorted Key: posts.createat
         Full-sort Groups: 311  Sort Method: quicksort  Average Memory: 45kB  Peak Memory: 45kB
         Buffers: shared hit=13951
         ->  Nested Loop Left Join  (cost=1.00..7619846.74 rows=61217938 width=464) (actual time=0.059..26.840 rows=10001 loops=1)
               Buffers: shared hit=13951
               ->  Index Scan using idx_posts_create_at on posts  (cost=0.57..6087529.97 rows=61217938 width=461) (actual time=0.040..10.548 rows=10001 loops=1)
                     Index Cond: (createat >= '1687424888405'::bigint)
                     Filter: (ROW(createat, (id)::text) > ROW('1687424888405'::bigint, 'tpomh9yu1tffmdp6dopobwuc9h'::text))
                     Rows Removed by Filter: 2
                     Buffers: shared hit=9955
               ->  Memoize  (cost=0.43..0.70 rows=1 width=30) (actual time=0.001..0.001 rows=1 loops=10001)
                     Cache Key: posts.channelid
                     Cache Mode: logical
                     Hits: 9002  Misses: 999  Evictions: 0  Overflows: 0  Memory Usage: 151kB
                     Buffers: shared hit=3996
                     ->  Index Scan using channels_pkey on channels  (cost=0.42..0.69 rows=1 width=30) (actual time=0.007..0.007 rows=1 loops=999)
                           Index Cond: ((id)::text = (posts.channelid)::text)
                           Buffers: shared hit=3996
 Planning:
   Buffers: shared hit=112
 Planning Time: 0.471 ms
 Execution Time: 34.716 ms
(26 rows)

* Go back to the old query for MySQL

As one could expect, the two databases have completely opposite
behaviours, and for MySQL, the old query is significantly faster than
the new one, exactly for the same reason than in Postgres, but the other
way around: the old query uses an Index range scan while the new one
would use a Filter.

Old query (0.18s):

mysql> EXPLAIN ANALYZE SELECT Posts.*, Channels.TeamId FROM Posts USE INDEX(idx_posts_create_at_id) LEFT JOIN Channels ON Posts.ChannelId = Channels.Id WHERE Posts.CreateAt > 1557752415221 OR (Posts.CreateAt = 1557752415221 AND Posts.Id > 'ad59ire57tfwmjr5r8xqxc75qw') ORDER BY Posts.CreateAt ASC, Posts.Id ASC LIMIT 10000;
--------------
EXPLAIN ANALYZE SELECT Posts.*, Channels.TeamId FROM Posts USE INDEX(idx_posts_create_at_id) LEFT JOIN Channels ON Posts.ChannelId = Channels.Id WHERE Posts.CreateAt > 1557752415221 OR (Posts.CreateAt = 1557752415221 AND Posts.Id > 'ad59ire57tfwmjr5r8xqxc75qw') ORDER BY Posts.CreateAt ASC, Posts.Id ASC LIMIT 10000
--------------

+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| EXPLAIN                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                   |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| -> Limit: 10000 row(s)  (cost=7472068.91 rows=10000) (actual time=0.063..164.174 rows=10000 loops=1)
    -> Nested loop left join  (cost=7472068.91 rows=5559093) (actual time=0.062..163.450 rows=10000 loops=1)
        -> Index range scan on Posts using idx_posts_create_at_id over (CreateAt = 1557752415221 AND 'ad59ire57tfwmjr5r8xqxc75qw' < Id) OR (1557752415221 < CreateAt), with index condition: ((Posts.CreateAt > 1557752415221) or ((Posts.CreateAt = 1557752415221) and (Posts.Id > 'ad59ire57tfwmjr5r8xqxc75qw')))  (cost=1357066.29 rows=5559093) (actual time=0.043..97.358 rows=10000 loops=1)
        -> Single-row index lookup on Channels using PRIMARY (Id=Posts.ChannelId)  (cost=1.00 rows=1) (actual time=0.006..0.006 rows=1 loops=10000)
 |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.18 sec)

New query (5s):

mysql> EXPLAIN ANALYZE SELECT Posts.*, Channels.TeamId FROM Posts USE INDEX(idx_posts_create_at_id) LEFT JOIN Channels ON Posts.ChannelId = Channels.Id WHERE (Posts.CreateAt, Posts.Id) > (1557752415221, 'ad
59ire57tfwmjr5r8xqxc75qw') ORDER BY Posts.CreateAt ASC, Posts.Id ASC LIMIT 10000;
--------------
EXPLAIN ANALYZE SELECT Posts.*, Channels.TeamId FROM Posts USE INDEX(idx_posts_create_at_id) LEFT JOIN Channels ON Posts.ChannelId = Channels.Id WHERE (Posts.CreateAt, Posts.Id) > (1557752415221, 'ad59ire57tfwmjr5r8xqxc75qw') ORDER BY Posts.CreateAt ASC, Posts.Id ASC LIMIT 10000
--------------

+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| EXPLAIN                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| -> Limit: 10000 row(s)  (cost=11119405.48 rows=10000) (actual time=5295.106..5455.285 rows=10000 loops=1)
    -> Nested loop left join  (cost=11119405.48 rows=10000) (actual time=5295.105..5454.572 rows=10000 loops=1)
        -> Filter: ((Posts.CreateAt,Posts.Id) > (1557752415221,'ad59ire57tfwmjr5r8xqxc75qw'))  (cost=221.48 rows=10000) (actual time=5295.078..5388.668 rows=10000 loops=1)
            -> Index scan on Posts using idx_posts_create_at_id  (cost=221.48 rows=10000) (actual time=0.055..5314.753 rows=600000 loops=1)
        -> Single-row index lookup on Channels using PRIMARY (Id=Posts.ChannelId)  (cost=1.00 rows=1) (actual time=0.006..0.006 rows=1 loops=10000)
 |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (5.46 sec)
```

* Add comment explaining the difference between DBs

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-03-21 12:57:29 +01:00
Agniva De Sarker
7679f51e3e MM-57193: Standardize the cache names (#26441)
We use the same name for hit/miss/invalidation metrics.

The downside is that now we have conflated a full cache purge
and a single cache key invalidation into a single label, but
the advantage is that it's easier to monitor the usage of a single
cache.

And moreover, duplicating the same string across multiple places
increases the chances of making a mistake. Centralizing the cache
name only at instantiation simplifies the code.

After this, we would be ready to use Redis.

https://mattermost.atlassian.net/browse/MM-57193

```release-note
NONE
```
2024-03-19 09:59:00 +05:30
Elias Nahum
7e9cd04a8b Channel Bookmarks (#25449)
* create ChannelBookmarks table

* ChannelBookmark model

* channel bookamrks Store layer

* add GetBookmarksForAllChannelByIdSince

* add channel bookmarks to test store

* Add channel bookmarks to app layer

* remove index for createAt in channel bookmarks migrations

* remove createAt from select channel bookmark query and enable store delete bookmark test

* update reponse of UpdateBookmark

* rename db migration files

* channel bookmarks store update sort order

* channel bookmarks app layer update sort order

* fix lint & tests

* Fix lint and introduce util functions to insert / remove from slice

* remove model etag

* i18n

* defer remove file info after test run

* Fix tests passing the request context

* fix migrations

* fix TestRetry

* Add bookmark permissions (#25560)

* Adds channel bookmarks permissions

* Fix linter

* Remove unnecessary empty lines

* Remove scss change as it's not necessary anymore

* Fix mock store

* Fix mock store and add role entry

* Fix test

* Adds cypress test and update permissions migration to update admin roles

* Adds channel bookmarks roles to default admin roles

* Adds bookmark permissions to default role permissions constant in webapp

* Update mmctl test

* Update permission test after normalising the roles

* fix store tests

* fix app layer tests

* Add new bookmark endpoint (#25624)

* Adds channel bookmarks api scaffold and create endpoint

* Applies review comments to the API docs

* Adds websocket test to create channel bookmark

---------

Co-authored-by: Mattermost Build <build@mattermost.com>

* MM-54426 exclude Channel Bookmarks files from data retention (#25656)

* Augment channel APIs to include bookmarks (#25567)

* update files docs for server 9.4

* Adds update channel bookmark endpoint (#25653)

* Adds update channel bookmark sort order endpoint (#25686)

* Adds update channel bookmark endpoint

* Updates edit app method to return the right deleted bookmark and adds tests

* Adds the update channel bookmark sort order endpoint

* Fix repeated test after merge

* Assign right permissions to each test

* Update store and app layer to return specific errors and add tests

* Adds delete channel bookmark endpoint (#25693)

* Updates edit app method to return the right deleted bookmark and adds tests

* Fix repeated test after merge

* Updates edit app method to return the right deleted bookmark and adds tests

* Adds delete channel bookmark endpoint

* Adds list channel bookmarks endpoint (#25700)

* Add channel moderation to bookmarks (#25716)

* fix migrations index

* fix getChannelsForTeamForUser

* fix getChannelsForTeamForUser

* fix bad merge client4

* fix file api with bookmark permission

* add ChannelBookmarks feature flag

* add missing translations

* Set DB column for type as enum

* use custom type for bookmark query using sqlx

* use transaction when saving bookmark

* return NewErrNotFound instead of Sql.ErrNoRows

* use squirrel for IN query

* add a limit of 1K for records in GetBookmarksForAllChannelByIdSince

* UpdateSortOrder with one single query instead of multiple updates

* fix shadow declaration

* fix channel bookmarks permission string definition in admin console

* fix another shadow declaration

* Fix model conversion

* add SplitSliceInChunks

* remove include bookmarks in channels api

* Cap amount of bookmarks per channel

* add etag back to get channels

* feedback review

* update file info when replacing a bookmark file

* return 501 not implemented when the license is not available

* add detail message when getting channel member on bookmark api

* start audit before permission check on create bookmark api

* use require.Eventuallyf for testing WS events

* remove unnecessary log in app layer

* use require instead of assert to avoid panics

* enforce limit when querying bookmarks since

* prevent to create/update bookmark if file is already attached

* fix lint

* delete file when a bookmark is deleted

* Dot allow to set a fileId and a url at the same time to a bookmark

* fix query to delete a file that belongs to a bookmark

* do not patch the bookmark type

* Server side FeatureFlag check (#26145)

* use ff in server, set ff to false

* turn on FF for unit tests

* defer unset FF for unit tests

* turn ff on for testing

* only allow attaching files that were uploaded for bookmark

* Set feature flag off as default

* fix lint

* update email templates as PR failed

* revert templates

* force the assignment of ID when creating a bookmark

* Fix unit tests

---------

Co-authored-by: Miguel de la Cruz <miguel@mcrx.me>
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Caleb Roseland <caleb@calebroseland.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
2024-03-12 08:36:05 -06:00
Agniva De Sarker
bca5ab9a1f MM-57084: Use cache for GetAllProfiles (#26391)
UserStore.GetAllProfiles is a very frequent call. We cache it
when there are no options passed which is the default case.

https://mattermost.atlassian.net/browse/MM-57084

```release-note
NONE
```
2024-03-11 10:34:20 +05:30
Harshil Sharma
4fda7e6f34 Moved some common SQL function to public utils as they are used in plugins (#26412)
* Moved some common SQL function tu public utls as they are used in plugins

* goimported file

* Added tests

* Created sub-package

* MOved SetupConnection to public sql utils
2024-03-11 09:54:23 +05:30
Agniva De Sarker
204c728b08 MM-56879: Migrate caches from store layer to cache layer (#26255)
There were 3 remaining caches which were there in the store layer.
We migrate them to make the store layer fully free
from any caches.

https://mattermost.atlassian.net/browse/MM-56879

```release-note
NONE
```

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-03-07 10:55:28 +05:30
Agniva De Sarker
f9861b8666 MM-56987: Improve cache API (#26298)
This is in preparation to make the codebase
ready to use Redis. In Redis, iterating all
the keys at once is an expensive operation.
It is recommended to work on batches of keys.

Remove the Len method as it was unused.

I tried to repurpose the Keys method to iterate
on keys rather than returning all keys at once, but
it has other complicacies because the code calls
other cache functions on those keys, so to handle
the LRU cache properly, it becomes slightly more
painful.

For now, we keep it like this and rather collect
all keys from Redis and then return.

https://mattermost.atlassian.net/browse/MM-56987

```release-note
NONE
```
2024-03-01 09:46:09 +05:30
Maria A Nunez
e9b9d4ff60 Deprecate admin advisor (#26045)
* Deprecate admin advisor

* Webapp portion

* More webapp deprecation

* More cleanup

* Linting

* emoved metric ack dialog from annoucenemet bar

* Cleanued up uninsed i18n strings

* Updated test

* fixed types

* Updating server test

* Updated i18n

* Updated cypress test:

* Updated cypress test:

---------

Co-authored-by: harshil Sharma <harshilsharma63@gmail.com>
2024-02-26 09:05:00 +05:30
Aswath S
dc8fc773dc MM-55733 add request context to Channelstore.Save method (#26141) 2024-02-24 13:08:51 +01:00
Aditya Pratap Singh Hada
6541708a93 [MM-55735] Add Request Context to PostStore.Save (#26048) 2024-02-20 17:07:20 +01:00
Trang
f130615011 [MM-55737] Add Request Context to UserStore.Save (#26109) 2024-02-16 17:12:49 +01:00
Agniva De Sarker
f3c7510c70 MM-56878: Remove unused store method (#26230)
IsUserInChannelUseCache wasn't being used anywhere.

https://mattermost.atlassian.net/browse/MM-56878

```release-note
NONE
```
2024-02-15 19:20:48 +05:30
Ben Schumacher
de3e5aab25 [MM-53156] Remove Multi-Product architecture (#25669) 2024-02-15 13:01:44 +01:00
Agniva De Sarker
28cb8d0441 MM-56579: Include deactivated users from DM posts (#25985)
There were 2 separate bugs here:
1. We were not including deactivated users while fetching posts from
DM channels.
2. We were not respecting the includeArchivedChannels flag while
fetching DM channels.

We fix both of these issues here.

```release-note
Include posts from deactivated users in DM channel export. Also
respect the --include-archived-channels flag for DM channels.
```

https://mattermost.atlassian.net/browse/MM-56579

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-02-14 20:59:46 +05:30
Michael Kochell
4e071e861c Webapp - Outgoing OAuth Connections (#25507)
* added store

* make generated

* add missing license headers

* fix receiver name

* i18n

* i18n sorting

* update migrations from master

* make migrations-extract

* update retrylayer tests

* replaced sql query with id pagination

* fixed flaky tests

* missing columns

* missing columns on save/update

* typo

* improved tests

* remove enum from mysql colum

* add password credentials to store

* license changes

* OAuthOutgoingConnectionInterface

* Oauth -> OAuth

* make generated

* copied over installed_oauth_apps component and renamed things to installed_outgoing_oauth_connections

* merge migrations

* renamed migrations

* model change suggestions

* refactor test functionsn

* migration typo

* refactor store table names

* updated sanitize test

* cleanup merge

* refactor symbol

* "installed outgoing oauth connections" page works

* move things into a nested folder

* add and edit page stubs work

* list endpoint

* oauthoutgoingconnection -> outgoingoauthconnection

* signature change

* i18n update

* granttype typo

* naming

* api list

* uppercase typo

* i18n

* missing license header

* fixed path in comments

* updated openapi definitions

* changes to support selecting command request url

* sanitize connections

* make generated

* test license and no feature flag

* removed t.fatal

* updated testhelper calls

* yaml schema fixes

* switched interface name

* suggested translation

* missing i18n translation

* management permission

* moved permission initalization to proper place

* endpoints

* put tests

* error check typo

* fixed specific enttity urls

* tests

* read permission check

* updated openapi definitions

* i18n

* GetConnectionByAudience method

* notes

* replaced GetConnectionsByAudience with a filter

* added custom oauth token object

* updated interface and usage

* properly set enterprise interface

* move retrieval logic to impl

* webhook tests

* translations

* i18n: updates

* address comments

* endpoint and tests

* i18n

* api docs

* fixed endpoint path

* sq.like

* use filter object instead of parameters

* set url values if not empty

* typos

* converted some components to function components, and move around files

* correctly check token url

* restore flag to previous value

* added command oauth handler

* update enterprise imports

* migrate last component to function component

* Added enterprise import

* refactor permissions and add necessary webapp code

* Check correct flag in permission tree

* allow partial updates

* sort i18n webapp

* missing test modification

* fixed webapp i18n sorting

* allow validating stored connections

* added missing translation

* fix finished adding connection link and text on result page

* added missing permission to smoke tests

* missing role in smoke test

* updated translations

* updated translations

* support editing client secret on existing connection

* fix some i18n strings

* updated translations

* better error messages

* progress on using react select for command request url while maintaining typed in value

* remove writeheader, test

* HasValidGrantType

* end early to avoid nil pointer errors

* move slash command request url input box into its own component

* wrap components related to oauth connections in config check

* fix tests

* i18n-extract

* change some i18n strings to say "Outgoing OAuth 2.0 Connections"

* remove debug code

* fixed i18n

* updated i18n file

* feature configuration backend

* typo

* add system console setting

* Revert "typo"

This reverts commit 669da23e8ee47525ccaa6f59cbbd20bf8a121191.

* Revert "updated i18n file"

This reverts commit d0882c0dd7587533f0d0f7a7b7b190684186158a.

* Revert "fixed i18n"

This reverts commit 3108866bc19139182dfd094921c56cdefc4695ea.

* fixed i18n

* updated i18n file

* typo

* updated i18n

* updated i18n

* updated i18n

* updated version to 9.6

* replace feature flag with system console configuration

* i18n

* updated tests

* pr feedback

* fix styling of disabled text box

* fix styling of action links in integration console

* server changes for validation feature

* webapp changes for validation feature

* pencil icon styling

* styling fixes for oauth audience correct configuration message

* fix sanitize test

* remove max lengths from outgoing oauth connection form

* use config var in webapp instead of feature flag

* change asterisks to bullets

* update api docs for validate endpoint

* feedback from ux review

* fix lint, types, tests

* fix stylelint

* implement validation button under the token url input

* support wildcard for matching audience urls

* updates for styling

* update snapshots

* add doc links for the outgoing oauth connections feature

* change doc links to use permalink

* add docs link to system console

* fix: use limitedreader in json decoding

* fix: form error in validation

* management permission can read now

* updated api documentation

* doc typo

* require one permission to read only

* fix api connection list audience filter

* fix audience matching and add loading indicator

* fix team permissions on outgoing oauth connection api calls

* fix api doc and test, for adding team id to query params

* handle read permissions by adding a team in the payload

* missing teamid query parameter in test

* change validate button logic to not require audience urls to be filled out

* fix redux type

---------

Co-authored-by: Felipe Martin <me@fmartingr.com>
2024-02-09 14:49:49 -05:00
compiledsound
1dcd251418 MM-55736 Add Request Context to TeamStore.SaveMember (#26061)
Co-authored-by: compiledsound <>
2024-02-07 09:00:13 +01:00
Arun
419b30b569 [MM-55734] Add Request Context to ChannelStore.SaveMember (#26039)
* added rctx to reuired files

* added more file changes after issues on startup

* added context to draft_store

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-02-02 15:15:17 +01:00
Utsav Ladani
3ac6edb406 [GH-25484] Fix draft removal on post deletion (#25715)
* [GH-25484] Fix draft removal on post deletion

* [GH-25484] Add batch migration to remove orphan drafts

* [GH-25484] Fix tests of migration and draft store

* [GH-25484] Remove translation file changes.

* [GH-25484] Remove translation file changes.

---------

Co-authored-by: Devin Binnie <52460000+devinbinnie@users.noreply.github.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com>
2024-01-29 09:55:34 -05:00
Agniva De Sarker
f246cd26b7 MM-56457: Update LastUpdateAt in UpdateMemberNotifyProps (#25984)
PR https://github.com/mattermost/mattermost/pull/18114
mistakenly left out updating the lastUpdateAt of the field.

We fix that here.

https://mattermost.atlassian.net/browse/MM-56457
```release-note
NONE
```
2024-01-23 16:31:25 -05:00
Devin Binnie
8dbe7aed79 [MM-56362] Disable post stats for MySQL (#25981) 2024-01-23 10:34:23 -05:00
Matthew Straughn
ae33de57ff Mm 16623 fix all initialism errors in channels/app and channels/api4 and /store dir (#25787) 2024-01-22 14:35:16 +01:00
Agniva De Sarker
8364e30a2a MM-55524: Include deactivated users in DM export (#25695)
We were incorrectly excluding deactivated users
while getting channel members for a DM channel, whereas
we were actually exporting all users in the users array.

We fix this and also correctly honor the includeArchivedChannels
flag as well.

https://mattermost.atlassian.net/browse/MM-55524

```release-note
Include deactivated members in a favorited DM channel export.
```


---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-01-20 08:47:47 +05:30
Devin Binnie
bbe432b3b0 [MM-56398] Fix date range query for user reporting (#25960)
* [MM-56398] Fix date range query for user reporting

* Missing debug stuff

* Fix tests
2024-01-18 14:06:30 -05:00
Harrison Healey
fd3a3081d4 Remove accidentally included trace logs (#25939) 2024-01-17 16:18:01 -05:00
Doug Lauder
a07097ed57 Plugin API hook for Shared Channel file attachment sync (#25874)
* option for auto inviting plugin to all shared channels.

* auto-invite remotes to shared channels when flag set

* fix unit test

* immediately ping new remotes; fix unique siteurl bug

* make i18n-extract

* fix translations

* plugin hooks for file attachments

* hook for profile image sync

* fix profile image sync

* fix unit test

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2024-01-16 15:48:51 +01:00
Harrison Healey
4d96c11314 MM-56083 Add PatchMultipleMembersNotifyProps plugin API (#25690)
* Add ChannelStore.UpdateMultipleMembersNotifyProps

* Make UpdateMultipleMembersNotifyProps return updated values from the DB

* Add UpdateChannelMembersNotifications plugin API

* Extract i18n

* Fix style

* Make layers

* Change to PatchMultipleMembersNotifyProps

* Add limit to PatchChannelMembersNotifyProps

* Add additional unit tests

* Address feedback

* Lowercase decodeJSON

* Have PatchMultipleMembersNotifyProps update LastUpdateAt

* Fix tests that relied on unreliable return order

* Fix i18n
2024-01-11 13:24:52 -05:00
Agniva De Sarker
04cf1ed114 Bump up the minimum required MySQL version (#25465)
MySQL 5.7 is at end of life.

https://mattermost.atlassian.net/browse/MM-55589

```release-note
We bump up minimum MySQL version to be 8.0.0
```


Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
2024-01-11 22:18:20 +05:30