Граф коммитов

1237 Коммитов

Автор SHA1 Сообщение Дата
Miguel de la Cruz
2decc2ccdb Prevent synced CPA values to be updated from the API (#30687)
* Prevents the API from updating synced CPA values

The patch functions for CPA values now accept a parameter that checks
if they should allow for synced values to be updated, and prevent
those updates if necessary

* Fix linter

* Fix parameter after merge

---------

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-04-30 18:43:05 +02:00
Ben Schumacher
df3560ed9c [MM-61086] Fix errcheck linter issue in batch_worker_test.go (#30644) 2025-04-29 11:08:38 +02:00
Ben Schumacher
f1ddeec2f6 [MM-61778] Fix errcheck issues in web_hub_test.go (#30676)
Co-authored-by: Claude <noreply@anthropic.com>
2025-04-29 10:23:53 +02:00
Ben Schumacher
2f5b70bfa2 [MM-29108] Fix errcheck issues in server/channels/app/team.go (#30683)
Co-authored-by: Claude <noreply@anthropic.com>
2025-04-29 09:03:06 +02:00
Ben Schumacher
d0c4780eb1 [MM-28761] Fix errcheck linter issues in hosted_purchase_screening worker (#30624) 2025-04-29 08:57:52 +02:00
Ben Schumacher
5ee308bd83 [MM-61467] Fix errcheck linter issues in post_test.go (#30686) 2025-04-29 08:50:47 +02:00
Agniva De Sarker
efde5e2717 [AI assisted] MM-62755: Refactor scanning to map to a util (#30780)
With some neat generics, I was able to refactor
the scanning to a util function. I used it to
refactor 3 places and also removed an unnecessary method.

Claude was quite good here.

https://mattermost.atlassian.net/browse/MM-62755
```release-note
NONE
```
2025-04-28 19:21:12 +05:30
Ben Schumacher
cd5523f5fb MM-29092 Fix error handling in auto_environment.go (#30610)
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Caleb Roseland <caleb@calebroseland.com>
2025-04-28 15:05:31 +02:00
Ben Schumacher
eea39a5f23 [MM-61087] Fix errcheck linter issues in channels/jobs/helper_test.go (#30643)
Co-authored-by: Claude <noreply@anthropic.com>
2025-04-28 15:02:10 +02:00
Ben Schumacher
6a228877b7 [MM-28762] Fix errcheck issues in jobs.go (#30708)
Co-authored-by: Claude <noreply@anthropic.com>
2025-04-28 14:57:40 +02:00
Ben Schumacher
6fc60c8e5f [MM-61105] Fix errcheck linter errors in config_test.go (#30689) 2025-04-28 14:53:45 +02:00
Agniva De Sarker
29f7c895b8 MM-63878: Fix INSERT IGNORE in saveChannelT (#30850)
INSERT IGNORE will ignore ALL errors in the INSERT statement.
This is not what was intended. The right way is to do a
redundant update on duplicate key.

It's not great, but that's how MySQL wants us to do it.

https://mattermost.atlassian.net/browse/MM-63878
```release-note
NONE
```
2025-04-24 21:29:09 +05:30
Agniva De Sarker
131cf039bb MM-63756: Added index to sidebarchannels table (#30724)
The (s SqlChannelStore) getSidebarCategoriesT gets called quite frequently.
- Team switch
- WS reconnect
- Category created
- Category updated
- Category deleted

Of these 1 and 2 are probably the most commonly called sources. Based on that,
the sidebarChannels table is not that well-optimized. Even though
the query time might be reasonable, without an index, it has to churn a lot of
DB CPU for a sequential scan.

We add a new index to optimize this.

CREATE INDEX idx_sidebarchannels_categoryid ON sidebarchannels(categoryid);

```
Before:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Sort  (cost=40854.18..40854.19 rows=4 width=193) (actual time=251.635..251.646 rows=204 loops=1)
   Sort Key: sidebarcategories.sortorder, sidebarchannels.sortorder
   Sort Method: quicksort  Memory: 65kB
   Buffers: shared hit=1203 read=23668
   ->  Nested Loop  (cost=8.87..40854.14 rows=4 width=193) (actual time=251.345..251.455 rows=204 loops=1)
         Buffers: shared hit=1203 read=23668
         ->  Nested Loop  (cost=0.41..9.47 rows=1 width=54) (actual time=0.068..0.074 rows=1 loops=1)
               Buffers: shared hit=5
               ->  Seq Scan on teams  (cost=0.00..1.03 rows=1 width=27) (actual time=0.024..0.026 rows=1 loops=1)
                     Filter: (((id)::text = '3ee5y5ok6jgxicrmqstdnghmfr'::text) AND (deleteat = 0))
                     Rows Removed by Filter: 1
                     Buffers: shared hit=1
               ->  Index Scan using teammembers_pkey on teammembers  (cost=0.41..8.43 rows=1 width=27) (actual time=0.039..0.043 rows=1 loops=1)
                     Index Cond: (((teamid)::text = '3ee5y5ok6jgxicrmqstdnghmfr'::text) AND ((userid)::text = 'tc3p1yqw67d8idcp3g98awexqe'::text))
                     Filter: (deleteat = 0)
                     Buffers: shared hit=4
         ->  Hash Right Join  (cost=8.45..40844.62 rows=4 width=193) (actual time=251.274..251.361 rows=204 loops=1)
               Hash Cond: ((sidebarchannels.categoryid)::text = (sidebarcategories.id)::text)
               Buffers: shared hit=1198 read=23668
               ->  Seq Scan on sidebarchannels  (cost=0.00..37514.77 rows=1265277 width=100) (actual time=0.043..99.345 rows=1265444 loops=1)
                     Buffers: shared hit=1194 read=23668
               ->  Hash  (cost=8.44..8.44 rows=1 width=158) (actual time=0.047..0.047 rows=6 loops=1)
                     Buckets: 1024  Batches: 1  Memory Usage: 10kB
                     Buffers: shared hit=4
                     ->  Index Scan using idx_sidebarcategories_userid_teamid on sidebarcategories  (cost=0.42..8.44 rows=1 width=158) (actual time=0.029..0.037 rows=6 loops=1)
                           Index Cond: (((userid)::text = 'tc3p1yqw67d8idcp3g98awexqe'::text) AND ((teamid)::text = '3ee5y5ok6jgxicrmqstdnghmfr'::text))
                           Buffers: shared hit=4
 Planning:
   Buffers: shared hit=9
 Planning Time: 1.215 ms
 Execution Time: 251.755 ms
(31 rows)

After:
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Sort  (cost=1544.53..1544.54 rows=4 width=192) (actual time=0.834..0.859 rows=204 loops=1)
   Sort Key: sidebarcategories.sortorder, sidebarchannels.sortorder
   Sort Method: quicksort  Memory: 65kB
   Buffers: shared hit=58
   ->  Nested Loop Left Join  (cost=8.53..1544.49 rows=4 width=192) (actual time=0.066..0.252 rows=204 loops=1)
         Buffers: shared hit=58
         ->  Nested Loop  (cost=0.83..17.93 rows=1 width=157) (actual time=0.042..0.098 rows=6 loops=1)
               Buffers: shared hit=34
               ->  Nested Loop  (cost=0.42..9.48 rows=1 width=157) (actual time=0.030..0.049 rows=6 loops=1)
                     Buffers: shared hit=10
                     ->  Index Scan using idx_sidebarcategories_userid_teamid on sidebarcategories  (cost=0.42..8.44 rows=1 width=157) (actual time=0.018..0.022 rows=6 loops=1)
                           Index Cond: (((userid)::text = 'tc3p1yqw67d8idcp3g98awexqe'::text) AND ((teamid)::text = '3ee5y5ok6jgxicrmqstdnghmfr'::text))
                           Buffers: shared hit=4
                     ->  Seq Scan on teams  (cost=0.00..1.03 rows=1 width=27) (actual time=0.002..0.003 rows=1 loops=6)
                           Filter: (((id)::text = '3ee5y5ok6jgxicrmqstdnghmfr'::text) AND (deleteat = 0))
                           Rows Removed by Filter: 1
                           Buffers: shared hit=6
               ->  Index Scan using teammembers_pkey on teammembers  (cost=0.41..8.43 rows=1 width=27) (actual time=0.007..0.007 rows=1 loops=6)
                     Index Cond: (((teamid)::text = '3ee5y5ok6jgxicrmqstdnghmfr'::text) AND ((userid)::text = 'tc3p1yqw67d8idcp3g98awexqe'::text))
                     Filter: (deleteat = 0)
                     Buffers: shared hit=24
         ->  Bitmap Heap Scan on sidebarchannels  (cost=7.69..1522.35 rows=421 width=100) (actual time=0.012..0.017 rows=34 loops=6)
               Recheck Cond: ((categoryid)::text = (sidebarcategories.id)::text)
               Heap Blocks: exact=6
               Buffers: shared hit=24
               ->  Bitmap Index Scan on idx_sidebarchannels_categoryid  (cost=0.00..7.58 rows=421 width=0) (actual time=0.010..0.010 rows=34 loops=6)
                     Index Cond: ((categoryid)::text = (sidebarcategories.id)::text)
                     Buffers: shared hit=18
 Planning:
   Buffers: shared hit=18
 Planning Time: 0.543 ms
 Execution Time: 0.968 ms
(32 rows)
```

I have also looked at potentially re-ordering the JOINs to make
sidebarchannels and sidebarcategories JOIN earlier, but that didn't give
a major benefit.

Also looked at adding a compound index with (categoryid, sortorder) to improve
sorting performance, but that didn't give a major benefit from what the single
column index already gives.

The `completePopulatingCategoryChannelsT` query also partially benefits
from this. But the Postgres optimizer sometimes selects the index on categoryId
and sometimes on ChannelId, both giving equivalent performance. So there's no major
improvement there, but at the same time, no regression as well.

```
Original:
[bigdb] # EXPLAIN (ANALYZE, BUFFERS) SELECT Id FROM ChannelMembers LEFT JOIN Channels ON Channels.Id=ChannelMembers.ChannelId WHERE (ChannelMembers.UserId = 'tc3p1yqw67d8idcp3g98awexqe' AND Channels.Type IN ('D'
                                                                                                                                                                                                                ,'G') AND Channels.DeleteAt = 0 AND NOT EXISTS ( SELECT 1 FROM SidebarChannels JOIN SidebarCategories on SidebarChannels.CategoryId=SidebarCategories.Id WHERE (SidebarChannels.ChannelId = ChannelMembers.ChannelI
                                                                                                                                                                                                                                                                                                                                                                                d AND SidebarCategories.UserId = 'tc3p1yqw67d8idcp3g98awexqe' AND SidebarCategories.TeamId = '3ee5y5ok6jgxicrmqstdnghmfr') )) ORDER BY DisplayName ASC;
                                                                                            QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Sort  (cost=5864.68..5865.84 rows=463 width=40) (actual time=9.008..9.022 rows=39 loops=1)
   Sort Key: channels.displayname
   Sort Method: quicksort  Memory: 27kB
   Buffers: shared hit=2112
   ->  Nested Loop Anti Join  (cost=1.96..5844.18 rows=463 width=40) (actual time=0.188..8.932 rows=39 loops=1)
         Buffers: shared hit=2112
         ->  Nested Loop  (cost=0.99..3476.66 rows=463 width=67) (actual time=0.159..7.952 rows=39 loops=1)
               Buffers: shared hit=1956
               ->  Index Only Scan using idx_channelmembers_user_id_channel_id_last_viewed_at on channelmembers  (cost=0.56..40.78 rows=470 width=27) (actual time=0.036..0.467 rows=437 loops=1)
                     Index Cond: (userid = 'tc3p1yqw67d8idcp3g98awexqe'::text)
                     Heap Fetches: 45
                     Buffers: shared hit=208
               ->  Memoize  (cost=0.43..7.69 rows=1 width=40) (actual time=0.016..0.016 rows=0 loops=437)
                     Cache Key: channelmembers.channelid
                     Cache Mode: logical
                     Hits: 0  Misses: 437  Evictions: 0  Overflows: 0  Memory Usage: 42kB
                     Buffers: shared hit=1748
                     ->  Index Scan using channels_pkey on channels  (cost=0.42..7.68 rows=1 width=40) (actual time=0.015..0.015 rows=0 loops=437)
                           Index Cond: ((id)::text = (channelmembers.channelid)::text)
                           Filter: ((type = ANY ('{D,G}'::channel_type[])) AND (deleteat = 0))
                           Rows Removed by Filter: 1
                           Buffers: shared hit=1748
         ->  Nested Loop  (cost=0.97..5.10 rows=1 width=27) (actual time=0.023..0.023 rows=0 loops=39)
               Buffers: shared hit=156
               ->  Index Only Scan using sidebarchannels_pkey on sidebarchannels  (cost=0.55..4.56 rows=1 width=92) (actual time=0.022..0.022 rows=0 loops=39)
                     Index Cond: (channelid = (channelmembers.channelid)::text)
                     Heap Fetches: 0
                     Buffers: shared hit=156
               ->  Index Scan using sidebarcategories_pkey on sidebarcategories  (cost=0.42..0.48 rows=1 width=65) (never executed)
                     Index Cond: ((id)::text = (sidebarchannels.categoryid)::text)
                     Filter: (((userid)::text = 'tc3p1yqw67d8idcp3g98awexqe'::text) AND ((teamid)::text = '3ee5y5ok6jgxicrmqstdnghmfr'::text))
 Planning:
   Buffers: shared hit=48 dirtied=1
 Planning Time: 2.222 ms
 Execution Time: 9.142 ms
(35 rows)

New:
[bigdb] # EXPLAIN (ANALYZE, BUFFERS) SELECT Id FROM ChannelMembers LEFT JOIN Channels ON Channels.Id=ChannelMembers.ChannelId WHERE (ChannelMembers.UserId = 'tc3p1yqw67d8idcp3g98awexqe' AND Channels.Type IN ('D'
                                                                                                                                                                                                                ,'G') AND Channels.DeleteAt = 0 AND NOT EXISTS ( SELECT 1 FROM SidebarChannels JOIN SidebarCategories on SidebarChannels.CategoryId=SidebarCategories.Id WHERE (SidebarChannels.ChannelId = ChannelMembers.ChannelId AND SidebarCategories.UserId = 'tc3p1yqw67d8idcp3g98awexqe' AND SidebarCategories.TeamId = '3ee5y5ok6jgxicrmqstdnghmfr') )) ORDER BY DisplayName ASC;
                                                                                            QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 Sort  (cost=5059.95..5061.11 rows=463 width=40) (actual time=12.072..12.086 rows=39 loops=1)
   Sort Key: channels.displayname
   Sort Method: quicksort  Memory: 27kB
   Buffers: shared hit=1984
   ->  Nested Loop Anti Join  (cost=9.10..5039.45 rows=463 width=40) (actual time=0.751..12.009 rows=39 loops=1)
         Join Filter: ((sidebarchannels.channelid)::text = (channelmembers.channelid)::text)
         Rows Removed by Join Filter: 7839
         Buffers: shared hit=1984
         ->  Nested Loop  (cost=0.99..3476.66 rows=463 width=67) (actual time=0.161..7.579 rows=39 loops=1)
               Buffers: shared hit=1956
               ->  Index Only Scan using idx_channelmembers_user_id_channel_id_last_viewed_at on channelmembers  (cost=0.56..40.78 rows=470 width=27) (actual time=0.036..0.449 rows=437 loops=1)
                     Index Cond: (userid = 'tc3p1yqw67d8idcp3g98awexqe'::text)
                     Heap Fetches: 45
                     Buffers: shared hit=208
               ->  Memoize  (cost=0.43..7.69 rows=1 width=40) (actual time=0.016..0.016 rows=0 loops=437)
                     Cache Key: channelmembers.channelid
                     Cache Mode: logical
                     Hits: 0  Misses: 437  Evictions: 0  Overflows: 0  Memory Usage: 42kB
                     Buffers: shared hit=1748
                     ->  Index Scan using channels_pkey on channels  (cost=0.42..7.68 rows=1 width=40) (actual time=0.014..0.014 rows=0 loops=437)
                           Index Cond: ((id)::text = (channelmembers.channelid)::text)
                           Filter: ((type = ANY ('{D,G}'::channel_type[])) AND (deleteat = 0))
                           Rows Removed by Filter: 1
                           Buffers: shared hit=1748
         ->  Materialize  (cost=8.11..1535.03 rows=4 width=27) (actual time=0.003..0.046 rows=201 loops=39)
               Buffers: shared hit=28
               ->  Nested Loop  (cost=8.11..1535.01 rows=4 width=27) (actual time=0.099..0.383 rows=201 loops=1)
                     Buffers: shared hit=28
                     ->  Index Scan using idx_sidebarcategories_userid_teamid on sidebarcategories  (cost=0.42..8.44 rows=1 width=65) (actual time=0.047..0.057 rows=6 loops=1)
                           Index Cond: (((userid)::text = 'tc3p1yqw67d8idcp3g98awexqe'::text) AND ((teamid)::text = '3ee5y5ok6jgxicrmqstdnghmfr'::text))
                           Buffers: shared hit=4
                     ->  Bitmap Heap Scan on sidebarchannels  (cost=7.69..1522.35 rows=421 width=92) (actual time=0.028..0.040 rows=34 loops=6)
                           Recheck Cond: ((categoryid)::text = (sidebarcategories.id)::text)
                           Heap Blocks: exact=6
                           Buffers: shared hit=24
                           ->  Bitmap Index Scan on idx_sidebarchannels_categoryid  (cost=0.00..7.58 rows=421 width=0) (actual time=0.023..0.023 rows=34 loops=6)
                                 Index Cond: ((categoryid)::text = (sidebarcategories.id)::text)
                                 Buffers: shared hit=18
 Planning:
   Buffers: shared hit=51
 Planning Time: 2.240 ms
 Execution Time: 12.210 ms
(42 rows)
```

Analysis on MySQL for completion:
```
Before:
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| -> Sort: SidebarCategories.SortOrder, SidebarChannels.SortOrder  (actual time=277.675..277.675 rows=4 loops=1)
    -> Stream results  (cost=138558.36 rows=1287808) (actual time=242.506..277.650 rows=4 loops=1)
        -> Left hash join (<hash>(SidebarChannels.CategoryId)=<hash>(SidebarCategories.Id)), extra conditions: (SidebarChannels.CategoryId = SidebarCategories.Id)  (cost=138558.36 rows=1287808) (actual time=242.498..277.626 rows=4 loops=1)
            -> Index lookup on SidebarCategories using idx_sidebarcategories_userid_teamid (UserId='qdggj9pyobgkjpj8htwzizks1r', TeamId='xmh7bupzajnudqf3h4mm76qapy')  (cost=1.40 rows=4) (actual time=0.092..0.094 rows=4 loops=1)
            -> Hash
                -> Table scan on SidebarChannels  (cost=8394.55 rows=321952) (actual time=0.123..106.334 rows=300002 loops=1)
 |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

After:
----------------------------------------------------------------+
| -> Sort: SidebarCategories.SortOrder, SidebarChannels.SortOrder  (actual time=0.739..0.742 rows=4 loops=1)
    -> Stream results  (cost=6.80 rows=7) (actual time=0.468..0.703 rows=4 loops=1)
        -> Nested loop left join  (cost=6.80 rows=7) (actual time=0.456..0.673 rows=4 loops=1)
            -> Index lookup on SidebarCategories using idx_sidebarcategories_userid_teamid (UserId='qdggj9pyobgkjpj8htwzizks1r', TeamId='xmh7bupzajnudqf3h4mm76qapy')  (cost=4.38 rows=4) (actual time=0.302..0.313 rows=4 loops=1)
            -> Index lookup on SidebarChannels using idx_sidebarchannels_categoryid (CategoryId=SidebarCategories.Id)  (cost=0.48 rows=2) (actual time=0.085..0.087 rows=0 loops=4)
 |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
```

Timing wise, it takes around 2s to add the index on a table with 1.2M rows
for Postgres. And it takes around 5s on MySQL on a table with 300K rows.
It looks like it takes longer on MySQL, but since both migrations are
non-locking, it should be fine.

https://mattermost.atlassian.net/browse/MM-63756
```release-note
NONE
```
2025-04-24 12:11:28 +05:30
Jesse Hallam
011f179831 avoid SELECT * in preference store (#30835) 2025-04-23 15:06:04 -04:00
Pablo Vélez
6ae0efd285 MM61173 - settings modal base creation (#30338)
* MM-61173 - channel settings modal: base modal, initial commit, file creation and base component

* new enhancements to the base modal creation

* revert changes on textbox_links and edit channel header

* fix types and add back unintentioned deleted value

* add the preview textbox component

* extract logic for info tab into its own component

* add the purpose input to the window

* move other component logic to its own component and code clean up

* ability to update channel type

* more advances on the archive channel tab

* fix unit test in textbox

* fix translations

* do not show the archive modal in default channel

* fix issue with url editor not being resetted on undo action

* adjust text and styling for the header and purpose inputs

* remove textboxlinks and use button eye icon

* adjust test and preview button style

* add unit test to channel patch

* move logic from parent modal to info tab component

* fix border issues and focus back to preview textareas

* prevent saving changes when pressing enter when selecting an icon

* enhance input component to cover limits validations and enhances tests

* set default error message for save changes panel

* add props to provide custom value to the buttons

* remove channel input errors on reset button click

* create new component settings textbox

* rename component to advanced textbox and add unit tests

* styling of the info tab and add error state to advanced textbox

* add logic to prevent tab switch with unsaved changes

* adjust url error logic and code clean up

* code clean up and enhance comments

* add char min length to advanced textbox logic

* add the channel settings modal to the new menu

* add new test files and fix reset error

* remove unused error variables

* adjust translations and remove unncesary import

* enhance permissions for archive channels and manage channel settings

* Adjust permission tree so channel admins can convert from private to public

* enhance the test suit around channel conversion type

* fix some e2e tests and solve channel input name issue

* fix unit test by interacting first with the input element

* adjust e2e tests to channel settings modal changes

* remove commented tests and implement pr feedback

* adjust more pr feedback to the code

* more pr feedback enhancements

* further enhancements to tab navigation, and adjust more e2e tests

* remove unused components and fix e2e tests

* revert unnecessary permissions changes

* Add name label to textboxes

* adjust e2e and unit tests

* revert min lenght change value and adjust tests and snapshots

* Channel banner settings (#30721)

* Added channel banner setting header

* Updated section styling

* handled animation

* handled min and max lengths

* cleanup

* color change fix

* general improvements

* Fixed API test

* removed unused param className

* added e2e tests

* test: add channel settings configuration tab test file

* Based on the context, here's a concise commit message for this change:

feat: Add comprehensive tests for ChannelSettingsConfigurationTab

* added some more tests

* CI

* reverted package-lock.json changes in Playwright

* remove extra border from advaced textbox

* adjust styling for name label in advance texbox and restart preview state on modal close

* sync package.lock in playwright

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Harshil Sharma <harshilsharma63@gmail.com>
Co-authored-by: Harshil Sharma <18575143+harshilsharma63@users.noreply.github.com>
2025-04-23 12:49:54 +02:00
Jesse Hallam
f6aeca7e50 avoid SELECT * in jobs store (#30832) 2025-04-23 12:11:09 +05:30
Jesse Hallam
3c9b2511bc avoid SELECT * in audit store (#30829) 2025-04-23 08:37:06 +02:00
Jesse Hallam
94bc32b50c avoid SELECT * in cluster discovery store (#30830) 2025-04-23 05:08:43 +08:00
Jesse Hallam
b095ba22b8 avoid SELECT * in notify admin store (#30834) 2025-04-22 17:27:33 -03:00
Jesse Hallam
018c909e2f avoid SELECT * in command stores (#30831)
* avoid SELECT * in command stores

* Define column names once and reuse as columns list in selects and inserts

* Define column names once and reuse with ExecBuilder
2025-04-22 19:37:10 +00:00
Jesse Hallam
981d1d869a avoid SELECT * in channel member history store (#30828) 2025-04-22 15:33:32 -04:00
Jesse Hallam
eb8aaba1bf avoid SELECT * in link metadata store (#30833)
* avoid SELECT * in link metadata store

* Address PR comment: define column names once and reuse
2025-04-22 19:29:09 +00:00
Jesse Hallam
701ddc896a MM-63791: guest permissions to teams (#30789)
* improve th.CreateGuestAndClient

* test coverage for guest user access to teams

* restrict guest access to public teams unless a member
2025-04-22 10:12:22 -03:00
Agniva De Sarker
d8dbb6cc22 MM-56548: [AI assisted]Add support for incremental thread loading using UpdateAt timestamp (#30486)
Every time we load the RHS, we used to load the FULL thread always. Although
the actual ThreadViewer React component is virtualized, and the server side
API call is paginated, we still went through all the pages, to get the full
thread and passed it on to the ThreadViewer. This would be for first loads,
and subsequent loads of the same thread.

This was a bug originally, but then it was a necessity after we applied websocket event scope because
now we won't get emoji reactions of a thread if the user is not on the thread.

To fix that, we enhance the thread loading functionality by adding support for fetching
thread updates based on the UpdateAt timestamp. Now, for subsequent loads,
we only get the changed posts in a thread. The implementation:

- Adds new API parameters: fromUpdateAt and updatesOnly to the GetPostThread endpoint
- Updates database queries to support sorting and filtering by UpdateAt
- Implements thread state management to track the last update timestamp
- Adds client-side support to use incremental loading for improved performance
- Ensures proper validation for parameter combinations and error handling

This change enables more efficient thread loading, particularly for long threads
with frequent updates, by only fetching posts that have been updated since the
last view.

Caveats: For delta updates, the SQL query won't use the best index possible
because we have an index for (CreateAt, Id), but no index for (UpdateAt, Id).
However, from my tests, it is not as bad as it looks:

```
[loadtest] # EXPLAIN (ANALYZE, BUFFERS) SELECT * FROM Posts WHERE Posts.DeleteAt = 0 AND Posts.RootId = 'qbr5gctu9iyg8c36hpcq6f3w8e' AND Posts.UpdateAt > 1623445795824 ORDER BY UpdateAt ASC, Id ASC LIMIT 61;
                                                                   QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------
 Limit  (cost=8.31..8.31 rows=1 width=216) (actual time=0.047..0.049 rows=0 loops=1)
   Buffers: shared hit=2
   ->  Sort  (cost=8.31..8.31 rows=1 width=216) (actual time=0.044..0.045 rows=0 loops=1)
         Sort Key: updateat, id
         Sort Method: quicksort  Memory: 25kB
         Buffers: shared hit=2
         ->  Index Scan using idx_posts_root_id_delete_at on posts  (cost=0.28..8.30 rows=1 width=216) (actual time=0.031..0.032 rows=0 loops=1)
               Index Cond: (((rootid)::text = 'qbr5gctu9iyg8c36hpcq6f3w8e'::text) AND (deleteat = 0))
               Filter: (updateat > '1623445795824'::bigint)
               Buffers: shared hit=2
 Planning:
   Buffers: shared hit=3
 Planning Time: 0.508 ms
 Execution Time: 0.106 ms
(14 rows)
```

We still get an index scan with index cond. Although there's a filter element, but atleast we get the whole thread with the index.
My thinking is that while the whole thread might be large, but after that, updates on a thread should be incremental.
Therefore, we should be okay without adding yet another index on the posts table.

This is just the first step in what could be potentially improved further.

1. We shouldn't even be loading the full thread always. But rather let the virtualized viewer
load more posts on demand.
2. If a post has been just reacted to, then we need not send the whole post down, but just the
reaction. This further saves bandwidth.

https://mattermost.atlassian.net/browse/MM-56548

TBD: Add load-test coverage to update the thread loading code

```release-note
NONE
```
---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-04-22 10:43:13 +05:30
Jesse Hallam
b59f10cbbd MM-62158: Group Store, explicit aliases (#30741) 2025-04-17 17:37:28 -03:00
Jesse Hallam
4a93939359 MM-63728: Add license load metric endpoint and UI indicator (#30700)
* Add license load metric endpoint and UI indicator

Adds an API endpoint to calculate and return license usage as a load metric, and displays this metric in the About dialog. The metric is calculated as (MAU/licensed users)*100.

Additionally:
- Renamed function to be consistent with API endpoint name
- Added proper i18n strings for error messages and UI elements

* Fix TypeScript null check in about_build_modal.tsx

* MM-63728: Update OpenAPI documentation for license load metric

Update the OpenAPI documentation and code comments to correctly describe the license load metric calculation as using a multiplier of 1000 instead of percentage.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* MM-63728: Use float for license load metric calculation

Modify the license load metric calculation to use floats throughout the computation process while still returning an integer result. This maintains the existing API but improves the precision of the calculation.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* improve tests manually

* Update server/channels/api4/license_test.go

Co-authored-by: Doug Lauder <wiggin77@warpmail.net>

* Update server/channels/api4/license_test.go

Co-authored-by: Doug Lauder <wiggin77@warpmail.net>

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
2025-04-17 17:29:46 -03:00
Miguel de la Cruz
3df7bfca88 Improves validation and sanitization for CPA fields and values (#30694)
This change automatically removes options and sync attributes when
sanitizing fields that don't support them. As per values, it returns
an error when the value for a text type field is longer than the 64
characters limit we're currently applying.

The PR fixes a bug on the create CPA field endpoint that was causing
the attrs of the CPAField not to be decoded correctly.

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
2025-04-16 16:04:30 +02:00
Nick Misasi
495a49b896 Feature/audit certificate upload (#30223)
* feat: Add certificate upload option for audit logging settings

* Commit current changes

* Additions

* MM-62944 Fix fileupload settings not being clickable

* Support for uploading a cert for experimental audit logging cert. Pre cloud implementation in the backend

* Forgot to add new hook

* Add support for setting custom audit log certifcates in Cloud

* Permissions

* I18n

* Change order

* Linter fixes

* Linter fixes, add openapi spec

* additions for openapi

* More openapi fixes because it won't run locally

* Undo, cursor went rogue

* newline fix

* Align types properly

* Fix i18n

* Fix i18n AGAIN

* Fix error

* Update api/v4/source/audit_logging.yaml

---------

Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-04-16 09:34:18 -04:00
Ben Schumacher
efd17a37ea [MM-28948] Fix errcheck linter issues in import_functions.go (#30615)
Co-authored-by: Claude <noreply@anthropic.com>
2025-04-15 09:53:38 +02:00
Ben Schumacher
b7f89984db [MM-61076] Fix errcheck issues in server/channels/web/saml.go (#30612)
Co-authored-by: Claude <noreply@anthropic.com>
2025-04-15 09:52:28 +02:00
Ben Schumacher
b8ad438c0a [MM-61515] Fix errcheck linter issues in webhook_test.go (#30684) 2025-04-15 09:50:22 +02:00
Agniva De Sarker
848bac2bae MM-53739: replace store.NewErrNotFound with errors.Wrap in category queries (#30712)
It was a mistake to return ANY error as NewErrNotFound. Because this
can be a DB timeout, or any other network error.

Returning NewErrNotFound would categorize it as 404, eventually printing
the error in the DEBUG level. Changing it to normal error fixes this.

https://mattermost.atlassian.net/browse/MM-63739

```release-note
NONE
```
2025-04-14 21:11:59 +05:30
Jesse Hallam
b84d913243 flaky test: Avoid team names that start with "A". (#30651)
This avoids a non-zero chance we append "pi" from the `model.NewId` and
end up with an invalid prefix "Api".

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-04-14 12:47:38 +00:00
Jesse Hallam
949a19efc1 MM-62156: Avoid SELECT * in retention_policy_store.go (#30458)
* MM-62156: Avoid SELECT * in retention_policy_store.go

- Modified subQueryIN function to use specific column name instead of SELECT *
- Improved code comments to explain the change
- Maintained same functionality while avoiding SELECT *

Fixes: https://mattermost.atlassian.net/browse/MM-62156

🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>

* simplify subQueryIN comments

* inline part of subQueryIN for greater clarity

---------

Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-04-14 09:21:13 -03:00
Arya Khochare
0f860f0512 Fixed errcheck issues in server/channels/app/integration_action.go (#29040)
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
2025-04-14 13:56:05 +02:00
catalintomai
c23f44fe8e MM-63316: Guest access to channel (#30467) 2025-04-14 13:07:50 +02:00
catalintomai
04676582cd MM-63342:Bot accounts OAuth gating (#30466) 2025-04-14 11:51:46 +02:00
kasyap dharanikota
ae046fb34e fix: handle error from InvalidateAllCaches in slack.go (#30606)
* fix: hanlde error from InvalidateAllCaches in slack.go

* change signature of InvalidateAllCaches to *model.AppError

* return  err from InvalidateAllCaches everywhere

* Formatting

---------

Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
2025-04-14 10:24:21 +02:00
Ben Schumacher
4d0109feeb [MM-61082] Fix errcheck issues in license_test.go (#30617)
* [MM-28754] Fix errcheck issues in license_test.go

- Properly handle error returned from os.WriteFile
- Remove license_test.go from errcheck exceptions list in .golangci.yml

Fixes https://github.com/mattermost/mattermost/issues/28754

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Remove empty line

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-04-14 09:28:28 +02:00
Pablo Vélez
02c7678438 MM-63590 - validate user has proper permission when updating team privacy (#30650)
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-04-12 17:29:50 +02:00
Ben Schumacher
b8bf2e235d [MM-61463] Fix errcheck issues in post_helpers_test.go (#30609)
* [MM-61463] Fix errcheck issues in post_helpers_test.go

- Added proper error handling for System.Save calls
- Removed post_helpers_test.go from errcheck ignore list in .golangci.yml

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Add back         channels/app/plugin_test.go|

* Fix server/.golangci.yml

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-04-11 13:51:08 +02:00
Ben Schumacher
545de88486 [MM-61079] Fix error checking in webhook.go (#30611)
* MM-28751 Fix error checking in webhook.go

- Implement proper error checking for r.ParseForm()
- Implement proper error checking for r.ParseMultipartForm()
- Implement proper error checking for w.Write()
- Remove webhook.go exception from .golangci.yml

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

* Fix translation

---------

Co-authored-by: Claude <noreply@anthropic.com>
2025-04-11 12:17:43 +02:00
Arya Khochare
2e2782e4bf Fixed errcheck issues in server/channels/app/file_bench_test.go (#29003)
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-04-11 12:12:59 +02:00
Ben Schumacher
748f8227e3 Fix errcheck error in plugin_test.go by properly handling error from th.App.ch.RemovePlugin (#30608) 2025-04-11 10:42:16 +02:00
David Krauser
f53625d59f [MM-63693] Don't log PING websocket events (#30669) 2025-04-10 16:24:23 -04:00
Jesse Hallam
42274b9eee MM-63200: unrestricted local admin (#30295)
* use SessionHasPermissionToCheckRestrictedAdmin

* allow unrestricted config edits from localmode

* check model.PermissionManageSystem for getLatestVersion

* simplify/clarify RequestTrialLicense semantics

* rename for clarity

* whitespace from linter

---------

Co-authored-by: Mattermost Build <build@mattermost.com>
2025-04-10 22:22:03 +03:00
Miguel de la Cruz
ca9fd45408 Adds a mechanism to delete CPA values for a given user (#30330)
* Adds a mechanism to delete CPA values for a given user

This requires improving the Property Value service to enable delete
all values for a given target, so a new method was created that allows
to delete filtering by targetType and targetID (required) and
optionally for a specific groupID in case the caller wants to affect
all values for a target (useful in case you remove a post for example
and want to delete all values pointing to that post regardless of the
feature they belong to) or only those that belong to a specific
feature.

* Fix property value tests

* Fix after merge and update method name

* Fix linter

---------

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-04-10 19:22:05 +02:00
Miguel de la Cruz
3ab0da1648 Adds direct participants to the channel invite (#30404)
* Adds direct participants to the channel invite

The channel invite now contains the sanitized users that are local to
the node that is sending the invite. In the event that the receiving
server doesn't have those users in its local database, it can create
them from the invite and correctly generate the DM or GM with them as
members.

* Use IsRemote instead of directly checking user attributes

---------

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-04-10 19:19:59 +02:00
catalintomai
e7e80634d6 MM-43598: Bulk export not exporting non-thread DMs from deactivated users (#30427) 2025-04-10 15:12:33 +02:00
Miguel de la Cruz
0c8e30da4d Move the sanitization and validation of CPA values to the model (#30653)
* Move the sanitization and validation of CPA values to the model

* Fix CI

* Use proper IDs instead of strings

---------

Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
Co-authored-by: Mattermost Build <build@mattermost.com>
2025-04-10 11:31:40 +02:00