Every time we load the RHS, we used to load the FULL thread always. Although
the actual ThreadViewer React component is virtualized, and the server side
API call is paginated, we still went through all the pages, to get the full
thread and passed it on to the ThreadViewer. This would be for first loads,
and subsequent loads of the same thread.
This was a bug originally, but then it was a necessity after we applied websocket event scope because
now we won't get emoji reactions of a thread if the user is not on the thread.
To fix that, we enhance the thread loading functionality by adding support for fetching
thread updates based on the UpdateAt timestamp. Now, for subsequent loads,
we only get the changed posts in a thread. The implementation:
- Adds new API parameters: fromUpdateAt and updatesOnly to the GetPostThread endpoint
- Updates database queries to support sorting and filtering by UpdateAt
- Implements thread state management to track the last update timestamp
- Adds client-side support to use incremental loading for improved performance
- Ensures proper validation for parameter combinations and error handling
This change enables more efficient thread loading, particularly for long threads
with frequent updates, by only fetching posts that have been updated since the
last view.
Caveats: For delta updates, the SQL query won't use the best index possible
because we have an index for (CreateAt, Id), but no index for (UpdateAt, Id).
However, from my tests, it is not as bad as it looks:
```
[loadtest] # EXPLAIN (ANALYZE, BUFFERS) SELECT * FROM Posts WHERE Posts.DeleteAt = 0 AND Posts.RootId = 'qbr5gctu9iyg8c36hpcq6f3w8e' AND Posts.UpdateAt > 1623445795824 ORDER BY UpdateAt ASC, Id ASC LIMIT 61;
QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=8.31..8.31 rows=1 width=216) (actual time=0.047..0.049 rows=0 loops=1)
Buffers: shared hit=2
-> Sort (cost=8.31..8.31 rows=1 width=216) (actual time=0.044..0.045 rows=0 loops=1)
Sort Key: updateat, id
Sort Method: quicksort Memory: 25kB
Buffers: shared hit=2
-> Index Scan using idx_posts_root_id_delete_at on posts (cost=0.28..8.30 rows=1 width=216) (actual time=0.031..0.032 rows=0 loops=1)
Index Cond: (((rootid)::text = 'qbr5gctu9iyg8c36hpcq6f3w8e'::text) AND (deleteat = 0))
Filter: (updateat > '1623445795824'::bigint)
Buffers: shared hit=2
Planning:
Buffers: shared hit=3
Planning Time: 0.508 ms
Execution Time: 0.106 ms
(14 rows)
```
We still get an index scan with index cond. Although there's a filter element, but atleast we get the whole thread with the index.
My thinking is that while the whole thread might be large, but after that, updates on a thread should be incremental.
Therefore, we should be okay without adding yet another index on the posts table.
This is just the first step in what could be potentially improved further.
1. We shouldn't even be loading the full thread always. But rather let the virtualized viewer
load more posts on demand.
2. If a post has been just reacted to, then we need not send the whole post down, but just the
reaction. This further saves bandwidth.
https://mattermost.atlassian.net/browse/MM-56548
TBD: Add load-test coverage to update the thread loading code
```release-note
NONE
```
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* Add license load metric endpoint and UI indicator
Adds an API endpoint to calculate and return license usage as a load metric, and displays this metric in the About dialog. The metric is calculated as (MAU/licensed users)*100.
Additionally:
- Renamed function to be consistent with API endpoint name
- Added proper i18n strings for error messages and UI elements
* Fix TypeScript null check in about_build_modal.tsx
* MM-63728: Update OpenAPI documentation for license load metric
Update the OpenAPI documentation and code comments to correctly describe the license load metric calculation as using a multiplier of 1000 instead of percentage.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* MM-63728: Use float for license load metric calculation
Modify the license load metric calculation to use floats throughout the computation process while still returning an integer result. This maintains the existing API but improves the precision of the calculation.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* improve tests manually
* Update server/channels/api4/license_test.go
Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
* Update server/channels/api4/license_test.go
Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
This change automatically removes options and sync attributes when
sanitizing fields that don't support them. As per values, it returns
an error when the value for a text type field is longer than the 64
characters limit we're currently applying.
The PR fixes a bug on the create CPA field endpoint that was causing
the attrs of the CPAField not to be decoded correctly.
Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
* feat: Add certificate upload option for audit logging settings
* Commit current changes
* Additions
* MM-62944 Fix fileupload settings not being clickable
* Support for uploading a cert for experimental audit logging cert. Pre cloud implementation in the backend
* Forgot to add new hook
* Add support for setting custom audit log certifcates in Cloud
* Permissions
* I18n
* Change order
* Linter fixes
* Linter fixes, add openapi spec
* additions for openapi
* More openapi fixes because it won't run locally
* Undo, cursor went rogue
* newline fix
* Align types properly
* Fix i18n
* Fix i18n AGAIN
* Fix error
* Update api/v4/source/audit_logging.yaml
---------
Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
It was a mistake to return ANY error as NewErrNotFound. Because this
can be a DB timeout, or any other network error.
Returning NewErrNotFound would categorize it as 404, eventually printing
the error in the DEBUG level. Changing it to normal error fixes this.
https://mattermost.atlassian.net/browse/MM-63739
```release-note
NONE
```
This avoids a non-zero chance we append "pi" from the `model.NewId` and
end up with an invalid prefix "Api".
Co-authored-by: Mattermost Build <build@mattermost.com>
* MM-62156: Avoid SELECT * in retention_policy_store.go
- Modified subQueryIN function to use specific column name instead of SELECT *
- Improved code comments to explain the change
- Maintained same functionality while avoiding SELECT *
Fixes: https://mattermost.atlassian.net/browse/MM-62156🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* simplify subQueryIN comments
* inline part of subQueryIN for greater clarity
---------
Co-authored-by: Claude <noreply@anthropic.com>
Co-authored-by: Mattermost Build <build@mattermost.com>
* fix: hanlde error from InvalidateAllCaches in slack.go
* change signature of InvalidateAllCaches to *model.AppError
* return err from InvalidateAllCaches everywhere
* Formatting
---------
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
* [MM-61463] Fix errcheck issues in post_helpers_test.go
- Added proper error handling for System.Save calls
- Removed post_helpers_test.go from errcheck ignore list in .golangci.yml
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Add back channels/app/plugin_test.go|
* Fix server/.golangci.yml
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Adds a mechanism to delete CPA values for a given user
This requires improving the Property Value service to enable delete
all values for a given target, so a new method was created that allows
to delete filtering by targetType and targetID (required) and
optionally for a specific groupID in case the caller wants to affect
all values for a target (useful in case you remove a post for example
and want to delete all values pointing to that post regardless of the
feature they belong to) or only those that belong to a specific
feature.
* Fix property value tests
* Fix after merge and update method name
* Fix linter
---------
Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
Co-authored-by: Mattermost Build <build@mattermost.com>
* Adds direct participants to the channel invite
The channel invite now contains the sanitized users that are local to
the node that is sending the invite. In the event that the receiving
server doesn't have those users in its local database, it can create
them from the invite and correctly generate the DM or GM with them as
members.
* Use IsRemote instead of directly checking user attributes
---------
Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
Co-authored-by: Mattermost Build <build@mattermost.com>
* Move the sanitization and validation of CPA values to the model
* Fix CI
* Use proper IDs instead of strings
---------
Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
Co-authored-by: Mattermost Build <build@mattermost.com>
Fixes https://github.com/mattermost/mattermost/issues/29102
- Fix error handling in fileutils.FindDir() call by properly checking the returned boolean value
- Add proper error handling for SetPhase2PermissionsMigrationStatus calls
- Remove channels/app/support_packet_test.go from the errcheck exception list in .golangci.yml
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-authored-by: Claude <noreply@anthropic.com>
* allow opening angle bracket before links
* cut off link before an angle bracket(both open & close)
* add more test cases for ParseURLAutolink func
* add more test cases for ParseWWWAutolink func
* add more test cases for TrimTrailingCharactersFromLink func
* add more test cases for GetFirstLinkAndImages func
* add test cases for isAllowedBeforeWWWLink func
* allow closing angle bracket and opening paranthesis before links
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
This PR adds functionality to search by a global search prefix.
This allows Mattermost to be used across multiple data centers
with multiple Elasticsearch instances synchronized using
cross-cluster replication.
While here, we also add tests cases to cover for some missing
search interface methods.
For now, no system console setting is exposed. Because IndexPrefix
is also not exposed. It can be added later if a need arises.
https://mattermost.atlassian.net/browse/MM-63298
```release-note
A new config setting ElasticsearchSettings.GlobalSearchPrefix is added
which can be used to search across multiple indices having a common prefix.
This is useful in a scenario with multiple Elasticsearch instances, where
multiple instances are writing to different indices with different prefixes
using the ElasticsearchSettings.IndexPrefix setting.
```
Co-authored-by: Mattermost Build <build@mattermost.com>
* MM-62157: Avoid SELECT * in user_store.go
- Replace raw SQL query "SELECT * FROM Users" with query builder in Update method
- Replace raw SQL query "SELECT * from Users" with query builder in ClearAllCustomRoleAssignments
- Change Select("*") to Select("data.*") in GetUserReport method
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* MM-62157: Avoid more instances of SELECT * in user_store.go
- Replace raw SQL query "Users.*, cm.ChannelId" with usersQuery in getUsersInGroupChannels
- Replace "Users.*" with getUsersColumns() in GetUserReport method
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
* Adds a groupID filter to the property service methods
This allows the property service caller to directly ensure that a
given call is only going to affect a field or value that belongs to a
given group, instead of (for example) retrieving a property value
before deleting it by id to ensure that the value belongs to a
specific property group. The groupID filter is optional and has no
effect if called with the empty string value.
The changes also remove references to input sanitization on trimming
the whitespace for the CPA field names and validate at the API level
the input for the field patch endpoint.
* Fix linter
---------
Co-authored-by: Miguel de la Cruz <miguel@ctrlz.es>
Co-authored-by: Mattermost Build <build@mattermost.com>
* MM-62162: Replace SELECT * in user_access_token_store.go
- Replaced all SELECT * queries with explicit column selection
- Used query builder instead of raw SQL strings for all queries
- Added reusable userAccessTokensSelectQuery in the store constructor
- Added comprehensive test for pagination and IsActive flag
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Fix code formatting in user_access_token_store test file
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* MM-62162: Update UserAccessToken Search method to use explicit column names
- Fixed "ambiguous column" errors by explicitly naming columns with table qualifiers
- Added test for the Search functionality
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* MM-62162: Update UserAccessToken Search method to use explicit column names
- Fixed "ambiguous column" errors by explicitly naming columns with table qualifiers
- Maintained exact semantics of original query's LIKE filters
- Refactored to use query builder instead of raw SQL
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* MM-62162: Update UserAccessToken Search method to use reusable query
- Modified userAccessTokensSelectQuery to use fully qualified column names
- Updated Search method to use the existing query builder
- Fixed "ambiguous column" errors by using full column qualifiers
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* MM-62162: Remove unnecessary comment in user_access_token_store.go
Removed redundant comment in Search method explaining the use of qualified column names, as the code is self-documenting.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* MM-62162: Use t.Cleanup() for token cleanup in userAccessTokenPagination test
* MM-62162: Handle error in Cleanup function
* linting
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* MM-62154: Avoid SELECT * in webhook_store.go
- Defined column list functions for both IncomingWebhooks and OutgoingWebhooks tables
- Converted all uses of SELECT * to use the column list functions
- Used the QueryBuilder pattern for SQL queries for better consistency and readability
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* MM-62154: Improve webhook store implementation using query builders
- Refactored webhook store to follow session_store.go pattern
- Added select query builders in the constructor
- Used pre-built queries throughout the store implementation for better maintainability and performance
- Removed redundant incomingWebhookColumns() and outgoingWebhookColumns() functions
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* MM-62154: Fix whitespace style issues in webhook_store.go
- Ran gofmt to remove trailing whitespace
- Ensures consistent code formatting
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* MM-62154: Remove unnecessary int(0) casting
- Simplified DeleteAt=0 condition by removing redundant int casting
- Maintains same behavior while making code cleaner
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* MM-62160: Avoid SELECT * in user_terms_of_service.go
- Added userTermsOfServiceSelectQuery in the constructor
- Replaced raw SQL query with query builder pattern
- Used explicit column selection instead of SELECT *
- Made the implementation more resilient to schema changes
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* MM-62160: Also avoid SELECT * in terms_of_service_store.go
- Added termsOfServiceSelectQuery field to store struct
- Replaced SELECT * with explicit column selection
- Updated GetLatest and Get methods to use the query builder pattern
- Made the implementation more resilient to schema changes
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* MM-62155: Avoid SELECT * in team_store.go
- Added explicit column lists for all SELECT queries
- Created teamSelectQuery for reused queries
- Replaced raw SQL queries with query builder pattern
* Refactor getTeamMembersWithSchemeSelectQuery to use teamMembersQuery builder
Instead of duplicating TeamMembers columns, use the existing builder to avoid redundancy.
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* Remove redundant comments from getTeamMembersWithSchemeSelectQuery
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
---------
Co-authored-by: Claude <noreply@anthropic.com>
