* Return an error seperately from Response
* Remove BuildErrorResponse
* Drop Response.Error from model/client4.go
* Migrate require.Nil checks
* Migrate require.NotNil checks
* More manual fixes
* Move error check out of CheckOKStatus and CheckCreatedStatus
* Move error check out of CheckForbiddenStatus
* Move error check out of CheckUnauthorizedStatus
* Move error check out of CheckNotFoundStatus
* Move error check out of CheckBadRequestStatus
* Move error check out of CheckNotImplementedStatus and CheckRequestEntityTooLargeStatus
* Move error check out of CheckInternalErrorStatus
* Move error check out of CheckServiceUnavailableStatus
* Remove error check from checkHTTPStatus
* Remove remaining references to Response.Error
* Check previously unchecked errors
* Manually fix compile and linter errors
* Return error in CreateWebSocket methods
* Return error instead of *AppError in DoApi methods
* Manually fix bad replacments
* Conistently return Response and error
* Use err instead of seperate bool return value to indicate success
* Reduce ussage of model.AppError in web/oauth_test.go
* Remove client4.Must
* Check error in buf.ReadFrom
* Fix failing tests
* MM-22845: Added support for permalink previews.
* MM-22845: Adds license to new file.
* MM-22845: Adds endpoint to retrieve multiple posts by id.
* MM-22845: Fix for deleted post.
* MM-22845: Adds config setting for permalink previews.
* MM-22845: Adds API test for new endpoint.
* MM-22845: Fix typo.
* MM-22845: Tests that post create or updated via App get the previewed_post prop.
* MM-22845: Tests for matching permalinks.
* MM-22845: Adds PreparePostForClient test for permalink previews.
* MM-22845: Embeds entire post in permalink metadata.
* MM-22845: Filter WS message payload of created and edited post based on permissions.
* MM-22845: Runs app layer generator.
* MM-22845: Lint check fix.
* MM-22845: Adds feature flag.
* MM-22845: Clones WS message.
* MM-22845: Removes knowledge of permalink from LinkMetadata table. Removes knowledge of user id from post embedding methods in favour of a 'sanitize' method/step.
* MM-22845: Handle nil post metadata.
* MM-22845: Switch to cloning post.
* MM-22845: Removes unused code.
* MM-22845: Refactor.
* MM-22845: Reverts whitespace change.
* MM-22845: Removes unnecessary code.
* MM-22845: Removes unnecessary function.
* MM-22845: Warn but don't error if permalinked referenced post or channel is not found.
* MM-22845: Fix for clone method.
* MM-22845: Fix for clone method.
* MM-22845: Updates translations.
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-37299 - add the invite users feature flag value for AB testing
* make the ff a string value
* add suggested changes
Co-authored-by: Pablo Velez Vidal <pablo.velez@mattermost.com>
https://mattermost.atlassian.net/browse/MM-22051
```release-note
Removed the following methods/functions:
(ad *AccessData) ToJson()
(ar *AccessResponse) ToJson()
(ar *AnalyticsRow) ToJson()
(ar AnalyticsRows) ToJson()
(o *Audit) ToJson()
(o Audits) ToJson()
(ad *AuthData) ToJson()
(ar *AuthorizeRequest) ToJson()
(o *ChannelPatch) ToJson()
(o *ChannelsWithCount) ToJson()
(o *ChannelCounts) ToJson()
(o *ChannelData) ToJson()
(o *ChannelMembers) ToJson()
(o *ChannelUnread) ToJson()
(o *ChannelUnreadAt) ToJson()
(o *ChannelStats) ToJson()
(o *ChannelView) ToJson()
(o *ChannelViewResponse) ToJson()
(o *ClusterDiscovery) ToJson()
(ci *ClusterInfo) ToJson()
(cs *ClusterStats) ToJson()
(o *Command) ToJson()
CommandListToJson(l []*Command) string
(o *CommandArgs) ToJson()
(cmr *CommandMoveRequest) ToJson()
(o *CommandResponse) ToJson()
(c *Compliance) ToJson()
(c Compliances) ToJson()
(o *Config) ToJson()
EmojiListToJson(emojiList []*Emoji)
```
* MM-25516: Changed to byte slice instead of string for cluster messages
https://mattermost.atlassian.net/browse/MM-25116
Testing:
Manually tested.
Load-tested with Cluster Controller.
I looked into changing the serialization method to use msgpack,
but the ClusterMessage struct was mainly used for only 3 fields
which didn't lead to much of a CPU time improvement, whereas
actually led to more allocations using msgpack. Hence, I chose
to remain with JSON.
```
name old time/op new time/op delta
ClusterMarshal-8 3.51µs ± 1% 3.10µs ± 2% -11.59% (p=0.000 n=9+10)
name old alloc/op new alloc/op delta
ClusterMarshal-8 776B ± 0% 1000B ± 0% +28.87% (p=0.000 n=10+10)
name old allocs/op new allocs/op delta
ClusterMarshal-8 12.0 ± 0% 13.0 ± 0% +8.33% (p=0.000 n=10+10)
```
```release-note
Changed the field type of Data in model.ClusterMessage to []byte from string.
```
* Trigger CI
```release-note
NONE
```
* Cleanup model/utils.go
- Removed some functions which were unsued. (The unused linter was somehow failing to catch this).
- Moved some functions inside other packages.
- Removed two duplicate instances of the same function - RemoveDuplicateStrings and UniqueStrings.
- Moved some regexes to global vars to prevent compilation every time.
```release-note
NONE
```
* fix lint
```release-note
NONE
```
* bring back unused exception
```release-note
NONE
```
* fix tests
```release-note
NONE
```
* Address review comments
```release-note
NONE
```
* MM-21357: Use typed constant for channel types
https://mattermost.atlassian.net/browse/MM-21357
```release-note
- Introduced a new type ChannelType for all channel types.
- Updated the Client4.UpdateChannelPrivacy method to ChannelType.
```
* Address review comments
```release-note
NONE
```
* telemetry fix
```release-note
NONE
```
* Move some variables out of model
There were some stuff that's not necessary to be in model
and can just remain unexported variables.
```release-note
NONE
```
* Add license
```release-note
NONE
```
Summary:
The CRT backend was first released in version 5.29.0. Since then, the behaviour of the CRT feature has been refined, several bugs have been fixed, and a few sql columns have been added. Before these various fixes went in, the threads and channel membership tables have accumulated incorrect mention and unreads data. This PR fixes some of this bad historical data.
Summary of fixes:
- Marks threads as read for users where the last reply time of the thread is earlier than the time the user viewed the channel. Marking a thread means setting the mention count to zero and setting the last viewed at time of the the thread as the last viewed at time of the channel. This is done through a "sql migration"
- Fix channel counts, i.e. the total message count, total root message count, mention count, and mention count in root messages for users who have viewed the channel after the last post in the channel. This is done as a "sql migration"
Ticket Link:
Parts of
https://mattermost.atlassian.net/browse/MM-35494https://mattermost.atlassian.net/browse/MM-35345
* pre-checkout commit
* add API endpoints for retention policies
* allow deleting multiple teams/channels from a policy in a single request
* pre-checkout commit
* add auditing in API functions
* add permission checks
* update the store layers
* update storetest
* add check constraint on PostDuration column
* pre-checkout commit
* add query to delete posts under the scope of a granular retention policy
* add suggestions from sbishel
* allow clients to specify channels/teams when creating a new policy
* remove foreign keys referencing Channels and Teams tables
* add checks for whether teams and channels exist
* pre-checkout commit
* remove data referencing the Posts table
* pre-checkout commit
* write data store tests
* sort results of buildGetPoliciesQuery
* add missing test cases for teams
* pre-checkout commit
* add Client4 methods for data retention policy endpoints
* add uint and uint64 to app/layer_generators
* make granular policies override global policies
* fix lint errors
* pre-checkout commit
* add license to top of files
* add tests for data store layer
* add missing test cases for store layer
* run make i18n-extract
* add query to delete ChannelMemberHistory
* work in progress
* add test for old reply to old post
* fix lint error
* use COALESCE on each Posts column
* begin implementing orphaned rows worker
* split PR
* pre-checkout commit
* use RetentionPolicyWithTeamAndChannelCounts
* update app and api layers
* run make i18n-extract
* add RetentionPolicy to retrylayer_test.go
* Revert "split PR"
This reverts commit b316f03dd307a30deae931944ca7e4a1cc904605.
* fix errors caused by revert
* add suggestions from sbishel
* fix copy-paste error
* fix lint errors
* pre-checkout commit
* add function to delete orphaned rows
* use -1 for infinite retention
* remove check constraint
* copy i18n entries from master
* re-run tests with newer enterprise branch
* add team data to channel list
* add search for channels and teams in a policy
* add store tests for channel and team search
* add suggestions from mkraft
* run make einterfaces-mocks
* fix lint errors
* add suggestions from mkraft
* move removeOrphanedRows method to wireup branch
* Revert "move removeOrphanedRows method to wireup branch"
This reverts commit 94605c9b4a5378ffa44a3dec4d3f8e3306b9d33e.
* use DeleteOrphanedRows where possible
* run make i18n-extract
* use COMPLIANCE permissions
* run make migrations-bindadta
* clean up teams before test
* fix tests for TestRetentionPolicyStore
* add API endpoints for mobile
* fix lint error
* fix some of the lint errors
* move user/data_retention endpoints to data_retention.go
* Revert "fix some of the lint errors"
This reverts commit b5b2dc27566c427187db942c5c0afe319e7679c4.
* add exclude_policy_constrained parameter for /channels and /teams
* fix lint errors
* add policy_id field to GET endpoints for channels and teams
* use PolicyWithTeamID in RetentionPolicy layer
* fix lint errors
* run make i18n-extract
* update mock call in telemetry_test.go
* return status:OK in JSON instead of 204
* pre-checkout commit
* add policy_id field on channels/teams
* fix lint errors
* use sq.Eq instead of '?'
* use new subsection permissions
* update channels and teams endpoints to use new subsection permissions
* add extra search opts for channels in a policy
* fix lint errors
* allow negative post duration in patch
* remove DELETE FROM query in retention policy tests
* use *int64 for PostDuration
* re-run CI tests
* use 3-step deletion strategy for each table
* fix lint errors
* run make store-layers
* re-run CI tests
* add test with channel, team and global policies
* use common function for SQL queries
* add pagination test
* use struct for args to common SQL function
* fix lint errors
* run make i18n-extract
* check if Channels.TeamId is "" or nil
* use three OR clauses
* write separate genericRetentionPoliciesDeletion function
* add config setting for BatchSize
* add telemetry for BatchSize
* use feature flag
* add old i18n messages back in
* re-run CI tests
* update call signature in storetest
* MM-30831: Adds constant for retention default batch size.
* MM-30831: Removes comment re: optimization.
* MM-30831: Converts days to milliseconds.
* MM-30831: Reverts change to test.
* Revert "MM-30831: Reverts change to test."
This reverts commit 6d14275a1ceae682bb9e17ec69b39252b44e0c0c.
* Revert "MM-30831: Converts days to milliseconds."
This reverts commit a0cb6ec09d854a05194c1daee1c5333f260231c3.
* MM-30831: Fixes tests.
* MM-30381: Fix for change to method sig.
Co-authored-by: Max Erenberg <max.erenberg@mattermost.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Martin Kraft <martin@upspin.org>
