* Add playbooks related permissions.
* Add RolesGrantPermission to pluginapi
* Fixing scopes.
* New defaults.
* Fix defaults
* Fix tests.
* Fix migration.
* More test and migration fixes.
* Need to add everything to system admin too.
* Move to 63
* Feedback fixes.
* Fix system manager editing playbook permissions.
* MM-36862: removes participant from thread
Removing a participant upon last reply deleted from thread didn't work
reliably, a suspect on this is the replica lag, since we are first
deleting the post and then counting non-deleted posts of the participant
to decide on whether to delete or not.
The findings that led to this conclusion is that the reply count gets
updated but the participant is not removed (participant removal depends
on the number of replies this participant has in the thread.)
This commit fixes that by removing first the participant and then
deleting the post.
So we delete the participant if they have 1 post in that thread, and
then we delete the post, so now they have no posts in the thread.
* Makes deleting posts transactional
This commit makes deleting a post transactional and also tries to fix
permanent deletion of posts.
Currently when we permanently delete all posts by a user we don't update
the threads reply count nor the participant's array. This commit tries
to fix that.
* Adds comments on deleting posts
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Summary
The addition of the TotalMsgCountRoot and MsgCountRoot columns to support CRT caused several issues with previously read threads and channels being marked as unread. Previously we attempted to fix this purely in a SQL migration [MM-35345][MM-35494] fixes for incorrect mentions and unreads for threads and channels #17803 but that turned out to be too heavy and it was decided to break up some of the fixes into async jobs.
This PR implements an async job to mark channels as read if there are no user posts since the last time the user viewed the channel.
Ticket Link
https://mattermost.atlassian.net/browse/MM-37013
* Show private channels in autocomplete
This is supported in all Engines:
MySQL, Postgres, Bleve, Elasticsearch.
https://mattermost.atlassian.net/browse/MM-18496
```release-note
Private channels will now appear in channel autocomplete.
If you are using Bleve or ElasticSearch, you will have to reindex
the channels again to populate them with the new attributes.
```
A large chunk of this work has been based on the earlier
effort at https://github.com/mattermost/mattermost-server/pull/17804.
Full credit goes to https://github.com/arvinDarmawan.
* Add comment
```release-note
NONE
```
* Adding more tests
```release-note
NONE
```
* fix more tests
```release-note
NONE
```
* tmp
```release-note
NONE
```
* more fixes
```release-note
NONE
```
* add tests
```release-note
NONE
```
* Add review comments from previous PR
```release-note
NONE
```
* Add API to return all channels from all team
```release-note
NONE
```
* Added support for bleve and ES
```release-note
NONE
```
* Streaming response for GetAllChannels
```release-note
NONE
```
* Fix tests
```release-note
NONE
```
* Trigger CI
```release-note
NONE
```
* fix tests
```release-note
NONE
```
* Addressing review comments
```release-note
NONE
```
* Fix lint
```release-note
NONE
```
* Removing flaky test
```release-note
NONE
```
* Address comments
```release-note
NONE
```
* Trigger CI
```release-note
NONE
```
* Added /users/<userid>/channel_members endpoint
```release-note
NONE
```
* Minor edit
```release-note
NONE
```
* Improve embedding
```release-note
NONE
```
* Fix lint error
```release-note
NONE
```
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-36862: should remove user from participants
When deleting a reply in a thread we should also delete the participant
from the participants array. This should happen if they have no other
replies in that thread.
This commit fixes that.
* Adds warning logs
* Delete Post: fetches participants only when needed
* Minor refactor
* Remove participant should check for error in count
* Fixes error when binary_parameters=true
* Fixes: removes ArrayToJSON so we can handle the error
* Satisfies the linter
* Better error handling
* Satisfy govet
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Add functionality to cleanup old jobs
Historically, we never cleaned up old jobs from the DB
leading to them being accumulated forever.
This PR introduces functionality to cleanup old jobs
older than a defined threshold.
The functionality is set to false by default and has
to be enabled for it to work.
```release-note
2 new config settings were added.
JobSettings.CleanupOldJobs: This indicates whether to clean up old jobs
from the DB or not. Default is false.
JobSettings.CleanupJobsThresholdHours: This defines the time gap in hours beyond
which older jobs will be removed. This has no effect if the above config
setting is set to false. Default is -1
```
* fix copy pasta
```release-note
NONE
```
* address review comments
```release-note
NONE
```
* Fix lint
```release-note
NONE
```
* Use single config option for everything
```release-note
NONE
```
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-36743: adds last_root_post_at in channels table
Channel recency for CRT users should not count replies,
this commit solves that issue by adding a new column to the channels
table: LastRootPostAt.
With that new info CRT users can have recent channels to work as
expected.
* Adds the 'LastRootPostAt' to the SQL scripts
* Fixes tests
* Adds LastRootPostAt migration to v6.1
* Fixes index
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Add API endpoint and adapt search to allow multi-team search
* Refactor handler, refactor sql query to use squirrel, rename app and store functions and add tests
* Fix lint
* Fix search engines and remove unneeded comments
* Fix test
* Remove user from channel after test
* MM-35017: order and inclusion of participants
From now on root poster is not considered a thread participant,
only repliers are participants.
Ordering: A second reply to the thread from the same user should place
the user last at the participants list.
* Fixes shadow
* Fixes tests
* Order By CreateAt for participants of old threads
* Removes unnecessary check on root id
* Removes GetParticipantProfilesByIds it's not needed
At first GetParticipantProfilesByIds was created because GetProfileByIds
gets users by order of username.
This was not needed, we can order by user ids in Go.
This commit removes GetParticipantProfilesByIds and replaces it by
GetProfileByIds as it was done before.
* Fixes participants order for pre CRT threads
Older threads (before CRT) are being converted to CRT threads upon
reply.
This commit considers the order of participants to be inserted to
the new thread.
* Adds tests for thread participants
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
We properly truncate the date part from the createAt time.
The SQL query is improved now to directly return the bool.
Fixed the test to properly test the feature.
https://mattermost.atlassian.net/browse/MM-38132
```release-note
NONE
```
* CRT: desktop thread notifications
* Fixes go lint
* Adds default for desktop CRT notifications
* Adds email and push notifications for CRT threads
Adds user ids of thread followers with CRT to crtMentions so they will get
notified appropriately.
* Minor change
* Refactor a bit
CRTMentions.addMention had a bug on the return and de-duplication.
This commit fixes duplicate notifications by looking up if the user is to be
notified on CRT on both email and push notifications.
* Minor refactor
* Changes according to review comments
- Fixes adding to followers a user that had explicitly unfollowed a
thread.
- Simplified send email according to email_threads option
- Send mentions and followers in separate arrays via the websocket
- Fixes push notifications message for push_threads
* Adds a comment on a buggy use case
* Updates comment to correct ticket link
* Fixes when user notifications is set to all
There was a bug where if user had set notifications to all
then they would receive desktop notifications even for non following threads.
A similar bug existed in push notifications, where if a user has set it
to all the threads setting would still be considered.
This commit fixes that by adding users to notificationsForCRT
StringArray when they have the non thread setting to 'all'.
* Fixes notifications to users unfollowing threads
Users which had previously explicitly unfollowed a thread
should not receive notifications about those threads.
* Update store mocks
* Fixes push notifications for CRT
Push notification about replies for CRT users should have a title of
"Reply to Thread".
CRT users with global user setting to 'UserNotifyAll' should not get
notifications for unfollowed threads.
This commit fixes those issues.
* Fixes i18n error
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
- We move logging statements to the upper layer.
Store functions are low-level methods and should return error
upwards rather than logging.
- Used IN instead of any (array ()) which is equivalent.
- Made the delay to be of type time.Duration and un-exported it.
- Unexported the batch size constant.
```release-note
NONE
```
* MM-21357: Use typed constant for channel types
https://mattermost.atlassian.net/browse/MM-21357
```release-note
- Introduced a new type ChannelType for all channel types.
- Updated the Client4.UpdateChannelPrivacy method to ChannelType.
```
* Address review comments
```release-note
NONE
```
* telemetry fix
```release-note
NONE
```
* nested sql query for reply_count in getSingle post method of post_store.go
* added tests for replyCount in GetSingle api
* updated the post object message field in GetSingle unit test to use model.NewRandomString
Summary:
The CRT backend was first released in version 5.29.0. Since then, the behaviour of the CRT feature has been refined, several bugs have been fixed, and a few sql columns have been added. Before these various fixes went in, the threads and channel membership tables have accumulated incorrect mention and unreads data. This PR fixes some of this bad historical data.
Summary of fixes:
- Marks threads as read for users where the last reply time of the thread is earlier than the time the user viewed the channel. Marking a thread means setting the mention count to zero and setting the last viewed at time of the the thread as the last viewed at time of the channel. This is done through a "sql migration"
- Fix channel counts, i.e. the total message count, total root message count, mention count, and mention count in root messages for users who have viewed the channel after the last post in the channel. This is done as a "sql migration"
Ticket Link:
Parts of
https://mattermost.atlassian.net/browse/MM-35494https://mattermost.atlassian.net/browse/MM-35345
* pre-checkout commit
* add API endpoints for retention policies
* allow deleting multiple teams/channels from a policy in a single request
* pre-checkout commit
* add auditing in API functions
* add permission checks
* update the store layers
* update storetest
* add check constraint on PostDuration column
* pre-checkout commit
* add query to delete posts under the scope of a granular retention policy
* add suggestions from sbishel
* allow clients to specify channels/teams when creating a new policy
* remove foreign keys referencing Channels and Teams tables
* add checks for whether teams and channels exist
* pre-checkout commit
* remove data referencing the Posts table
* pre-checkout commit
* write data store tests
* sort results of buildGetPoliciesQuery
* add missing test cases for teams
* pre-checkout commit
* add Client4 methods for data retention policy endpoints
* add uint and uint64 to app/layer_generators
* make granular policies override global policies
* fix lint errors
* pre-checkout commit
* add license to top of files
* add tests for data store layer
* add missing test cases for store layer
* run make i18n-extract
* add query to delete ChannelMemberHistory
* work in progress
* add test for old reply to old post
* fix lint error
* use COALESCE on each Posts column
* begin implementing orphaned rows worker
* split PR
* pre-checkout commit
* use RetentionPolicyWithTeamAndChannelCounts
* update app and api layers
* run make i18n-extract
* add RetentionPolicy to retrylayer_test.go
* Revert "split PR"
This reverts commit b316f03dd307a30deae931944ca7e4a1cc904605.
* fix errors caused by revert
* add suggestions from sbishel
* fix copy-paste error
* fix lint errors
* pre-checkout commit
* add function to delete orphaned rows
* use -1 for infinite retention
* remove check constraint
* copy i18n entries from master
* re-run tests with newer enterprise branch
* add team data to channel list
* add search for channels and teams in a policy
* add store tests for channel and team search
* add suggestions from mkraft
* run make einterfaces-mocks
* fix lint errors
* add suggestions from mkraft
* move removeOrphanedRows method to wireup branch
* Revert "move removeOrphanedRows method to wireup branch"
This reverts commit 94605c9b4a5378ffa44a3dec4d3f8e3306b9d33e.
* use DeleteOrphanedRows where possible
* run make i18n-extract
* use COMPLIANCE permissions
* run make migrations-bindadta
* clean up teams before test
* fix tests for TestRetentionPolicyStore
* add API endpoints for mobile
* fix lint error
* fix some of the lint errors
* move user/data_retention endpoints to data_retention.go
* Revert "fix some of the lint errors"
This reverts commit b5b2dc27566c427187db942c5c0afe319e7679c4.
* add exclude_policy_constrained parameter for /channels and /teams
* fix lint errors
* add policy_id field to GET endpoints for channels and teams
* use PolicyWithTeamID in RetentionPolicy layer
* fix lint errors
* run make i18n-extract
* update mock call in telemetry_test.go
* return status:OK in JSON instead of 204
* pre-checkout commit
* add policy_id field on channels/teams
* fix lint errors
* use sq.Eq instead of '?'
* use new subsection permissions
* update channels and teams endpoints to use new subsection permissions
* add extra search opts for channels in a policy
* fix lint errors
* allow negative post duration in patch
* remove DELETE FROM query in retention policy tests
* use *int64 for PostDuration
* re-run CI tests
* use 3-step deletion strategy for each table
* fix lint errors
* run make store-layers
* re-run CI tests
* add test with channel, team and global policies
* use common function for SQL queries
* add pagination test
* use struct for args to common SQL function
* fix lint errors
* run make i18n-extract
* check if Channels.TeamId is "" or nil
* use three OR clauses
* write separate genericRetentionPoliciesDeletion function
* add config setting for BatchSize
* add telemetry for BatchSize
* use feature flag
* add old i18n messages back in
* re-run CI tests
* update call signature in storetest
* MM-30831: Adds constant for retention default batch size.
* MM-30831: Removes comment re: optimization.
* MM-30831: Converts days to milliseconds.
* MM-30831: Reverts change to test.
* Revert "MM-30831: Reverts change to test."
This reverts commit 6d14275a1ceae682bb9e17ec69b39252b44e0c0c.
* Revert "MM-30831: Converts days to milliseconds."
This reverts commit a0cb6ec09d854a05194c1daee1c5333f260231c3.
* MM-30831: Fixes tests.
* MM-30381: Fix for change to method sig.
Co-authored-by: Max Erenberg <max.erenberg@mattermost.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Martin Kraft <martin@upspin.org>
