* MM-35017: order and inclusion of participants
From now on root poster is not considered a thread participant,
only repliers are participants.
Ordering: A second reply to the thread from the same user should place
the user last at the participants list.
* Fixes shadow
* Fixes tests
* Order By CreateAt for participants of old threads
* Removes unnecessary check on root id
* Removes GetParticipantProfilesByIds it's not needed
At first GetParticipantProfilesByIds was created because GetProfileByIds
gets users by order of username.
This was not needed, we can order by user ids in Go.
This commit removes GetParticipantProfilesByIds and replaces it by
GetProfileByIds as it was done before.
* Fixes participants order for pre CRT threads
Older threads (before CRT) are being converted to CRT threads upon
reply.
This commit considers the order of participants to be inserted to
the new thread.
* Adds tests for thread participants
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
We properly truncate the date part from the createAt time.
The SQL query is improved now to directly return the bool.
Fixed the test to properly test the feature.
https://mattermost.atlassian.net/browse/MM-38132
```release-note
NONE
```
* Added a sqlXPassThrough layer
This layer embeds the original sqlx object and wraps some
key methods to avoid some boilerplate.
- Add timeout to all methods from the timeout in settings.
- Added trace functionality.
- Added auto-rebind capabilities so that each store method
doesn't need to call rebind every time.
- Added auto-lowering of query string to support NamedExec
for both MySQL and Postgres
All methods are not wrapped as all are likely not to be called
from our code. As and when they are called, support will be added.
```release-note
NONE
```
* fix lint issues
```release-note
NONE
```
* address review comments
```release-note
NONE
```
* incorporate suggestions
```release-note
NONE
```
* Prepare support for sqlx
Just setting up some basic skeleton to start
replacing gorp queries with sqlx.
There is still an issue with efficiently writing
cross-platform named queries, because :FieldName
needs to be :fieldname for Postgres.
The positional params can be rebound depending on
driver name. But named queries can't from my
investigation. Will look into this.
```release-note
NONE
```
* Refactor mapper
```release-note
NONE
```
* Forgot to init slice
```release-note
NONE
```
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* CRT: desktop thread notifications
* Fixes go lint
* Adds default for desktop CRT notifications
* Adds email and push notifications for CRT threads
Adds user ids of thread followers with CRT to crtMentions so they will get
notified appropriately.
* Minor change
* Refactor a bit
CRTMentions.addMention had a bug on the return and de-duplication.
This commit fixes duplicate notifications by looking up if the user is to be
notified on CRT on both email and push notifications.
* Minor refactor
* Changes according to review comments
- Fixes adding to followers a user that had explicitly unfollowed a
thread.
- Simplified send email according to email_threads option
- Send mentions and followers in separate arrays via the websocket
- Fixes push notifications message for push_threads
* Adds a comment on a buggy use case
* Updates comment to correct ticket link
* Fixes when user notifications is set to all
There was a bug where if user had set notifications to all
then they would receive desktop notifications even for non following threads.
A similar bug existed in push notifications, where if a user has set it
to all the threads setting would still be considered.
This commit fixes that by adding users to notificationsForCRT
StringArray when they have the non thread setting to 'all'.
* Fixes notifications to users unfollowing threads
Users which had previously explicitly unfollowed a thread
should not receive notifications about those threads.
* Update store mocks
* Fixes push notifications for CRT
Push notification about replies for CRT users should have a title of
"Reply to Thread".
CRT users with global user setting to 'UserNotifyAll' should not get
notifications for unfollowed threads.
This commit fixes those issues.
* Fixes i18n error
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
- We move logging statements to the upper layer.
Store functions are low-level methods and should return error
upwards rather than logging.
- Used IN instead of any (array ()) which is equivalent.
- Made the delay to be of type time.Duration and un-exported it.
- Unexported the batch size constant.
```release-note
NONE
```
Summary
If a user has unfollowed a thread, another user's reply in the thread should not cause the first user to re-follow the thread. The first user will be re-followed if they are mentioned in the thread.
Simplify and make flexible the logic surrounding following and remove some duplicate code.
Ticket Link
https://mattermost.atlassian.net/browse/MM-36544https://mattermost.atlassian.net/browse/MM-37439
* MM-21357: Use typed constant for channel types
https://mattermost.atlassian.net/browse/MM-21357
```release-note
- Introduced a new type ChannelType for all channel types.
- Updated the Client4.UpdateChannelPrivacy method to ChannelType.
```
* Address review comments
```release-note
NONE
```
* telemetry fix
```release-note
NONE
```
* nested sql query for reply_count in getSingle post method of post_store.go
* added tests for replyCount in GetSingle api
* updated the post object message field in GetSingle unit test to use model.NewRandomString
Summary
Fix unreads query for threads to check deleted option
Normalize unreads query to look for threads after given time
Fix migration to only run once, and increment timestamp by one so that threads are definitely marked as read.
Ticket Link
Fixes issues with https://mattermost.atlassian.net/browse/MM-35345
And possibly some issues with thread unreads.
Summary:
The CRT backend was first released in version 5.29.0. Since then, the behaviour of the CRT feature has been refined, several bugs have been fixed, and a few sql columns have been added. Before these various fixes went in, the threads and channel membership tables have accumulated incorrect mention and unreads data. This PR fixes some of this bad historical data.
Summary of fixes:
- Marks threads as read for users where the last reply time of the thread is earlier than the time the user viewed the channel. Marking a thread means setting the mention count to zero and setting the last viewed at time of the the thread as the last viewed at time of the channel. This is done through a "sql migration"
- Fix channel counts, i.e. the total message count, total root message count, mention count, and mention count in root messages for users who have viewed the channel after the last post in the channel. This is done as a "sql migration"
Ticket Link:
Parts of
https://mattermost.atlassian.net/browse/MM-35494https://mattermost.atlassian.net/browse/MM-35345
* pre-checkout commit
* add API endpoints for retention policies
* allow deleting multiple teams/channels from a policy in a single request
* pre-checkout commit
* add auditing in API functions
* add permission checks
* update the store layers
* update storetest
* add check constraint on PostDuration column
* pre-checkout commit
* add query to delete posts under the scope of a granular retention policy
* add suggestions from sbishel
* allow clients to specify channels/teams when creating a new policy
* remove foreign keys referencing Channels and Teams tables
* add checks for whether teams and channels exist
* pre-checkout commit
* remove data referencing the Posts table
* pre-checkout commit
* write data store tests
* sort results of buildGetPoliciesQuery
* add missing test cases for teams
* pre-checkout commit
* add Client4 methods for data retention policy endpoints
* add uint and uint64 to app/layer_generators
* make granular policies override global policies
* fix lint errors
* pre-checkout commit
* add license to top of files
* add tests for data store layer
* add missing test cases for store layer
* run make i18n-extract
* add query to delete ChannelMemberHistory
* work in progress
* add test for old reply to old post
* fix lint error
* use COALESCE on each Posts column
* begin implementing orphaned rows worker
* split PR
* pre-checkout commit
* use RetentionPolicyWithTeamAndChannelCounts
* update app and api layers
* run make i18n-extract
* add RetentionPolicy to retrylayer_test.go
* Revert "split PR"
This reverts commit b316f03dd307a30deae931944ca7e4a1cc904605.
* fix errors caused by revert
* add suggestions from sbishel
* fix copy-paste error
* fix lint errors
* pre-checkout commit
* add function to delete orphaned rows
* use -1 for infinite retention
* remove check constraint
* copy i18n entries from master
* re-run tests with newer enterprise branch
* add team data to channel list
* add search for channels and teams in a policy
* add store tests for channel and team search
* add suggestions from mkraft
* run make einterfaces-mocks
* fix lint errors
* add suggestions from mkraft
* move removeOrphanedRows method to wireup branch
* Revert "move removeOrphanedRows method to wireup branch"
This reverts commit 94605c9b4a5378ffa44a3dec4d3f8e3306b9d33e.
* use DeleteOrphanedRows where possible
* run make i18n-extract
* use COMPLIANCE permissions
* run make migrations-bindadta
* clean up teams before test
* fix tests for TestRetentionPolicyStore
* add API endpoints for mobile
* fix lint error
* fix some of the lint errors
* move user/data_retention endpoints to data_retention.go
* Revert "fix some of the lint errors"
This reverts commit b5b2dc27566c427187db942c5c0afe319e7679c4.
* add exclude_policy_constrained parameter for /channels and /teams
* fix lint errors
* add policy_id field to GET endpoints for channels and teams
* use PolicyWithTeamID in RetentionPolicy layer
* fix lint errors
* run make i18n-extract
* update mock call in telemetry_test.go
* return status:OK in JSON instead of 204
* pre-checkout commit
* add policy_id field on channels/teams
* fix lint errors
* use sq.Eq instead of '?'
* use new subsection permissions
* update channels and teams endpoints to use new subsection permissions
* add extra search opts for channels in a policy
* fix lint errors
* allow negative post duration in patch
* remove DELETE FROM query in retention policy tests
* use *int64 for PostDuration
* re-run CI tests
* use 3-step deletion strategy for each table
* fix lint errors
* run make store-layers
* re-run CI tests
* add test with channel, team and global policies
* use common function for SQL queries
* add pagination test
* use struct for args to common SQL function
* fix lint errors
* run make i18n-extract
* check if Channels.TeamId is "" or nil
* use three OR clauses
* write separate genericRetentionPoliciesDeletion function
* add config setting for BatchSize
* add telemetry for BatchSize
* use feature flag
* add old i18n messages back in
* re-run CI tests
* update call signature in storetest
* MM-30831: Adds constant for retention default batch size.
* MM-30831: Removes comment re: optimization.
* MM-30831: Converts days to milliseconds.
* MM-30831: Reverts change to test.
* Revert "MM-30831: Reverts change to test."
This reverts commit 6d14275a1ceae682bb9e17ec69b39252b44e0c0c.
* Revert "MM-30831: Converts days to milliseconds."
This reverts commit a0cb6ec09d854a05194c1daee1c5333f260231c3.
* MM-30831: Fixes tests.
* MM-30381: Fix for change to method sig.
Co-authored-by: Max Erenberg <max.erenberg@mattermost.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Martin Kraft <martin@upspin.org>
* MM-34434: Added 'is_trial' boolean to all trial license requests and to the License struct.
* MM-34434: Generalized the concept of a license request.
* MM-34434: Verifies JSON field of license instance is set.
* MM-34434: Added missing client param.
* MM-34434: Added some tests of the request trial API endpoint.
* MM-34434: Removed comment.
* fix broken test (#17348)
* Add missing wrapped errors (#17339)
* Improve document extraction and including a document extraction command (#17183)
* Add extract documents content command
* Adding the extraction command and making the pure go pdf library as secondary option
* Improving the memory usage and docextractor interface
* Enable content extraction by default in all the instances
* Tiny improvement on archive indexing
* Adding App interface generation and the opentracing layer
* Fixing linter errors
* Addressing PR review comments
* Addressing PR review comments
* Update en.json (#17356)
Automatic Merge
* adding new feature flag (#17308)
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Bump no_output_timeout to 2 hours (#17358)
* log invalid username (#17345)
Automatic Merge
* MM-34434: Added missing client param.
MM-34434: Added some tests of the request trial API endpoint.
MM-34434: Removed comment.
* MM-34434: Switched to a hard-coded true value.
* MM-34434: Reverts test change.
* MM-34434: Removes unnecessary field.
* MM-34434: Tests that is_trial is hard-coded by TrialLicenseRequest.
* MM-34434: Removed accidental commit.
* MM-34434: Removes unnecessary is_trial key from JSON payload.
* MM-34434: Reverts to old pointer receiver variable name.
* MM-34434: Removes test.
* #MM-34437 Initialized license service
* ##MM-34437 Verified at all points if server is trial elligible
* WIp
* #MM-34437 removed unused commented code
* MM-34437 make a log less severe
* #MM-34437 generated einterface mocks
* #MM-34437 added license on new file
* #MM-34437 removed unused translation
* #MM-34437 some refactoring
* Update api4/license.go
* Update api4/license.go
* #MM-34437 made a variable name consistent
* #MM-34437 Added mocks for lince validator
* #M--34437 Added license validator test framework
* #MM-34437 Renamed isTrial method to isTrialLicense to avoid conflict with newlya dded field
* #M--34437 Allowed sales-sanctioned trials
* #MM-34437 fixed trial license API tests
* Added tests for add license API
* #MM-34437 fixed ValidateLicense test
* #MM-34437 Added util tests
* #MM-34437 using NoError for checking no error
* #MM-34437 using NoError for checking no error
* Added dummy piblic key for testing
* Fixed tests
* #MM-34437 udpaetd trial license URL for testing
* #MM-34437 adjusted times for licences generated through admin portal
* Reverted test-only changes
Co-authored-by: Martin Kraft <martin@upspin.org>
Co-authored-by: Hossein <hahmadia@users.noreply.github.com>
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
Co-authored-by: Jesús Espino <jespinog@gmail.com>
Co-authored-by: Amy Blais <amy_blais@hotmail.com>
Co-authored-by: Ben Cooke <benkcooke@gmail.com>
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Co-authored-by: Max Erenberg <max.erenberg@mattermost.com>
This PR builds up on the pass-through DB driver to a fully functioning DB driver implementation via our RPC layer.
To keep things separate from the plugin RPC API, and have the ability to move fast with changes, a separate field Driver is added to MattermostPlugin. Typically the field which is required to be compatible are the API and Helpers. It would be well-documented that Driver is purely for internal use by Mattermost plugins.
A new Driver interface was created which would have a client and server implementation. Every object (connection, statement, etc.) is created and added to a map on the server side. On the client side, the wrapper structs hold the object id, and communicate via the RPC API using this id.
When the server gets the object id, it picks up the appropriate object from its map and performs the operation, and sends back the data.
Some things that need to be handled are errors. Typical error types like pq.Error and mysql.MySQLError are registered with encoding/gob. But for error variables like sql.ErrNoRows, a special integer is encoded with the ErrorString struct. And on the cilent side, the integer is checked, and the appropriate error variable is returned.
Some pending things:
- Context support. This is tricky. Since context.Context is an interface, it's not possible to marshal it. We have to find a way to get the timeout value from the context and pass it.
- RowsColumnScanType(rowsID string, index int) reflect.Type API. Again, reflect.Type is an interface.
- Master/Replica API support.
* Revert "Revert "[MM-8497] Ability to set Do Not Disturb for a specified period of time (#16067)" (#17657)"
This reverts commit ff383990f8.
* add debug log for recurring function
* add feature flag for dnd timed status
* refactoring changes
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
