Граф коммитов

108 Коммитов

Автор SHA1 Сообщение Дата
mkraft
7fa6b321ae Custom groups (#18839)
* WIP

* adding initial creategroup endpoint

* fetching by group source

* fixing startup error

* updating create endpoint to take an array of user_ids, this will allow us to create the group with one request

* adding delete group endpoint and appropriate test

* adding source param for getGroups

* adding add members and delete members endpoints

* locking down crud endpoints to only be allowed for custom groups

* user search stuff

* allowing remoteid be null by changing field to pointer

* code cleanup and store level tests

* adding new tests and removing unused endpoint

* resolving conflicts

* Adds authz check for group.

* Adds authz checks to groups APIs.

* Updated create group authz tests.

* Updates delete group tests.

* Tests create group.

* Adds some tests and validations.

* adding new parameter so I can get users not in a group

* Fixed all lint warnings.

* Fix type.

* fixing search users not in group

* Fixes some lint errors.

* Moves entry in JSON array.

* Fixed SQL query.

* Fixes permission migration test.

* Fixes migration test.

* Fixes some group store tests.

* Fix test.

* Fix test.

* Revert lint change.

* Migrated CreateWithUserIds to sqlx.

* Adds tests for GetMember; migrates implementation to sqlx.

* Tests GetNonMemberUsersPage and hanles wrong group id.

* Fixes test.

* Switches GetMaster to GetMasterX.

* Switches GetReplica to GetReplicaX.

* Fixes logic.

* Fixes shadow declaration.

* Adds include_member_count to get group API endpoint.

* Adds filter_has_member param to getGroups.

* Fixes.

* Removes array of group sources.

* fixing error

* Testing reverting CreateWithUserIds back to gorp.

* Added websocket event for CreateGroupWithUserIds.

* Changed a few response status codes. Switched to correct permission.

* Added member count to ws payload for group when updating or creating.

* Adds feature flag checks for custom groups.

* Added middleware function to require license. Added config to disable custom groups.

* Change for function signature change of executePossiblyEmptyQuery.

* Lint fixes.

* Adds telemetry none comment.

* Adds translations.

* Migrated to sqlx.

* Temp. removal of translation.

* Fixed typo.

* Added an intermediary model to query with a field that is now ignored by sqlx on read queries.

* Re-used existing store struct.

* Inludes member count.

* Fix for merge error.'

* Require license for group endpoints.

* Updates translations.

* Fix shadow declaration.

* Renames permissions. Switches to new method to retrieve remoteid.

* Added WS events for upsert and delete member(s).

* Added new store error type ErrUniqueConstraint.

* Added EnableCustonGroups to the client config.

* Sanitized some user records.

* Added parameter to include_total_count for listing groups.

* Added translations.

* adding deleteAt field to getByUsers query

* Revert sanitize.

* Added uniqueness constraint error to UpdateGroup.

* Removed the FutureFeatures flag so that the feature is not enabled on old Enterprise licenses.

* Renamed function.

* Updates authz check for user search related to groups.

* Removed debug statement.

* Removed unused app method.

* Added telemetry for enable_custom_groups.

* Returns early from nil license.

* Updates test.

* Returned early to avoid nesting in (*SqlGroupStore).checkUserExist. Switched to reading from replica in (*SqlGroupStore).GetMember. Handled JSON marshal error in (*Client4).UpsertGroupMembers

* Switched to SanitizeProfile.

* Switched to model.NewInt.

* Switched from status NotImplemented to Forbidden for missing license.

* Removed deactivated users from 'exists' set.

* Revert gotool update.

* Ignored lint error that I think is invalid.

* Added the approprate access tag for disabling custom groups.

* Revert change to response status.

* Fixed refactor mistake.

* Limited the group member WS events to individual users.

* Removed WS event of deleted groups.

* Updated license check for searchUsers endpoint.

* Switched from license feature to license sku.

* Update app/group.go

Co-authored-by: Claudio Costa <cstcld91@gmail.com>

* Update app/group.go

Co-authored-by: Claudio Costa <cstcld91@gmail.com>

* Remove linter ignore comment.

* Added function to create sku-specific license.

* Fixed typo. Removed comment.

* Fixed for wrong type.

* Added missing param to client. Removed unnecessary props setting. Added test for retrieving groups by source.

* Updated some tests now that we're validating group membership not created for deactivated user.

* Fix for groups endpoint returning all group types by default.

* Changes constant names. Adds migration for all users to manage custom group members.

* Removes requirement for manage_system permission to filter user search by group.

* Added migration mock.

* Removes default permissions from custom_group_user role.

* Fixes migration.

* Fixes emoji migration test.

* fixing issue with member counts

* fixing search issue for deleted members

Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MBP.ht.home>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.fritz.box>
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
2022-02-17 12:34:39 -05:00
Agniva De Sarker
6898b3d70f Optimize TeamsUnread (#19507)
The GetTeamsUnreadForUser call would be called for every team switch.
In CRT mode, it would make a separate store call for every team, which
would run the 3 aggregate SQL queries in GetThreadsForUser.

This is suboptimal because the complexity is linearly proportional
to the number of teams.

We make the following optimizations:
1. Change the query to a single one which aggregates all teams.
2. The query originally used just 2 out of the 3 queries, so one
query was fully redundant. We remove that query in the new one.
3. Further analysis was done whether it makes sense to run the 2
queries synchronously or not. The load-tests didn't show any degradation
in running them concurrently, so we keep the same behavior.

```release-note
NONE
```
2022-02-17 09:24:39 +05:30
Kitae Kim
ff288b488c [GH-19150] Remove strings in all channel types for constants (#19535)
Automatic Merge
2022-02-11 12:04:18 +02:00
Ben Cooke
88968f9e17 Growth spike guest accounts (#19437)
* tools updates

* Revert "tools updates"

This reverts commit 6293297b55803c5a263e200ebd80192899666ae9.

* new endpoint to get users that should potentially be guests

* checking authservice to ensure they were an email signup

* adding tests for new endpoint

* fixing translation issue

* permissions for new endpoint

* fixing tests

* fixing when domain array is empty

* fixing when domain array is empty

* removing bots from request

Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MBP.ht.home>
Co-authored-by: mkraft <martinkraft@gmail.com>
2022-02-10 15:36:14 -05:00
Kyriakos Z
457aeb8e7e MM-36589: provide previous values for unreads (#18492)
* MM-36589: provide previous values for unreads

To successfully figure out the new counts of mentions or unread replies
for CRT we need to provide previous values alongside with the new.
This is needed so we'll know how many to subtract from the total.

This commit provides those numbers upon publishing websocket ThreadUpdated
and ThreadReadChanged events.

* Fixes errors

* Removes unneeded lines

* Adds GetThreadUnreadReplyCount store method

Uses the new store method to get unread replies instead of
GetThreadForUser.

* Tests, and some changes

- Adds api4 tests to test ws events
- Uses sqlx instead of gorp for new store method
- Fixes case where previous_unread_replies could be a negative value

* Refactors tests and adds more cases

* Fixes previous and current unread counts for commenter

When a user posts a reply to a thread the unread counts had a couple of
issues.
UnreadMentions where not zeroed out, and previous unread counts where
not set correctly.

This commit tries to fix that by marking the thread as read for the
current poster after we set previous unread counts to the websocket
event data.

Also MaintainMembership should zero out UnreadMentions when we are
setting the thread as read.

* Oops

* Fixes tests by updating when maintaining membership

OK, so some tests broke because we zero UnreadMentions in the membership
when we UpdateViewedTimestamp, since the new timestamp is always now.

Some tests broke because MaintainMembership for the commenter so that
the thread is read each time commenter posts moved further down in the
SendNotification method.

BOTH those test cases are fixed with this commit. To be sincere though I
don't understand why the second one is fixed by this.

* SystemAdminUser was not part of the channel

Some tests are failing because SystemAdminUser is not part of the
team and channels.
This commit adds user to team and channels, in an effort to fix
api4/user_tests

* Fixes tests

* Fixes tests

* Addresses review comments

* Fix if clause

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2022-02-09 15:41:30 -05:00
Kirill Krotov
440790dad6 [MM-39631] mirgate user store to sqlx (#19403)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2022-02-09 19:56:16 +05:30
Agniva De Sarker
a733fb840d GraphQL (part 1): Store layer (#19465)
We add the necessary store methods to use GraphQL

```release-note
NONE
```
2022-02-02 22:44:22 +05:30
Ashish Bhate
a55bd001b4 Revert "[MM-37013] Async job to fix CRT channel unreads (#18340)" (#19319)
This reverts commit 0da249c651.
2022-01-12 18:51:47 +05:30
Ibrahim Serdar Acikgoz
6595b31f23 Replace go-migrate with morph (#19116) 2022-01-08 10:16:07 +02:00
Kirill Krotov
0dbc74f6d5 [GH-18965] - Pass expire time from outside Cleanup method (#19008)
Pass expire time from outside Cleanup method

Fixes #18965
2021-12-22 14:57:30 +05:30
Ashish Bhate
0da249c651 [MM-37013] Async job to fix CRT channel unreads (#18340)
Summary
The addition of the TotalMsgCountRoot and MsgCountRoot columns to support CRT caused several issues with previously read threads and channels being marked as unread. Previously we attempted to fix this purely in a SQL migration [MM-35345][MM-35494] fixes for incorrect mentions and unreads for threads and channels #17803 but that turned out to be too heavy and it was decided to break up some of the fixes into async jobs.
This PR implements an async job to mark channels as read if there are no user posts since the last time the user viewed the channel. 

Ticket Link
https://mattermost.atlassian.net/browse/MM-37013
2021-11-18 15:31:18 +05:30
Agniva De Sarker
e24f22745e [MM-18946] [MM-26721] [MM-6842] Cross team search+private channel autocomplete (#18468)
* Show private channels in autocomplete

This is supported in all Engines:
MySQL, Postgres, Bleve, Elasticsearch.

https://mattermost.atlassian.net/browse/MM-18496

```release-note
Private channels will now appear in channel autocomplete.
If you are using Bleve or ElasticSearch, you will have to reindex
the channels again to populate them with the new attributes.
```

A large chunk of this work has been based on the earlier
effort at https://github.com/mattermost/mattermost-server/pull/17804.

Full credit goes to https://github.com/arvinDarmawan.

* Add comment

```release-note
NONE
```

* Adding more tests

```release-note
NONE
```

* fix more tests

```release-note
NONE
```

* tmp

```release-note
NONE
```

* more fixes

```release-note
NONE
```

* add tests

```release-note
NONE
```

* Add review comments from previous PR

```release-note
NONE
```

* Add API to return all channels from all team

```release-note
NONE
```

* Added support for bleve and ES

```release-note
NONE
```

* Streaming response for GetAllChannels

```release-note
NONE
```

* Fix tests

```release-note
NONE
```

* Trigger CI
```release-note
NONE
```

* fix tests

```release-note
NONE
```

* Addressing review comments

```release-note
NONE
```

* Fix lint

```release-note
NONE
```

* Removing flaky test

```release-note
NONE
```

* Address comments

```release-note
NONE
```

* Trigger CI
```release-note
NONE
```

* Added /users/<userid>/channel_members endpoint

```release-note
NONE
```

* Minor edit

```release-note
NONE
```

* Improve embedding

```release-note
NONE
```

* Fix lint error

```release-note
NONE
```

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-10-26 11:30:59 +05:30
Agniva De Sarker
99e6039472 Add functionality to cleanup old jobs (#18646)
* Add functionality to cleanup old jobs

Historically, we never cleaned up old jobs from the DB
leading to them being accumulated forever.

This PR introduces functionality to cleanup old jobs
older than a defined threshold.

The functionality is set to false by default and has
to be enabled for it to work.

```release-note
2 new config settings were added.

JobSettings.CleanupOldJobs: This indicates whether to clean up old jobs
from the DB or not. Default is false.

JobSettings.CleanupJobsThresholdHours: This defines the time gap in hours beyond
which older jobs will be removed. This has no effect if the above config
setting is set to false. Default is -1
```

* fix copy pasta

```release-note
NONE
```

* address review comments

```release-note
NONE
```

* Fix lint

```release-note
NONE
```

* Use single config option for everything

```release-note
NONE
```

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-10-15 11:03:54 +05:30
Daniel Espino García
02a9ef3f82 [MM-38216] Add API endpoint and adapt search to allow multi-team search (#18371)
* Add API endpoint and adapt search to allow multi-team search

* Refactor handler, refactor sql query to use squirrel, rename app and store functions and add tests

* Fix lint

* Fix search engines and remove unneeded comments

* Fix test

* Remove user from channel after test
2021-09-23 14:43:09 +02:00
Agniva De Sarker
7bbcf86531 Update channel member notify props to use native JSON (#18114)
* Update channel member notify props to use native JSON

Created a new store method that patches the notify props field.

https://community-daily.mattermost.com/plugins/focalboard/workspace/zyoahc9uapdn3xdptac6jb69ic?id=285b80a3-257d-41f6-8cf4-ed80ca9d92e5&v=495cdb4d-c13a-4992-8eb9-80cfee2819a4&c=91d08676-4a0e-4f02-8dce-d24d4fc56449

```release-note
NONE
```

* cleanup

```release-note
NONE
```

* forgot to commit

```release-note
NONE
```

* Fix edge case

```release-note
NONE
```

* address review comments

```release-note
NONE
```

* fix incorrect order

```release-note
NONE
```

* Address review comments

```release-note
NONE
```

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-08-23 10:04:30 +05:30
Kyriakos Z
a0c5d8feab MM-36234,MM-37030,MM-37031: CRT, desktop thread notifications (#18088)
* CRT: desktop thread notifications

* Fixes go lint

* Adds default for desktop CRT notifications

* Adds email and push notifications for CRT threads

Adds user ids of thread followers with CRT to crtMentions so they will get
notified appropriately.

* Minor change

* Refactor a bit

CRTMentions.addMention had a bug on the return and de-duplication.
This commit fixes duplicate notifications by looking up if the user is to be
notified on CRT on both email and push notifications.

* Minor refactor

* Changes according to review comments

- Fixes adding to followers a user that had explicitly unfollowed a
  thread.
- Simplified send email according to email_threads option
- Send mentions and followers in separate arrays via the websocket
- Fixes push notifications message for push_threads

* Adds a comment on a buggy use case

* Updates comment to correct ticket link

* Fixes when user notifications is set to all

There was a bug where if user had set notifications to all
then they would receive desktop notifications even for non following threads.

A similar bug existed in push notifications, where if a user has set it
to all the threads setting would still be considered.

This commit fixes that by adding users to notificationsForCRT
StringArray when they have the non thread setting to 'all'.

* Fixes notifications to users unfollowing threads

Users which had previously explicitly unfollowed a thread
should not receive notifications about those threads.

* Update store mocks

* Fixes push notifications for CRT

Push notification about replies for CRT users should have a title of
"Reply to Thread".

CRT users with global user setting to 'UserNotifyAll' should not get
notifications for unfollowed threads.

This commit fixes those issues.

* Fixes i18n error

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-08-19 17:28:46 +03:00
Claudio Costa
04b27ce93c [MM-28985] Remove pointers to slice (part 1) (#18034)
* Remove pointers to slice (part 1)

* Remove use of pointers to slice from model package (#18045)

* Fix after merge
2021-08-17 11:18:33 +02:00
Agniva De Sarker
132f114793 Modified updateUserNotifyProps to directly update the field (#18097)
* Modified updateUserNotifyProps to directly update the field

The method was only being used during import and it unnecessarily
made multiple queries to the DB.

Changed to a separate query that just updated the props field.

https://community-daily.mattermost.com/plugins/focalboard/workspace/zyoahc9uapdn3xdptac6jb69ic?id=285b80a3-257d-41f6-8cf4-ed80ca9d92e5&v=495cdb4d-c13a-4992-8eb9-80cfee2819a4&c=e4f9a891-85d6-4886-8590-1e327f7f8b8f

```release-note
NONE
```

* invalidating cache

```release-note
NONE
```
2021-08-17 10:51:41 +02:00
Claudio Costa
308a88efb1 [MM-37898] Exclude bots from initial user store emptiness check (#18139)
* Exclude bots from initial user store emptiness check

* Add test
2021-08-14 07:40:16 +02:00
Agniva De Sarker
d1b164ab1f Improve session cleanup (#18123)
- We move logging statements to the upper layer.
Store functions are low-level methods and should return error
upwards rather than logging.
- Used IN instead of any (array ()) which is equivalent.
- Made the delay to be of type time.Duration and un-exported it.
- Unexported the batch size constant.

```release-note
NONE
```
2021-08-13 20:15:24 +05:30
Claudio Costa
bd65e8daf9 Replace db count query in user signup process (#18072) 2021-08-12 11:23:21 +02:00
Agniva De Sarker
c4b4e1bc38 MM-36271: Bump major version to 6.0 (#17973)
https://mattermost.atlassian.net/browse/MM-36271

```release-note
We bump the major version to 6.0
```
2021-07-22 12:21:47 +05:30
Agniva De Sarker
da7d71ccf7 MM-21357: Use typed constant for channel types (#17928)
* MM-21357: Use typed constant for channel types

https://mattermost.atlassian.net/browse/MM-21357

```release-note
- Introduced a new type ChannelType for all channel types.
- Updated the Client4.UpdateChannelPrivacy method to ChannelType.
```

* Address review comments

```release-note
NONE
```

* telemetry fix

```release-note
NONE
```
2021-07-21 16:38:39 +02:00
Miguel de la Cruz
5b99df7bcd [MM-33395] Invalidate email tokens (#17069)
* [MM-33395] Invalidate existing verify email tokens when creating a new one

* Update store layers

* Addressing review comments

* Fix linter

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-07-09 14:31:34 +02:00
Max Erenberg
58d5d51f7a [MM-30831] granular data retention wireup (#17417)
* pre-checkout commit

* add API endpoints for retention policies

* allow deleting multiple teams/channels from a policy in a single request

* pre-checkout commit

* add auditing in API functions

* add permission checks

* update the store layers

* update storetest

* add check constraint on PostDuration column

* pre-checkout commit

* add query to delete posts under the scope of a granular retention policy

* add suggestions from sbishel

* allow clients to specify channels/teams when creating a new policy

* remove foreign keys referencing Channels and Teams tables

* add checks for whether teams and channels exist

* pre-checkout commit

* remove data referencing the Posts table

* pre-checkout commit

* write data store tests

* sort results of buildGetPoliciesQuery

* add missing test cases for teams

* pre-checkout commit

* add Client4 methods for data retention policy endpoints

* add uint and uint64 to app/layer_generators

* make granular policies override global policies

* fix lint errors

* pre-checkout commit

* add license to top of files

* add tests for data store layer

* add missing test cases for store layer

* run make i18n-extract

* add query to delete ChannelMemberHistory

* work in progress

* add test for old reply to old post

* fix lint error

* use COALESCE on each Posts column

* begin implementing orphaned rows worker

* split PR

* pre-checkout commit

* use RetentionPolicyWithTeamAndChannelCounts

* update app and api layers

* run make i18n-extract

* add RetentionPolicy to retrylayer_test.go

* Revert "split PR"

This reverts commit b316f03dd307a30deae931944ca7e4a1cc904605.

* fix errors caused by revert

* add suggestions from sbishel

* fix copy-paste error

* fix lint errors

* pre-checkout commit

* add function to delete orphaned rows

* use -1 for infinite retention

* remove check constraint

* copy i18n entries from master

* re-run tests with newer enterprise branch

* add team data to channel list

* add search for channels and teams in a policy

* add store tests for channel and team search

* add suggestions from mkraft

* run make einterfaces-mocks

* fix lint errors

* add suggestions from mkraft

* move removeOrphanedRows method to wireup branch

* Revert "move removeOrphanedRows method to wireup branch"

This reverts commit 94605c9b4a5378ffa44a3dec4d3f8e3306b9d33e.

* use DeleteOrphanedRows where possible

* run make i18n-extract

* use COMPLIANCE permissions

* run make migrations-bindadta

* clean up teams before test

* fix tests for TestRetentionPolicyStore

* add API endpoints for mobile

* fix lint error

* fix some of the lint errors

* move user/data_retention endpoints to data_retention.go

* Revert "fix some of the lint errors"

This reverts commit b5b2dc27566c427187db942c5c0afe319e7679c4.

* add exclude_policy_constrained parameter for /channels and /teams

* fix lint errors

* add policy_id field to GET endpoints for channels and teams

* use PolicyWithTeamID in RetentionPolicy layer

* fix lint errors

* run make i18n-extract

* update mock call in telemetry_test.go

* return status:OK in JSON instead of 204

* pre-checkout commit

* add policy_id field on channels/teams

* fix lint errors

* use sq.Eq instead of '?'

* use new subsection permissions

* update channels and teams endpoints to use new subsection permissions

* add extra search opts for channels in a policy

* fix lint errors

* allow negative post duration in patch

* remove DELETE FROM query in retention policy tests

* use *int64 for PostDuration

* re-run CI tests

* use 3-step deletion strategy for each table

* fix lint errors

* run make store-layers

* re-run CI tests

* add test with channel, team and global policies

* use common function for SQL queries

* add pagination test

* use struct for args to common SQL function

* fix lint errors

* run make i18n-extract

* check if Channels.TeamId is "" or nil

* use three OR clauses

* write separate genericRetentionPoliciesDeletion function

* add config setting for BatchSize

* add telemetry for BatchSize

* use feature flag

* add old i18n messages back in

* re-run CI tests

* update call signature in storetest

* MM-30831: Adds constant for retention default batch size.

* MM-30831: Removes comment re: optimization.

* MM-30831: Converts days to milliseconds.

* MM-30831: Reverts change to test.

* Revert "MM-30831: Reverts change to test."

This reverts commit 6d14275a1ceae682bb9e17ec69b39252b44e0c0c.

* Revert "MM-30831: Converts days to milliseconds."

This reverts commit a0cb6ec09d854a05194c1daee1c5333f260231c3.

* MM-30831: Fixes tests.

* MM-30381: Fix for change to method sig.

Co-authored-by: Max Erenberg <max.erenberg@mattermost.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Martin Kraft <martin@upspin.org>
2021-06-23 07:55:12 -04:00
Harshil Sharma
e4aa729a0c MM-34437 Preventing infinite trial requests (#17472)
* MM-34434: Added 'is_trial' boolean to all trial license requests and to the License struct.

* MM-34434: Generalized the concept of a license request.

* MM-34434: Verifies JSON field of license instance is set.

* MM-34434: Added missing client param.

* MM-34434: Added some tests of the request trial API endpoint.

* MM-34434: Removed comment.

* fix broken test (#17348)

* Add missing wrapped errors (#17339)

* Improve document extraction and including a document extraction command (#17183)

* Add extract documents content command

* Adding the extraction command and making the pure go pdf library as secondary option

* Improving the memory usage and docextractor interface

* Enable content extraction by default in all the instances

* Tiny improvement on archive indexing

* Adding App interface generation and the opentracing layer

* Fixing linter errors

* Addressing PR review comments

* Addressing PR review comments

* Update en.json (#17356)

Automatic Merge

* adding new feature flag (#17308)

Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>

* Bump no_output_timeout to 2 hours (#17358)

* log invalid username (#17345)

Automatic Merge

* MM-34434: Added missing client param.

MM-34434: Added some tests of the request trial API endpoint.

MM-34434: Removed comment.

* MM-34434: Switched to a hard-coded true value.

* MM-34434: Reverts test change.

* MM-34434: Removes unnecessary field.

* MM-34434: Tests that is_trial is hard-coded by TrialLicenseRequest.

* MM-34434: Removed accidental commit.

* MM-34434: Removes unnecessary is_trial key from JSON payload.

* MM-34434: Reverts to old pointer receiver variable name.

* MM-34434: Removes test.

* #MM-34437 Initialized license service

* ##MM-34437 Verified at all points if server is trial elligible

* WIp

* #MM-34437 removed unused commented code

* MM-34437 make a log less severe

* #MM-34437 generated einterface mocks

* #MM-34437 added license on new file

* #MM-34437 removed unused translation

* #MM-34437 some refactoring

* Update api4/license.go

* Update api4/license.go

* #MM-34437 made a variable name consistent

* #MM-34437 Added mocks for lince validator

* #M--34437 Added license validator test framework

* #MM-34437 Renamed isTrial method to isTrialLicense to avoid conflict with newlya dded field

* #M--34437 Allowed sales-sanctioned trials

* #MM-34437 fixed trial license API tests

* Added tests for add license API

* #MM-34437 fixed ValidateLicense test

* #MM-34437 Added util tests

* #MM-34437 using NoError for checking no error

* #MM-34437 using NoError for checking no error

* Added dummy piblic key for testing

* Fixed tests

* #MM-34437 udpaetd trial license URL for testing

* #MM-34437 adjusted times for licences generated through admin portal

* Reverted test-only changes

Co-authored-by: Martin Kraft <martin@upspin.org>
Co-authored-by: Hossein <hahmadia@users.noreply.github.com>
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
Co-authored-by: Jesús Espino <jespinog@gmail.com>
Co-authored-by: Amy Blais <amy_blais@hotmail.com>
Co-authored-by: Ben Cooke <benkcooke@gmail.com>
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Co-authored-by: Max Erenberg <max.erenberg@mattermost.com>
2021-06-17 17:37:34 +05:30
darkLord19
b0bdd23d2c [MM-8497] Ability to set Do Not Disturb for a specified period of time (#17680)
* Revert "Revert "[MM-8497] Ability to set Do Not Disturb for a specified period of time (#16067)" (#17657)"

This reverts commit ff383990f8.

* add debug log for recurring function

* add feature flag for dnd timed status

* refactoring changes

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-06-16 14:38:26 -04:00
Claudio Costa
6483abd263 [MM-36268] Fix replication lag error on post reply (#17752)
* Fix replication lag error on post reply

* Improve further by using a db transaction
2021-06-16 16:45:34 +02:00
Joram Wilander
d0778486ad MM-35396 Refactor GetThreadForUser store func to take membership as argument to prevent replica lag issues and reduce joins in query (#17754)
* Update store function GetThreadForUser to use master DB to fix replica lag

* Refactor GetThreadForUser store func to take membership as argument to prevent replica lag issues and reduce joins in query

* Add translation

* Fix test

* Updates per feedback

* Minor clean-up per feedback
2021-06-14 12:33:08 -04:00
Eli Yukelzon
46649292f8 MM-34758 Collapsed Reply Threads without mobile support (#17424)
Summary
added support for legacy clients accessing server
added collapsed_threads_supported param to viewChannel API and setPostUnread API

Ticket Link
https://mattermost.atlassian.net/browse/MM-34758

Related Webapp PR
mattermost/mattermost-webapp#7933
2021-05-26 20:40:25 +05:30
Jesse Hallam
ff383990f8 Revert "[MM-8497] Ability to set Do Not Disturb for a specified period of time (#16067)" (#17657)
This reverts commit 77d42568f9.
2021-05-21 09:38:27 -03:00
Doug Lauder
2b02b03497 MM-35133 trigger sync user immediately after change (#17579)
- ensure changes to user profile sync immediately
- refactor sync send
2021-05-20 12:07:40 -04:00
Max Erenberg
9ef41a55e2 [MM-7968] restrict creation of direct channels to team members (#17222)
* restrict creation of direct channels to team members

* run make i18n-extract

* add suggestions from hahmadia

* place common-team-check logic in app layer

* use flat SQL query

* show more specific error message to user

* MM-7968: Fmt file.

* MM-7968: Fix for moved session field.

Co-authored-by: Max Erenberg <max.erenberg@mattermost.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Martin Kraft <martinkraft@gmail.com>
Co-authored-by: Martin Kraft <martin@upspin.org>
2021-05-19 08:45:03 -04:00
darkLord19
77d42568f9 [MM-8497] Ability to set Do Not Disturb for a specified period of time (#16067)
* Add support for timed DND status

- accept a date time value in api query when dnd mode for user needs to be unset
- Create a new function to handle SetDNDStatus calls
- Create a scheduled task to unset dnd mode to wahtever mode was before setting it to DND

* update schema version

* Model changes to make fields more intuitive

- move dndendtime to status model
- add new field prev status in status to keep track of previous status of user
- update db migration function
- make use of prevstatus and dndendtime from status model

* set prev status and dndendtime appropriately after unsetting dnd mode

* add json tag for dndendtime

* unset dnd status only if not changed manually by user

* update dnd statuses after server restart

* make app-layers

* fix failing tests

* don't create sched task when setting status to DND

* get only expired statuses from db

- convert end time from any timezone to utc
- store dnd end time in unix format for usability reasons

* run update dnd status only on leader

* make mocks

* fix tests

* run UpdateDNDStatusOfUsers as recurring task

* save all statuses at once in db and update UpdateDNDStatusOfUsers logic

* add app method to get timezone of user

* store dnd end time in context.Params

* set max size of prevstatus

* update status model to take endtime input as string and store in db as unix time(int64)

* Add tests for SetStatusDoNotDisturbTimed

* if dnd_end_time is not passed the call old api to set dnd mode

* fix tests

* new plugin api to use new timed dnd mode

* get and update rows in a single db query

* dnd end time will be stored in request body and not route param

* exclude statuses which has dndendtimeunix < 0

* update and get the updated dnd statuses in single db query

* add updated status to cache

* DNDEndTimeUnix and PrevStatus need not to be visible to users

* update db schema version for migration

* Keep Status and PrevStatus varchar size same

* add test to verify status is restored after dnd end time expires

* expect endtime in utc from client

- remove store method GetTimezone as no longer needed
- add documentation for SetStatusDoNotDisturbTimed

* reduce sleep time for dnd timed restore test

* more appropriate name for new api to update user status

* update db migration function

* parse and validate time before potentially triggering db query to get status of user

* add migration changes in to existing upgrade function

* not supporting un-timed dnd status via api

* don't call Srv.Store directly, call via app layer

* rename dndendtime to statuscleartime to make it suitable for custom status usage as well

* Revert "rename dndendtime to statuscleartime to make it suitable for custom status usage as well"

This reverts commit fa69152d9a3db18f1c59b34c878fb7ce494440b5.

* mysql doesn't support RETURNING clause so add tx to get and update statuses

* add UpdateDNDStatusOfUsers mock in tests

* update store mock import path

* add mock in storelib

* Add status mocks to empty store

* Close the task during server shutdown

* Do not cancel a nil task

* update squirrel queries

* remove untimed dnd test

* start recurring task to unset statuses on leadership change

* set dndTask to nil after cancelling it upon server shutdown

* new recurring task which starts at nearest rounded time of the interval

* mock Get() call for status

* return updated statuses in case of mysql

* remove unneccessary code

* add Get() mock to empty store

* fix mocking for once and all

* address review comments

fix mysql updateStatus fn

protect dndTask with mutex

minor refactors

* move runDNDStatusExpireJob to server.go and pass App as arg instead of method receiver

* frontend will send endtime in unix epoch format so get rid of double representation

* scan for all fields and not just two

* add some tests and fix review comments

* remove extra sql query and create needed result in go

* add storetest for UpdateExpiredDNDStatuses

* add migrations to latest version

* update min supported version

* add comment to fix a bug in future

* update test to expect 1 status in return

* rename UpdateUserStatusWithDNDTimeout to SetUserStatusTimedDND

* rename DNDEndTimeUnix to DNDEndTime

* cast int to int64 for equality

* fix tests and error handling

* move updating values to retrieved statuses fields outside sql transaction

* move migrations to 5.36

Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-05-07 18:20:55 +02:00
Max Erenberg
9f4902e188 [MM-33908] Resync LDAP Groups to Teams and Channels (#17372)
* add includeRemovedMembers flag

* fix API call in client4.go

* remove check for 'since'

* add comments

* run make app-layers

* re-run CI tests

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-05-04 12:37:36 -04:00
Max Erenberg
3e5b0a7d7f [MM-29666] compliance export skipped posts (#17445)
* add MessageExportCursor

* sort by PostUpdateAt and PostId

* re-run CI tests

* remove panic debugging line

Co-authored-by: Martin Kraft <martin@upspin.org>
2021-05-04 09:08:09 -04:00
Eli Yukelzon
ba302cf512 MM-35125 CRT: Mention badge on threads doesn't appear until refresh (#17504) 2021-04-27 10:18:20 +03:00
Max Erenberg
9cd50a4e22 [MM-33853] remove CSV row limit in compliance monitoring (#17185) 2021-04-23 09:19:13 -04:00
darkLord19
f6505e1ee6 MM-31339: Send only one direct message reply within one calendar day. (#17181)
* don't send auto response if already responded today

* update query to get posts from channel for given user and Updatetime requires value in milli seconds

* regenerate mocks and layers

* update function to return true/false on existence of auto responded post in channel and add tests

* add store tests

* bubble up error and propagate upstream

* fix error handling logic

* use require instead of assert

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>

* rename variable for better redability and logging fixes

* update comment explaining function

* use new function to generate test ids

* add comments to clarify NewTestId copies

* add translations for error id

* fix translation

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Saturnino Abril <saturnino.abril@gmail.com>
2021-04-22 12:33:45 +08:00
Ibrahim Serdar Acikgoz
28ff4dc8d0 [MM-32281] SqlRoleStore/GetByName: add context to allow read from master (#17176)
* role_store/GetByName: add context

* propagate context in the app layer

* propagate context in the app layer

* add missing import

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-04-21 18:01:02 +03:00
Eli Yukelzon
56198a99c6 MM-34871 CRT: Participants of thread include non-replying followers (and past followers) (#17447)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-04-21 09:48:30 +03:00
Max Erenberg
f36f5c74b1 Mm 30807 granular data retention scaffold (#17464) 2021-04-20 13:16:40 -04:00
Jesús Espino
368b642105 Fixing problem with read-replica on indexing files (#17460) 2021-04-20 12:30:43 +02:00
Agniva De Sarker
e0efdd708b Revert "Mm 30807 granular data retention scaffold (#16891)" (#17437)
This reverts commit 3ea75332e7.
2021-04-18 22:41:50 +05:30
Max Erenberg
3ea75332e7 Mm 30807 granular data retention scaffold (#16891)
create the necessary tables, models and APIs for the granular data retention policy feature
2021-04-16 11:32:09 -04:00
Eli Yukelzon
f90209c8a3 MM-34609 Mark-as-unread on a post in a thread should cause auto-follow (#17343)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-04-16 10:26:08 +03:00
Doug Lauder
9799fe9be6 MM-34549 shared channels; add users to channel that were already sync'd (#17361)
Fixes a bug and adds a feature for shared channels:
- The Bug: when creating new shared channels, users that had already been sync'd via another channel were not added to the new channel's member list, since the users were not sync'd again. This PR sync's users per channel.
- The Feature: support custom statuses
2021-04-14 14:59:26 -04:00
Max Erenberg
869da7a78b [MM-32044] Reset SAML auth data (#17161)
Automatic Merge
2021-04-13 00:46:30 +02:00
Hossein
daec0b3201 All Sections to SubSections (#16917)
* initial

* Revert "initial"

This reverts commit 3d631aeecdc2324cd5d17c0ecdc431a8ccc15060.

* [MM-32352] Add Experimental Subsections BACKEND (#16887)

Automatic Merge

* update appiface

* Fix app layers

* Ancillary Permissions on backend (#17061)

Automatic Merge

* [MM-32799] Add About Section (#17015)

* Add About Section

* add mock key

* Update role.go

* Update role.go

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>

* [MM-33437] Fix config access tags for experimental settings (#17111)

Automatic Merge

* [MM-32794] Reporting Sub Section (#17035)

* test

* revert

* add permissions

* add new permission stuff

* add store mock

* fix bad merge

* gofmt fix

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>

* [MM-32343] Environment SubSection (#17054)

* pre-checkout commit

* fix permission for testSiteURL

* pre-merge commit

* increase size of Permissions column in Roles table

* add entry for ENVIRONMENT to testlib/store.go

* use TEXT for Permissions column in Roles table

* use environment subsection permissions for API endpoints

* use subsections permissions for /config/environment

* add suggestions from hahmadia

* update tests to use subsection permissions

* add permissions column back in

* comment out code in upgradeDatabaseToVersion534

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>

* MM-32351: Add Compliance Subsections (#17023)

* add subsections for compliance sectin

* add to mock functions

* updates for read job

* fixes

* fix test

* update tests

* update tests

* another test fix

* some cleanup

* update mlog

* fix linting

* Fix bad merges

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Hossein <hahmadia@users.noreply.github.com>
Co-authored-by: Hossein Ahmadian-Yazdi <hyazdi1997@gmail.com>

* MM-32347 Site Subsections (#17095)

* Init

* Added migration key in testlib store

* Fix syntax error

* fix bad merge

* fix lint

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>

* MM-32350 Integrations (#17097)

* implement server subsections

* fix tests

* update test

* go fmt

Co-authored-by: Hossein Ahmadian-Yazdi <hyazdi1997@gmail.com>

* patch forgotten endpoints

* Adding subsection permissions for Authentication (#17087)

* adding new permissions, migrations to do

* permission migrations and ancilary permissions

* running make app-layers

* fixing tests and lint

* adding permissions to saml

* ldap write permissions

* running make app-layers

* fixing conflict

* making app layers

* clean up and fix tests

* change job type

* fix js error, if site url not returned

Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Hossein Ahmadian-Yazdi <hyazdi1997@gmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>

* Update permissions_migrations.go

* gofmt

* upgrade to 535

* gofmt

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Max Erenberg <max.erenberg@mattermost.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
Co-authored-by: Anurag Shivarathri <anurag6713@gmail.com>
Co-authored-by: Ben Cooke <benkcooke@gmail.com>
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
2021-04-06 10:39:48 -04:00
Haardik Dharma
343c51830f Fix initialism errors (PR-2) (#17032)
* Fix initialism errors

* Fix check-mocks test

* Revert mlog and filestore packages

* Update plugin_hooks_test.go

* Update opentracinglayer.go

* Regenerate mocks and check store layers

* Revert plugin's context changes

* Update context.go

* Update plugin_requests.go

* Update plugin_hooks_test.go

* Regenerate mocks

* Regenerate mocks and store layers

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-04-03 12:43:22 +05:30