Граф коммитов

70 Коммитов

Автор SHA1 Сообщение Дата
mkraft
7fa6b321ae Custom groups (#18839)
* WIP

* adding initial creategroup endpoint

* fetching by group source

* fixing startup error

* updating create endpoint to take an array of user_ids, this will allow us to create the group with one request

* adding delete group endpoint and appropriate test

* adding source param for getGroups

* adding add members and delete members endpoints

* locking down crud endpoints to only be allowed for custom groups

* user search stuff

* allowing remoteid be null by changing field to pointer

* code cleanup and store level tests

* adding new tests and removing unused endpoint

* resolving conflicts

* Adds authz check for group.

* Adds authz checks to groups APIs.

* Updated create group authz tests.

* Updates delete group tests.

* Tests create group.

* Adds some tests and validations.

* adding new parameter so I can get users not in a group

* Fixed all lint warnings.

* Fix type.

* fixing search users not in group

* Fixes some lint errors.

* Moves entry in JSON array.

* Fixed SQL query.

* Fixes permission migration test.

* Fixes migration test.

* Fixes some group store tests.

* Fix test.

* Fix test.

* Revert lint change.

* Migrated CreateWithUserIds to sqlx.

* Adds tests for GetMember; migrates implementation to sqlx.

* Tests GetNonMemberUsersPage and hanles wrong group id.

* Fixes test.

* Switches GetMaster to GetMasterX.

* Switches GetReplica to GetReplicaX.

* Fixes logic.

* Fixes shadow declaration.

* Adds include_member_count to get group API endpoint.

* Adds filter_has_member param to getGroups.

* Fixes.

* Removes array of group sources.

* fixing error

* Testing reverting CreateWithUserIds back to gorp.

* Added websocket event for CreateGroupWithUserIds.

* Changed a few response status codes. Switched to correct permission.

* Added member count to ws payload for group when updating or creating.

* Adds feature flag checks for custom groups.

* Added middleware function to require license. Added config to disable custom groups.

* Change for function signature change of executePossiblyEmptyQuery.

* Lint fixes.

* Adds telemetry none comment.

* Adds translations.

* Migrated to sqlx.

* Temp. removal of translation.

* Fixed typo.

* Added an intermediary model to query with a field that is now ignored by sqlx on read queries.

* Re-used existing store struct.

* Inludes member count.

* Fix for merge error.'

* Require license for group endpoints.

* Updates translations.

* Fix shadow declaration.

* Renames permissions. Switches to new method to retrieve remoteid.

* Added WS events for upsert and delete member(s).

* Added new store error type ErrUniqueConstraint.

* Added EnableCustonGroups to the client config.

* Sanitized some user records.

* Added parameter to include_total_count for listing groups.

* Added translations.

* adding deleteAt field to getByUsers query

* Revert sanitize.

* Added uniqueness constraint error to UpdateGroup.

* Removed the FutureFeatures flag so that the feature is not enabled on old Enterprise licenses.

* Renamed function.

* Updates authz check for user search related to groups.

* Removed debug statement.

* Removed unused app method.

* Added telemetry for enable_custom_groups.

* Returns early from nil license.

* Updates test.

* Returned early to avoid nesting in (*SqlGroupStore).checkUserExist. Switched to reading from replica in (*SqlGroupStore).GetMember. Handled JSON marshal error in (*Client4).UpsertGroupMembers

* Switched to SanitizeProfile.

* Switched to model.NewInt.

* Switched from status NotImplemented to Forbidden for missing license.

* Removed deactivated users from 'exists' set.

* Revert gotool update.

* Ignored lint error that I think is invalid.

* Added the approprate access tag for disabling custom groups.

* Revert change to response status.

* Fixed refactor mistake.

* Limited the group member WS events to individual users.

* Removed WS event of deleted groups.

* Updated license check for searchUsers endpoint.

* Switched from license feature to license sku.

* Update app/group.go

Co-authored-by: Claudio Costa <cstcld91@gmail.com>

* Update app/group.go

Co-authored-by: Claudio Costa <cstcld91@gmail.com>

* Remove linter ignore comment.

* Added function to create sku-specific license.

* Fixed typo. Removed comment.

* Fixed for wrong type.

* Added missing param to client. Removed unnecessary props setting. Added test for retrieving groups by source.

* Updated some tests now that we're validating group membership not created for deactivated user.

* Fix for groups endpoint returning all group types by default.

* Changes constant names. Adds migration for all users to manage custom group members.

* Removes requirement for manage_system permission to filter user search by group.

* Added migration mock.

* Removes default permissions from custom_group_user role.

* Fixes migration.

* Fixes emoji migration test.

* fixing issue with member counts

* fixing search issue for deleted members

Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MBP.ht.home>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.fritz.box>
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
2022-02-17 12:34:39 -05:00
Christopher Speller
5ec78546e2 MM-38150 Adding playbooks permissions (#19096)
* Add playbooks related permissions.

* Add RolesGrantPermission to pluginapi

* Fixing scopes.

* New defaults.

* Fix defaults

* Fix tests.

* Fix migration.

* More test and migration fixes.

* Need to add everything to system admin too.

* Move to 63

* Feedback fixes.

* Fix system manager editing playbook permissions.
2021-12-07 09:43:33 -08:00
Agniva De Sarker
5ad7fb5b6f Remove job server dependency searchengine (#18908)
* Remove job server dependency searchengine

The job server field was not being used. We remove it
as part of a cleanup of server initialization refactor.

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4?c=8a171d5e-4398-4bc9-8fbf-8b9b14f37069

```release-note
NONE
```

* Fix missing field

```release-note
NONE
```
2021-11-03 18:44:57 +01:00
Daniel Espino García
5928e4f9e0 Fix race condition in telemetry (#18734)
* Fix race condition in telemetry

* Fix tests
2021-10-20 09:58:25 +02:00
Ibrahim Serdar Acikgoz
be203d9bf3 Initial team service implementation (#18279)
* initial service implementation

* reflect review comments
2021-10-13 15:20:40 +03:00
Doug Lauder
a4507327a7 MM-36764 mlog refactor (#18118)
Refactor mlog
- simplify mlog by removing redundant code
- remove Zap dependency
- update unit test helpers
- update logging config
- update auditing
2021-08-17 16:08:04 -04:00
Agniva De Sarker
0d075c32db MM-22051: Remove ToJson from model (part 1) (#18005)
https://mattermost.atlassian.net/browse/MM-22051

```release-note
Removed the following methods/functions:
(ad *AccessData) ToJson()
(ar *AccessResponse) ToJson()
(ar *AnalyticsRow) ToJson()
(ar AnalyticsRows) ToJson()
(o *Audit) ToJson()
(o Audits) ToJson()
(ad *AuthData) ToJson()
(ar *AuthorizeRequest) ToJson()
(o *ChannelPatch) ToJson()
(o *ChannelsWithCount) ToJson()
(o *ChannelCounts) ToJson()
(o *ChannelData) ToJson()
(o *ChannelMembers) ToJson()
(o *ChannelUnread) ToJson()
(o *ChannelUnreadAt) ToJson()
(o *ChannelStats) ToJson()
(o *ChannelView) ToJson()
(o *ChannelViewResponse) ToJson()
(o *ClusterDiscovery) ToJson()
(ci *ClusterInfo) ToJson()
(cs *ClusterStats) ToJson()
(o *Command) ToJson()
CommandListToJson(l []*Command) string
(o *CommandArgs) ToJson()
(cmr *CommandMoveRequest) ToJson()
(o *CommandResponse) ToJson()
(c *Compliance) ToJson()
(c Compliances) ToJson()
(o *Config) ToJson()
EmojiListToJson(emojiList []*Emoji)
```
2021-07-28 13:15:46 +05:30
Agniva De Sarker
c4b4e1bc38 MM-36271: Bump major version to 6.0 (#17973)
https://mattermost.atlassian.net/browse/MM-36271

```release-note
We bump the major version to 6.0
```
2021-07-22 12:21:47 +05:30
Agniva De Sarker
623f8dc9af MM-21357: Use typed constants ClusterEvent (#17920)
* MM-21357: Use typed constants ClusterEvent

Use typed constants for:
- model.ClusterEvent

https://mattermost.atlassian.net/browse/MM-21357

```release-note
Use typed constant for model.ClusterEvent
```

* trigger CI
```release-note
NONE
```
2021-07-21 16:38:39 +02:00
Ben Schumacher
97ccf0bdf6 [MM-16751] golint model (#17896) 2021-07-21 16:38:39 +02:00
Ibrahim Serdar Acikgoz
f69b28610a user service: add oauth and profile images (#17784)
* users: add cache to service

* reflect review comments

* add oauth

* move profile picture genaration

* reflect review comments

* move default bot image to users package

* add missing wraps and apply suggestions

* add comment for app.GetSession
2021-06-18 19:25:03 +03:00
Ibrahim Serdar Acikgoz
24fb0033f4 users service: add sessions (#17744)
* users: add cache to service

* reflect review comments
2021-06-14 18:08:00 +03:00
Hossein
abdba7acc3 [MM-35861] Add missing migration transformation (#17647) 2021-05-19 17:00:51 -04:00
Jesús Espino
df695115be Removing FilesSearch feature flag (#17548)
* Removing FilesSearch feature flag

* Fixing tests

* Adding an improvement on plain text extraction

* Adding tests for plain text extraction

* Removed unneeded conversion

* Adding missed license

* Remove the feature flag from the migration

* Fixing some tests

* Updating i18n/en.json file
2021-04-30 23:21:26 +02:00
Doug Lauder
e2b9cb98aa MM-33903 rename slash commands & permission (#17494)
- rename slash commands
    - "remote" -> "secure-connection"
    - "share" -> "share-channel"
- change status icons to text (translated)
- remove channel invite "Description" field
- rename permission "manage_remote_clusters" -> "manage_secure_connections"
2021-04-30 14:59:29 -04:00
Claudio Costa
6af032d06a [MM-35077] Add basic support for plugin intra-cluster communication (#17495)
* Add basic support for plugin intra-cluster communication

* Some renaming for added clarity

* Allow sending cluster event to specific nodes

* Improve naming and documentation

* Improve logging
2021-04-28 19:59:32 +02:00
Hossein
daec0b3201 All Sections to SubSections (#16917)
* initial

* Revert "initial"

This reverts commit 3d631aeecdc2324cd5d17c0ecdc431a8ccc15060.

* [MM-32352] Add Experimental Subsections BACKEND (#16887)

Automatic Merge

* update appiface

* Fix app layers

* Ancillary Permissions on backend (#17061)

Automatic Merge

* [MM-32799] Add About Section (#17015)

* Add About Section

* add mock key

* Update role.go

* Update role.go

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>

* [MM-33437] Fix config access tags for experimental settings (#17111)

Automatic Merge

* [MM-32794] Reporting Sub Section (#17035)

* test

* revert

* add permissions

* add new permission stuff

* add store mock

* fix bad merge

* gofmt fix

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>

* [MM-32343] Environment SubSection (#17054)

* pre-checkout commit

* fix permission for testSiteURL

* pre-merge commit

* increase size of Permissions column in Roles table

* add entry for ENVIRONMENT to testlib/store.go

* use TEXT for Permissions column in Roles table

* use environment subsection permissions for API endpoints

* use subsections permissions for /config/environment

* add suggestions from hahmadia

* update tests to use subsection permissions

* add permissions column back in

* comment out code in upgradeDatabaseToVersion534

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>

* MM-32351: Add Compliance Subsections (#17023)

* add subsections for compliance sectin

* add to mock functions

* updates for read job

* fixes

* fix test

* update tests

* update tests

* another test fix

* some cleanup

* update mlog

* fix linting

* Fix bad merges

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Hossein <hahmadia@users.noreply.github.com>
Co-authored-by: Hossein Ahmadian-Yazdi <hyazdi1997@gmail.com>

* MM-32347 Site Subsections (#17095)

* Init

* Added migration key in testlib store

* Fix syntax error

* fix bad merge

* fix lint

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>

* MM-32350 Integrations (#17097)

* implement server subsections

* fix tests

* update test

* go fmt

Co-authored-by: Hossein Ahmadian-Yazdi <hyazdi1997@gmail.com>

* patch forgotten endpoints

* Adding subsection permissions for Authentication (#17087)

* adding new permissions, migrations to do

* permission migrations and ancilary permissions

* running make app-layers

* fixing tests and lint

* adding permissions to saml

* ldap write permissions

* running make app-layers

* fixing conflict

* making app layers

* clean up and fix tests

* change job type

* fix js error, if site url not returned

Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Hossein Ahmadian-Yazdi <hyazdi1997@gmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>

* Update permissions_migrations.go

* gofmt

* upgrade to 535

* gofmt

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Max Erenberg <max.erenberg@mattermost.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
Co-authored-by: Anurag Shivarathri <anurag6713@gmail.com>
Co-authored-by: Ben Cooke <benkcooke@gmail.com>
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
2021-04-06 10:39:48 -04:00
Doug Lauder
02196e04fa MM-27493 Shared channels (MVP) (#17301)
Remote Cluster Service
- provides ability for multiple Mattermost cluster instances to create a trusted connection with each other and exchange messages
- trusted connections are managed via slash commands (for now)
- facilitates features requiring inter-cluster communication, such as Shared Channels
Shared Channels Service
- provides ability to shared channels between one or more Mattermost cluster instances (using trusted connection)
- sharing/unsharing of channels is managed via slash commands (for now)
2021-04-01 13:44:56 -04:00
Jesús Espino
95b0809850 Extracting html templates into a library (#16946)
* Extracting html templates into a library

* Moving tests to the right place

* Fixing tests

* Addressing PR review comments

* Addressing PR review comments

* Replacing attomic with RWMutex

* Returning errors as channel for Templates watcher

* Address PR review comments

* Other small fixes

* Simplifying NewWithWatcher

* Addressing PR review comments

* Making error handling on rendering templates more robust

* Fixing tests

* Changing how we return errors

* Fixing shadow variables

* Addressing PR review comments

* Logging errors from the outside of sendNotificationEmail

* Fixing lock in shutdown

* Fixing the resource copy for commands tests temporary directories

* Removing unused import

* A couple of tiny fixes
2021-03-12 18:46:43 +01:00
Martin Raymond Kraft
58dce5930e [MM-31094] Replication Lag (#16888)
* MM-31094: Adds tooling to develop and test using a MySQL instance with replication lag. Adds some lazy lookups to fallback to master if results are not found.

* MM-31094: Removes mysql-read-replica from default docker services.

* MM-31094: Switches (store..SessionStore).Get and (store.TeamStore).GetMember to using context.Context.

* MM-31094: Updates (store.UsersStore).Get to use context.

* MM-31094: Updates (store.PostStore).Get to use context.

* MM-31094: Removes feature flag and config setting.

* MM-31094: Rolls back some master reads.

* MM-31094: Rolls a non-cache read.

* MM-31094: Removes feature flag from the store.

* MM-31094: Removes unused constant and struct field.

* MM-31094: Removes some old feature flag references.

* MM-31094: Fixes some tests.

* MM-31094: App layers fix.

* MM-31094: Fixes mocks.

* MM-31094: Don't reparse flag.

* MM-31094: No reparse.

* MM-31094: Removed unused FeatureFlags field.

* MM-31094: Removes unnecessary feature flags variable declarations.

* MM-31094: Fixes copy-paste error.

* MM-31094: Fixes logical error.

* MM-30194: Removes test method from store.

* Revert "MM-30194: Removes test method from store."

This reverts commit d5a6e8529bd5f4d993824c828e239d009b05e567.

* MM-31094: Conforming to make's strange syntax.

* MM-31094: Configures helper for read replica with option.

* MM-31094: Adds some missing ctx's.

* MM-31094: WIP

* MM-31094: Updates test names.

* MM-31094: WIP

* MM-31094: Removes unnecessary master reads.

* MM-31094: ID case changes out of scope.

* MM-31094: Removes unused context.

* MM-31094: Switches to a helper. Removes some var naming changes. Fixes a merge error.

* MM-31094: Removes SQLITE db driver ref.

* MM-31094: Layer generate fix.

* MM-31094: Removes unnecessary changes.

* MM-31094: Moves test method.

* MM-31094: Re-add previous fix.

* MM-31094: Removes make command for dev.

* MM-31094: Fix for login.

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-03-12 12:37:30 -05:00
Jesús Espino
9cc5089af8 Moving mlog to corelibs (#16915)
* Moving mlog to corelibs

* Regenerating app layers

* Fix golangci-lint problem

* Fixing golangci-lint errors

* Renaming from corelibs to shared

* Renaming from corelibs to shared

* Fixing import

* Fixing merge problems

* Fixing build
2021-03-05 09:18:37 +01:00
Jesús Espino
78355ae2a7 Moving filesstore services into shared folder (#16940)
* Moving filesstore services into shared folder

* Fixing app-layers generation

* Renaming from filesstore to filestore
2021-03-02 14:37:21 +01:00
Jesús Espino
85293fcf41 Add the API for search files (#15605)
* Adding search files api

* Fixing golangci-lint

* Adding bulk-indexing and improving a bit the name indexing for bleve and elasticsearch

* Add content extraction config migration

* Fixing a problem with document extraction

* Unapplying certain changes moved to other PR

* Fixing tests

* Making extract content app migration a private method

* Addressing PR review comments

* Addressing PR review comments

* Adding feature flag

* Removing debug string

* Fixing imports

* Fixing linting errors

* Do not migrate the config if the feature flag is not enabled

* Fix tests

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-02-26 07:41:05 +01:00
Jesús Espino
a3de71fba4 Reducing the filestore dependencies from the rest of the source code (#16816)
* Reducing the filestore dependencies from the rest of the source code

* Making more generic config conversion to FileBackendSettings

* Fixing usage of the NewFileBackend function

* Fixing more usages of the NewFileBackend function

* Fix some linter errors

* Fix more linter errors

* Fixing some unit tests

* Fixing linter problem

* Addressing PR review comments

* Simplifing the CopyFile for tests

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-02-15 10:09:28 +01:00
Hossein
01f264cd62 [MM-31790] Support Packet Generation BACKEND (#16667)
* init commit

* clean up the code

* make mocks

* fix translations

* mocks and lint fixes

* add tests

* little fixes

* Update i18n/en.json

Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>

* Update i18n/en.json

Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>

* Update i18n/en.json

Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>

* Update i18n/en.json

Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>

* Update i18n/en.json

Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>

* Address Comments

* fix i18n

* update api endpoint

* add enable file and file level for conditional show of banner

* Address Comments

* Make it more clear about returns

* Create zip file utility function

* update en.json

* address comments

* write tests

* check for data in test

* remove warning string

* Correct expected and actual

* set database through environment variables

* reset environment variable at end of test

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
2021-02-01 15:18:52 -05:00
Hossein
4d5ca92a04 [MM-28000] Fix system manager can download export files without right permissions (#16569)
Automatic Merge
2021-01-12 20:45:17 +01:00
Agniva De Sarker
e89b26e8f3 goimports (#16640)
* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w`

* added goimports lint check to .golangci.yml

* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w` for a corner case

* make app-layers, *-mocks and store-layers for ci check

Co-authored-by: Mahmudul Haque <mahmudulhaque@protonmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-01-07 22:42:43 +05:30
Ibrahim Serdar Acikgoz
8036ba86e7 [MM-31398] upgrade error level to critical in case of panic or os.Exit (#16584)
* upgrade error level to critical in case of panic or os.Exit

* reflect revivew comments

* fix error capitalization
2020-12-23 13:11:35 +03:00
Hossein
d666bb8b3d Add Billing Permissions (#16364)
* add billing permissions

* update store.go

* Update permission.go

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-12-08 12:41:28 -05:00
Agniva De Sarker
d137cf0aa5 MM-31082: Use a base path for test assets (#16478)
* MM-31082: Use a base path for test assets

Tests can be run from other tools using a different base path.
We need to respect the MM_SERVER_PATH variable for that.

https://mattermost.atlassian.net/browse/MM-31082

```release-note
NONE
```

* do not append mattermost-server if env var is set

* fix typo
2020-12-04 20:38:07 +05:30
Jesús Espino
a74fe05695 Removing supplier concept from the sql store (#16355)
* Removing supplier concept from the sql store

* Removing other metions to supplier

* Fixing gofmt

* Fixing gofmt

* Renaming NewSqlStore to New

* Fixing tests

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-12-02 12:06:23 +01:00
Ibrahim Serdar Acikgoz
213345fa2c add manage_remote_clusters permission (#16252) 2020-11-18 18:40:12 +03:00
Agniva De Sarker
c82c37a36e MM-28067: Optimize app server startup in tests (#16263)
* MM-28067: Optimize app server startup in tests

For every test, we would wipe out the database and then start
the server again. This would run through all the migrations
and create new rows in Systems and Roles table every time.
As a result, this was consuming a lot of setup time for every test.

We optimize this by preloading the DB with dummy data in the roles
and systems table so that the server code just skips over those migrations.
This is completely forward compatible and adding new migrations does not
need to generate the sql files again. Only in case of schema changes to the
roles or systems table, this would need to be done.

It is unlikely the Systems table schema will get changed. The Roles table
might change in future but it's a comparatively rare event. Given the reduction
in CI runtime we are seeing, it's a worthy optimization.

We also apply some more optimizations:
- Coalesce multiple UpdateConfig calls into a single one. Each UpdateConfig call
has to do a json marshal which would take precious CPU cycles. It's much more
efficient to do everything in a single call.
- Remove unnecessary debug.FreeOSMemory in reload config. This was an artifact
from old days and is no longer required.

Numbers:
Results show a full **2 minutes** shaved off the test runtime. Earlier, tests
would take around 16 minutes. Now they take 14 minutes.

```release-note
NONE
```

* fix app package
2020-11-16 13:55:32 +01:00
Farhan Munshi
c9a4a475d3 [MM-28717] Refactor applyMultiRoleFilters to use sq builder (#15500)
* Refactor apply multi role filters and add role filters to get all profiles

* Add some tests

* Fix tests

* Fix lint

* Trigger CI

* Rename param to make more sense

* Tie get filtered user stats to usermanagement read users

* Dont filter out other system roles when searching for team members or team admins only filter out system admins

* add new permissions

* add migration

* fix test

* remove system roles as default permissions

* implement changes discussed with dennis

* add read only and fix i18n

* use model consts instead of strings

* turn the permissions into pseudo constants

* Update read only default permissions

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Hossein Ahmadian-Yazdi <hyazdi1997@gmail.com>
2020-11-13 10:57:57 -05:00
Agniva De Sarker
6c71fbaebd MM-29478: Fix race in TestNotifyClusterEvent (#16240)
The messages slice was accessed in a race manner. We protect it
with a mutex

```release-note
NONE
```

https://mattermost.atlassian.net/browse/MM-29478
2020-11-09 19:25:51 +05:30
Agniva De Sarker
72432ab3ff MM-27909: Add manage_shared_channels permission (#15601)
* MM-27909: Add manage_shared_channels permission

We add a new permission to manage shared channels.

It's a channel scoped permission and only the system admin
has that by default.

https://mattermost.atlassian.net/browse/MM-27909

* change to system scoped

* Trigger CI

* Trigger CI

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-10-26 15:54:50 +05:30
Farhan Munshi
2baf95df1e [MM-27274] Add permission to convert public channel to private (#15370)
* Add permission to convert public channel to private

* Rename to PERMISSION_CONVERT_PUBLIC_CHANNEL_TO_PRIVATE

* Fix tests

* Update definitions for roles that have manage_team

* Fix tests

* Add convert private channel to public permission

* Add to channel scoped perms

* Update permission checks on channel privacy endpoint

* Trigger CI
2020-09-03 13:41:07 -04:00
Miguel de la Cruz
49a75059c5 [MM-27919] Allow i18n and resources to be loaded using an environment variable (#15341)
* [MM-27919] Allow i18n and resources to be loaded using an environment variable

* Improve error message

* Modify environment variable name to be less ambiguous

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-08-25 17:29:40 +02:00
Martin Kraft
8354206e5c MM-25543: New Admin Roles (#14960)
* MM-23832: Initial set of changes

* MM-23832: further iteration

* MM-23832: further iteration

* MM-23832: further iteration

* MM-23832: Fixes merge.

* create migration for new Roles

* MM-23832: Renames some roles.

* MM-23832: Adds ability to see logs.

* MM-23832: Removes manage roles from restricted admin.

* MM-23832: Make authentication section read-only for restricted admin.

* MM-23832: Allow restricted admin to purge caches.

* MM-23832: Adds ability to recycle DB connections.

* MM-23832: Adds ability to purge indexes.

* MM-23832: Adds ability to test email and S3 config.

* MM-23832: Adds abilituy to read job status.

* MM-23832: Adds ability to read plugin statuses.

* MM-23832: Renames Restricted Admin to System Manager.

* MM-23832: Adds manage team roles to system_user_manager.

* MM-23832: Updates some permissions.

* MM-23832: Allow get all channels and get moderations.

* MM-23832: Adds some permissions to User Manager.

* MM-23832: Remove write users from user manager.

* MM-23832: Changes permissions for the usermanagement > users sysconsole section.

* MM-23832: Removes read_settings and write_settings permissions. Ensures the usermanagement parent permissions encompass the sub-permissions.

* MM-23832: Updates permissions.

* MM-23832: Changes some permissions checks, adds new permissions to roles.

* MM-23832: Adds ability to update a role.

* MM-23832: Permissions updates.

* MM-23832: Removes write access to plugins for system manager.

* MM-23832: Removes read compliance from new roles.

* MM-23832: Adds mock for new roles creation migration.

* MM-23832: Changes to variadic param.

* MM-23832: Removes some duplication in the permissions model. Renames some permissions constants.

* MM-23832: Updates some migrations.

* MM-23832: Removes some unnecessary constants.

* MM-23832: Changes back to old app method name.

* MM-23832: Fixes incorrect permission check.

* MM-23832: Changes write to read permission check.

* MM-23832: Removes the authentication permission from link/unlink group.

* MM-23832: Enable testing LDAP with read permissions.

* MM-23832: Make testing elasticsearch a read permission.

* MM-23832: Warn metrics are associated to any system console read permissions.

* MM-23832: Updates some permissions checks.

* MM-23832: Removes non-systemconsole permissions from roles.

* MM-23832: Update default permission assignment of sysadmin.

* MM-23832: Fixes incorrect permission check. Removes some unused stuff.

* MM-23832: Update permission to check.

* MM-23832: Switches to struct tags.

* MM-23832: Adds some docs for the permissions tag.

* MM-23832: Removes whitespace.

* MM-23832: Combines system admin restricted access with other acess-control tag.

* MM-23832: Fixes some tests.

* MM-23832: Clarifies docs, does not assume prior permission check in '-' access value case.

* MM-23832: Updates to correct access tag value.

* MM-23832: Adds test of the config settings tag access.

* MM-23832: Undoes whitespace change.

* MM-23832: Removes comment.

* MM-23832: Adds the permissions to the new roles rather than using OR conditions on the permissions checks.

* MM-23832: Removes or condition on permission check.

* MM-23832: Updates mapping.

* MM-23832: Typo fix.

* MM-23832: Adds new 'read_jobs' permission.

* MM-23832: Add read_jobs to all roles with manage_jobs.

* MM-23832: Adds new permission read_other_users_teams.

* MM-23832: Adds read filtering of config.

* MM-23932: Change tag value.

* MM-23832: Fixes some tests. Adds test for read config access tag.

* MM-23832: Adds permissions to list teams.

* MM-23832: Removes the '-' tag value. Adds a new permission read_channel_groups. Updates a permission check.

* MM-23832: Removes unnecessary parent permission for user_management. Fixes permission check change error.

* MM-23832: Removes unused parameter to filter/merge function.

* MM-23832: Renames migration name.

* MM-23832: Fix for godoc.

* MM-23832: Fixes tests.

* MM-23832: Only makes a map once rather than every function call. Doesn't require access tag on config field structs. Reverts one test update and fixes another.

* MM-23832: Removes all of the unnecessary uses of (*App).SessionHasPermissionToAny since removing the user_management parent permission.

* MM-23832: Updates constant type.

* MM-23832: Removes unnecessary comment.

* MM-23832: Renames permissions.

* MM-23832: Fix for permission name changes.

* MM-23832: Adds missing config access tags. Adds some requirec ancillary permissions for write_usermanagement_teams.

* MM-23832: Adds local API endpoint for getting config.

* MM-23832: If tag value is blank or restrict_sys_admin_write then don't do the permission check.

* MM-23832: nil check for strings prior to dereferencing.

* MM-23832: Fix for config display logic.

* MM-23832: Updates godoc.

* MM-23832: Delays the unrestricted check for parity with other permissions checks if the channel id does not exist.

* MM-23832: Removes tautology.

* MM-23832: Re-adds status code check.

* MM-23832: Adds new permission to edit brand image.

* MM-23832: Exports variable for use by mmctl.

* MM-23832: Initialize exported map for use by mmctl.

* MM-23832: Accept deprecated permissions as valid.

* MM-23832: Adds missing permissions to archive a channel.

* MM-23832: Adds missing permissions for managing team.

* MM-23832: Properly filters config values in patch and update API responses.

* MM-23832: Fixes license viewing and writing permissions.

* MM-23832: Require license to assign 'new system roles'.

* MM-23832: Adds translation keys.

* MM-23832: Updates translation order.

* MM-27529: Splits read_channel_groups into read_public_channel_groups and read_private_channel_groups.

* MM-23832: Prevent read-only permissions from editing site url test parameter.

* MM-23832: Prevent read permissions from sniffing ports and elastic password.

* MM-23832: Adds missing permission required for write user management channels.

* MM-23832: Allows new roles to search for channels.

* MM-23832: Adds ability for system_manager to manage jobs.

* MM-23832: Cluster status access by sysconsole permission, not manage_system.

* MM-23832: Adds 'add_user_to_team' permission to sysconsole write usermanagement teams.

* MM-23832: Fixes lint.

* MM-23832: Test fix.

* MM-23832: Test fix.

Co-authored-by: Catalin Tomai <catalin.tomai@mattermost.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-08-21 16:49:31 -04:00
Jesús Espino
7fe6c94eda Adding Upgrade to Enterprise version feature (#14539)
* Adding Upgrade to Enterprise version feature

* Addressing PR review comments, and adding some minor improvements

* Add tests file

* Addressing PR comments

* fix linter checks

* Storing and exposing the upgraded from TE info

* Fix showing errors on mac

* A more appropiate status code for not-supported upgrade

* Fixing tests

* Handling permissions errors

* More server logging around upgrade failures

* Apply text changes suggested from code review

Co-authored-by: Eric Sadur <57730300+esadur@users.noreply.github.com>

* Address PR review comments

* Only allow to restart the system after an upgrade

* Verify file signature before upgrade

* Adding limit to the downloaded file

* Simplifying the upgrade binary process with backup in memory

* Fixing backup/restore mechanism for the binary file

* Improve file permissions handling

* Askin the permissions for the right place (the parent directory)

* Fixing tests

* Addressing PR review comments

* Fix license headers

* Fixing retry layer

* Making it work on windows builds

* Adding license header

* Fixing 2 tests

* Fixing tests that need UpgradeFromTE System key mock

* Extracting i18n translation

* Apply suggestions from code review

Co-authored-by: Eric Sadur <57730300+esadur@users.noreply.github.com>

* Improving how the errors are written

* Fixing another error text

* Removing unneeded translation

* Fixing upgrade status strings

* Update i18n/en.json

Co-authored-by: Eric Sadur <57730300+esadur@users.noreply.github.com>

* Fixing tests

Co-authored-by: Eric Sadur <57730300+esadur@users.noreply.github.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-08-21 20:23:04 +02:00
Rodrigo Villablanca
20e44399c7 SystemStore migration to return plain errors (#14835)
* SystemStore migration to return plain errors

* Fix nilness

* Fix translations

* Fix merge

* Fix layers

* Fix merge errors

* Lint: remove unnecessary use of sprint

* Fix merge errors

* Fix i18n

Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
2020-08-13 20:32:57 +05:30
Agniva De Sarker
3cf05efdd9 MM-25404: Add GetHealthScore metric for the cluster (#14767)
* MM-25404: Add GetHealthScore metric for the cluster

This PR adds the necessary function to the cluster interface
so that they can be called from App code.

* Add mocks

* Remove fakeapp

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-06-24 18:46:33 +05:30
Miguel de la Cruz
4fde004a5b [MM-23535] Add disable database search flag and return empty posts results if set (#14245)
* [MM-23535] Add disable database search flag and return empty posts results if set

* Add UpdateConfig function for the SearchStore and hook it into the app lifecycle

* Add the config listener in the server instance instead of using FakeApp

* Instantiate searchlayer as a pointer to avoid passing around copies of it
2020-05-13 14:00:57 +02:00
Eli Yukelzon
7800116429 MM-23093 Implement Server Setup telemetry - server configuration (#14374)
* added advanced first day diagnostics reporting

* typo

* config corrected

* defaults

* moved from config to system db table

* missing file

* added error handling

* tests

* typos

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-04-30 18:18:12 +03:00
Jesse Hallam
636d168b84 MM-23926: avoid setting last viewed for bots (#14253)
The current code path for `CreatePostAsUser` tries to update the `LastViewedAt` for bots, which logs a warning message if the bot isn't actually in the channel.

This pull request changes the semantics of bot posting to not update the bot's `LastViewedAt` timestamp, avoiding this log altogether. It matches the semantics of `from_webhook`, but notably makes bots slightly less like users in that they no longer "read" channels when they post. This seems reasonable, but I'm both looking for validation of this semantic change in addition to the code review.

Fixes: https://mattermost.atlassian.net/browse/MM-23926

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-04-22 12:28:31 -03:00
Farhan Munshi
ab338e8417 [MM-23019] Create use_group_mentions permission and migrate existing roles (#13987)
* MM-22962 Create use_group_mentions permission and give to all non guest roles that can create post

* Add use_group_mentions to team admin role for test

* Trigger CI

* MM-22962 Remove old migration keys
2020-03-30 23:08:48 -04:00
Jesse Hallam
f149ada16a MM-23261 plugin stderr debug logs (#14166)
* explicitly assert panic as error log

* Revert "[MM-18150] plugin panic trace should not be lost (#13559)"

This reverts commit 5d928b4f94, while leaving the unit tests intact
and now asserting debug logs instead.

* missing license header
2020-03-30 15:00:45 -03:00
Miguel de la Cruz
4fe25b1cdd [MM-21551] Add search tests structure to test the search engines (#14031)
* WIP

* Adding bleve to go modules

* WIP

* Adding missing files from searchengine implementation

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* User and channel indexing and searches implemented

* Make bleve tests run with in-memory indexes

* Implement post index and deletion tests

* Initial commits for the search layer

* Removing unnecesary indexing

* WIP

* WIP

* More fixes for tests

* Adding the search layer

* Finishing the migration of searchers to the layer

* Removing unnecesary code

* Allowing multiple engines active at the same time

* WIP

* Add simple post search

* Print information when using bleve

* Adding some debugging to understand better how the searches are working

* Making more dynamic config of search engines

* Add post search basics

* Adding the Purge API endpoint

* Fixing bleve config updates

* Adding missed file

* Regenerating search engine mocks

* Adding missed v5 to modules imports

* fixing i18n

* Fixing some test around search engine

* Removing all bleve traces

* Cleaning up the vendors directory and go.mod/go.sum files

* Regenerating timer layer

* Adding properly the license

* Fixing govet shadow error

* Fixing some tests

* Fixing TestSearchPostsFromUser

* Fixing another test

* Fixing more tests

* Fixing more tests

* Removing SearchEngine redundant text from searchengine module code

* Fixing some reindexing problems in members updates

* Fixing tests

* Addressing PR comments

* Reverting go.mod and go.sum

* Addressing PR comments

* Fixing tests compilation

* Fixing govet

* Adding search engine stop method

* Being more explicit on where we use includeDeleted

* Adding GetSqlSupplier test helper method

* Mocking elasticsearch start function

* Fixing tests

* Search tests

* Fix tests

* Fix mod

* Fixing searchEngine for test helpers with store mocks

* Remove loglines

* Fix i18n strings

* Migrate search posts tests

* Fix linter

* Do not run search tests if -short flag is enabled

* Migrate back store tests that didn't belong to the searchlayer

* Fix scopelint issues

Co-authored-by: Jesús Espino <jespinog@gmail.com>
2020-03-30 19:17:40 +02:00
Martin Kraft
4d99aa22ba MM-22212 & MM-22208: Read from the higher-scoped scheme if the permission is non-moderated. (#13813)
* MM-22212: Read non-moderated permissions from higher-scoped scheme.

* MM-2212: Corrects test count in comment.

* MM-22212: Adds godoc comment.

* MM-2212: Switches to the channel roles check in a few more places.

* MM-22212: Refactors and fixes.

* MM-22212: Reverts change, no longer required.

* MM-22212: Removes translation.

* MM-22212: Un-comments merged new permission.

* MM-22212: Un-comments merged new permission.

* MM-22212: Performance tweak.

* MM-22212: Fixes some fmting.

* MM-22212: Add unit test for newly-added store methods.

* MM-22212: Renames app method.

* MM-22212: Re-uses existing function to find string in slice.

* MM-22212: Keeps 'higher-scoped' terminology for consistency.

* MM-22212: Refactors based on PR feedback.

* MM-22212: Fix for some bad merging.

* MM-22212: Renamed some things.

* MM-22212: Use an 'else' instead of a 'continue' for readability.

* MM-22212: Caches (*SqlRoleStore).ChannelRolesUnderTeamRole.

* MM-22212: Adds mock to new cache store.

* MM-22212: Adds missing open tracing app layer methods.

* MM-22212: Adds migration to add moderated permissions to channel_admin if present on channel_user.

* MM-22212: Migrates team schemes. Removes unused AppError.

* MM-22212: Fix for for if.

* MM-22212: Fixes iterator.

* MM-22212: Updates open tracing generated methods.

* MM-22212: Fix mocks.

* MM-22212: Change migration key name.

* MM-22212: Switched to data structure from other branch.

* MM-22212: Fixes tests after adding 'use_channel_mentions' to the channel_admin role.

* MM-22212: Adds tracking of channel moderation.

* Revert "MM-22212: Adds tracking of channel moderation."

This reverts commit 23689efa22c112e4ba37f6a212535dd7ebfb63db.

* MM-22212: Switch some functions to methods and vice versa.

* MM-22212: Fix for refactor bug not notifiying websocket about changed role.

* MM-22212: Adds test for public/private 'manage_members' handling.

* MM-22122 Fix manage channel members edge case for public and private channels (#14049)

* MM-22212: Adds moderated permission to team_admin.

* MM-22212: Updates migration.

* MM-22212: Revert unnecessary update to default roles.

* Add channel scheme updated event when channel scheme is deleted or created (#14057)

* MM-22212: Adds newline.

* MM-22212: Migration fix.

* MM-22212: Fix for migration.

* MM-22212: Test fix.

Co-authored-by: Farhan Munshi <3207297+fm2munsh@users.noreply.github.com>
2020-03-23 13:44:20 -04:00
Jesús Espino
c66e182b08 Adding the new search engine abstraction (#13304)
* WIP

* Adding bleve to go modules

* WIP

* Adding missing files from searchengine implementation

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* User and channel indexing and searches implemented

* Make bleve tests run with in-memory indexes

* Implement post index and deletion tests

* Initial commits for the search layer

* Removing unnecesary indexing

* WIP

* WIP

* More fixes for tests

* Adding the search layer

* Finishing the migration of searchers to the layer

* Removing unnecesary code

* Allowing multiple engines active at the same time

* WIP

* Add simple post search

* Print information when using bleve

* Adding some debugging to understand better how the searches are working

* Making more dynamic config of search engines

* Add post search basics

* Adding the Purge API endpoint

* Fixing bleve config updates

* Adding missed file

* Regenerating search engine mocks

* Adding missed v5 to modules imports

* fixing i18n

* Fixing some test around search engine

* Removing all bleve traces

* Cleaning up the vendors directory and go.mod/go.sum files

* Regenerating timer layer

* Adding properly the license

* Fixing govet shadow error

* Fixing some tests

* Fixing TestSearchPostsFromUser

* Fixing another test

* Fixing more tests

* Fixing more tests

* Removing SearchEngine redundant text from searchengine module code

* Fixing some reindexing problems in members updates

* Fixing tests

* Addressing PR comments

* Reverting go.mod and go.sum

* Addressing PR comments

* Fixing tests compilation

* Fixing govet

* Adding search engine stop method

* Being more explicit on where we use includeDeleted

* Adding GetSqlSupplier test helper method

* Mocking elasticsearch start function

* Fixing tests

Co-authored-by: Miguel de la Cruz <miguel@mcrx.me>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-13 15:33:18 +01:00