Граф коммитов

302 Коммитов

Автор SHA1 Сообщение Дата
Rodrigo Villablanca
1d4a7c0552 TeamStore migration part 2 (#15396)
* Migration completed

* Fix tests

* Reduce to one line

* Fix: change to plain error

* Fix imports

* Trigger CI

* Fix i18

* Fix merge with master

* Trigger CI
2020-09-16 21:54:39 -03:00
Rodrigo Villablanca
9ee9c78412 ChannelStore migration Part 2 (#15405)
* Migration completed

* Several corrections in tests

* Fix imports

* Fix some errors after testing

* Trigger CI

* Fix tests

* Suggestions

* Suggestions

* Add license

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-15 23:18:30 +05:30
Rodrigo Villablanca
2cb655ed67 Migration of ChannelStore Part 1 (#15235)
* Lint: remove unnecessary use of sprintf

* Fix i18n

* Returning the right error (InternalServerError)

* Doing some suggestions

* Fix store layers

* Fix missed translation

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-03 11:24:46 +05:30
Rodrigo Villablanca
54f86e7fb1 Migration of TeamStore Part 1 (#15246)
Automatic Merge
2020-09-03 00:29:57 -04:00
Martin Kraft
8354206e5c MM-25543: New Admin Roles (#14960)
* MM-23832: Initial set of changes

* MM-23832: further iteration

* MM-23832: further iteration

* MM-23832: further iteration

* MM-23832: Fixes merge.

* create migration for new Roles

* MM-23832: Renames some roles.

* MM-23832: Adds ability to see logs.

* MM-23832: Removes manage roles from restricted admin.

* MM-23832: Make authentication section read-only for restricted admin.

* MM-23832: Allow restricted admin to purge caches.

* MM-23832: Adds ability to recycle DB connections.

* MM-23832: Adds ability to purge indexes.

* MM-23832: Adds ability to test email and S3 config.

* MM-23832: Adds abilituy to read job status.

* MM-23832: Adds ability to read plugin statuses.

* MM-23832: Renames Restricted Admin to System Manager.

* MM-23832: Adds manage team roles to system_user_manager.

* MM-23832: Updates some permissions.

* MM-23832: Allow get all channels and get moderations.

* MM-23832: Adds some permissions to User Manager.

* MM-23832: Remove write users from user manager.

* MM-23832: Changes permissions for the usermanagement > users sysconsole section.

* MM-23832: Removes read_settings and write_settings permissions. Ensures the usermanagement parent permissions encompass the sub-permissions.

* MM-23832: Updates permissions.

* MM-23832: Changes some permissions checks, adds new permissions to roles.

* MM-23832: Adds ability to update a role.

* MM-23832: Permissions updates.

* MM-23832: Removes write access to plugins for system manager.

* MM-23832: Removes read compliance from new roles.

* MM-23832: Adds mock for new roles creation migration.

* MM-23832: Changes to variadic param.

* MM-23832: Removes some duplication in the permissions model. Renames some permissions constants.

* MM-23832: Updates some migrations.

* MM-23832: Removes some unnecessary constants.

* MM-23832: Changes back to old app method name.

* MM-23832: Fixes incorrect permission check.

* MM-23832: Changes write to read permission check.

* MM-23832: Removes the authentication permission from link/unlink group.

* MM-23832: Enable testing LDAP with read permissions.

* MM-23832: Make testing elasticsearch a read permission.

* MM-23832: Warn metrics are associated to any system console read permissions.

* MM-23832: Updates some permissions checks.

* MM-23832: Removes non-systemconsole permissions from roles.

* MM-23832: Update default permission assignment of sysadmin.

* MM-23832: Fixes incorrect permission check. Removes some unused stuff.

* MM-23832: Update permission to check.

* MM-23832: Switches to struct tags.

* MM-23832: Adds some docs for the permissions tag.

* MM-23832: Removes whitespace.

* MM-23832: Combines system admin restricted access with other acess-control tag.

* MM-23832: Fixes some tests.

* MM-23832: Clarifies docs, does not assume prior permission check in '-' access value case.

* MM-23832: Updates to correct access tag value.

* MM-23832: Adds test of the config settings tag access.

* MM-23832: Undoes whitespace change.

* MM-23832: Removes comment.

* MM-23832: Adds the permissions to the new roles rather than using OR conditions on the permissions checks.

* MM-23832: Removes or condition on permission check.

* MM-23832: Updates mapping.

* MM-23832: Typo fix.

* MM-23832: Adds new 'read_jobs' permission.

* MM-23832: Add read_jobs to all roles with manage_jobs.

* MM-23832: Adds new permission read_other_users_teams.

* MM-23832: Adds read filtering of config.

* MM-23932: Change tag value.

* MM-23832: Fixes some tests. Adds test for read config access tag.

* MM-23832: Adds permissions to list teams.

* MM-23832: Removes the '-' tag value. Adds a new permission read_channel_groups. Updates a permission check.

* MM-23832: Removes unnecessary parent permission for user_management. Fixes permission check change error.

* MM-23832: Removes unused parameter to filter/merge function.

* MM-23832: Renames migration name.

* MM-23832: Fix for godoc.

* MM-23832: Fixes tests.

* MM-23832: Only makes a map once rather than every function call. Doesn't require access tag on config field structs. Reverts one test update and fixes another.

* MM-23832: Removes all of the unnecessary uses of (*App).SessionHasPermissionToAny since removing the user_management parent permission.

* MM-23832: Updates constant type.

* MM-23832: Removes unnecessary comment.

* MM-23832: Renames permissions.

* MM-23832: Fix for permission name changes.

* MM-23832: Adds missing config access tags. Adds some requirec ancillary permissions for write_usermanagement_teams.

* MM-23832: Adds local API endpoint for getting config.

* MM-23832: If tag value is blank or restrict_sys_admin_write then don't do the permission check.

* MM-23832: nil check for strings prior to dereferencing.

* MM-23832: Fix for config display logic.

* MM-23832: Updates godoc.

* MM-23832: Delays the unrestricted check for parity with other permissions checks if the channel id does not exist.

* MM-23832: Removes tautology.

* MM-23832: Re-adds status code check.

* MM-23832: Adds new permission to edit brand image.

* MM-23832: Exports variable for use by mmctl.

* MM-23832: Initialize exported map for use by mmctl.

* MM-23832: Accept deprecated permissions as valid.

* MM-23832: Adds missing permissions to archive a channel.

* MM-23832: Adds missing permissions for managing team.

* MM-23832: Properly filters config values in patch and update API responses.

* MM-23832: Fixes license viewing and writing permissions.

* MM-23832: Require license to assign 'new system roles'.

* MM-23832: Adds translation keys.

* MM-23832: Updates translation order.

* MM-27529: Splits read_channel_groups into read_public_channel_groups and read_private_channel_groups.

* MM-23832: Prevent read-only permissions from editing site url test parameter.

* MM-23832: Prevent read permissions from sniffing ports and elastic password.

* MM-23832: Adds missing permission required for write user management channels.

* MM-23832: Allows new roles to search for channels.

* MM-23832: Adds ability for system_manager to manage jobs.

* MM-23832: Cluster status access by sysconsole permission, not manage_system.

* MM-23832: Adds 'add_user_to_team' permission to sysconsole write usermanagement teams.

* MM-23832: Fixes lint.

* MM-23832: Test fix.

* MM-23832: Test fix.

Co-authored-by: Catalin Tomai <catalin.tomai@mattermost.com>
Co-authored-by: Scott Bishel <scott.bishel@mattermost.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-08-21 16:49:31 -04:00
Harrison Healey
52611a1761 Move channel category code into its own files (#15319)
* Move channel category code into its own files

* Move channel category tests into app/channel_category_test.go
2020-08-21 15:07:55 -04:00
Claudio Costa
e6c1d5f75b Fix panic on DB error (#15310)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-08-21 18:38:14 +02:00
Doug Lauder
095e5d788b MM-27868 fix panic when user joins teams (#15268)
Server panics when user joins team and one of the default channels has been archived.

This can be triggered by archiving "Off-Topic" or any of the channels listed in `TeamSettings.ExperimentalDefaultChannels`.
2020-08-17 08:20:04 -04:00
Ashish Bhate
d76039db23 [MM-25659] ability to permanent delete channel through client (#15202)
Summary

    Ability to permanent delete channel through client

Ticket Link

    https://mattermost.atlassian.net/browse/MM-25659
2020-08-14 14:12:39 +05:30
Rodrigo Villablanca
5566395032 First part of PostStore migration (#15123)
Automatic Merge
2020-08-12 13:35:57 -04:00
Ibrahim Serdar Acikgoz
3f19a8c011 [MM-27376] api4/channel: add move channel to local mode (#15200)
* api4/channel: add move channel to local mode

* app/channel: reflect review comments
2020-08-10 17:44:13 +02:00
Farhan Munshi
d4dac31b04 MM-27407 Return archived channels even if view archived config is false for sysadmin endpoints (#15149)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-08-04 12:02:12 +05:30
Rodrigo Villablanca
1c5b6522e3 WebhookStore migration (#15042)
* Migration completed

* Fix tests

Co-authored-by: Jesús Espino <jespinog@gmail.com>
2020-07-31 15:53:10 +02:00
Agniva De Sarker
77f7a97bee MM-25563: Add endpoint to fetch only archived channels (#15031)
Automatic Merge
2020-07-27 13:31:39 +02:00
Farhan Munshi
ed34468996 [MM-25990] Add filters to search all channels (#15009)
* MM-25990 Add filters to search all channels endpoint and clean up tests

* Add deleted filter

* Remove redundant private public query
2020-07-23 10:46:33 -04:00
Agniva De Sarker
aad940e104 MM-27187: Use the correct page offset for cache clear methods (#15094)
* MM-27187: Use the correct page offset for cache clear methods

We were just using page and incrementing by 1. This would fetch pages
one by one like (1-100, 2-102, 3-103) rather than (1-100,100-200,200-300).

We fix that to update the correct page offset.

* Trigger CI
2020-07-23 19:55:16 +05:30
Ibrahim Serdar Acikgoz
bf664997a7 [MM-21464] api4/channel: add ability to force move channel by removing non-members (#14887)
* api4/channel: add ability to force move channel by removing non-members

* app/channel: add log to move channel for non taget team members

* app/channel: add tets for remove non-team members for channel

* Update api4/channel_test.go

Co-authored-by: Eli Yukelzon <reflog@gmail.com>

Co-authored-by: Eli Yukelzon <reflog@gmail.com>
2020-07-16 11:26:19 +03:00
Harrison Healey
302e59a0fe MM-26753 Create initial sidebar categories on demand if migration hasn't ran (#14981)
* MM-26753 Change CreateInitialSidebarCategories to only take a user ID

* MM-26753 Create initial sidebar categories on demand if migration hasn't ran

* Wait for sidebar categories to be loaded in case of replication lag
2020-07-08 12:07:42 -04:00
Harrison Healey
de6a57cdc3 Sidebar V2 Phase 2 (#14967)
* parent 48256721c4 (#14358)

author Eli Yukelzon <reflog@gmail.com> 1585814774 +0300
committer Eli Yukelzon <reflog@gmail.com> 1589111022 +0300

Sidebar caregories implemented

Apply suggestions from code review

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>

Update store/sqlstore/channel_store.go

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>

Update store/sqlstore/channel_store.go

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>

code review suggestions

status messages

edge case

bugs...

timeout reverse

* MM-25126 Add a separate default sorting method for categories (#14575)

* MM-25158 Add user to initial sidebar categories when joining team (#14570)

* MM-25281  Place new categories in the correct position (#14609)

* MM-25277  Return channels that aren't in a category as part of the Channels/Direct Messages categories (#14601)

* MM-25276  Remove categories when leaving a team (#14600)

* Remove categories when leaving a team

* layers

* corrected cleanup function

* lint

* .

* corrected errors in postgres

* .

* MM-25280  Ensure that the "update category order" API call only contains real category IDs and isn't missing any IDs (#14626)

* Ensure that the "update category order" API call only contains real category IDs and isn't missing any IDs

* tests

* correct status code

* MM-25278  Change "update category" API to return 400 when changing unsupported fields (#14599)

* MM-25279  Change category migration to only populate channels in Favorites (#14627)

* MM-25157 Add API to delete custom categories  (#14574)

* MM-25157 Add API to delete custom categories

* get categories fix

* maxorder fix

* Use correct websocket event when deleting category

* Fix tests and remove debug code

* Actually use the right websocket event this time

* test cleanup

* Update test for new category order

Co-authored-by: Eli Yukelzon <reflog@gmail.com>

* MM-24914 Various fixes for sidebar channel handling (#14756)

* Fix checking for channel membership when reordering channels

* Remove unique constraint on SidebarCategories

* Set column sizes for SidebarCategories and SidebarChannels tables

* Allow changing the sorting method for non-DM categories

* Fix nil pointers in error handling

* Fix orphaned channels from other team being returned in Channels category

* Fix non-orphaned channels being duplicated in the Channels category

* Remove unique constraint on SidebarChannels

* Fix category/name of favorites preferences

* Fix testSidebarChannelsMigration

* Rename err to nErr and appErr to err

* Fix channel order returned by GetSidebarCategories on MySQL

* Fix adding/removing favorites preferences

* Remove leftover TODO

* Change SidebarCategoryType enums to use full names (#14786)

* Change SidebarCategoryType enums to use full names

* Fix Channels constant

* Remove leftover debug code

* MM-24914 Fix updateCategory endpoint returning the wrong type (#14795)

* MM-24914 Make some changes to UpdateSidebarCategories (#14806)

* Fix orphaned DMs not always being returned

* MM-24914 Make some changes to UpdateSidebarCategories

* Run updateSidebarCategoryOrderT in a transaction

* Fix deleting SidebarChannels based on order of arguments to UpdateSidebarCategories

* bump for api testing

* bump for api testing

* Change CreateInitialSidebarCategories to return a plain error

* Change MigrateSidebarCategories to return a plain error

* Remove usage of UpdateColumns when updating sidebar categories (#14843)

* Remove usage of UpdateColumns when changing category order

* Add a random test case

* Remove usage of UpdateColumns when updating sidebar categories (#14843)

* Remove usage of UpdateColumns when changing category order

* Add a random test case

* Remove usage of UpdateColumns when updating sidebar categories (#14843)

* Remove usage of UpdateColumns when changing category order

* Add a random test case

* MM-26343 Make CreateInitialSidebarCategories idempotent (#14870)

* Fix bad merge

* Fix another bad merge

* Fix unintentionally removed i18n string

Co-authored-by: Eli Yukelzon <reflog@gmail.com>
2020-07-06 18:20:35 -04:00
Ashish Bhate
af8b914c6c MM-23596: Ability to list private channels for team (#14925)
Summary:
store, app, api and go driver support for listing private channels

Ticket Link:
https://mattermost.atlassian.net/browse/MM-23596
2020-07-06 12:34:29 +05:30
Rodrigo Villablanca
0118db9d23 Migration of AuditStore to return plain errors (#14825)
Automatic Merge
2020-06-26 06:26:35 +02:00
Agniva De Sarker
a1547cfac5 MM-19548: Add a deadlock retry function for SaveChannel (#14857)
* MM-19548: Add a deadlock retry function for SaveChannel

A deadlock has been seen to occur in the upsertPublicChannelT method
during bulk import.

Here is a brief excerpt:

*** (1) TRANSACTION:
TRANSACTION 3141, ACTIVE 1 sec inserting
INSERT INTO
                            PublicChannels(Id, DeleteAt, TeamId, DisplayName, Name, Header, Purpose)
                        VALUES
                            (?, ?, ?, ?, ?, ?, ?)
                        ON DUPLICATE KEY UPDATE
                            DeleteAt = ?,
                            TeamId = ?,
                            DisplayName = ?,
                            Name = ?,
                            Header = ?,
                            Purpose = ?
*** (1) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 76 page no 4 n bits 104 index Name of table `mydb`.`PublicChannels` trx id 3141 lock_mode X locks gap before rec insert intention waiting
** (2) TRANSACTION:
TRANSACTION 3140, ACTIVE 1 sec inserting
mysql tables in use 1, locked 1
5 lock struct(s), heap size 1136, 3 row lock(s), undo log entries 2
MySQL thread id 50, OS thread handle 140641523848960, query id 3226 172.17.0.1 mmuser update
INSERT INTO
                            PublicChannels(Id, DeleteAt, TeamId, DisplayName, Name, Header, Purpose)
                        VALUES
                            (?, ?, ?, ?, ?, ?, ?)
                        ON DUPLICATE KEY UPDATE
                            DeleteAt = ?,
                            TeamId = ?,
                            DisplayName = ?,
                            Name = ?,
                            Header = ?,
                            Purpose = ?
*** (2) HOLDS THE LOCK(S):
RECORD LOCKS space id 76 page no 4 n bits 104 index Name of table `mydb`.`PublicChannels` trx id 3140 lock_mode X locks gap before rec

*** (2) WAITING FOR THIS LOCK TO BE GRANTED:
RECORD LOCKS space id 76 page no 4 n bits 104 index Name of table `mydb`.`PublicChannels` trx id 3140 lock_mode X locks gap before rec insert intention waiting

*** WE ROLL BACK TRANSACTION (1)

Following is my analysis:

From the deadlock output, it can be seen that it's due to a gap lock.
And that's clear because the index is Name which is a multi-column index using Name and TeamId.
But interestingly, both transactions seem to be inserting the same data, which is what is puzzling me.
The multi-column index on Name and TeamId will guarantee that they are always unique. And from looking at the code,
it does not seem possible to me that it will try to insert the same data from 2 different transactions.
But even if they do, why does tx 2 try to acquire the same lock again when it already has that ?

Here is what I think the order of events happening

Tx 2 gets a gap lock.
Tx 1 tries to get the same gap lock.
Tx 2 tries to again get the same gap lock ?

The last step is what is puzzling me. Why does an UPSERT statement acquire 2 gap locks ? From my reading of https://dev.mysql.com/doc/refman/8.0/en/innodb-locks-set.html:

> INSERT ... ON DUPLICATE KEY UPDATE differs from a simple INSERT in that an exclusive lock rather than a shared lock is placed on the row to be updated when a duplicate-key error occurs. An exclusive index-record lock is taken for a duplicate primary key value. An exclusive next-key lock is taken for a duplicate unique key value.

From what I understand, the expectation is that there will be one X lock and one gap lock is taken.
But that's not what the deadlock output seems to say.

The general advice on the internet seems to be that deadlocks will happen and not all of them can be understood.

For now, we add a generic deadlock retry function at the store package which can be reused by other queries too.

P.S.: This is a verbatim copy of my investigation posted at https://dba.stackexchange.com/questions/268652/mysql-deadlock-upsert-query-acquiring-gap-lock-twice

Testing:

This is ofcourse hard to test because it is impossible to reproduce this. I have tested this by manually returning an error
and confirming that it indeed retries.

WARN[2020-06-19T11:18:24.9585676+05:30] A deadlock happened. Retrying.                caller="sqlstore/channel_store.go:568" error="Error 1213: mydeadlock"
WARN[2020-06-19T11:18:24.959158+05:30] A deadlock happened. Retrying.                caller="sqlstore/channel_store.go:568" error="Error 1213: mydeadlock"
WARN[2020-06-19T11:18:24.9595072+05:30] A deadlock happened. Retrying.                caller="sqlstore/channel_store.go:568" error="Error 1213: mydeadlock"
WARN[2020-06-19T11:18:24.9595451+05:30] Deadlock happened 3 times. Giving up          caller="sqlstore/channel_store.go:579"
ERRO[2020-06-19T11:18:24.9596426+05:30] Unable to save channel.                       caller="mlog/log.go:175" err_details="Error 1213: mydeadlock" err_where=CreateChannel http_code=500 ip_addr="::1" method=POST path=/api/v4/channels request_id=745bsj13b7f6mnmsbn3t97grbw user_id=xcof1ipipbrfxpfjf6x4p6kx9e

* Fix tests

* Address review comments

* Address review comments

* Add forgotten test

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-06-23 08:44:39 +05:30
Ibrahim Serdar Acikgoz
124014ad9c [MM-23719] api4/channel: add move channel to a team endpoint (#14246)
* api4: add move channel method

* api4: add tests for move channel, model: add move channel to client4.go

* add api.channel.move_channel.type.invalid message

* model/client4: remove a redundant line

* api4/channel: add tests for gm and private channel types

* app/channel: update move channel comment

* app/channel: add extra check if a users joins to channel during movement

* app/channel: log errors for post move channel

* app/channel: remove deactivated members by default while moving a ch.

* model/client: update move channel command

* fix vet errors

* app/channel: add missing webhook updates

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-06-22 16:57:49 +03:00
Martin Kraft
6158e91308 MM-25261: Add 'include_deleted' parameter to get all channels API req… (#14721)
* MM-25261: Add 'include_deleted' parameter to get all channels API request.

* MM-25261: Adds test for deleted channels.

* MM-25261: Switches to single liner.

* MM-25261: Adhere to beta config setting for viewing archived channels.

* MM-25261: Test fix.

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-06-19 16:53:37 -04:00
Rodrigo Villablanca
d391fd6231 Migration of ChannelStore.GetMoreChannels to return plain error (#14811)
Automatic Merge
2020-06-19 20:26:35 +02:00
Rodrigo Villablanca
e342b5a2f2 Migrate multiples methods from ChannelStore to return error interface (#14708)
Automatic Merge
2020-06-16 10:56:35 +02:00
Rodrigo Villablanca
41d9c673cf Migrate GetChannels method from ChannelStore to return error interface (#14711)
* Migrate GetChannels method from ChannelStore to return error interface

* Fix testing

* Changed error type: ErrInvalidInput -> ErrNotFound

* Added note about error migrations

* Fix en.json

* Fix i18n

Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
2020-06-11 14:02:03 +05:30
Rodrigo Villablanca
9b0ae49b55 Migrate GetDeleted method from ChannelStore to return error interface (#14710)
* Migrate GetDeleted method from ChannelStore to return error interface

* Moving i18n translations to the correct place

Co-authored-by: Jesús Espino <jespinog@gmail.com>
2020-06-11 09:37:37 +05:30
Rodrigo Villablanca
cac154e62b Migrate GetAllChannels and GetAllChannelsCount from ChannelStore to r… (#14765)
* Migrate GetAllChannels and GetAllChannelsCount from ChannelStore to return plain errors

* Moving i18n translations to the correct place

Co-authored-by: Jesús Espino <jespinog@gmail.com>
2020-06-09 21:48:01 +05:30
Rodrigo Villablanca
6f28f3526d Migrate PermanentDelete method from ChannelStore to return error inte… (#14706)
Automatic Merge
2020-06-06 11:47:18 +02:00
Rodrigo Villablanca
ac32b2da41 Migrate Delete/Restore/SetDeleteAt methods from ChannelStore to return error interface (#14700)
Automatic Merge
2020-06-05 04:32:16 -07:00
Agniva De Sarker
21b753b38d [MM-25314] Migrate AppError from channel_member_history_store.go (#14693)
* Migrate AppError from channel_member_history_store

* Migrate AppError from channel_member_history_store

* Migrate AppError from channel_member_history_store

* Migrate AppError from channel_member_history_store

* Migrating error keys

* Review fixes.

* Regenerating i18n

* Add some i18n strings

Co-authored-by: Dante Pippi <6619666+dantepippi@users.noreply.github.com>
Co-authored-by: Jesús Espino <jespinog@gmail.com>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-06-04 13:08:51 +05:30
Rodrigo Villablanca
85a69d6112 Migrate Get/GetFromMaster methods from ChannelStore to return error interface (#14688)
* Advances

* Migration finished

* Rename err to normalized error

* fix imports

* Renamed key

* Renamed key

* Suggestions

* Fix i18n

* Fix tests

Co-authored-by: Jesús Espino <jespinog@gmail.com>
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
2020-06-02 21:58:29 +05:30
Rodrigo Villablanca
60cc775cf6 Migrate Update method from ChannelStore to return idiomatic plain error (#14687)
* Partial advances

* Update migrated

* Fix imports

* Suggestions

* Suggestions

* Updating i18n

Co-authored-by: Jesús Espino <jespinog@gmail.com>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-06-02 20:50:34 +05:30
Martin Kraft
e83cc7357c MM-24517: Add newly created channel admin role to all clients' redux state. (#14635)
* MM-24517: Add newly created channel admin role to redux state.

* MM-24517: Adds comment; fixes lint errors.
2020-05-28 14:38:27 -04:00
Agniva De Sarker
14f7118dde MM-22706: pass along set_online flag in websocket response (#14591)
* MM-22706: pass along set_online flag in websocket response

To let the client know whether a user has created a post without
being online or not, we get the set_online query param and
pass it down to the websocket event being passed down to the client.

With this PR, the "data" field of the `posted` event will contain
a `set_online` boolean field set to true/false depending on the
query_param set_online value set in the createPost call.

* Setting to false for auto responder

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-05-27 17:35:02 +05:30
Agniva De Sarker
090743de86 MM-24132: Migrate AppError from SaveDirectChannel/channel_store.go (#14318)
* MM-24135: Migrate AppError from SaveChannel/channel_store.go

This is the first POC of migration of store app errors to plain error.

We create a few basic error types in the store package and use
them to return the errors from store methods. In the app layer,
we inspect the error and re-create the exact app errors. This lets
us preserve the same error content, but yet move to plain errors.

Since this is a gradual migration, this means that the error inspection
code will be duplicated across the app layer whenever a store method
is invoked. But all of that should go away once we start propagating
the errors higher up the hierarchy.

There have been a significant amount of changes in the storetest and searchtest
layer, primarily because we have to rename the err variable now that it is of
a different type.

* Addressed review comments

* MM-24132: Migrate AppError from SaveDirectChannel/channel_store.go

This PR migrates 2 new methods SaveDirectChannel and CreateDirectChannel
to return error instead of AppError.

We also need to handle the error internally in SaveMultipleMember for now
until that is migrated too.

* Fix layers

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-05-14 23:41:05 +05:30
Agniva De Sarker
729a84a3e6 MM-24135: Migrate AppError from SaveChannel/channel_store.go (#14299)
* MM-24135: Migrate AppError from SaveChannel/channel_store.go

This is the first POC of migration of store app errors to plain error.

We create a few basic error types in the store package and use
them to return the errors from store methods. In the app layer,
we inspect the error and re-create the exact app errors. This lets
us preserve the same error content, but yet move to plain errors.

Since this is a gradual migration, this means that the error inspection
code will be duplicated across the app layer whenever a store method
is invoked. But all of that should go away once we start propagating
the errors higher up the hierarchy.

There have been a significant amount of changes in the storetest and searchtest
layer, primarily because we have to rename the err variable now that it is of
a different type.

* Addressed review comments

* Made all appError origins to be CreateChannel

* Remove typed internal error

* Fix translations

* fix layer generation

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-05-12 21:23:41 +05:30
Farhan Munshi
fff93cb73f [MM-24987] Dont call GetSchemeRolesForChannel and instead load the scheme from channel (#14514)
* MM-24987 Dont call a.GetSchemeRolesForChannel and instead load the scheme using the channel directly

* Trigger CI

* MM-24987 Add a test for concurrent patch to channel moderation

* MM-24987 Cleanup
2020-05-08 17:50:45 -04:00
Farhan Munshi
7bc630a600 [MM-23727] Make channel validation consistent on the server (#14230)
* MM-23727 Ensure user ids not allowed in channel name:

MM-23727 Move channel name validation to model level

* MM-23727 Update wording
2020-04-26 12:38:33 -04:00
Martin Kraft
f92d3fa518 MM-23876: Fix for patching channel moderations with a null team schem… (#14239)
* MM-23876: Fix for patching channel moderations with a null team scheme channel guest role.

* MM-23876: Tests the moderations response.
2020-04-07 17:35:03 +05:30
Agniva De Sarker
ef209c5e0b MM-23620: Handle error from GetUser (#14204)
* MM-23620: Handle error from GetUser

In case of high DB load, the DB will start to throw errors.
Unless we handle the error appropriately, the server will crash.

* Removing unnecessary lines
2020-04-06 21:07:30 +05:30
Martin Kraft
19cf3e85b7 MM-23770: Fix for blank DefaultChannelGuestRole on team schemes. (#14210)
* MM-23770: Fix for blank DefaultChannelGuestRole on team schemes.

* MM-23770: Adds test for an team scheme with a blank DefaultChannelGuestRole field.

* MM-23770: Fix for unexpected Schemes.Get.
2020-04-02 08:09:23 -04:00
Martin Kraft
4d99aa22ba MM-22212 & MM-22208: Read from the higher-scoped scheme if the permission is non-moderated. (#13813)
* MM-22212: Read non-moderated permissions from higher-scoped scheme.

* MM-2212: Corrects test count in comment.

* MM-22212: Adds godoc comment.

* MM-2212: Switches to the channel roles check in a few more places.

* MM-22212: Refactors and fixes.

* MM-22212: Reverts change, no longer required.

* MM-22212: Removes translation.

* MM-22212: Un-comments merged new permission.

* MM-22212: Un-comments merged new permission.

* MM-22212: Performance tweak.

* MM-22212: Fixes some fmting.

* MM-22212: Add unit test for newly-added store methods.

* MM-22212: Renames app method.

* MM-22212: Re-uses existing function to find string in slice.

* MM-22212: Keeps 'higher-scoped' terminology for consistency.

* MM-22212: Refactors based on PR feedback.

* MM-22212: Fix for some bad merging.

* MM-22212: Renamed some things.

* MM-22212: Use an 'else' instead of a 'continue' for readability.

* MM-22212: Caches (*SqlRoleStore).ChannelRolesUnderTeamRole.

* MM-22212: Adds mock to new cache store.

* MM-22212: Adds missing open tracing app layer methods.

* MM-22212: Adds migration to add moderated permissions to channel_admin if present on channel_user.

* MM-22212: Migrates team schemes. Removes unused AppError.

* MM-22212: Fix for for if.

* MM-22212: Fixes iterator.

* MM-22212: Updates open tracing generated methods.

* MM-22212: Fix mocks.

* MM-22212: Change migration key name.

* MM-22212: Switched to data structure from other branch.

* MM-22212: Fixes tests after adding 'use_channel_mentions' to the channel_admin role.

* MM-22212: Adds tracking of channel moderation.

* Revert "MM-22212: Adds tracking of channel moderation."

This reverts commit 23689efa22c112e4ba37f6a212535dd7ebfb63db.

* MM-22212: Switch some functions to methods and vice versa.

* MM-22212: Fix for refactor bug not notifiying websocket about changed role.

* MM-22212: Adds test for public/private 'manage_members' handling.

* MM-22122 Fix manage channel members edge case for public and private channels (#14049)

* MM-22212: Adds moderated permission to team_admin.

* MM-22212: Updates migration.

* MM-22212: Revert unnecessary update to default roles.

* Add channel scheme updated event when channel scheme is deleted or created (#14057)

* MM-22212: Adds newline.

* MM-22212: Migration fix.

* MM-22212: Fix for migration.

* MM-22212: Test fix.

Co-authored-by: Farhan Munshi <3207297+fm2munsh@users.noreply.github.com>
2020-03-23 13:44:20 -04:00
Farhan Munshi
911e636ec3 [MM-22368] Log Channel Moderation events (#14042)
* MM-22368 Log Channel Moderation events

* Modify wording

* Use structured fields

* Update app/channel.go
2020-03-19 19:31:18 +05:30
Jesús Espino
c66e182b08 Adding the new search engine abstraction (#13304)
* WIP

* Adding bleve to go modules

* WIP

* Adding missing files from searchengine implementation

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* User and channel indexing and searches implemented

* Make bleve tests run with in-memory indexes

* Implement post index and deletion tests

* Initial commits for the search layer

* Removing unnecesary indexing

* WIP

* WIP

* More fixes for tests

* Adding the search layer

* Finishing the migration of searchers to the layer

* Removing unnecesary code

* Allowing multiple engines active at the same time

* WIP

* Add simple post search

* Print information when using bleve

* Adding some debugging to understand better how the searches are working

* Making more dynamic config of search engines

* Add post search basics

* Adding the Purge API endpoint

* Fixing bleve config updates

* Adding missed file

* Regenerating search engine mocks

* Adding missed v5 to modules imports

* fixing i18n

* Fixing some test around search engine

* Removing all bleve traces

* Cleaning up the vendors directory and go.mod/go.sum files

* Regenerating timer layer

* Adding properly the license

* Fixing govet shadow error

* Fixing some tests

* Fixing TestSearchPostsFromUser

* Fixing another test

* Fixing more tests

* Fixing more tests

* Removing SearchEngine redundant text from searchengine module code

* Fixing some reindexing problems in members updates

* Fixing tests

* Addressing PR comments

* Reverting go.mod and go.sum

* Addressing PR comments

* Fixing tests compilation

* Fixing govet

* Adding search engine stop method

* Being more explicit on where we use includeDeleted

* Adding GetSqlSupplier test helper method

* Mocking elasticsearch start function

* Fixing tests

Co-authored-by: Miguel de la Cruz <miguel@mcrx.me>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-13 15:33:18 +01:00
Agniva De Sarker
c8b60c2d75 MM-22783: Fix scopelint issues (#13969)
* Fix scopelint issues

* Incorporating review comments

* Keeping the order of linters alphabetical

* Fix issues after merge

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-13 08:44:39 +05:30
Hossein Ahmadian-Yazdi
34668511cf fix read only channel when join channel (#13946)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-06 15:58:14 -05:00
Farhan Munshi
79c786bc0c [MM-22206] Add PATCH channel moderations (PUT /moderations/patch) (#13845)
* MM-22205 Add get channel moderations endpoint

* MM-22206 Add patch channel moderations endpoint

Add api tests for patch channel moderations

* MM-22205 Ensure ordered permissions returned and create struct ChannelModeratedRoles

* MM-22206 Use structs instead of map

* MM-22206 Add test cases for GetChannelModeratedPermissions

* MM-22206 Add tests for ChannelModeratedPermissionsChangedByPatch

* MM-22206 Use NewBool instead of defining booleans

* MM-22206 Tie Channel Mentions to Create Posts when building Channel Moderations

* Revert "MM-22206 Tie Channel Mentions to Create Posts when building Channel Moderations"

This reverts commit a0bfc95f1732955c5ef5fc3e4b05ea8ab954acea.

* MM-22206 Review changes

Modify GetSchemeRolesForChannel to return named variables
Change calls to SessionHasPermissionToChannel to SessionHasPermissionTo
Add a CreateChannelScheme method
Add a DeleteChannelScheme method
Move GetChannelModeratedPermissions to Role model

* Fix lint

* Add ChannelModeration methods to App interface

* MM-22206 Rename method to GetTeamSchemeChannelRoles

* MM-22206 Check CHANNEL_MODERATED_PERMISSIONS_MAP for existing permission before iterating through it

* Modify wording to higherScoped to match #13813

* MM-22206 Delete channel scheme between tests

* MM-22206 Fix tests

* Actually patch role

* MM-22206 Shadow declaration of err
2020-03-05 10:04:34 -05:00
Jesús Espino
54bb6d10f3 Making InvalidateCacheForUser public again (#13997) 2020-03-04 16:16:27 +01:00