The code was writing to master and immediately after that reading
from a replica causing it to fail intermittently.
Since this is not a very high-traffic table, it should be safe to read
from master always.
* MM-24135: Migrate AppError from SaveChannel/channel_store.go
This is the first POC of migration of store app errors to plain error.
We create a few basic error types in the store package and use
them to return the errors from store methods. In the app layer,
we inspect the error and re-create the exact app errors. This lets
us preserve the same error content, but yet move to plain errors.
Since this is a gradual migration, this means that the error inspection
code will be duplicated across the app layer whenever a store method
is invoked. But all of that should go away once we start propagating
the errors higher up the hierarchy.
There have been a significant amount of changes in the storetest and searchtest
layer, primarily because we have to rename the err variable now that it is of
a different type.
* Addressed review comments
* Made all appError origins to be CreateChannel
* Remove typed internal error
* Fix translations
* fix layer generation
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
This test writes directly to a connection
which causes panics and more frustration in an already fragile CI.
Since this anyways checks an edge condition, and will anyways be
removed in v6, let's remove this for now and let CI be happy.
After registering the conn in the hub, we proceeded to send a
direct message to the user. We had changed it to send the direct message
in the same hub goroutine that handles the registration. This was the correct
behavior and fixes chances of having panics due to sending to closed channels.
However, often fixing something unearths some deeper underlying bug. This was
such a case :)
The issue was that register channel had a buffer size of 1. And we were sending
a direct message after registration. In the code to send direct message, we
were checking if the user has been registered or not, and if not, then skip it.
Therefore, since the register channel buffer was 1, it could very well be that
the select case would pick up the direct message send case first - in which
case it would not have been registered, and therefore no hello message would be sent.
The fix is to unbuffer the register and unregister channels. There does not seem
to be a valid reason to make these buffered channels. They are meant to be
synchronous operations, because the code following them assumes that the user
has been registered.
While here, we also remove all the time.Sleeps before waiting on the Response channel
because they are not required at all. Waiting on a channel is already blocking.
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
* add a since parameter to getGroups api
* update for lint error
* when using since, return deleted groups as well.
* update flaky test, groups have same create time
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* add getGroupsByUserId to API layer
* update for lint errors
* add check for contextId = userId or ManageSystem Permission
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-23935 extend session expiry on user activity
- if user types anything before a session expires the session will be extended to now + session length
- ensures new session expiries are not written to DB too frequently
- new session store func for updating session ExpiresAt
- session length defaults for mobile and web/ldap changed from 180 days to 30 days
This corrects an issue found when running multiple clustered
Mattermost servers and using the optional `get_server_status`
check on the ping API endpoint. This is done by allowing each
server to write and read from its own health check key in the
system table.
* MM-23264 Add api endpoint for get groups with members in channel
Add store tests
Add tests for api func
Gofmt
Apply changes from code review
* MM-23264 Make store layers
* MM-23264 Check read permission on channel member counts
* Trigger CI
* Remove unnecessary check for PERMISSION_LIST_TEAM_CHANNELS
In the autocompleteChannelsForTeamForSearch method we're checking for
the PERMISSION_LIST_TEAM_CHANNELS permission in order to avoid filtering
channels in the autocomplete search but this check is not necessary.
Now we're going directly to the database to search for those channels
in this specific method and we're filtering by channel membership and
team so there is no chance that we are going to filter undesired
channels to the user.
[Here](https://github.com/mattermost/mattermost-server/blob/v5.22.0/store/sqlstore/channel_store.go#L2014)
is the query where you can see the filtering we're making
Add auditing to server CLI.
Also:
- simplify auditing in API layer
- reduce number of AddMeta calls
- have models serialize themselves
- more consistent field naming
Filter blacklisted Unicode characters from:
user: first name, last name, nickname, bot description, username
team: name, display name, description, company name
channel: name, display name
* MM-23770: Fix for blank DefaultChannelGuestRole on team schemes.
* MM-23770: Adds test for an team scheme with a blank DefaultChannelGuestRole field.
* MM-23770: Fix for unexpected Schemes.Get.
* MM-23484 - add simple validation to GetUsersStatusesByIds
* MM-23484 - return early in validation, add extra test case
* MM-23484 - add test case for empty array
Co-authored-by: Fedor Vitkovskiy <fedorvitkovskiy@pop-os.localdomain>
* WIP
* Adding bleve to go modules
* WIP
* Adding missing files from searchengine implementation
* WIP
* WIP
* WIP
* WIP
* WIP
* WIP
* User and channel indexing and searches implemented
* Make bleve tests run with in-memory indexes
* Implement post index and deletion tests
* Initial commits for the search layer
* Removing unnecesary indexing
* WIP
* WIP
* More fixes for tests
* Adding the search layer
* Finishing the migration of searchers to the layer
* Removing unnecesary code
* Allowing multiple engines active at the same time
* WIP
* Add simple post search
* Print information when using bleve
* Adding some debugging to understand better how the searches are working
* Making more dynamic config of search engines
* Add post search basics
* Adding the Purge API endpoint
* Fixing bleve config updates
* Adding missed file
* Regenerating search engine mocks
* Adding missed v5 to modules imports
* fixing i18n
* Fixing some test around search engine
* Removing all bleve traces
* Cleaning up the vendors directory and go.mod/go.sum files
* Regenerating timer layer
* Adding properly the license
* Fixing govet shadow error
* Fixing some tests
* Fixing TestSearchPostsFromUser
* Fixing another test
* Fixing more tests
* Fixing more tests
* Removing SearchEngine redundant text from searchengine module code
* Fixing some reindexing problems in members updates
* Fixing tests
* Addressing PR comments
* Reverting go.mod and go.sum
* Addressing PR comments
* Fixing tests compilation
* Fixing govet
* Adding search engine stop method
* Being more explicit on where we use includeDeleted
* Adding GetSqlSupplier test helper method
* Mocking elasticsearch start function
* Fixing tests
* Search tests
* Fix tests
* Fix mod
* Fixing searchEngine for test helpers with store mocks
* Remove loglines
* Fix i18n strings
* Migrate search posts tests
* Fix linter
* Do not run search tests if -short flag is enabled
* Migrate back store tests that didn't belong to the searchlayer
* Fix scopelint issues
Co-authored-by: Jesús Espino <jespinog@gmail.com>
* MM-22212: Read non-moderated permissions from higher-scoped scheme.
* MM-2212: Corrects test count in comment.
* MM-22212: Adds godoc comment.
* MM-2212: Switches to the channel roles check in a few more places.
* MM-22212: Refactors and fixes.
* MM-22212: Reverts change, no longer required.
* MM-22212: Removes translation.
* MM-22212: Un-comments merged new permission.
* MM-22212: Un-comments merged new permission.
* MM-22212: Performance tweak.
* MM-22212: Fixes some fmting.
* MM-22212: Add unit test for newly-added store methods.
* MM-22212: Renames app method.
* MM-22212: Re-uses existing function to find string in slice.
* MM-22212: Keeps 'higher-scoped' terminology for consistency.
* MM-22212: Refactors based on PR feedback.
* MM-22212: Fix for some bad merging.
* MM-22212: Renamed some things.
* MM-22212: Use an 'else' instead of a 'continue' for readability.
* MM-22212: Caches (*SqlRoleStore).ChannelRolesUnderTeamRole.
* MM-22212: Adds mock to new cache store.
* MM-22212: Adds missing open tracing app layer methods.
* MM-22212: Adds migration to add moderated permissions to channel_admin if present on channel_user.
* MM-22212: Migrates team schemes. Removes unused AppError.
* MM-22212: Fix for for if.
* MM-22212: Fixes iterator.
* MM-22212: Updates open tracing generated methods.
* MM-22212: Fix mocks.
* MM-22212: Change migration key name.
* MM-22212: Switched to data structure from other branch.
* MM-22212: Fixes tests after adding 'use_channel_mentions' to the channel_admin role.
* MM-22212: Adds tracking of channel moderation.
* Revert "MM-22212: Adds tracking of channel moderation."
This reverts commit 23689efa22c112e4ba37f6a212535dd7ebfb63db.
* MM-22212: Switch some functions to methods and vice versa.
* MM-22212: Fix for refactor bug not notifiying websocket about changed role.
* MM-22212: Adds test for public/private 'manage_members' handling.
* MM-22122 Fix manage channel members edge case for public and private channels (#14049)
* MM-22212: Adds moderated permission to team_admin.
* MM-22212: Updates migration.
* MM-22212: Revert unnecessary update to default roles.
* Add channel scheme updated event when channel scheme is deleted or created (#14057)
* MM-22212: Adds newline.
* MM-22212: Migration fix.
* MM-22212: Fix for migration.
* MM-22212: Test fix.
Co-authored-by: Farhan Munshi <3207297+fm2munsh@users.noreply.github.com>
* Remove unncessary recover() calls
A recover which is not called inside a defer will always return nil.
So there's no use of calling it.
And even if we did call it inside a defer, recovering and immediately
panicking does not make sense either. So we just remove it.
* Fixed another instance
* New auditing API outputting to syslog via TLS
* New config section for specifying remote syslog server IP, port, and cert.
* Legacy audit API retained for access history feature
There is a race at the DB level where the goroutine c.App.SyncRolesAndMembership
would set the the SchemeAdmin attribute to false. If this happens after
UpdateUserToTeamAdmin happens and before the Login, then the user does not
have the required permissions to unlink the group and it fails.
Thanks to @streamer45 for spotting the issue.
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-20913 add client API to fetch server busy state and timeout
* deprecate current API that only returns timeout
* required for mmctl MM-20913
* restore deleted unit test for GetServerBusyExpires
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-22205 Add get channel moderations endpoint
* MM-22206 Add patch channel moderations endpoint
Add api tests for patch channel moderations
* MM-22205 Ensure ordered permissions returned and create struct ChannelModeratedRoles
* MM-22206 Use structs instead of map
* MM-22206 Add test cases for GetChannelModeratedPermissions
* MM-22206 Add tests for ChannelModeratedPermissionsChangedByPatch
* MM-22206 Use NewBool instead of defining booleans
* MM-22206 Tie Channel Mentions to Create Posts when building Channel Moderations
* Revert "MM-22206 Tie Channel Mentions to Create Posts when building Channel Moderations"
This reverts commit a0bfc95f1732955c5ef5fc3e4b05ea8ab954acea.
* MM-22206 Review changes
Modify GetSchemeRolesForChannel to return named variables
Change calls to SessionHasPermissionToChannel to SessionHasPermissionTo
Add a CreateChannelScheme method
Add a DeleteChannelScheme method
Move GetChannelModeratedPermissions to Role model
* Fix lint
* Add ChannelModeration methods to App interface
* MM-22206 Rename method to GetTeamSchemeChannelRoles
* MM-22206 Check CHANNEL_MODERATED_PERMISSIONS_MAP for existing permission before iterating through it
* Modify wording to higherScoped to match #13813
* MM-22206 Delete channel scheme between tests
* MM-22206 Fix tests
* Actually patch role
* MM-22206 Shadow declaration of err
* Summary
Fixing inconsistencies of the app_error messages. Now all end with a
period and there is the same user experience all over the system.
We agreed on this change on the community server a while ago and I just
came around finally doing it.
Ticket Link
N/A
Checklist
Includes text changes and localization file (.../i18n/en.json and
.../webapp/i18n/en.json) updates
* Update i18n/en.json
Approving changes
Co-Authored-By: Jason Blais <13119842+jasonblais@users.noreply.github.com>
* Update i18n/en.json
Approving changes
Co-Authored-By: Jason Blais <13119842+jasonblais@users.noreply.github.com>
* Update i18n/en.json
Approving changes
Co-Authored-By: Jason Blais <13119842+jasonblais@users.noreply.github.com>
* Fix test failures due string differences
Co-authored-by: Jason Blais <13119842+jasonblais@users.noreply.github.com>
Co-authored-by: Miguel de la Cruz <mgdelacroix@gmail.com>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* Cache test users to speed up testing
Users were recreated for many tests and hashing passwords takes a lot of
time. Testing /api4 takes around 120s less and /app 60s less.
* fix build
* simplifying a bit the approach
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Jesús Espino <jespinog@gmail.com>
* MM-22282 Add Ephemeral response when using mentions without permission and add new prop to disable mention highlights on client
* MM-22622 Make test name test actually what it does and fix comment style
* MM-22622 Check ephemeral post created when post create with mentions on API
* MM-22622 More tests for App>CreatePost
* MM-22622 Make DisableMentionHighlights more concise and rename ephemeral post
* MM-22622 Dont send ephemeral message for system message created by user
* Trigger build
I could not figure out the actual root cause for this.
But a possible area of concern is that somewhere in the call stack,
the login failed partially. Therefore, the session did not have the teamMembers
populated. And the permission check failed therefore.
Adding the error checks should let us know in future if this is not the case.
Another possible case is the app.ClearTeamMembersCache which gets called in
the goroutine app.SyncRolesAndMembership. If this races with the session,
then it's possible that the teamMember is wiped off from the session. But this
happens after the permission check. So it's not very likely.
Also, while we are here, I found that (*app).ClearTeamMembersCache
does not log the internal error bubbled up. This prevents us from understanding
what actually happened. Added that logging.
* Introducing unit (not integration) tests for the app layer
* Initial support for unit tests at the API
* Adding unit tests support to the store layer
* Add unit tests support in commands
* Adding last tests needed for run unit tests properly
* Fixing govet
* Removing some duplication
* Fixing tests
* Fixing tests
* Not compiling test helpers with the main module for api
* Revert "Not compiling test helpers with the main module for api"
This reverts commit 36a199bbe0f7503f665f5d6c7b41c53aabc2e54f.
* Fixing tests
* Fixing unit tests
* More consistency between api4/apiteslib.go and app/helper_test.go
* Renaming things to make more obvious the new Setup functions purpose
* Reverting change in go.sum
* Start with empty mock for app layer
* Start with empty mock for api layer
* Start with empty mock for web layer
* Renaming SetupWithStoreMockConfig to SetupConfigWithStoreMock
* Fixing tests on web package
* Removing unnecesary function
