* MM-23935 extend session expiry on user activity
- if user types anything before a session expires the session will be extended to now + session length
- ensures new session expiries are not written to DB too frequently
- new session store func for updating session ExpiresAt
- session length defaults for mobile and web/ldap changed from 180 days to 30 days
* MM-23264 Add api endpoint for get groups with members in channel
Add store tests
Add tests for api func
Gofmt
Apply changes from code review
* MM-23264 Make store layers
* MM-23264 Check read permission on channel member counts
* Trigger CI
* MM-23017 Check group mentions as part of notification logic
* Add nil groups to existing test cases
* MM-23017 Add tests for insertGroupMention and addGroupMention
* MM-23017 Add tests for getExplicitMentions that have groups
* Add tests for group store GetMemberUsersNotInChannel
* MM-23017 Add tests for AllowGroupMentions
* MM-23017 Fix error message name
* MM-23017 Swap Checks to Name
* MM-23017 Code review fixes
* Rename var and fix allowGroupMentions test
* MM-23017 Use GetMemberUsersInTeam inside of insertGroupMentions
* MM-23017 use group mentions permission
* Actually call GetMemberUsersInTeam
* Remove unnecessary new line
* Uncomment filter allow reference
* MM-23017 Fix group channel notifications
* Update store layer
* MM-23017 Improve test coverage for group channels
* Trigger CI
* Trigger CI
* Add database server version to telemetry
Also added a new query in the store to retrieve the database version
* Add test for the GetDbVersion function
* More drivers in the tests
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* store/cache: add cache to user layer
* store/localcache: fix cache data access
* app/login: add invalidate for user
* store/user: move user sanitization to cache
* store/user: remove sanitize checks form the tests
* store/user: remove sanitization from store and cache layer
* store/cache: remove unnecessary error wraps
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* WIP
* Adding bleve to go modules
* WIP
* Adding missing files from searchengine implementation
* WIP
* WIP
* WIP
* WIP
* WIP
* WIP
* User and channel indexing and searches implemented
* Make bleve tests run with in-memory indexes
* Implement post index and deletion tests
* Initial commits for the search layer
* Removing unnecesary indexing
* WIP
* WIP
* More fixes for tests
* Adding the search layer
* Finishing the migration of searchers to the layer
* Removing unnecesary code
* Allowing multiple engines active at the same time
* WIP
* Add simple post search
* Print information when using bleve
* Adding some debugging to understand better how the searches are working
* Making more dynamic config of search engines
* Add post search basics
* Adding the Purge API endpoint
* Fixing bleve config updates
* Adding missed file
* Regenerating search engine mocks
* Adding missed v5 to modules imports
* fixing i18n
* Fixing some test around search engine
* Removing all bleve traces
* Cleaning up the vendors directory and go.mod/go.sum files
* Regenerating timer layer
* Adding properly the license
* Fixing govet shadow error
* Fixing some tests
* Fixing TestSearchPostsFromUser
* Fixing another test
* Fixing more tests
* Fixing more tests
* Removing SearchEngine redundant text from searchengine module code
* Fixing some reindexing problems in members updates
* Fixing tests
* Addressing PR comments
* Reverting go.mod and go.sum
* Addressing PR comments
* Fixing tests compilation
* Fixing govet
* Adding search engine stop method
* Being more explicit on where we use includeDeleted
* Adding GetSqlSupplier test helper method
* Mocking elasticsearch start function
* Fixing tests
* Search tests
* Fix tests
* Fix mod
* Fixing searchEngine for test helpers with store mocks
* Remove loglines
* Fix i18n strings
* Migrate search posts tests
* Fix linter
* Do not run search tests if -short flag is enabled
* Migrate back store tests that didn't belong to the searchlayer
* Fix scopelint issues
Co-authored-by: Jesús Espino <jespinog@gmail.com>
* Members Save Multiple
* Multiple Channel Members implemented
* Working version with member multi save and multi update
* Adding tests to the save multiple team members store methods
* Tests passing for SaveMember
* Adding remove members tests
* Working on channel store tests
* More work on testing properly SaveMember and SaveMultipleMembers on channels
* More work on testing properly SaveMember and SaveMultipleMembers on teams
* Adding tests for update multiple members for team
* Adding tests for update multiple members for channel
* Fixing some tets
* Fixing tests
* Fixing tests
* Fixing i18n
* Addressing govet problems
* Addressing govet problems
* Addressing govet problems
* Addressing govet problems
* Addressing PR comments (and removing TODO)
* Addressing PR comments
* Fixed broken test
* Removing unnecesary i18n translation string
* MM-22212: Read non-moderated permissions from higher-scoped scheme.
* MM-2212: Corrects test count in comment.
* MM-22212: Adds godoc comment.
* MM-2212: Switches to the channel roles check in a few more places.
* MM-22212: Refactors and fixes.
* MM-22212: Reverts change, no longer required.
* MM-22212: Removes translation.
* MM-22212: Un-comments merged new permission.
* MM-22212: Un-comments merged new permission.
* MM-22212: Performance tweak.
* MM-22212: Fixes some fmting.
* MM-22212: Add unit test for newly-added store methods.
* MM-22212: Renames app method.
* MM-22212: Re-uses existing function to find string in slice.
* MM-22212: Keeps 'higher-scoped' terminology for consistency.
* MM-22212: Refactors based on PR feedback.
* MM-22212: Fix for some bad merging.
* MM-22212: Renamed some things.
* MM-22212: Use an 'else' instead of a 'continue' for readability.
* MM-22212: Caches (*SqlRoleStore).ChannelRolesUnderTeamRole.
* MM-22212: Adds mock to new cache store.
* MM-22212: Adds missing open tracing app layer methods.
* MM-22212: Adds migration to add moderated permissions to channel_admin if present on channel_user.
* MM-22212: Migrates team schemes. Removes unused AppError.
* MM-22212: Fix for for if.
* MM-22212: Fixes iterator.
* MM-22212: Updates open tracing generated methods.
* MM-22212: Fix mocks.
* MM-22212: Change migration key name.
* MM-22212: Switched to data structure from other branch.
* MM-22212: Fixes tests after adding 'use_channel_mentions' to the channel_admin role.
* MM-22212: Adds tracking of channel moderation.
* Revert "MM-22212: Adds tracking of channel moderation."
This reverts commit 23689efa22c112e4ba37f6a212535dd7ebfb63db.
* MM-22212: Switch some functions to methods and vice versa.
* MM-22212: Fix for refactor bug not notifiying websocket about changed role.
* MM-22212: Adds test for public/private 'manage_members' handling.
* MM-22122 Fix manage channel members edge case for public and private channels (#14049)
* MM-22212: Adds moderated permission to team_admin.
* MM-22212: Updates migration.
* MM-22212: Revert unnecessary update to default roles.
* Add channel scheme updated event when channel scheme is deleted or created (#14057)
* MM-22212: Adds newline.
* MM-22212: Migration fix.
* MM-22212: Fix for migration.
* MM-22212: Test fix.
Co-authored-by: Farhan Munshi <3207297+fm2munsh@users.noreply.github.com>
* Optimize reactions table
Change reactions primary key to (PostId, UserId, EmojiName) so fetching
reactions for post will use primary key lookup instead of table scan.
* fix db version
* review fixes
* update database schema in scripts/
* WIP
* Adding bleve to go modules
* WIP
* Adding missing files from searchengine implementation
* WIP
* WIP
* WIP
* WIP
* WIP
* WIP
* User and channel indexing and searches implemented
* Make bleve tests run with in-memory indexes
* Implement post index and deletion tests
* Initial commits for the search layer
* Removing unnecesary indexing
* WIP
* WIP
* More fixes for tests
* Adding the search layer
* Finishing the migration of searchers to the layer
* Removing unnecesary code
* Allowing multiple engines active at the same time
* WIP
* Add simple post search
* Print information when using bleve
* Adding some debugging to understand better how the searches are working
* Making more dynamic config of search engines
* Add post search basics
* Adding the Purge API endpoint
* Fixing bleve config updates
* Adding missed file
* Regenerating search engine mocks
* Adding missed v5 to modules imports
* fixing i18n
* Fixing some test around search engine
* Removing all bleve traces
* Cleaning up the vendors directory and go.mod/go.sum files
* Regenerating timer layer
* Adding properly the license
* Fixing govet shadow error
* Fixing some tests
* Fixing TestSearchPostsFromUser
* Fixing another test
* Fixing more tests
* Fixing more tests
* Removing SearchEngine redundant text from searchengine module code
* Fixing some reindexing problems in members updates
* Fixing tests
* Addressing PR comments
* Reverting go.mod and go.sum
* Addressing PR comments
* Fixing tests compilation
* Fixing govet
* Adding search engine stop method
* Being more explicit on where we use includeDeleted
* Adding GetSqlSupplier test helper method
* Mocking elasticsearch start function
* Fixing tests
Co-authored-by: Miguel de la Cruz <miguel@mcrx.me>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* Adding SaveMultiple to posts
* Improving tests
* fixing i18n
* Fixing tests
* Improving testing on top of Save and SaveMultiple
* Fixing shadow variables
* Addressing some PR comments
* More clear update post test
* Addressing some PR comments
* Addressing some PR comments and simplifying the code
* Improting replies in bulk too
* Fixing reply count and processing last imported replies
* Adding OverwriteMultiple to posts aggregating everything in the same transaction
* Adding 2 pending tests to implement
* Adding tests for overwrite multiple posts
* Adding tests for TeamStore.GetByNames method
* Fixing shadow variables
* Addressing PR comments
* Extracting i18n strings
* Fixing tests
* Fixing tests
* Adding more test cases
* Using a variable instead of a fake timestamp
* allow ExpireInSeconds < 0
Allow `ExpireInSeconds < 0` for use with `KVSetWithOptions`. While this has no practical use in reality, it's much easier to thoroughly unit tests the underlying functionality if we can match the semantics of CompareAndSet. Strictly speaking, this is a breaking change, but not relative to the advertised semantics. Anyway, it's also not entirely unreasonable to treat a negative `ExpireInSeconds` as having already expired vs. marking it as never expired.
* updated tests, to break apart
* honour expiry in CompareAndSet
* honour expiry in CompareAndDelete
* honour expiry in List
* fail unique constraint exception for SaveOrUpdate
A unique constraint error on a `SaveOrUpdate` should not be ignored: we did not save or update the requested value, as someone else managed to write the record first.
Note this is handled differently in `CompareAndSet`, where we correctly swallow the error and return `false` to indicate we did not successfully save the value.
* unexport DEFAULT_PLUGIN_KEY_FETCH_LIMIT
* s/InternalServerError/BadRequest/ for failed SaveOrUpdate
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* constants and options for getting files
* Method to get files with options
* Add i18n strings for en
* Add API methods for getting files with options
* gofmt -s file
* explicitly set create at in tests
* use greater than nanosecond time difference for tests
* use gte instead of gt for getting files by created time
* use created at time as default sort order for getting file infos
* use explicit inline strings instead of format strings
* join tables only when required
* use if as secondary sort, and update tests
* update field docs to reflect previous changes
* make page and perPage get options as required
* add json struct tags to GetFileOptions
* bump minimum server versioni
* remove sorting by username and channelname
* use bool for sort order type
* use FileInfo prefix instead of just File
* clearer comments
* use zero-based page numbering
* test filtering by user and channel
* remove unnecessary whitespace
* use int instead of uint for page and perPage arguments
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-21946 Ensure deleted groups are not returned from the groups API
* MM-21946 Only append to query at the end
* MM-21946 Add DeleteAt check to the top of the GetGroups function and clean up the formatting a bit
* MM-21946 Move the From statement into the next block
* MM-20644: Add users to teams as a SchemeAdmin based on a new configuration field on GroupTeams and GroupChannels records.
* MM-20644: Adds SchemeAdmin to mapping of the GroupSyncable struct fields.
* MM-2064: Adds test to ensure SchemeAdmin field value is mapped.
* MM-20644: Adds missing index creation for fresh DBs.
* MM-20644: Duplicates UpdateMembersRole across Team and Channel stores. Adds tests.
* MM-20644: Fixes some old method name references.
* MM-20644: Moves variable declaration; removes Println statement.
* MM-21157: Triggers role sync when a syncable is linked, unlinked, or updated.
* MM-20644: Use a SQL query instead of two to update Team and Channel members.
* MM-20644: Fixes tests; updates query.
* MM-21157: Removes second invocation of function because of refactor that performs it all in a single query.
* MM-21157: Switches a few queries to Squirrel.
* MM-21157: SQL-formats some strings.
* MM-21157: Select with list.
* MM-21157: Converts some more sql to squirrel.
* MM-21157: Fix incorrect conflict resolutions.
* MM-21157: Fix incorrect conflict resolutions.
* MM-21157: Adds missing mocks.
* MM-21157: Clears cache upon syncing roles.
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
`KVCompareAndSet(key, sameValue, sameValue)` can fail spuriously on MySQL if the underlying `UPDATE` requires no actual changes. As per the [MySQL documentation](https://dev.mysql.com/doc/refman/8.0/en/mysql-affected-rows.html), we can't rely on rows affected in this case:
> For UPDATE statements, the affected-rows value by default is the number of rows actually changed. If you specify the CLIENT_FOUND_ROWS flag to mysql_real_connect() when connecting to mysqld, the affected-rows value is the number of rows “found”; that is, matched by the WHERE clause.
It's not tenable to change `CLIENT_FOUND_ROWS` for the all connection, so handle this case in the code instead by running a `SELECT` after the fact. Note that `KVCompareAndSet` has no guarantee of atomicity in this case, but neither would `CompareAndSwap` on which this is method was inspired.
Finally, note that no changes are required for Postgres, which has sane semantics as the default.
Fixes: https://mattermost.atlassian.net/browse/MM-21328
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-20644: Add users to teams as a SchemeAdmin based on a new configuration field on GroupTeams and GroupChannels records.
* MM-20644: Adds SchemeAdmin to mapping of the GroupSyncable struct fields.
* MM-2064: Adds test to ensure SchemeAdmin field value is mapped.
* MM-20644: Adds missing index creation for fresh DBs.
* MM-20644: Duplicates UpdateMembersRole across Team and Channel stores. Adds tests.
* MM-20644: Fixes some old method name references.
* MM-20644: Moves variable declaration; removes Println statement.
* MM-20644: Use a SQL query instead of two to update Team and Channel members.
* MM-20644: Fixes tests; updates query.
* MM-20644: Fix permission check for patching a group syncable.
* MM-20644: Fixes test for change of permissions verification in group patch API request.
* MM-20644: Fix for ORM select vs insert.
* MM-20644: Linting fixes.
* MM-20644: Fixes some tests.
* MM-20644: Skips changing the role of guests.
* MM-20644: Added syncableID filtering
* Ensure both DeleteAt and UpdateAt are set to same value
* fix another point this could happen
* Revert "fix another point this could happen"
This reverts commit 5deb102e782bcc1da103b1b10715c066d9cc6130.
* Refactor to use structured logging
* Properly formatted with gofmt
* created interface Cache, but construction of cache is still coupled.
* Implementing cache factories to build caches
* Simple redis implementation without error handling. Keys and values by default are string
* refactor NewLocalCacheLayer to inject cache factory
* Removed redis impl to focus on cache abstraction
* CacheFactory injected on sqlsupplier and saved in Store struct
* remove useless private method
* replace concrete declaration of lru cache to cache abstraction
* discard spaces
* Renamed factory to provider because concrete implementations of factories may hold an state (such as a redis client for example)
* refactor to include all caches in the same package cache and subpackages
* method close cache. This method will be used for concrete implementations that need to release resources (close a connection, etc)
* closing cacheprovider and releasing resources while closing sql store
* fixed merge conflict fail
* gofmt files
* remove unused property from post_store
* naming refactor to avoid stutter. Added godocs on interface
* Store doesnt know anything about the cache and provider. Cache provider will be built after loading config and injected in localCacheLayer
* fixed broken test
* cache provider initialized before RunOldAppInitialization which initializes the localcachelayer
* move statusCache to server to initialize it with the new cache provider
* update terms_service and channel_layer to have new cacheProvider
* gofmt
* Add Connect method to the cache provider
* mock cacheprovider in user_layer_test
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-21209: Use the LRU cache for UserStore.Get call too
We already have userProfileByIdsCache to store the user profiles by Id.
It just wasn't being used for the (*UserStore).Get method. We add a wrapper
method in LocalCacheUserStore to intercept that call and check for the
presence of the user Id in the cache.
There is no need to add any code for invalidation as all of that is already
present.
* Fix nil check for rootstore
* Fix TestUserStore test
Added an invalidate call. The invalidation was being done
from the app level. Hence we have to do it manually here for the test.
