* Fixing reply count on new posts
* Fixing tests
* Fixing post reply count on getPostsAround
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-25040: Only return team-associated groups if the team is group-constrained.
MM-25040: Prevents associating a group to a channel if the team doesn't have the group first.
* MM-25040: Fix lints.
* MM-25040: Still add the groupteam if the team is not group-constrained.
* MM-25040: Wraps groupteam upsert in else branch for efficiency.
* MM-25040: Removes unnecessary page iteration.
* MM-25040: Fix typo.
* MM-25040: Moves filtering to SQL.
* MM-25040: Updates tests, check pagination.
* MM-25040: Fix lint error.
* MM-25040: Adds some more group store tests.
* MM-25040: Fix for wrong test parameter.
* Fixing behavior of Replicas and SearchReplicas in canary environments
* Trying to fix tests
* Revert "Trying to fix tests"
This reverts commit 3531da961844f5cb8557efcd3b570b06f362c6df.
* Revert "Fixing behavior of Replicas and SearchReplicas in canary environments"
This reverts commit 0c05901c843e4ccd60c8320fb4e0123b1bacf430.
* Revert "Disable read/search db replicas in TE/E0 (#14400)"
This reverts commit ef5ac519d9.
* Making the store aware of the license
* Readding the unit tests
* Fixing sqlstor supplier tests
* Adding mutex to ensure license write consistency and fixing tests
* Fixing tests
* Fixing tests
* Shuting down server properly during tests
* Trying to fix tests
* Trying to fix the tests
* Skipping flaky tests
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* make Group.Name a pointer to allow null
* fix unit tests
* fix build error
* fix unit test
* ensure Name field not nil
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
We use the same optimization used in MM-23369 to prevent
mysql from using the index in the sort query.
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-24135: Migrate AppError from SaveChannel/channel_store.go
This is the first POC of migration of store app errors to plain error.
We create a few basic error types in the store package and use
them to return the errors from store methods. In the app layer,
we inspect the error and re-create the exact app errors. This lets
us preserve the same error content, but yet move to plain errors.
Since this is a gradual migration, this means that the error inspection
code will be duplicated across the app layer whenever a store method
is invoked. But all of that should go away once we start propagating
the errors higher up the hierarchy.
There have been a significant amount of changes in the storetest and searchtest
layer, primarily because we have to rename the err variable now that it is of
a different type.
* Addressed review comments
* MM-24132: Migrate AppError from SaveDirectChannel/channel_store.go
This PR migrates 2 new methods SaveDirectChannel and CreateDirectChannel
to return error instead of AppError.
We also need to handle the error internally in SaveMultipleMember for now
until that is migrated too.
* MM-24133: Migrate AppError from bot_store.go
* Fix errors
* Fix err
* Fix bad return
* Fix vet errors
* Fix incorrect error check
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
The code was writing to master and immediately after that reading
from a replica causing it to fail intermittently.
Since this is not a very high-traffic table, it should be safe to read
from master always.
* MM-24135: Migrate AppError from SaveChannel/channel_store.go
This is the first POC of migration of store app errors to plain error.
We create a few basic error types in the store package and use
them to return the errors from store methods. In the app layer,
we inspect the error and re-create the exact app errors. This lets
us preserve the same error content, but yet move to plain errors.
Since this is a gradual migration, this means that the error inspection
code will be duplicated across the app layer whenever a store method
is invoked. But all of that should go away once we start propagating
the errors higher up the hierarchy.
There have been a significant amount of changes in the storetest and searchtest
layer, primarily because we have to rename the err variable now that it is of
a different type.
* Addressed review comments
* MM-24132: Migrate AppError from SaveDirectChannel/channel_store.go
This PR migrates 2 new methods SaveDirectChannel and CreateDirectChannel
to return error instead of AppError.
We also need to handle the error internally in SaveMultipleMember for now
until that is migrated too.
* Fix layers
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-24135: Migrate AppError from SaveChannel/channel_store.go
This is the first POC of migration of store app errors to plain error.
We create a few basic error types in the store package and use
them to return the errors from store methods. In the app layer,
we inspect the error and re-create the exact app errors. This lets
us preserve the same error content, but yet move to plain errors.
Since this is a gradual migration, this means that the error inspection
code will be duplicated across the app layer whenever a store method
is invoked. But all of that should go away once we start propagating
the errors higher up the hierarchy.
There have been a significant amount of changes in the storetest and searchtest
layer, primarily because we have to rename the err variable now that it is of
a different type.
* Addressed review comments
* Made all appError origins to be CreateChannel
* Remove typed internal error
* Fix translations
* fix layer generation
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* add a since parameter to getGroups api
* update for lint error
* when using since, return deleted groups as well.
* update flaky test, groups have same create time
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* MM-23935 extend session expiry on user activity
- if user types anything before a session expires the session will be extended to now + session length
- ensures new session expiries are not written to DB too frequently
- new session store func for updating session ExpiresAt
- session length defaults for mobile and web/ldap changed from 180 days to 30 days
Given that this query is part of the top 5 most used queries
we want to move it to use raw queries instead of gorp so we
can get rid of the reflection overhead
* MM-23264 Add api endpoint for get groups with members in channel
Add store tests
Add tests for api func
Gofmt
Apply changes from code review
* MM-23264 Make store layers
* MM-23264 Check read permission on channel member counts
* Trigger CI
* MM-23017 Check group mentions as part of notification logic
* Add nil groups to existing test cases
* MM-23017 Add tests for insertGroupMention and addGroupMention
* MM-23017 Add tests for getExplicitMentions that have groups
* Add tests for group store GetMemberUsersNotInChannel
* MM-23017 Add tests for AllowGroupMentions
* MM-23017 Fix error message name
* MM-23017 Swap Checks to Name
* MM-23017 Code review fixes
* Rename var and fix allowGroupMentions test
* MM-23017 Use GetMemberUsersInTeam inside of insertGroupMentions
* MM-23017 use group mentions permission
* Actually call GetMemberUsersInTeam
* Remove unnecessary new line
* Uncomment filter allow reference
* MM-23017 Fix group channel notifications
* Update store layer
* MM-23017 Improve test coverage for group channels
* Trigger CI
* Trigger CI
* Add database server version to telemetry
Also added a new query in the store to retrieve the database version
* Add test for the GetDbVersion function
* More drivers in the tests
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* store/cache: add cache to user layer
* store/localcache: fix cache data access
* app/login: add invalidate for user
* store/user: move user sanitization to cache
* store/user: remove sanitize checks form the tests
* store/user: remove sanitization from store and cache layer
* store/cache: remove unnecessary error wraps
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
* WIP
* Adding bleve to go modules
* WIP
* Adding missing files from searchengine implementation
* WIP
* WIP
* WIP
* WIP
* WIP
* WIP
* User and channel indexing and searches implemented
* Make bleve tests run with in-memory indexes
* Implement post index and deletion tests
* Initial commits for the search layer
* Removing unnecesary indexing
* WIP
* WIP
* More fixes for tests
* Adding the search layer
* Finishing the migration of searchers to the layer
* Removing unnecesary code
* Allowing multiple engines active at the same time
* WIP
* Add simple post search
* Print information when using bleve
* Adding some debugging to understand better how the searches are working
* Making more dynamic config of search engines
* Add post search basics
* Adding the Purge API endpoint
* Fixing bleve config updates
* Adding missed file
* Regenerating search engine mocks
* Adding missed v5 to modules imports
* fixing i18n
* Fixing some test around search engine
* Removing all bleve traces
* Cleaning up the vendors directory and go.mod/go.sum files
* Regenerating timer layer
* Adding properly the license
* Fixing govet shadow error
* Fixing some tests
* Fixing TestSearchPostsFromUser
* Fixing another test
* Fixing more tests
* Fixing more tests
* Removing SearchEngine redundant text from searchengine module code
* Fixing some reindexing problems in members updates
* Fixing tests
* Addressing PR comments
* Reverting go.mod and go.sum
* Addressing PR comments
* Fixing tests compilation
* Fixing govet
* Adding search engine stop method
* Being more explicit on where we use includeDeleted
* Adding GetSqlSupplier test helper method
* Mocking elasticsearch start function
* Fixing tests
* Search tests
* Fix tests
* Fix mod
* Fixing searchEngine for test helpers with store mocks
* Remove loglines
* Fix i18n strings
* Migrate search posts tests
* Fix linter
* Do not run search tests if -short flag is enabled
* Migrate back store tests that didn't belong to the searchlayer
* Fix scopelint issues
Co-authored-by: Jesús Espino <jespinog@gmail.com>
* Remove goroutines from session_store.go
There isn't much benefit of running these 2 queries concurrently
versus just serially. These queries are not in the hot path anyways.
Maybe just simpler to keep it serial.
* Incorporate review comments
* Members Save Multiple
* Multiple Channel Members implemented
* Working version with member multi save and multi update
* Adding tests to the save multiple team members store methods
* Tests passing for SaveMember
* Adding remove members tests
* Working on channel store tests
* More work on testing properly SaveMember and SaveMultipleMembers on channels
* More work on testing properly SaveMember and SaveMultipleMembers on teams
* Adding tests for update multiple members for team
* Adding tests for update multiple members for channel
* Fixing some tets
* Fixing tests
* Fixing tests
* Fixing i18n
* Addressing govet problems
* Addressing govet problems
* Addressing govet problems
* Addressing govet problems
* Addressing PR comments (and removing TODO)
* Addressing PR comments
* Fixed broken test
* Removing unnecesary i18n translation string
* MM-23369: Allow mysql to choose a better index
When the ORDER BY clause contains a column which is in the WHERE
clause and also part of an index, mysql tries to use that specific
index to avoid sorting. This is inspite of the fact that
there may be other indices which are better for scanning the table
and then doing a sort.
Essentially, mysql becomes dumb and scans a lot of rows to avoid
sorting. Whereas, it could have scanned a lot less rows and do the
sorting in no time.
To fix this, we use the other columns in the ORDER BY clause as well
which are part of the index. This causes no change in the results
because the other columns are an EQUAL condition check, but this
lets mysql use the right index. Because now mysql sees that it has
to order by other columns too, so it better use the other index
to scan and then do the sorting.
This does not affect tables of smaller size because the LIMIT of
rows is always 1. And mysql will stop sorting the moment it gets
the first row. So sorting is not the overhead at all.
Therefore, this seems like an optimal fix.
References:
https://dev.mysql.com/doc/refman/5.7/en/table-scan-avoidance.htmlhttps://code.openark.org/blog/mysql/7-ways-to-convince-mysql-to-use-the-right-indexhttps://dev.mysql.com/doc/refman/5.7/en/limit-optimization.html
* Added a comment to clarify things in code
* Incorporating review comments
