Summary
- Attempt at fixing the AutocompleteInTeam/empty_string test which is flaky on MySQL-5.6. This is likely an index_merge_intersection optimization issue. See https://community.mattermost.com/core/pl/uzgygwa9e3d4pns14ayfs7mobw
for more discussion. The fix here simply modifies the where clause to change the query plan so index_merge_intersection is not used, while also not introducing a temporary table or a filesort.
Ticket Link
- https://mattermost.atlassian.net/browse/MM-26837
* MM-27040: Fix flaky tests due to same UpdateAt timestamp
Update queries are not guaranteed to always update the row.
Since we are only bumping the UpdateAt timestamps, if 2 update
requests hit concurrently within the same millisecond, then one
is bound to fail.
We fix all cases in the codebase where we were updating the UpdateAt
field to not check for count != 1, but rather count > 1.
A similar fix was already done in 3f46cf6f60.
* Remove incorrect test
Even after specifying all the 3 columns to coerce MySQL into choosing
the right multi-column index, it sometimes uses 2 separate indices
and does an index_merge of them.
This creates problems because the DeleteAt=0 search is essentially
the entire Posts table, and causes a disastrously bad performance
than even choosing the wrong index (idx_create_at).
The problem with this approach is that we hardcode the decision
to a specific index when MySQL was free to choose the right index
depending on table statistics. However, there does not appear to be
a case where this index can cause regressions than using some other index.
Another option here was to set optimizer_switch="index_merge_intersection=off"
at a session level for a transaction and then switch it back on after the
query is done.
However, this can cause some unintentional consequences because this setting
is only available at a session level and not at a query level.
There is no need to set something at the session level when an index hint suffices.
https://mattermost.atlassian.net/browse/MM-27575
* Refactor team store to use squirrel
* Fixing or expression
* Refactor team store to use squirrel
* Refactor team store to use squirrel
* Refactor team store to use squirrel
* Refactor team store to use squirrel
* Removing shadow declaration
* Fix error messages
* Improving Update() and Delete() calls
* Update store/sqlstore/team_store.go
Using squirrel to build 'in' clause
Co-authored-by: Miguel de la Cruz <mgdelacroix@gmail.com>
* Add missing translation
Co-authored-by: Dante Pippi <dantepippi@gmai.com>
Co-authored-by: Miguel de la Cruz <mgdelacroix@gmail.com>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
* Add team filters to search teams
Remove unneeded logs
Add team filters to search teams
* Use bool pointers for filters
Re-add include group constrained
Fix lint
Return the union of filters
* Fix direct/group channel false positives
* Move public structures to model package
* Expose CheckIntegrity as a local API method
* Remove extra file
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-26410 Allow moving channels into Favorites when they're favorited in prefs
* MM-26410 Fix management of Favorites category when updating preferences
* MM-26410 Add management of Favorites category when deleting preferences
* Address feedback 1
* Remove WHERE (1=1) from query
* Remove unnecessary sq.Expr
* Rewrite query to use left join
* Remove redundant where statement and add some more tests
* Fix linting issues
* Rename addChannelToFavoritesCategory to addChannelToFavoritesCategory
* MM-27041: Do not run tests against both DBs in CI
We already run the entire test suite for both mysql and postgres in parallel in CI.
So we just run the tests for the current database set.
* Fix incorrect env var
* Migrate command webhook store AppError to error
* Migrate command webhook store AppError to error
* Migrate command webhook store AppError to error
* Migrate command webhook store AppError to error
* Migrate command webhook store AppError to error
* Migrate command webhook store AppError to error
* Migrate command webhook store AppError to error
* Changes requested in the review.
* Changing http status
* fix i18n
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Updating a Group or a GroupMember only changed the UpdateAt or CreateAt times
respectively. And it threw an error if number of rows changed was not 1.
However, it can happen that 2 calls happen so fast that 1 milisecond does not pass,
or even 2 concurrent calls at the same time might happen so that model.GetMillis
return the same timestamp. In those cases, the number of rows updated can be 0.
The error should just check if the number is greater than 1, instead of not equal to 1.
This makes it more robust and correct.
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-26574 Add role filters to user search and get
* Add ability to get filtered user stats
Add support for include bots
* Add tests for user count with filters
Add tests
* Apply changes from code review
* Fix guest filtering
* Fix up tests related to guests
* Clean role names
* Trigger CI
* Trigger CI
* Use squirrel to build query for plugin store
* Typo of PluginkeyValueStore to PluginKeyValueStore
* wrong parameter of queryBuilder in Get method
* Use casting to int for comparison on ExpireAt
* Revert query for CompareAndSet and CompareAndDelete temporarily
* Delete query of expired value
* Update query when oldValue is not nil
* Check count query when there is no row affected
* Delete query on CompareAndDelete
* Put squirrel in separate import group
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
* Use Raw SQL query instead of GORP
As part of performance improvements we're moving the top
used queries to avoid using gorp and get rid of the overhead
caused by the reflection usage
* TokenStore migration to return plain errors
* Fix translations
* Fix: returned error is ignored and http.StatusBadRequest is always returned
* Fix
* Fix translations
* Suggestions
* Changed from BadRequest to NotFound
* Setting the correct http status
* Changed test to expect 404 status
* Fix error
* Remove use of Gorp in ChannelStore.GetAllChannelMembersForUser
In order to improve performance, we're removing gorp calls in the
most used queries in the platform and this is one of them
* MM-26753 Change CreateInitialSidebarCategories to only take a user ID
* MM-26753 Create initial sidebar categories on demand if migration hasn't ran
* Wait for sidebar categories to be loaded in case of replication lag
* MM-26541 parse channel member counts using like instead of using JSON as mysql versions do not support json extract
* Use a constant for default timezone
* Update wording and comments
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* parent 48256721c4 (#14358)
author Eli Yukelzon <reflog@gmail.com> 1585814774 +0300
committer Eli Yukelzon <reflog@gmail.com> 1589111022 +0300
Sidebar caregories implemented
Apply suggestions from code review
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Update store/sqlstore/channel_store.go
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
Update store/sqlstore/channel_store.go
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
code review suggestions
status messages
edge case
bugs...
timeout reverse
* MM-25126 Add a separate default sorting method for categories (#14575)
* MM-25158 Add user to initial sidebar categories when joining team (#14570)
* MM-25281 Place new categories in the correct position (#14609)
* MM-25277 Return channels that aren't in a category as part of the Channels/Direct Messages categories (#14601)
* MM-25276 Remove categories when leaving a team (#14600)
* Remove categories when leaving a team
* layers
* corrected cleanup function
* lint
* .
* corrected errors in postgres
* .
* MM-25280 Ensure that the "update category order" API call only contains real category IDs and isn't missing any IDs (#14626)
* Ensure that the "update category order" API call only contains real category IDs and isn't missing any IDs
* tests
* correct status code
* MM-25278 Change "update category" API to return 400 when changing unsupported fields (#14599)
* MM-25279 Change category migration to only populate channels in Favorites (#14627)
* MM-25157 Add API to delete custom categories (#14574)
* MM-25157 Add API to delete custom categories
* get categories fix
* maxorder fix
* Use correct websocket event when deleting category
* Fix tests and remove debug code
* Actually use the right websocket event this time
* test cleanup
* Update test for new category order
Co-authored-by: Eli Yukelzon <reflog@gmail.com>
* MM-24914 Various fixes for sidebar channel handling (#14756)
* Fix checking for channel membership when reordering channels
* Remove unique constraint on SidebarCategories
* Set column sizes for SidebarCategories and SidebarChannels tables
* Allow changing the sorting method for non-DM categories
* Fix nil pointers in error handling
* Fix orphaned channels from other team being returned in Channels category
* Fix non-orphaned channels being duplicated in the Channels category
* Remove unique constraint on SidebarChannels
* Fix category/name of favorites preferences
* Fix testSidebarChannelsMigration
* Rename err to nErr and appErr to err
* Fix channel order returned by GetSidebarCategories on MySQL
* Fix adding/removing favorites preferences
* Remove leftover TODO
* Change SidebarCategoryType enums to use full names (#14786)
* Change SidebarCategoryType enums to use full names
* Fix Channels constant
* Remove leftover debug code
* MM-24914 Fix updateCategory endpoint returning the wrong type (#14795)
* MM-24914 Make some changes to UpdateSidebarCategories (#14806)
* Fix orphaned DMs not always being returned
* MM-24914 Make some changes to UpdateSidebarCategories
* Run updateSidebarCategoryOrderT in a transaction
* Fix deleting SidebarChannels based on order of arguments to UpdateSidebarCategories
* bump for api testing
* bump for api testing
* Change CreateInitialSidebarCategories to return a plain error
* Change MigrateSidebarCategories to return a plain error
* Remove usage of UpdateColumns when updating sidebar categories (#14843)
* Remove usage of UpdateColumns when changing category order
* Add a random test case
* Remove usage of UpdateColumns when updating sidebar categories (#14843)
* Remove usage of UpdateColumns when changing category order
* Add a random test case
* Remove usage of UpdateColumns when updating sidebar categories (#14843)
* Remove usage of UpdateColumns when changing category order
* Add a random test case
* MM-26343 Make CreateInitialSidebarCategories idempotent (#14870)
* Fix bad merge
* Fix another bad merge
* Fix unintentionally removed i18n string
Co-authored-by: Eli Yukelzon <reflog@gmail.com>
