0ebd3e8085
For our MySQL customers, we have discovered that the query is not
able to choose the right plan by itself without adequate hints.
This is only for MySQL as we have confirmed from multiple customers
that Postgres takes the right index idx_sidebarcategories_userid_teamid
for the sidebarCategories table. And if it doesn't, then a VACUUM ANALYZE
fixes it.
But for MySQL, we have to do two things:
- Pass an index hint to let it use idx_sidebarcategories_userid_teamid.
- Pass an optimizer hint to materialize the sub-query. This is used
to materialize the doesNotHaveSidebarChannel sub-query into a temporary
table, letting MySQL reuse the contents of the table for further processing
in the parent sections of the query.
I have confirmed both locally and in the customer environment
that it gives a clear benefit.
*LOCAL*
OLD:
```
| -> Nested loop antijoin (cost=2889.85 rows=19767) (actual time=3.355..38.033 rows=15 loops=1)
-> Nested loop inner join (cost=66.65 rows=110) (actual time=0.410..1.689 rows=220 loops=1)
-> Filter: ((Channels.DeleteAt = 0) and (Channels.`Type` in ('O','P'))) (cost=25.25 rows=110) (actual time=0.394..0.886 rows=220 loops=1)
-> Index lookup on Channels using idx_channels_team_id_display_name (TeamId='team01'), with index condition: (Channels.Id is not null) (cost=25.25 rows=220) (actual time=0.389..0.793 rows=220 loops=1)
-> Single-row covering index lookup on ChannelMembers using PRIMARY (ChannelId=Channels.Id, UserId='user000') (cost=0.28 rows=1) (actual time=0.003..0.003 rows=1 loops=220)
-> Nested loop inner join (cost=4967.50 rows=180) (actual time=0.165..0.165 rows=1 loops=220)
-> Covering index lookup on SidebarChannels using PRIMARY (ChannelId=Channels.Id) (cost=7.86 rows=180) (actual time=0.055..0.062 rows=13 loops=220)
-> Filter: ((SidebarCategories.TeamId = 'team01') and (SidebarCategories.UserId = 'user000')) (cost=44.93 rows=1) (actual time=0.008..0.008 rows=0 loops=2881)
-> Single-row index lookup on SidebarCategories using PRIMARY (Id=SidebarChannels.CategoryId) (cost=44.93 rows=1) (actual time=0.006..0.006 rows=1 loops=2881)
|
```
NEW:
```
| -> Nested loop antijoin (cost=5879.73 rows=58021) (actual time=1.544..3.135 rows=15 loops=1)
-> Nested loop inner join (cost=66.65 rows=110) (actual time=0.421..1.778 rows=220 loops=1)
-> Filter: ((Channels.DeleteAt = 0) and (Channels.`Type` in ('O','P'))) (cost=25.25 rows=110) (actual time=0.405..0.945 rows=220 loops=1)
-> Index lookup on Channels using idx_channels_team_id_display_name (TeamId='team01'), with index condition: (Channels.Id is not null) (cost=25.25 rows=220) (actual time=0.400..0.859 rows=220 loops=1)
-> Single-row covering index lookup on ChannelMembers using PRIMARY (ChannelId=Channels.Id, UserId='user000') (cost=0.28 rows=1) (actual time=0.003..0.004 rows=1 loops=220)
-> Single-row index lookup on <subquery2> using <auto_distinct_key> (ChannelId=Channels.Id) (cost=130.37..130.37 rows=1) (actual time=0.006..0.006 rows=1 loops=220)
-> Materialize with deduplication (cost=130.35..130.35 rows=527) (actual time=1.118..1.118 rows=205 loops=1)
-> Filter: (SidebarChannels.ChannelId is not null) (cost=77.61 rows=527) (actual time=0.059..0.851 rows=523 loops=1)
-> Nested loop inner join (cost=77.61 rows=527) (actual time=0.058..0.786 rows=523 loops=1)
-> Covering index lookup on SidebarCategories using idx_sidebarcategories_userid_teamid (UserId='user000', TeamId='team01') (cost=2.81 rows=15) (actual time=0.025..0.031 rows=15 loops=1)
-> Covering index lookup on SidebarChannels using idx_sidebarchannels_categoryid (CategoryId=SidebarCategories.Id) (cost=1.70 rows=35) (actual time=0.032..0.046 rows=35 loops=15)
```
Performance improvement from 38ms to 3ms.
*CUSTOMER ENV* (with sensitive data wiped off)
OLD:
```
| -> Sort: channels.DisplayName (actual time=512..512 rows=5 loops=1)
-> Stream results (cost=3.28 rows=1.44) (actual time=223..512 rows=5 loops=1)
-> Nested loop antijoin (cost=3.28 rows=1.44) (actual time=223..512 rows=5 loops=1)
-> Nested loop inner join (cost=3.02 rows=0.3) (actual time=0.025..0.0878 rows=5 loops=1)
-> Covering index lookup on ChannelMembers using idx_channelmembers_user_id_channel_id_last_viewed_at (UserId='') (cost=0.916 rows=6) (actual time=0.0146..0.023 rows=6 loops=1)
-> Filter: ((channels.DeleteAt = 0) and (channels.TeamId = '') and (channels.`Type` in ('O','P'))) (cost=0.251 rows=0.05) (actual time=0.00999..0.0102 rows=0.833 loops=6)
-> Single-row index lookup on Channels using PRIMARY (Id=channelmembers.ChannelId) (cost=0.251 rows=1) (actual time=0.00778..0.00785 rows=1 loops=6)
-> Nested loop inner join (cost=2.85 rows=4.81) (actual time=102..102 rows=0 loops=5)
-> Covering index lookup on SidebarChannels using PRIMARY (ChannelId=channelmembers.ChannelId) (cost=2.01 rows=4.81) (actual time=0.0125..13.8 rows=24134 loops=5)
-> Filter: ((sidebarcategories.TeamId = '') and (sidebarcategories.UserId = '')) (cost=1.54 rows=1) (actual time=0.00359..0.00359 rows=0 loops=120671)
-> Single-row index lookup on SidebarCategories using PRIMARY (Id=sidebarchannels.CategoryId) (cost=1.54 rows=1) (actual time=0.00316..0.00319 rows=1 loops=120671)
```
NEW:
```
Here is the output
| -> Sort: channels.DisplayName (actual time=0.12..0.12 rows=5 loops=1)
-> Stream results (cost=3.45 rows=4.01) (actual time=0.0797..0.11 rows=5 loops=1)
-> Nested loop antijoin (cost=3.45 rows=4.01) (actual time=0.0769..0.106 rows=5 loops=1)
-> Nested loop inner join (cost=3.02 rows=0.3) (actual time=0.0291..0.0555 rows=5 loops=1)
-> Covering index lookup on ChannelMembers using idx_channelmembers_user_id_channel_id_last_viewed_at (UserId='') (cost=0.916 rows=6) (actual time=0.0145..0.0162 rows=6 loops=1)
-> Filter: ((channels.DeleteAt = 0) and (channels.TeamId = '') and (channels.`Type` in ('O','P'))) (cost=0.251 rows=0.05) (actual time=0.00611..0.00619 rows=0.833 loops=6)
-> Single-row index lookup on Channels using PRIMARY (Id=channelmembers.ChannelId) (cost=0.251 rows=1) (actual time=0.0053..0.00534 rows=1 loops=6)
-> Single-row index lookup on <subquery2> using <auto_distinct_key> (ChannelId=channelmembers.ChannelId) (cost=7.01..7.01 rows=1) (actual time=0.00956..0.00956 rows=0 loops=5)
-> Materialize with deduplication (cost=7..7 rows=13.4) (actual time=0.0451..0.0451 rows=0 loops=1)
-> Filter: (sidebarchannels.ChannelId is not null) (cost=5.66 rows=13.4) (actual time=0.0441..0.0441 rows=0 loops=1)
-> Nested loop inner join (cost=5.66 rows=13.4) (actual time=0.0439..0.0439 rows=0 loops=1)
-> Covering index lookup on SidebarCategories using idx_sidebarcategories_userid_teamid (UserId='', TeamId='') (cost=0.592 rows=3) (actual time=0.0105..0.0134 rows=3 loops=1)
-> Covering index lookup on SidebarChannels using idx_sidebarchannels_categoryid (CategoryId=sidebarcategories.Id) (cost=1.39 rows=4.46) (actual time=0.00999..0.00999 rows=0 loops=3)
```
Performance improvement from 512ms to 0.12ms.
https://mattermost.atlassian.net/browse/MM-64209
```release-note
NONE
```
Mattermost is an open source platform for secure collaboration across the entire software development lifecycle. This repo is the primary source for core development on the Mattermost platform; it's written in Go and React and runs as a single Linux binary with MySQL or PostgreSQL. A new compiled version is released under an MIT license every month on the 16th.
Deploy Mattermost on-premises, or try it for free in the cloud.
Learn more about the following use cases with Mattermost:
Other useful resources:
- Download and Install Mattermost - Install, setup, and configure your own Mattermost instance.
- Product documentation - Learn how to run a Mattermost instance and take advantage of all the features.
- Developer documentation - Contribute code to Mattermost or build an integration via APIs, Webhooks, slash commands, Apps, and plugins.
Table of contents
- Install Mattermost
- Native mobile and desktop apps
- Get security bulletins
- Get involved
- Learn more
- License
- Get the latest news
- Contributing
Install Mattermost
- Download and Install Mattermost Self-Hosted - Deploy a Mattermost Self-hosted instance in minutes via Docker, Ubuntu, or tar.
- Get started in the cloud to try Mattermost today.
- Developer machine setup - Follow this guide if you want to write code for Mattermost.
Other install guides:
- Deploy Mattermost on Docker
- Mattermost Omnibus
- Install Mattermost from Tar
- Ubuntu 20.04 LTS
- Kubernetes
- Helm
- Debian Buster
- RHEL 8
- More server install guides
Native mobile and desktop apps
In addition to the web interface, you can also download Mattermost clients for Android, iOS, Windows PC, macOS, and Linux.
Get security bulletins
Receive notifications of critical security updates. The sophistication of online attackers is perpetually increasing. If you're deploying Mattermost it's highly recommended you subscribe to the Mattermost Security Bulletin mailing list for updates on critical security releases.
Get involved
- Contribute to Mattermost
- Find "Help Wanted" projects
- Join Developer Discussion on a Mattermost server for contributors
- Get Help With Mattermost
Learn more
- API options - webhooks, slash commands, drivers, and web service
- See who's using Mattermost
- Browse over 700 Mattermost integrations
License
See the LICENSE file for license rights and limitations.
Get the latest news
- X - Follow Mattermost on X, formerly Twitter.
- Blog - Get the latest updates from the Mattermost blog.
- Facebook - Follow Mattermost on Facebook.
- LinkedIn - Follow Mattermost on LinkedIn.
- Email - Subscribe to our newsletter (1 or 2 per month).
- Mattermost - Join the ~contributors channel on the Mattermost Community Server.
- IRC - Join the #matterbridge channel on Freenode (thanks to matterircd).
- YouTube - Subscribe to Mattermost.
Contributing
Please see CONTRIBUTING.md. Join the Mattermost Contributors server to join community discussions about contributions, development, and more.