635cea19c7
In MySQL, tested in Community, times for the old and new queries are
mostly the same:
- Original : 6 s 250 ms (execution: 258 ms, fetching: 5 s 992 ms)
- New : 6 s 148 ms (execution: 148 ms, fetching: 6 s)
The output from EXPLAIN in Community is also similar for both queries:
- Original:
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
|----|-------------|------------|------------|--------|------------------------|------------------------|---------|----------------------|---------|----------|-----------------------|
| 1 | PRIMARY | <derived2> | | ALL | | | | | 10000 | 100.00 | Using filesort |
| 1 | PRIMARY | Channels | | eq_ref | PRIMARY | PRIMARY | 106 | PostsQuery.ChannelId | 1 | 100.00 | |
| 2 | DERIVED | Posts | | range | idx_posts_create_at_id | idx_posts_create_at_id | 115 | | 3108048 | 100.00 | Using index condition |
- New:
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
|----|-------------|----------|------------|--------|------------------------|------------------------|---------|----------------------------|---------|----------|-----------------------|
| 1 | SIMPLE | Posts | | range | idx_posts_create_at_id | idx_posts_create_at_id | 115 | | 3108050 | 100.00 | Using index condition |
| 1 | SIMPLE | Channels | | eq_ref | PRIMARY | PRIMARY | 106 | mattermost.Posts.ChannelId | 1 | 100.00 | |
In Postgres 12, this was tested locally with a 12M posts database, in a
machine with the following specs:
- CPU: i7-11800H (8C / 16T, 2.3 / 4.6GHz, 24MB)
- Memory: 32GB
- Storage: 1TB SSD M.2 2280 PCIe 4.0x4 Performance NVMe Opal2
The times of the queries in this setup are:
- Original query : 118.080 ms
- New query : 94.454 ms
The complete output from EXPLAIN ANALYZE in Postgres 12:
- Original query:
mattermost=> EXPLAIN ANALYZE
SELECT PostsQuery.*, Channels.TeamId
FROM (
SELECT *
FROM
Posts
WHERE Posts.CreateAt > 1629244800000
OR (Posts.CreateAt = 1629244800000 AND Posts.Id > '')
ORDER BY CreateAt ASC, Id ASC
LIMIT 10000
) AS PostsQuery
LEFT JOIN Channels ON PostsQuery.ChannelId = Channels.Id
ORDER BY CreateAt ASC, Id ASC;
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Sort (cost=156636.28..156661.28 rows=10000 width=375) (actual time=114.093..114.833 rows=10000 loops=1)
Sort Key: posts.createat, posts.id
Sort Method: external merge Disk: 3128kB
-> Hash Right Join (cost=136335.48..155971.89 rows=10000 width=375) (actual time=67.919..107.330 rows=10000 loops=1)
Hash Cond: ((channels.id)::text = (posts.channelid)::text)
-> Seq Scan on channels (cost=0.00..10902.68 rows=272168 width=28) (actual time=0.018..28.826 rows=272168 loops=1)
-> Hash (cost=135721.48..135721.48 rows=10000 width=374) (actual time=40.141..40.144 rows=10000 loops=1)
Buckets: 16384 Batches: 2 Memory Usage: 2244kB
-> Limit (cost=135596.48..135621.48 rows=10000 width=374) (actual time=36.799..38.169 rows=10000 loops=1)
-> Sort (cost=135596.48..135704.93 rows=43380 width=374) (actual time=30.641..31.699 rows=10000 loops=1)
Sort Key: posts.createat, posts.id
Sort Method: external merge Disk: 18840kB
-> Bitmap Heap Scan on posts (cost=827.91..132497.48 rows=43380 width=374) (actual time=1.856..8.630 rows=54398 loops=1)
Recheck Cond: ((createat > '1629244800000'::bigint) OR (createat = '1629244800000'::bigint))
Filter: ((createat > '1629244800000'::bigint) OR ((createat = '1629244800000'::bigint) AND ((id)::text > ''::text)))
Heap Blocks: exact=2674
-> BitmapOr (cost=827.91..827.91 rows=43380 width=0) (actual time=1.660..1.661 rows=0 loops=1)
-> Bitmap Index Scan on idx_posts_create_at (cost=0.00..801.78 rows=43379 width=0) (actual time=1.657..1.657 rows=54398 loops=1)
Index Cond: (createat > '1629244800000'::bigint)
-> Bitmap Index Scan on idx_posts_create_at (cost=0.00..4.44 rows=1 width=0) (actual time=0.002..0.002 rows=0 loops=1)
Index Cond: (createat = '1629244800000'::bigint)
Planning Time: 0.230 ms
JIT:
Functions: 14
Options: Inlining false, Optimization false, Expressions true, Deforming true
Timing: Generation 0.562 ms, Inlining 0.000 ms, Optimization 0.320 ms, Emission 5.863 ms, Total 6.745 ms
Execution Time: 118.080 ms
(27 rows)
- New query:
mattermost=> EXPLAIN ANALYZE
SELECT Posts.*, Channels.TeamId
FROM
Posts
LEFT JOIN Channels ON Posts.ChannelId = Channels.Id
WHERE Posts.CreateAt > 1629244800000
OR (Posts.CreateAt = 1629244800000 AND Posts.Id > '')
ORDER BY Posts.CreateAt ASC, Posts.Id ASC
LIMIT 10000;
QUERY PLAN
--------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=148443.93..149610.68 rows=10000 width=375) (actual time=83.610..92.492 rows=10000 loops=1)
-> Gather Merge (cost=148443.93..152661.73 rows=36150 width=375) (actual time=74.409..82.975 rows=10000 loops=1)
Workers Planned: 2
Workers Launched: 2
-> Sort (cost=147443.91..147489.10 rows=18075 width=375) (actual time=64.325..64.629 rows=3436 loops=3)
Sort Key: posts.createat, posts.id
Sort Method: external merge Disk: 5992kB
Worker 0: Sort Method: external merge Disk: 7144kB
Worker 1: Sort Method: external merge Disk: 7152kB
-> Parallel Hash Left Join (cost=12336.48..146152.66 rows=18075 width=375) (actual time=42.908..51.116 rows=18133 loops=3)
Hash Cond: ((posts.channelid)::text = (channels.id)::text)
-> Parallel Bitmap Heap Scan on posts (cost=827.91..132054.64 rows=18075 width=374) (actual time=2.448..5.417 rows=18133 loops=3)
Recheck Cond: ((createat > '1629244800000'::bigint) OR (createat = '1629244800000'::bigint))
Filter: ((createat > '1629244800000'::bigint) OR ((createat = '1629244800000'::bigint) AND ((id)::text > ''::text)))
Heap Blocks: exact=902
-> BitmapOr (cost=827.91..827.91 rows=43380 width=0) (actual time=2.111..2.112 rows=0 loops=1)
-> Bitmap Index Scan on idx_posts_create_at (cost=0.00..801.78 rows=43379 width=0) (actual time=2.105..2.105 rows=54398 loops=1)
Index Cond: (createat > '1629244800000'::bigint)
-> Bitmap Index Scan on idx_posts_create_at (cost=0.00..4.44 rows=1 width=0) (actual time=0.004..0.005 rows=0 loops=1)
Index Cond: (createat = '1629244800000'::bigint)
-> Parallel Hash (cost=9315.03..9315.03 rows=113403 width=28) (actual time=29.928..29.929 rows=90723 loops=3)
Buckets: 65536 Batches: 8 Memory Usage: 2688kB
-> Parallel Seq Scan on channels (cost=0.00..9315.03 rows=113403 width=28) (actual time=5.378..16.495 rows=90723 loops=3)
Planning Time: 0.460 ms
JIT:
Functions: 46
Options: Inlining false, Optimization false, Expressions true, Deforming true
Timing: Generation 2.291 ms, Inlining 0.000 ms, Optimization 1.518 ms, Emission 23.827 ms, Total 27.636 ms
Execution Time: 94.454 ms
(29 rows)
Mattermost is an open source platform for secure collaboration across the entire software development lifecycle. This repo is the primary source for core development on the Mattermost platform; it's written in Go and React and runs as a single Linux binary with MySQL or PostgreSQL. A new compiled version is released under an MIT license every month on the 16th.
Use it for free in Mattermost Cloud or deploy on-premises.
Learn more about the following use cases with Mattermost:
Other useful resources:
- Download and Install Mattermost - Install, setup, and configure your own Mattermost instance.
- Product documentation - Learn how to run a Mattermost instance and take advantage of all the features.
- Developer documentation - Contribute code to Mattermost or build an integration via APIs, Webhooks, slash commands, Apps, and plugins.
Table of contents
- Try out Mattermost
- Install Mattermost
- Native Mobile and Desktop Apps
- Get Security Bulletins
- Get Involved
- Learn More
- Get the Latest News
- Contributing
Install Mattermost
- Download and Install Mattermost Self-Hosted - Deploy a Mattermost Self-hosted instance in minutes via Docker, Ubuntu, or tar.
- Get started with Mattermost Cloud to use Mattermost instantly.
- Developer machine setup - Follow this guide if you want to write code for Mattermost.
Other install guides:
- Deploy Mattermost on Docker
- Mattermost Omnibus
- Install Mattermost from Tar
- Ubuntu 20.04 LTS
- Kubernetes
- Helm
- Debian Buster
- RHEL 8
- More server install guides
Native mobile and desktop apps
In addition to the web interface, you can also download Mattermost clients for Android, iOS, Windows PC, macOS, and Linux.
Get security bulletins
Receive notifications of critical security updates. The sophistication of online attackers is perpetually increasing. If you're deploying Mattermost it's highly recommended you subscribe to the Mattermost Security Bulletin mailing list for updates on critical security releases.
Get involved
- Contribute to Mattermost
- Find "Help Wanted" projects
- Join Developer Discussion on a Mattermost server for contributors
- Get Help With Mattermost
Learn more
- API options - webhooks, slash commands, drivers, and web service
- See who's using Mattermost
- Browse over 700 Mattermost integrations
License
See the LICENSE file for license rights and limitations.
Get the latest news
- Twitter - Follow Mattermost.
- Blog - Get the latest updates from the Mattermost blog.
- Facebook - Follow Mattermost.
- LinkedIn - Follow Mattermost.
- Email - Subscribe to our newsletter (1 or 2 per month).
- Mattermost - Join the ~contributors channel on the Mattermost Community Server.
- IRC - Join the #matterbridge channel on Freenode (thanks to matterircd).
Contributing
Please see CONTRIBUTING.md. Join the Mattermost Contributors server to join community discussions about contributions, development, and more.