We use the default_text_search_config param from the DB
and pass that as the search config instead of hardcoding
it to english.
The docs do say that if no param is passed, by default
it will automatically fall back to default_text_search_config.
But I have seen different query plans being created
when the parameter is not passed versus when it is passed.
I am not sure why this is happening, but to be safe,
I have taken the search config during startup and stored
it in the SqlStore struct.
For reference, this is what happens without a search config
passed:
```
[bigdb] # explain analyze SELECT *, (SELECT COUNT(*) FROM Posts WHERE Posts.RootId = (CASE WHEN q2.RootId = '' THEN q2.Id ELSE q2.RootId END) AND Posts.DeleteAt = 0) as ReplyCount FROM Posts q2 WHERE q2.DeleteAt = 0 AND q2.Type NOT LIKE 'system_%' AND to_tsvector(Message) @@ to_tsquery('sapiente') AND ChannelId IN (SELECT Id FROM Channels, ChannelMembers WHERE Id = ChannelId AND Channels.DeleteAt = 0 AND ChannelMembe
rs.UserId = 'tc3p1yqw67d8idcp3g98awexqe' AND (TeamId = '8ywxyw9ocp8smxrmjzrkqhrdwe' OR TeamId = '') AND Id IN ('h1x9asxr7idjpqfmg8q67us49h')) ORDER BY q2.CreateAt DESC LIMIT 100;
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=8200.93..8720.23 rows=9 width=382) (actual time=177.277..177.948 rows=100 loops=1)
-> Result (cost=8200.93..8720.23 rows=9 width=382) (actual time=177.274..177.939 rows=100 loops=1)
-> Sort (cost=8200.93..8200.95 rows=9 width=374) (actual time=177.245..177.271 rows=100 loops=1)
Sort Key: q2.createat DESC
Sort Method: top-N heapsort Memory: 161kB
-> Nested Loop Semi Join (cost=71.53..8200.79 rows=9 width=374) (actual time=5.910..176.815 rows=532 loops=1)
-> Bitmap Heap Scan on posts q2 (cost=70.56..8183.64 rows=9 width=374) (actual time=5.815..176.457 rows=532 loops=1)
Recheck Cond: ((channelid)::text = 'h1x9asxr7idjpqfmg8q67us49h'::text)
Filter: (((type)::text !~~ 'system_%'::text) AND (deleteat = 0) AND (to_tsvector((message)::text) @@ to_tsquery('sapiente'::text)))
Rows Removed by Filter: 5244
Heap Blocks: exact=944
-> Bitmap Index Scan on idx_posts_channel_id_update_at (cost=0.00..70.56 rows=1866 width=0) (actual time=1.883..1.883 rows=5776 loops=1)
Index Cond: ((channelid)::text = 'h1x9asxr7idjpqfmg8q67us49h'::text)
-> Materialize (cost=0.98..17.04 rows=1 width=27) (actual time=0.000..0.000 rows=1 loops=532)
-> Nested Loop (cost=0.98..17.03 rows=1 width=27) (actual time=0.080..0.080 rows=1 loops=1)
-> Index Scan using channels_pkey on channels (cost=0.42..8.45 rows=1 width=27) (actual time=0.041..0.042 rows=1 loops=1)
Index Cond: ((id)::text = 'h1x9asxr7idjpqfmg8q67us49h'::text)
Filter: ((deleteat = 0) AND (((teamid)::text = '8ywxyw9ocp8smxrmjzrkqhrdwe'::text) OR ((teamid)::text = ''::text)))
-> Index Only Scan using idx_channelmembers_user_id_channel_id_last_viewed_at on channelmembers (cost=0.56..8.58 rows=1 width=27) (actual time=0.034..0.034 rows=1 loops=1)
Index Cond: ((userid = 'tc3p1yqw67d8idcp3g98awexqe'::text) AND (channelid = 'h1x9asxr7idjpqfmg8q67us49h'::text))
Heap Fetches: 1
SubPlan 1
-> Aggregate (cost=57.68..57.69 rows=1 width=8) (actual time=0.006..0.006 rows=1 loops=100)
-> Index Only Scan using idx_posts_root_id_delete_at on posts (cost=0.56..54.44 rows=1294 width=0) (actual time=0.005..0.005 rows=3 loops=100)
Index Cond: ((rootid = (CASE WHEN ((q2.rootid)::text = ''::text) THEN q2.id ELSE q2.rootid END)::text) AND (deleteat = 0))
Heap Fetches: 0
Planning Time: 2.178 ms
Execution Time: 178.155 ms
```
We can see it using a top-N heapsort.
And this is what happens when an explicit search config
is passed:
```
[bigdb] # explain analyze SELECT *, (SELECT COUNT(*) FROM Posts WHERE Posts.RootId = (CASE WHEN q2.RootId = '' THEN q2.Id ELSE q2.RootId END) AND Posts.DeleteAt = 0) as ReplyCount FROM Posts q2 WHERE q2.DeleteAt = 0 AND q2.Type NOT LIKE 'system_%' AND to_tsvector('pg_catalog.english', Message) @@ to_tsquery('pg_catalog.english', 'sapiente') AND ChannelId IN (SELECT Id FROM Channels, ChannelMembers WHERE Id = ChannelId AND Channels.DeleteAt = 0 AND ChannelMembers.UserId = 'tc3p1yqw67d8idcp3g98awexqe' AND (TeamId = '8ywxyw9ocp8smxrmjzrkqhrdwe' OR TeamId = '') AND Id IN ('h1x9asxr7idjpqfmg8q67us49h')) ORDER BY q2.CreateAt DESC LIMIT 100;
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=1.54..10679.04 rows=100 width=382) (actual time=0.237..31.405 rows=100 loops=1)
-> Nested Loop Semi Join (cost=1.54..17512.64 rows=164 width=382) (actual time=0.236..31.387 rows=100 loops=1)
-> Index Scan Backward using idx_posts_channel_id_delete_at_create_at on posts q2 (cost=0.56..8032.81 rows=164 width=374) (actual time=0.121..30.124 rows=100 loops=1)
Index Cond: (((channelid)::text = 'h1x9asxr7idjpqfmg8q67us49h'::text) AND (deleteat = 0))
Filter: (((type)::text !~~ 'system_%'::text) AND (to_tsvector('english'::regconfig, (message)::text) @@ '''sapient'''::tsquery))
Rows Removed by Filter: 1088
-> Materialize (cost=0.98..17.04 rows=1 width=27) (actual time=0.001..0.001 rows=1 loops=100)
-> Nested Loop (cost=0.98..17.03 rows=1 width=27) (actual time=0.074..0.075 rows=1 loops=1)
-> Index Scan using channels_pkey on channels (cost=0.42..8.45 rows=1 width=27) (actual time=0.032..0.032 rows=1 loops=1)
Index Cond: ((id)::text = 'h1x9asxr7idjpqfmg8q67us49h'::text)
Filter: ((deleteat = 0) AND (((teamid)::text = '8ywxyw9ocp8smxrmjzrkqhrdwe'::text) OR ((teamid)::text = ''::text)))
-> Index Only Scan using idx_channelmembers_user_id_channel_id_last_viewed_at on channelmembers (cost=0.56..8.58 rows=1 width=27) (actual time=0.041..0.041 rows=1 loops=1)
Index Cond: ((userid = 'tc3p1yqw67d8idcp3g98awexqe'::text) AND (channelid = 'h1x9asxr7idjpqfmg8q67us49h'::text))
Heap Fetches: 1
SubPlan 1
-> Aggregate (cost=57.68..57.69 rows=1 width=8) (actual time=0.010..0.010 rows=1 loops=100)
-> Index Only Scan using idx_posts_root_id_delete_at on posts (cost=0.56..54.44 rows=1294 width=0) (actual time=0.009..0.009 rows=3 loops=100)
Index Cond: ((rootid = (CASE WHEN ((q2.rootid)::text = ''::text) THEN q2.id ELSE q2.rootid END)::text) AND (deleteat = 0))
Heap Fetches: 0
Planning Time: 0.401 ms
Execution Time: 31.466 ms
```
https://mattermost.atlassian.net/browse/MM-25554
```release-note
NONE
```
This PR adds the necessary resolvers to support
the graphQL front-end. Some additional methods
need to be added to the model structs to support
this. Comments have been made to clarify their purpose.
There is a slight duplication of code in the api layer.
But eventually that's going to go away when we move
away from the REST API entirely.
The schema is in api4/schema.graphqls and gets compiled
into the binary as an asset.
The GraphiQL editor url is at GET /api/v5/graphql.
Everything is behind the feature flag MM_FEATUREFLAGS_GRAPHQL.
```release-note
NONE
```
Remote Cluster Service
- provides ability for multiple Mattermost cluster instances to create a trusted connection with each other and exchange messages
- trusted connections are managed via slash commands (for now)
- facilitates features requiring inter-cluster communication, such as Shared Channels
Shared Channels Service
- provides ability to shared channels between one or more Mattermost cluster instances (using trusted connection)
- sharing/unsharing of channels is managed via slash commands (for now)
* MM-31094: Adds tooling to develop and test using a MySQL instance with replication lag. Adds some lazy lookups to fallback to master if results are not found.
* MM-31094: Removes mysql-read-replica from default docker services.
* MM-31094: Switches (store..SessionStore).Get and (store.TeamStore).GetMember to using context.Context.
* MM-31094: Updates (store.UsersStore).Get to use context.
* MM-31094: Updates (store.PostStore).Get to use context.
* MM-31094: Removes feature flag and config setting.
* MM-31094: Rolls back some master reads.
* MM-31094: Rolls a non-cache read.
* MM-31094: Removes feature flag from the store.
* MM-31094: Removes unused constant and struct field.
* MM-31094: Removes some old feature flag references.
* MM-31094: Fixes some tests.
* MM-31094: App layers fix.
* MM-31094: Fixes mocks.
* MM-31094: Don't reparse flag.
* MM-31094: No reparse.
* MM-31094: Removed unused FeatureFlags field.
* MM-31094: Removes unnecessary feature flags variable declarations.
* MM-31094: Fixes copy-paste error.
* MM-31094: Fixes logical error.
* MM-30194: Removes test method from store.
* Revert "MM-30194: Removes test method from store."
This reverts commit d5a6e8529bd5f4d993824c828e239d009b05e567.
* MM-31094: Conforming to make's strange syntax.
* MM-31094: Configures helper for read replica with option.
* MM-31094: Adds some missing ctx's.
* MM-31094: WIP
* MM-31094: Updates test names.
* MM-31094: WIP
* MM-31094: Removes unnecessary master reads.
* MM-31094: ID case changes out of scope.
* MM-31094: Removes unused context.
* MM-31094: Switches to a helper. Removes some var naming changes. Fixes a merge error.
* MM-31094: Removes SQLITE db driver ref.
* MM-31094: Layer generate fix.
* MM-31094: Removes unnecessary changes.
* MM-31094: Moves test method.
* MM-31094: Re-add previous fix.
* MM-31094: Removes make command for dev.
* MM-31094: Fix for login.
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* docker: prometheus and grafana for development
Add prometheus and grafana docker configuration when developing with access to the enterprise repository. This simplifies the setup for developers adding new metrics or even examining events from their development server. The services are disabled by default when no enterprise source is detected.
Grafana is provisioned automatically with the Prometheus datasource, as well as the canonical dashboards used with Mattermost. Furthermore, no authentication is required to access Grafana to simplify access from a development environment. The default home dashboard is customized to automatically show links to the provisioned dashboards. Dashboards can be saved, and login remains available via the default `admin/admin` credentials (which initiates a password reset), but no dashboard or system configuration is (currently) persisted if the container is destroyed.
Linux requires slightly special handling (well, really it's Docker for MacOS/Windows), in that `host.docker.internal` is the canonical way to resolve the host on MacOS/Windows, but on Linux it's usually sufficient to just use `localhost`. Until https://github.com/docker/for-linux/issues/264 is resolved, this PR includes code to customize the Prometheus configuration to point at the required address for the running platform.
* metrics: track active jobs by type
* metrics: active jobs chart
* metrics: server start and job annotations
* Update build/docker-compose.common.yml
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
* prometheus: resolve docker host via 172.17.0.1 instead
* skip enabling prometheus and grafana by default
* handle JOB_STATUS_ERROR and JOB_STATUS_CANCELED end states
* handle nil srv.metrics
* lookup job to determine type for metrics
* mocked unit tests for jobs
* goimports lint fixes
* missing license
* add instance to server start tag
* filter annotations by selected instance
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Add make command for running the app in an HA topology
* Tidy up conf files, remove unnecessary commands and add restart-haserver command
* Use the right flag for checking enterprise folder presence before ha setup
* Configurable dev environment
* Add a bit of documentation
* fixing gofmt
* A bit more doc
* Using variable
* Adding license header
* Moving LDAP_DATA variable to the default-config.mk file
* Adding another docker-compose for the makefile to not brake anybody workflow
* Moving dejavu to the config
* Fixing docker-compose.makefile.yaml for dejavu
* Adding keycloak support to the dev environment
* Address PR review comments
* Removing minio from default docker images
* Changing the default version of mysql to the oldest supported (5.6)
* Change the restart option to no for the dev environment
* Fixing restart option
* Reverting unneded changes
* Restoring 5.7 to check if test passes
* Going back to 5.6 mysql image
* Fixing tests on mysql 5.6
* Skipping flaky test
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Run all tests fasts in postgres
* Moving postgres config to a file
* Addressing PR review comments
* Testing against mysql in the CI to keep things less changed
* Revert "Testing against mysql in the CI to keep things less changed"
This reverts commit fc940c111db269a821c54e005d76df87cb70e031.
* Fixing a test broken in postgres but working in mysql
* Fixing some tests
* Fixing some config tests with postgres
* Fixing the rest of config tests
* Fixing govet error
* Fixing search tests for postgres
* configure MySQL to use tmpfs
* simplify use of COMPOSE_PROJECT_NAME, and set envs vs. overriding config.json, fix error output, --no-ansi
* cmd/mattermost/commands/*_test.go: ignore MM_SQLSETTINGS_DATASOURCE from the env
* clean up exit 1 semantics
