* Added webp package into app/emoji.go , also added a test
* fixed import order inside emoji.go
* Ran go mod tidy
* Conversion to png for Webp emojis that must be resized is implemented
* Deleted unused import
* Added reading mechanism for webp instead of creating
* fileutils and filepath are used while reading a webp file during test
* Deleted readtestwebp function and put it in emoji_test as requested
* Added webp image upload test inside api4/file_test.go
* Expect image is changed from false to true
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-44509 - subscription change not reflected after trial
* fix the tests
* receive the subscription id as param, defer the cache invalidation
* validate business email with body request
* update api request to accept json body payload
* fix unit tests
* fix tests after merge conflict fixes
Co-authored-by: Pablo Velez Vidal <pablo.velez@mattermost.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Add route endpoints, model, store functions, and tests for top threads
* Run make store-layers
* Make the following changes
- Fix top user threads query
- Fix passing parameters in api4/insights.go to handler in app
- Add top user threads test
* Add post-message, user_id, participants information to insights results
* model.TopThread.UserID -> model.TopThread.UserId, for compatibility with MySQL
* Rename name -> channel_name
* Add user information to response
* Link post in response, filter out deleted root posts from top threads
* Handle thread delete cases, add app tests for threads insights
* lint: fix typo
* lint: rename asserts
* lint: require.nil -> require.NoError
* Add integration tests for thread insights
* Add embeds and images to top posts
* Add license checks for top threads endpoints
* Query users in batch to populate post-creator
* Make the following changes
- Add license to test server in api4/
- Add tests for threads insights
- top team threads shouldn't include threads from other teams, DMs
- Test duration constraint
- Pagination testing for top threads in model/insights_test.go
* Add i18n-extract
* i18n fixes
* Add username, nickname to user_information
* Hide message, user_id, post_id, reply_count in depth=1 of top threads response
* Fix tests using response.reply_count to use response.post.reply_count
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* [MM-44667] Set InsecureSkipVerify for S3 when EnableInsecureOutgoingConnections is set
* Cleanup s3New, customTransport
* Lint fixes
* Cleanup s3store_test.go
* Using default transport
```release-note
NONE
```
Co-authored-by: Jake Gutierrez <jakegut0108@gmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
We use the default_text_search_config param from the DB
and pass that as the search config instead of hardcoding
it to english.
The docs do say that if no param is passed, by default
it will automatically fall back to default_text_search_config.
But I have seen different query plans being created
when the parameter is not passed versus when it is passed.
I am not sure why this is happening, but to be safe,
I have taken the search config during startup and stored
it in the SqlStore struct.
For reference, this is what happens without a search config
passed:
```
[bigdb] # explain analyze SELECT *, (SELECT COUNT(*) FROM Posts WHERE Posts.RootId = (CASE WHEN q2.RootId = '' THEN q2.Id ELSE q2.RootId END) AND Posts.DeleteAt = 0) as ReplyCount FROM Posts q2 WHERE q2.DeleteAt = 0 AND q2.Type NOT LIKE 'system_%' AND to_tsvector(Message) @@ to_tsquery('sapiente') AND ChannelId IN (SELECT Id FROM Channels, ChannelMembers WHERE Id = ChannelId AND Channels.DeleteAt = 0 AND ChannelMembe
rs.UserId = 'tc3p1yqw67d8idcp3g98awexqe' AND (TeamId = '8ywxyw9ocp8smxrmjzrkqhrdwe' OR TeamId = '') AND Id IN ('h1x9asxr7idjpqfmg8q67us49h')) ORDER BY q2.CreateAt DESC LIMIT 100;
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=8200.93..8720.23 rows=9 width=382) (actual time=177.277..177.948 rows=100 loops=1)
-> Result (cost=8200.93..8720.23 rows=9 width=382) (actual time=177.274..177.939 rows=100 loops=1)
-> Sort (cost=8200.93..8200.95 rows=9 width=374) (actual time=177.245..177.271 rows=100 loops=1)
Sort Key: q2.createat DESC
Sort Method: top-N heapsort Memory: 161kB
-> Nested Loop Semi Join (cost=71.53..8200.79 rows=9 width=374) (actual time=5.910..176.815 rows=532 loops=1)
-> Bitmap Heap Scan on posts q2 (cost=70.56..8183.64 rows=9 width=374) (actual time=5.815..176.457 rows=532 loops=1)
Recheck Cond: ((channelid)::text = 'h1x9asxr7idjpqfmg8q67us49h'::text)
Filter: (((type)::text !~~ 'system_%'::text) AND (deleteat = 0) AND (to_tsvector((message)::text) @@ to_tsquery('sapiente'::text)))
Rows Removed by Filter: 5244
Heap Blocks: exact=944
-> Bitmap Index Scan on idx_posts_channel_id_update_at (cost=0.00..70.56 rows=1866 width=0) (actual time=1.883..1.883 rows=5776 loops=1)
Index Cond: ((channelid)::text = 'h1x9asxr7idjpqfmg8q67us49h'::text)
-> Materialize (cost=0.98..17.04 rows=1 width=27) (actual time=0.000..0.000 rows=1 loops=532)
-> Nested Loop (cost=0.98..17.03 rows=1 width=27) (actual time=0.080..0.080 rows=1 loops=1)
-> Index Scan using channels_pkey on channels (cost=0.42..8.45 rows=1 width=27) (actual time=0.041..0.042 rows=1 loops=1)
Index Cond: ((id)::text = 'h1x9asxr7idjpqfmg8q67us49h'::text)
Filter: ((deleteat = 0) AND (((teamid)::text = '8ywxyw9ocp8smxrmjzrkqhrdwe'::text) OR ((teamid)::text = ''::text)))
-> Index Only Scan using idx_channelmembers_user_id_channel_id_last_viewed_at on channelmembers (cost=0.56..8.58 rows=1 width=27) (actual time=0.034..0.034 rows=1 loops=1)
Index Cond: ((userid = 'tc3p1yqw67d8idcp3g98awexqe'::text) AND (channelid = 'h1x9asxr7idjpqfmg8q67us49h'::text))
Heap Fetches: 1
SubPlan 1
-> Aggregate (cost=57.68..57.69 rows=1 width=8) (actual time=0.006..0.006 rows=1 loops=100)
-> Index Only Scan using idx_posts_root_id_delete_at on posts (cost=0.56..54.44 rows=1294 width=0) (actual time=0.005..0.005 rows=3 loops=100)
Index Cond: ((rootid = (CASE WHEN ((q2.rootid)::text = ''::text) THEN q2.id ELSE q2.rootid END)::text) AND (deleteat = 0))
Heap Fetches: 0
Planning Time: 2.178 ms
Execution Time: 178.155 ms
```
We can see it using a top-N heapsort.
And this is what happens when an explicit search config
is passed:
```
[bigdb] # explain analyze SELECT *, (SELECT COUNT(*) FROM Posts WHERE Posts.RootId = (CASE WHEN q2.RootId = '' THEN q2.Id ELSE q2.RootId END) AND Posts.DeleteAt = 0) as ReplyCount FROM Posts q2 WHERE q2.DeleteAt = 0 AND q2.Type NOT LIKE 'system_%' AND to_tsvector('pg_catalog.english', Message) @@ to_tsquery('pg_catalog.english', 'sapiente') AND ChannelId IN (SELECT Id FROM Channels, ChannelMembers WHERE Id = ChannelId AND Channels.DeleteAt = 0 AND ChannelMembers.UserId = 'tc3p1yqw67d8idcp3g98awexqe' AND (TeamId = '8ywxyw9ocp8smxrmjzrkqhrdwe' OR TeamId = '') AND Id IN ('h1x9asxr7idjpqfmg8q67us49h')) ORDER BY q2.CreateAt DESC LIMIT 100;
QUERY PLAN
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=1.54..10679.04 rows=100 width=382) (actual time=0.237..31.405 rows=100 loops=1)
-> Nested Loop Semi Join (cost=1.54..17512.64 rows=164 width=382) (actual time=0.236..31.387 rows=100 loops=1)
-> Index Scan Backward using idx_posts_channel_id_delete_at_create_at on posts q2 (cost=0.56..8032.81 rows=164 width=374) (actual time=0.121..30.124 rows=100 loops=1)
Index Cond: (((channelid)::text = 'h1x9asxr7idjpqfmg8q67us49h'::text) AND (deleteat = 0))
Filter: (((type)::text !~~ 'system_%'::text) AND (to_tsvector('english'::regconfig, (message)::text) @@ '''sapient'''::tsquery))
Rows Removed by Filter: 1088
-> Materialize (cost=0.98..17.04 rows=1 width=27) (actual time=0.001..0.001 rows=1 loops=100)
-> Nested Loop (cost=0.98..17.03 rows=1 width=27) (actual time=0.074..0.075 rows=1 loops=1)
-> Index Scan using channels_pkey on channels (cost=0.42..8.45 rows=1 width=27) (actual time=0.032..0.032 rows=1 loops=1)
Index Cond: ((id)::text = 'h1x9asxr7idjpqfmg8q67us49h'::text)
Filter: ((deleteat = 0) AND (((teamid)::text = '8ywxyw9ocp8smxrmjzrkqhrdwe'::text) OR ((teamid)::text = ''::text)))
-> Index Only Scan using idx_channelmembers_user_id_channel_id_last_viewed_at on channelmembers (cost=0.56..8.58 rows=1 width=27) (actual time=0.041..0.041 rows=1 loops=1)
Index Cond: ((userid = 'tc3p1yqw67d8idcp3g98awexqe'::text) AND (channelid = 'h1x9asxr7idjpqfmg8q67us49h'::text))
Heap Fetches: 1
SubPlan 1
-> Aggregate (cost=57.68..57.69 rows=1 width=8) (actual time=0.010..0.010 rows=1 loops=100)
-> Index Only Scan using idx_posts_root_id_delete_at on posts (cost=0.56..54.44 rows=1294 width=0) (actual time=0.009..0.009 rows=3 loops=100)
Index Cond: ((rootid = (CASE WHEN ((q2.rootid)::text = ''::text) THEN q2.id ELSE q2.rootid END)::text) AND (deleteat = 0))
Heap Fetches: 0
Planning Time: 0.401 ms
Execution Time: 31.466 ms
```
https://mattermost.atlassian.net/browse/MM-25554
```release-note
NONE
```
* MM-44642: Add a config key for Insights.
* MM-44642: Includes feature flag in config key/value conditional. Adds tests.
* MM-44642: Send the EnableCustomGroups and InsightsEnabled keys to unlicensed clients too.
* MM-44642: Adds some tests for Custom Groups config setting.
* if there is no trial end date, use the current date for the upgrade confirmation email
* changes
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Introduced search for channel id's
* Small fix to test case as it is not looking at count
* Introduced searching channels by id as a ChannelOpt. Defaulting to true for system console for the time being
* go fmt
* Modified whether to include search by id by passing it in as prop to search endpoint. Modified webapp to pass through search by id as well
* Fixed issue with full text search, as there is not always necessarily an index on Id.
* Modified test to verify count, and moved code inside conditional
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Improves groups membership telemetry.
* Removes unnecessary where clause (it's added by default).
* Cleans up data.
* Testing the relative group counts.
* Adds missing test mock.
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
