Граф коммитов

2803 Коммитов

Автор SHA1 Сообщение Дата
Kirill Krotov
0dbc74f6d5 [GH-18965] - Pass expire time from outside Cleanup method (#19008)
Pass expire time from outside Cleanup method

Fixes #18965
2021-12-22 14:57:30 +05:30
Allan Guwatudde
81409ee7c4 [MM-40407] - Do not show the Renew Now if the license id does not exist in the portal (#19188)
* [MM-40407] - Do not show the Renew Now if the license id does not exist in the portal

* improvements

* impl mock

* fix translations

* feedback impl

* fix typo

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-12-21 09:46:02 +03:00
Ben Schumacher
34c4e543f9 [MM-39999] Increase key length in plugin KV store to 150 (#19002) 2021-12-20 13:13:50 +01:00
Agniva De Sarker
8b155e34c7 Re-initialize plugins only when needed (#19158)
Previously, for every config change, we would re-initialize
plugins irrespective of whether there would be any actual
plugin related changed or not.

Now, we compute the difference between previous and new configs
and check whether there has been any changes to plugin related
configuration, and re-init plugins only if that's true.

This improves performance as plugin re-initialization is quite
costly.

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4/bed59298-27da-4b74-8be5-cdeb4c50c3e2
```release-note
NONE
```
2021-12-17 20:15:17 +05:30
Allan Guwatudde
0b1e9445c8 [MM-40682] - SIGSEGV after upgrade from 6.1.0 to 6.2.0 (#19192)
* [MM-40682] - SIGSEGV after upgrade from 6.1.0 to 6.2.0

* feedback impl
2021-12-17 09:39:10 +03:00
Allan Guwatudde
5aba3bb9e2 Improve gif processing (#19164)
* Improve gif processing

* feedback impl
2021-12-15 19:07:24 +03:00
Agniva De Sarker
0e00afd65f MM-40469: Handle invalid response codes from plugins (#19136)
* MM-40469: Handle invalid response codes from plugins

This was an interesting crash detected via Sentry.

Typically, any HTTP status code outside 100-999 range
will cause a crash in the HTTP server. And `ExecuteCommand`
is the only plugin hook which returns a model.AppError
instead of error.

So an incorrect plugin implementation could return
a status code of 0, and crash the server. We handle that
by rewriting any illegal response code to 500.

https://mattermost.atlassian.net/browse/MM-40469

```release-note
NONE
```

* added warning

```release-note
NONE
```
2021-12-09 14:17:41 +05:30
Christopher Speller
5ec78546e2 MM-38150 Adding playbooks permissions (#19096)
* Add playbooks related permissions.

* Add RolesGrantPermission to pluginapi

* Fixing scopes.

* New defaults.

* Fix defaults

* Fix tests.

* Fix migration.

* More test and migration fixes.

* Need to add everything to system admin too.

* Move to 63

* Feedback fixes.

* Fix system manager editing playbook permissions.
2021-12-07 09:43:33 -08:00
Agniva De Sarker
129f0aabd3 Remove the remaining fields from *app.Server (#19113)
We move these fields to Channels:
```
uploadLockMapMut sync.Mutex
uploadLockMap    map[string]bool
imgDecoder *imaging.Decoder
imgEncoder *imaging.Encoder
dndTaskMut sync.Mutex
dndTask    *model.ScheduledTask
```

I think this PR should conclue the initial phase
of migrating stuff from Server to Channels.

The remaining task would be to focus on continue
to create the remaining services from the
common things like users, teams, push notifications,
clustering for other products to consume.

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4/87df1e15-588e-49ff-8bd1-ffa9651b8c82

```release-note
NONE
```
2021-12-07 21:53:24 +05:30
Claudio Costa
6d361db638 [MM-40485] Enable receiving binary websocket messages (#19128)
* Enable receiving binary websocket messages

* Improve error message

* Prefer anonymous declaration

* Simplify

* Improve test

* Use MessagePack to clone WebSocketRequest struct

* Use short form

* Fix test
2021-12-07 15:24:18 +01:00
Agniva De Sarker
34db2e92cc Remove reliable websockets configuration (#19040)
The feature is stable enough to be removed
as a config knob.

For backwards compatiblity with older mobile
clients, we keep sending the config param
as true for client config requests.

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4/7m95frqbk9o8zxin6xo9c7iusoh

```release-note
NONE
```
2021-12-03 20:29:57 +05:30
Martin Kraft
7b28367928 MM-39433: Fixes (*App).getEmbedsAndImages parameter. (#19103) 2021-11-29 19:56:15 -05:00
Igor German
d3536087d7 Make getPrepackaged resilient to missing plugin icons (#18505)
Signed-off-by: Igor German <germanenak@gmail.com>

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-11-29 14:17:59 -04:00
Ben Cooke
27dacb82ce [MM-38239 & MM-39788] Recent files causing crash (#18942)
* wip

* adding tests for new endpoint

* tool updates

* new function for getting postsByIds

* fixing test

* adding limit of 1000 to post query

* fixing PR comments

* fixing permission logic

Co-authored-by: Collin <collineng@gmail.com>
Co-authored-by: Collin Eng <eng.engineereng@gmail.com>
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MBP.ht.home>
2021-11-26 11:51:32 -05:00
Agniva De Sarker
5ab3ad9bfd Move enterprise features under Channels (#19010)
* Move enterprise features under Channels

We move the EE features which are Channels related.

While here, we also move some code under *Server.Start()
from NewServer.

```release-note
NONE
```

* move saml and ldap back to server

```release-note
NONE
```

* fix test

```release-note
NONE
```

* try again

```release-note
NONE
```
2021-11-25 10:07:43 +05:30
Claudio Costa
3dea98ea4b Implement deep copy method for model.WebSocketEvent (#18977) 2021-11-22 09:54:19 +01:00
Ashish Bhate
0da249c651 [MM-37013] Async job to fix CRT channel unreads (#18340)
Summary
The addition of the TotalMsgCountRoot and MsgCountRoot columns to support CRT caused several issues with previously read threads and channels being marked as unread. Previously we attempted to fix this purely in a SQL migration [MM-35345][MM-35494] fixes for incorrect mentions and unreads for threads and channels #17803 but that turned out to be too heavy and it was decided to break up some of the fixes into async jobs.
This PR implements an async job to mark channels as read if there are no user posts since the last time the user viewed the channel. 

Ticket Link
https://mattermost.atlassian.net/browse/MM-37013
2021-11-18 15:31:18 +05:30
Agniva De Sarker
b7a7d8e7b6 MM-39612: Make acquiring and removing connections atomic (#18982)
* MM-39612: Make acquiring and removing connections atomic

The reconnect phase of a websocket was split into two parts:
one where we check if a connection with a given connectionID
exists or not. And second, where we remove that connection
and insert the new connection again in the index.

This would lead to a race where it would be possible
for 2 concurrent requests for the same connectionID to go through
which would lead to separate goroutines working on the same dead queue.

We simplify this by removing the connection from the index
in the check connection stage itself. And then just add that
during register phase.

And to distinguish between a fresh and an old connection, we add
a new field called reuseCount.

While here, we also cleanup some old comments and add more
in some places.

https://mattermost.atlassian.net/browse/MM-39612

```release-note
NONE
```

* remove unused method

```release-note
NONE
```

* race test

```release-note
NONE
```
2021-11-16 19:43:43 +05:30
Ben Schumacher
e8591537e0 [MM-39851] Add CRUD methods for user sessions to plugin API (#18958) 2021-11-16 15:08:20 +01:00
Allan Guwatudde
0ee2a3ca35 [MM-39060] - A/B Test: Reminder to Join Workspace email (#18894)
* [MM-39060] - A/B Test: Reminder to Join Workspace email

* Fix error and add error handling

* feedback impl

* run make i18n-extract

* improvements

* make i18n-extract

* setup ability to do telemetry on reminder invite emails

* improvement

* add telemetry

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-11-15 18:42:15 +03:00
Anurag Shivarathri
c603b1b989 fix (#18793) 2021-11-15 10:43:07 +05:30
Anurag Shivarathri
65c670a4c9 MM-36687 CRT push notifications (#18347)
* CRT changes

* Lint fix

* Moved postId check

* Moved postId check

* Lint fix

* Misc

* Added test case for badge count while clearing notifications

* Fixed count when CRT is on and added isCRTEnabled, teamId to isIdLoaded notifications

* test fix

* isCRTEnabledForUser capitalised

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-11-15 10:41:47 +05:30
Allan Guwatudde
e3240a2ddd [MM-40022] - Expose telemetry service in the server (#18976)
* [MM-40022] - Expose telemetry service in the server

* make app-layers

* feedback impl
2021-11-12 10:16:19 +03:00
Agniva De Sarker
a850704afe Move pluginCommands into Channels (#18974)
* Move pluginCommands into Channels

We move pluginCommands, pluginCommandsLock
into Channels.

We also move the plugin related route handlers
under Channels and move the init code under
NewChannels. To achieve this, the router initialization
is bumped up.

Along with it, we clean up some App methods
which were just wrappers over Channel methods.
Instead, we call the Channel method directly
to make things more readable and easy to understand.

```release-note
NONE
```

* fix tests

```release-note
NONE
```
2021-11-12 10:35:14 +05:30
Carlos Tadeu Panato Junior
bb5a74a42b [MM-34076] Add Plugin API User custom status (#17435)
* add missing client for custom user status

* add custom user status plugin api

* update based on feedback review

* add GetCustomStatus

* update interfaces
2021-11-10 11:11:20 -05:00
Agniva De Sarker
0a3e7f4d55 Ordering clarifications (#18964)
Automatic Merge
2021-11-09 10:15:15 +01:00
Berke Kalkan
e70c5a605a MM-32396 Create endpoint for listing all the roles (#16988)
* Create an endpoint for listing roles

* Add function to client model

* Create tests for listing roles endpoint

* Apply code review suggestions

* Restore GetAllRoles app method

* Use AppContext instead of App

* Minor fix

* Refactor according to new changes

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-11-08 16:38:41 +03:00
Agniva De Sarker
25257d6ef6 Move plugins under Channels (#18938)
* Move plugins under Channels

We move all plugin related fields under *Channels.
This essentially migrates several methods from being under
*Server to under *Channels.

We also move the plugin startup and shutdown code
to be under Channels.Start and Channels.Shutdown.

While here, we remove the getPluginPublicKeyFiles
method which was a one-line method which uselessly
returned an error.

Lastly, we fix the product initialization order which
was incorrect previously. Products are dependent on
the main server.

So startup should be products -> server.
And shutdown should be server -> products.

```release-note
NONE
```

* Added app layer

```release-note
NONE
```

* Incorporate suggestions

```release-note
NONE
```
2021-11-08 11:57:22 +05:30
Agniva De Sarker
3c3e1cb9e7 Remove DND run debug log (#18802)
This was being run every 5 minutes and unnecessarily clogs up
the logs. No other job makes a debug log before starting up.

```release-note
NONE
```
2021-11-05 09:27:19 +05:30
Agniva De Sarker
3bc151bbda Revert "Ordering clarifications (#18937)" (#18948)
Automatic Merge
2021-11-04 19:15:15 +01:00
Maria A Nunez
7e728d0b6f MM-39804 - Remove license expiration checks for cloud (#18921) 2021-11-04 12:26:28 -04:00
Agniva De Sarker
f921e69fe8 Ordering clarifications (#18937)
* Ordering clarifications

Clearly outline the dependency graph
during server bootup.

```release-note
NONE
```

* More clarifications

```release-note
NONE
```

* lint fix

```release-note
NONE
```
2021-11-04 19:22:23 +05:30
Martin Kraft
412e52894c MM-39420: Checks for presence of slice before index access. (#18826)
* MM-39420: Checks for presence of slice before index access.

* MM-39420: Sanitize all embeds with single method.

* MM-39420: Already checking len in method.
2021-11-04 09:41:19 -04:00
Agniva De Sarker
83b3090eb9 Fix PluginHealthCheckJob (#18927)
* Fix PluginHealthCheckJob

We were incorrectly calling to start PluginHealthCheckJob
without initializing the plugins environment.

To fix this, we move the code inside initPlugins right after
the environment is initialized.

To respond to config changes, we call it again from pluginsEnvironment.Shutdown
which gets called from ConfigListener when plugins are disabled. And initPlugins
is anyways called again from ConfigListener which plugins are enabled,
so we can avoid checking for PluginSettings.Enable again in the call.

We also rename the method to better indicate its nature.

During this, we also uncover and fix another bug where disabling
plugins would not shut down plugins at all because we were calling
s.GetPluginsEnvironment() directly which returns nil if plugins
were disabled. The approach we follow is to manually acquire the lock
whenever we need access to the struct ignoring config setting.
We fix that as well.

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4?c=6ef6178c-3512-4e57-8edd-1d2b66a09c9e
```release-note
NONE
```

* Fix test

```release-note
NONE
```
2021-11-04 08:54:03 +05:30
Agniva De Sarker
5ad7fb5b6f Remove job server dependency searchengine (#18908)
* Remove job server dependency searchengine

The job server field was not being used. We remove it
as part of a cleanup of server initialization refactor.

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4?c=8a171d5e-4398-4bc9-8fbf-8b9b14f37069

```release-note
NONE
```

* Fix missing field

```release-note
NONE
```
2021-11-03 18:44:57 +01:00
Agniva De Sarker
7a0131ee14 Use the correct error variable from provider.GetSSOSettings (#18920)
```release-note
NONE
```
2021-11-03 19:12:30 +05:30
Jesse Hallam
c1a27d8360 MM-39524: fix WebConn caching stale channel members (#18840)
When configured in a master/slave database environment, a read replica can sometimes return stale data to a `WebConn`, resulting in the user missing out on websocket events targetting that channel until the `WebConn` cache expires.

This is most easily reproducible by using Playbooks on community and starting a new run. The owner, or any automatically invited participants, typically find the websocket events dropped in that channel for up to 30 minutes, even after multiple page refreshes.

I've reproduced this locally, and while I've extended the unit tests, I note that they don't actually exercise this case given the need for a dedicated slave database during unit tests.

Fixes: https://mattermost.atlassian.net/browse/MM-39524
2021-10-28 11:08:09 -03:00
Doug Lauder
ceee3f4b5f re-config logger when config changes (#18860) 2021-10-28 08:06:34 -04:00
Kyriakos Z
4f2043bf97 MM-37934: supports channel override for CRT notifications (#18529)
* MM-37934: supports channel override for CRT notifications

Users with CRT 'ON' are enabled to override global CRT notification
settings per channel basis.

* Fixes brain freeze :o

* Fix push notification for CRT

* Some refactor and comments as per review comments

* Minor refactor

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-10-28 11:05:46 +03:00
Maria A Nunez
2ba20b5c82 MM-39591 - Restore custom message for guest invites (#18849) 2021-10-26 17:17:44 -04:00
Syerikjan Kh
1091710592 migrate from gorp to sqlx for Preference store (#18742)
Automatic Merge
2021-10-26 16:55:27 +02:00
Agniva De Sarker
e24f22745e [MM-18946] [MM-26721] [MM-6842] Cross team search+private channel autocomplete (#18468)
* Show private channels in autocomplete

This is supported in all Engines:
MySQL, Postgres, Bleve, Elasticsearch.

https://mattermost.atlassian.net/browse/MM-18496

```release-note
Private channels will now appear in channel autocomplete.
If you are using Bleve or ElasticSearch, you will have to reindex
the channels again to populate them with the new attributes.
```

A large chunk of this work has been based on the earlier
effort at https://github.com/mattermost/mattermost-server/pull/17804.

Full credit goes to https://github.com/arvinDarmawan.

* Add comment

```release-note
NONE
```

* Adding more tests

```release-note
NONE
```

* fix more tests

```release-note
NONE
```

* tmp

```release-note
NONE
```

* more fixes

```release-note
NONE
```

* add tests

```release-note
NONE
```

* Add review comments from previous PR

```release-note
NONE
```

* Add API to return all channels from all team

```release-note
NONE
```

* Added support for bleve and ES

```release-note
NONE
```

* Streaming response for GetAllChannels

```release-note
NONE
```

* Fix tests

```release-note
NONE
```

* Trigger CI
```release-note
NONE
```

* fix tests

```release-note
NONE
```

* Addressing review comments

```release-note
NONE
```

* Fix lint

```release-note
NONE
```

* Removing flaky test

```release-note
NONE
```

* Address comments

```release-note
NONE
```

* Trigger CI
```release-note
NONE
```

* Added /users/<userid>/channel_members endpoint

```release-note
NONE
```

* Minor edit

```release-note
NONE
```

* Improve embedding

```release-note
NONE
```

* Fix lint error

```release-note
NONE
```

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-10-26 11:30:59 +05:30
Agniva De Sarker
6bc6243131 Move some more atomic values under Channels (#18837)
* Move some more atomic values under Channels

The following fields were moved:

```
asymmetricSigningKey atomic.Value
clientConfig         atomic.Value
clientConfigHash     atomic.Value
limitedClientConfig  atomic.Value
```

And also moved the initialization order from NewServer
to Channels.Start to better reflect the order of things.

Removed AsymmetricSigningKey from ConfigService
as it was no longer used.

Removed calling regenerateClientConfig during startup explicitly
because it was anyways called from ensureAsymmetricSigningKey.

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4?c=87df1e15-588e-49ff-8bd1-ffa9651b8c82
```release-note
NONE
```

* Fix lint error

```release-note
NONE
```
2021-10-26 00:05:11 +05:30
Allan Guwatudde
246a0c297e [MM-39053] - A/B test for default notification settings (#18655)
* [MM-39053] - A/B test for default notification settings

* fix tests

* first name notifications on

* update

* Fix bad merge alignment

Co-authored-by: Maria A Nunez <maria.nunez@mattermost.com>
2021-10-21 14:00:25 +03:00
Joe
dea8610f1e [MM-35863] Add IsEnterpriseReady Plugin API (#18676)
Automatic Merge
2021-10-20 12:12:58 +02:00
Daniel Espino García
5928e4f9e0 Fix race condition in telemetry (#18734)
* Fix race condition in telemetry

* Fix tests
2021-10-20 09:58:25 +02:00
Agniva De Sarker
fdf1afadf3 Fix Flaky test UpdateUserEmail (#18750)
Since the email sending would run in a separate goroutine,
it was possible that during the second th.App.UpdateUser call
the goroutine would run much later.

So the secondToken would be the same as the firstToken,
and the test would fail.

The fix is to ensure that the token received is different
from the firstToken.

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4?c=254a9782-a59b-4d38-8886-4927f593e715

```release-note
NONE
```
2021-10-19 19:17:08 +05:30
Agniva De Sarker
2a07d4641a Remove global variables and fix racy test (#18733)
https://community.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4?c=65b44de0-1a9c-4643-84c1-5d803593f314

This PR improves on two things at once:
- It fixes the race condition by removing global variables.
- But instead of moving them to Server, it moves them inside Channels,
thereby making one more step towards the multi-product architecture.

```release-note
NONE
```
2021-10-18 19:38:16 +05:30
Agniva De Sarker
d949bd1638 Remove app initialization from WebHub (#18687)
- Make it a Server method.
- Pass Server instead of App.
- Remove global app instance from NewServer, rather
create local instances whenever needed.
- Remove App from Websocket router, and create dynamically
on every request.
- Remove HubStart and HubStop from App methods.
- Explicitly using s.Log instead of the global logger
to indicate dependency on Server. We could have passed the logger
explicitly but it doesn't look ideal.

```release-note
NONE
```
2021-10-18 12:36:13 +05:30
Allan Guwatudde
609fea0002 [MM-39205] Check that emails match during sign-up (#18635) 2021-10-18 09:34:10 +03:00