Граф коммитов

242 Коммитов

Автор SHA1 Сообщение Дата
Kirill Krotov
0dbc74f6d5 [GH-18965] - Pass expire time from outside Cleanup method (#19008)
Pass expire time from outside Cleanup method

Fixes #18965
2021-12-22 14:57:30 +05:30
Allan Guwatudde
81409ee7c4 [MM-40407] - Do not show the Renew Now if the license id does not exist in the portal (#19188)
* [MM-40407] - Do not show the Renew Now if the license id does not exist in the portal

* improvements

* impl mock

* fix translations

* feedback impl

* fix typo

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-12-21 09:46:02 +03:00
Agniva De Sarker
129f0aabd3 Remove the remaining fields from *app.Server (#19113)
We move these fields to Channels:
```
uploadLockMapMut sync.Mutex
uploadLockMap    map[string]bool
imgDecoder *imaging.Decoder
imgEncoder *imaging.Encoder
dndTaskMut sync.Mutex
dndTask    *model.ScheduledTask
```

I think this PR should conclue the initial phase
of migrating stuff from Server to Channels.

The remaining task would be to focus on continue
to create the remaining services from the
common things like users, teams, push notifications,
clustering for other products to consume.

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4/87df1e15-588e-49ff-8bd1-ffa9651b8c82

```release-note
NONE
```
2021-12-07 21:53:24 +05:30
Agniva De Sarker
5ab3ad9bfd Move enterprise features under Channels (#19010)
* Move enterprise features under Channels

We move the EE features which are Channels related.

While here, we also move some code under *Server.Start()
from NewServer.

```release-note
NONE
```

* move saml and ldap back to server

```release-note
NONE
```

* fix test

```release-note
NONE
```

* try again

```release-note
NONE
```
2021-11-25 10:07:43 +05:30
Ashish Bhate
0da249c651 [MM-37013] Async job to fix CRT channel unreads (#18340)
Summary
The addition of the TotalMsgCountRoot and MsgCountRoot columns to support CRT caused several issues with previously read threads and channels being marked as unread. Previously we attempted to fix this purely in a SQL migration [MM-35345][MM-35494] fixes for incorrect mentions and unreads for threads and channels #17803 but that turned out to be too heavy and it was decided to break up some of the fixes into async jobs.
This PR implements an async job to mark channels as read if there are no user posts since the last time the user viewed the channel. 

Ticket Link
https://mattermost.atlassian.net/browse/MM-37013
2021-11-18 15:31:18 +05:30
Agniva De Sarker
a850704afe Move pluginCommands into Channels (#18974)
* Move pluginCommands into Channels

We move pluginCommands, pluginCommandsLock
into Channels.

We also move the plugin related route handlers
under Channels and move the init code under
NewChannels. To achieve this, the router initialization
is bumped up.

Along with it, we clean up some App methods
which were just wrappers over Channel methods.
Instead, we call the Channel method directly
to make things more readable and easy to understand.

```release-note
NONE
```

* fix tests

```release-note
NONE
```
2021-11-12 10:35:14 +05:30
Agniva De Sarker
0a3e7f4d55 Ordering clarifications (#18964)
Automatic Merge
2021-11-09 10:15:15 +01:00
Agniva De Sarker
25257d6ef6 Move plugins under Channels (#18938)
* Move plugins under Channels

We move all plugin related fields under *Channels.
This essentially migrates several methods from being under
*Server to under *Channels.

We also move the plugin startup and shutdown code
to be under Channels.Start and Channels.Shutdown.

While here, we remove the getPluginPublicKeyFiles
method which was a one-line method which uselessly
returned an error.

Lastly, we fix the product initialization order which
was incorrect previously. Products are dependent on
the main server.

So startup should be products -> server.
And shutdown should be server -> products.

```release-note
NONE
```

* Added app layer

```release-note
NONE
```

* Incorporate suggestions

```release-note
NONE
```
2021-11-08 11:57:22 +05:30
Agniva De Sarker
3bc151bbda Revert "Ordering clarifications (#18937)" (#18948)
Automatic Merge
2021-11-04 19:15:15 +01:00
Maria A Nunez
7e728d0b6f MM-39804 - Remove license expiration checks for cloud (#18921) 2021-11-04 12:26:28 -04:00
Agniva De Sarker
f921e69fe8 Ordering clarifications (#18937)
* Ordering clarifications

Clearly outline the dependency graph
during server bootup.

```release-note
NONE
```

* More clarifications

```release-note
NONE
```

* lint fix

```release-note
NONE
```
2021-11-04 19:22:23 +05:30
Agniva De Sarker
83b3090eb9 Fix PluginHealthCheckJob (#18927)
* Fix PluginHealthCheckJob

We were incorrectly calling to start PluginHealthCheckJob
without initializing the plugins environment.

To fix this, we move the code inside initPlugins right after
the environment is initialized.

To respond to config changes, we call it again from pluginsEnvironment.Shutdown
which gets called from ConfigListener when plugins are disabled. And initPlugins
is anyways called again from ConfigListener which plugins are enabled,
so we can avoid checking for PluginSettings.Enable again in the call.

We also rename the method to better indicate its nature.

During this, we also uncover and fix another bug where disabling
plugins would not shut down plugins at all because we were calling
s.GetPluginsEnvironment() directly which returns nil if plugins
were disabled. The approach we follow is to manually acquire the lock
whenever we need access to the struct ignoring config setting.
We fix that as well.

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4?c=6ef6178c-3512-4e57-8edd-1d2b66a09c9e
```release-note
NONE
```

* Fix test

```release-note
NONE
```
2021-11-04 08:54:03 +05:30
Agniva De Sarker
5ad7fb5b6f Remove job server dependency searchengine (#18908)
* Remove job server dependency searchengine

The job server field was not being used. We remove it
as part of a cleanup of server initialization refactor.

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4?c=8a171d5e-4398-4bc9-8fbf-8b9b14f37069

```release-note
NONE
```

* Fix missing field

```release-note
NONE
```
2021-11-03 18:44:57 +01:00
Doug Lauder
ceee3f4b5f re-config logger when config changes (#18860) 2021-10-28 08:06:34 -04:00
Agniva De Sarker
6bc6243131 Move some more atomic values under Channels (#18837)
* Move some more atomic values under Channels

The following fields were moved:

```
asymmetricSigningKey atomic.Value
clientConfig         atomic.Value
clientConfigHash     atomic.Value
limitedClientConfig  atomic.Value
```

And also moved the initialization order from NewServer
to Channels.Start to better reflect the order of things.

Removed AsymmetricSigningKey from ConfigService
as it was no longer used.

Removed calling regenerateClientConfig during startup explicitly
because it was anyways called from ensureAsymmetricSigningKey.

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4?c=87df1e15-588e-49ff-8bd1-ffa9651b8c82
```release-note
NONE
```

* Fix lint error

```release-note
NONE
```
2021-10-26 00:05:11 +05:30
Agniva De Sarker
d949bd1638 Remove app initialization from WebHub (#18687)
- Make it a Server method.
- Pass Server instead of App.
- Remove global app instance from NewServer, rather
create local instances whenever needed.
- Remove App from Websocket router, and create dynamically
on every request.
- Remove HubStart and HubStop from App methods.
- Explicitly using s.Log instead of the global logger
to indicate dependency on Server. We could have passed the logger
explicitly but it doesn't look ideal.

```release-note
NONE
```
2021-10-18 12:36:13 +05:30
Agniva De Sarker
c27814393d Improve API4 initialization (#18680)
* Improve API4 initialization

- Refactored openGraphDataCache to be inside app layer.
- Moved the cache instance from global variable to be inside server.
- Moved out the app instantiation from the global commands package
to be instantiated on every call. Only the server instance is passed.
- Moved InitLocal to be called from inside Init.

```release-note
NONE
```

* Remove commented line

```release-note
NONE
```
2021-10-15 19:57:05 +05:30
Agniva De Sarker
43a99a5783 Move cloud init back to initEnterprise (#18668)
* Move cloud init back to initEnterprise

```release-note
NONE
```

* Remove debug lines

```release-note
NONE
```

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-10-15 19:29:06 +05:30
Agniva De Sarker
99e6039472 Add functionality to cleanup old jobs (#18646)
* Add functionality to cleanup old jobs

Historically, we never cleaned up old jobs from the DB
leading to them being accumulated forever.

This PR introduces functionality to cleanup old jobs
older than a defined threshold.

The functionality is set to false by default and has
to be enabled for it to work.

```release-note
2 new config settings were added.

JobSettings.CleanupOldJobs: This indicates whether to clean up old jobs
from the DB or not. Default is false.

JobSettings.CleanupJobsThresholdHours: This defines the time gap in hours beyond
which older jobs will be removed. This has no effect if the above config
setting is set to false. Default is -1
```

* fix copy pasta

```release-note
NONE
```

* address review comments

```release-note
NONE
```

* Fix lint

```release-note
NONE
```

* Use single config option for everything

```release-note
NONE
```

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-10-15 11:03:54 +05:30
darkLord19
98284a79eb remove timedDND feature flag (#18346) 2021-10-14 11:54:09 -04:00
Agniva De Sarker
850fef1ad1 Move ImageProxy inside Channels (#18656)
* Move ImageProxy inside Channels

```release-note
NONE
```

* Move back httpService

```release-note
NONE
```

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-10-14 13:57:04 +05:30
Ibrahim Serdar Acikgoz
be203d9bf3 Initial team service implementation (#18279)
* initial service implementation

* reflect review comments
2021-10-13 15:20:40 +03:00
Agniva De Sarker
f0ecdcc5f5 Move Channels into App (#18623)
* Move Channels into App

In this PR, we make Channels as part of App
instead of Server. This is part of the transition period
of moving fields from Server to Channels.

For now, Channels contains Server. So the hierarchy is

App -> Channels -> Server.

And as a first step, we also move httpService to Channels.

```release-note
NONE
```

* Fixing another test

```release-note
NONE
```

* new method

```release-note
NONE
```
2021-10-12 11:39:49 +05:30
Agniva De Sarker
561bc968c8 Creating empty Channels shell (#18611)
* Creating empty Channels shell

This is the first step in moving to a product-based
isolated architecture.

For now, Channels is empty and does not contain anything.

Next step is to change App to contain Channels instead
of Server. Some of the initialization code in NewServer
would need to be moved inside NewChannels.

This would complete the full pass-through mode of accessing
everything.

The last step would be to gradually move Channels related
fields from Server into Channels, keeping Server to be
just the global level struct.

```release-note
NONE
```

* fix vet failure

```release-note
NONE
```
2021-10-11 19:24:50 +05:30
Claudio Costa
e542842bd1 Disable admin support status job on server startup (#18341) 2021-09-14 09:02:08 +02:00
Doug Lauder
bb659d03fd Fix plugin logging missing context fields
Fix an issue where context fields logged by server on behalf of plugins didn't contain correct name/value pairs:
- bump Logr version to add key/value methods to sugar logger
- expose factory args when configuring logging with custom target types (needed for FocalBoard to create log target adapter that converts typed fields into slices of interface{} as per plugin logging API)
2021-09-13 10:12:18 -04:00
Ibrahim Serdar Acikgoz
b66fd1f89e move users service to app directory (#18257) 2021-08-24 10:10:40 +03:00
Doug Lauder
a4507327a7 MM-36764 mlog refactor (#18118)
Refactor mlog
- simplify mlog by removing redundant code
- remove Zap dependency
- update unit test helpers
- update logging config
- update auditing
2021-08-17 16:08:04 -04:00
Ben Schumacher
757dc96461 [MM-37772] Idiomatic naming (URL, URI, API) (#18128)
* s/Url/URL/g & s/Uri/URI/g

* s/Api/API/g
2021-08-16 19:46:44 +02:00
Agniva De Sarker
d1b164ab1f Improve session cleanup (#18123)
- We move logging statements to the upper layer.
Store functions are low-level methods and should return error
upwards rather than logging.
- Used IN instead of any (array ()) which is equivalent.
- Made the delay to be of type time.Duration and un-exported it.
- Unexported the batch size constant.

```release-note
NONE
```
2021-08-13 20:15:24 +05:30
Ben Schumacher
fd853e74a6 [MM-37755] Idiomatic naming (SMTP, TLS, TCP, XML, CSS, HTML, HTTP) (#18103) 2021-08-12 11:49:16 +02:00
Claudio Costa
ca755fdc34 [MM-37496] Temporarily disable admin advisor notifications (#18054)
Automatic Merge
2021-08-09 14:45:03 +02:00
Agniva De Sarker
c281d5eac6 MM-35754: Fail fast if templates directory is not found (#18053)
We used to log an error and move forward, which led to
panics later during server execution, because the templates
don't exist.

While technically speaking the server will run fine
because it's just one HTTP handler goroutine which panics,
but properly handling the error requires much more adjustments
in the code to check for nil templates.

It is better to fail earlier and force the application
to start in a clean slate.

https://mattermost.atlassian.net/browse/MM-37574

```release-note
NONE
```
2021-08-04 19:42:28 +05:30
Claudio Costa
a5463c8651 Implement content extraction job (#18022) 2021-08-04 11:10:53 +02:00
Claudio Costa
186475db3b [MM-35470] Disable config watching logic (#17913)
* Disable config watching logic

* Fix tests
2021-07-26 20:13:30 +02:00
Agniva De Sarker
c4b4e1bc38 MM-36271: Bump major version to 6.0 (#17973)
https://mattermost.atlassian.net/browse/MM-36271

```release-note
We bump the major version to 6.0
```
2021-07-22 12:21:47 +05:30
Claudio Costa
025a948905 Remove override of MaxNotificationsPerChannel (#17939) 2021-07-21 16:38:39 +02:00
Ben Schumacher
97ccf0bdf6 [MM-16751] golint model (#17896) 2021-07-21 16:38:39 +02:00
Ibrahim Serdar Acikgoz
41dc05a6bd Decouple emailservice from app package (#17827)
* decouple emailservice from app package

* fix some escaped errors

* move email package under app directory

* fix i18n

* reflect review comments
2021-07-19 23:26:06 +08:00
Allan Guwatudde
98d170894d [MM-36457] - Send email when license is up for renewal (#17816)
* [MM-36457] - Send email when license is up for renewal

* i18n-extract

* feedback impl

* feedback impl

* fix translations

* Change license key to test for testing purposes

* Revert license key

* Trying test license key once more

* revert license change

Co-authored-by: marianunez <maria.nunez@mattermost.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-07-06 18:50:19 +03:00
Ibrahim Serdar Acikgoz
f69b28610a user service: add oauth and profile images (#17784)
* users: add cache to service

* reflect review comments

* add oauth

* move profile picture genaration

* reflect review comments

* move default bot image to users package

* add missing wraps and apply suggestions

* add comment for app.GetSession
2021-06-18 19:25:03 +03:00
Harshil Sharma
e4aa729a0c MM-34437 Preventing infinite trial requests (#17472)
* MM-34434: Added 'is_trial' boolean to all trial license requests and to the License struct.

* MM-34434: Generalized the concept of a license request.

* MM-34434: Verifies JSON field of license instance is set.

* MM-34434: Added missing client param.

* MM-34434: Added some tests of the request trial API endpoint.

* MM-34434: Removed comment.

* fix broken test (#17348)

* Add missing wrapped errors (#17339)

* Improve document extraction and including a document extraction command (#17183)

* Add extract documents content command

* Adding the extraction command and making the pure go pdf library as secondary option

* Improving the memory usage and docextractor interface

* Enable content extraction by default in all the instances

* Tiny improvement on archive indexing

* Adding App interface generation and the opentracing layer

* Fixing linter errors

* Addressing PR review comments

* Addressing PR review comments

* Update en.json (#17356)

Automatic Merge

* adding new feature flag (#17308)

Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>

* Bump no_output_timeout to 2 hours (#17358)

* log invalid username (#17345)

Automatic Merge

* MM-34434: Added missing client param.

MM-34434: Added some tests of the request trial API endpoint.

MM-34434: Removed comment.

* MM-34434: Switched to a hard-coded true value.

* MM-34434: Reverts test change.

* MM-34434: Removes unnecessary field.

* MM-34434: Tests that is_trial is hard-coded by TrialLicenseRequest.

* MM-34434: Removed accidental commit.

* MM-34434: Removes unnecessary is_trial key from JSON payload.

* MM-34434: Reverts to old pointer receiver variable name.

* MM-34434: Removes test.

* #MM-34437 Initialized license service

* ##MM-34437 Verified at all points if server is trial elligible

* WIp

* #MM-34437 removed unused commented code

* MM-34437 make a log less severe

* #MM-34437 generated einterface mocks

* #MM-34437 added license on new file

* #MM-34437 removed unused translation

* #MM-34437 some refactoring

* Update api4/license.go

* Update api4/license.go

* #MM-34437 made a variable name consistent

* #MM-34437 Added mocks for lince validator

* #M--34437 Added license validator test framework

* #MM-34437 Renamed isTrial method to isTrialLicense to avoid conflict with newlya dded field

* #M--34437 Allowed sales-sanctioned trials

* #MM-34437 fixed trial license API tests

* Added tests for add license API

* #MM-34437 fixed ValidateLicense test

* #MM-34437 Added util tests

* #MM-34437 using NoError for checking no error

* #MM-34437 using NoError for checking no error

* Added dummy piblic key for testing

* Fixed tests

* #MM-34437 udpaetd trial license URL for testing

* #MM-34437 adjusted times for licences generated through admin portal

* Reverted test-only changes

Co-authored-by: Martin Kraft <martin@upspin.org>
Co-authored-by: Hossein <hahmadia@users.noreply.github.com>
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
Co-authored-by: Jesús Espino <jespinog@gmail.com>
Co-authored-by: Amy Blais <amy_blais@hotmail.com>
Co-authored-by: Ben Cooke <benkcooke@gmail.com>
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Co-authored-by: Max Erenberg <max.erenberg@mattermost.com>
2021-06-17 17:37:34 +05:30
darkLord19
b0bdd23d2c [MM-8497] Ability to set Do Not Disturb for a specified period of time (#17680)
* Revert "Revert "[MM-8497] Ability to set Do Not Disturb for a specified period of time (#16067)" (#17657)"

This reverts commit ff383990f8.

* add debug log for recurring function

* add feature flag for dnd timed status

* refactoring changes

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-06-16 14:38:26 -04:00
Ibrahim Serdar Acikgoz
24fb0033f4 users service: add sessions (#17744)
* users: add cache to service

* reflect review comments
2021-06-14 18:08:00 +03:00
Agniva De Sarker
723902f4cb MM-35030: Hoist GetDBVersion before creating store (#17764)
* MM-35030: Hoist GetDBVersion before creating store

The creation of the store included running the migrations
which means that SQL statements would be executed before
we could make the check for DB version.

We perform the DB check before running the migrations.

https://mattermost.atlassian.net/browse/MM-35030

```release-note
NONE
```

* fix lint error
2021-06-14 15:00:18 +05:30
Claudio Costa
39c3b8ebf9 [MM-29241] Image logic refactor (#17702)
* Image logic refactor

* Add missing translations

* Improve prepareImage

* Use iota

* Limit image encoder concurrency

* Unexport validation methods

* Avoid shortening on exported names

* Remove unnecessary complexity
2021-06-05 17:08:29 +08:00
Ibrahim Serdar Acikgoz
ac3bb2e811 user service: initial implementation (#17668)
* conceptual user service: initial commit

* reflect review comments

* fix i18n issues and some tests

* implement get user methods

* add license

* reflect review comments
2021-06-01 14:52:55 +03:00
Agniva De Sarker
a4f7df6f6e Remove all remaining occurences of fakeApp (#17661)
* Remove all remaining occurences of fakeApp

Migrated any remaining methods under server.

And for everything else, renamed fakeApp to app.
The word fakeApp is confusing and we should just call
it for what it is - an app.

https://focalboard-community.octo.mattermost.com/workspace/zyoahc9uapdn3xdptac6jb69ic?id=285b80a3-257d-41f6-8cf4-ed80ca9d92e5&v=495cdb4d-c13a-4992-8eb9-80cfee2819a4&c=639a0bc1-4401-43d5-81ec-0dd54e796d9a

```release-note
NONE
```

* fix tests
2021-05-24 10:24:51 +05:30
Jesse Hallam
ff383990f8 Revert "[MM-8497] Ability to set Do Not Disturb for a specified period of time (#16067)" (#17657)
This reverts commit 77d42568f9.
2021-05-21 09:38:27 -03:00
Doug Lauder
2b02b03497 MM-35133 trigger sync user immediately after change (#17579)
- ensure changes to user profile sync immediately
- refactor sync send
2021-05-20 12:07:40 -04:00