Граф коммитов

213 Коммитов

Автор SHA1 Сообщение Дата
Agniva De Sarker
d1b164ab1f Improve session cleanup (#18123)
- We move logging statements to the upper layer.
Store functions are low-level methods and should return error
upwards rather than logging.
- Used IN instead of any (array ()) which is equivalent.
- Made the delay to be of type time.Duration and un-exported it.
- Unexported the batch size constant.

```release-note
NONE
```
2021-08-13 20:15:24 +05:30
Ben Schumacher
fd853e74a6 [MM-37755] Idiomatic naming (SMTP, TLS, TCP, XML, CSS, HTML, HTTP) (#18103) 2021-08-12 11:49:16 +02:00
Claudio Costa
ca755fdc34 [MM-37496] Temporarily disable admin advisor notifications (#18054)
Automatic Merge
2021-08-09 14:45:03 +02:00
Agniva De Sarker
c281d5eac6 MM-35754: Fail fast if templates directory is not found (#18053)
We used to log an error and move forward, which led to
panics later during server execution, because the templates
don't exist.

While technically speaking the server will run fine
because it's just one HTTP handler goroutine which panics,
but properly handling the error requires much more adjustments
in the code to check for nil templates.

It is better to fail earlier and force the application
to start in a clean slate.

https://mattermost.atlassian.net/browse/MM-37574

```release-note
NONE
```
2021-08-04 19:42:28 +05:30
Claudio Costa
a5463c8651 Implement content extraction job (#18022) 2021-08-04 11:10:53 +02:00
Claudio Costa
186475db3b [MM-35470] Disable config watching logic (#17913)
* Disable config watching logic

* Fix tests
2021-07-26 20:13:30 +02:00
Agniva De Sarker
c4b4e1bc38 MM-36271: Bump major version to 6.0 (#17973)
https://mattermost.atlassian.net/browse/MM-36271

```release-note
We bump the major version to 6.0
```
2021-07-22 12:21:47 +05:30
Claudio Costa
025a948905 Remove override of MaxNotificationsPerChannel (#17939) 2021-07-21 16:38:39 +02:00
Ben Schumacher
97ccf0bdf6 [MM-16751] golint model (#17896) 2021-07-21 16:38:39 +02:00
Ibrahim Serdar Acikgoz
41dc05a6bd Decouple emailservice from app package (#17827)
* decouple emailservice from app package

* fix some escaped errors

* move email package under app directory

* fix i18n

* reflect review comments
2021-07-19 23:26:06 +08:00
Allan Guwatudde
98d170894d [MM-36457] - Send email when license is up for renewal (#17816)
* [MM-36457] - Send email when license is up for renewal

* i18n-extract

* feedback impl

* feedback impl

* fix translations

* Change license key to test for testing purposes

* Revert license key

* Trying test license key once more

* revert license change

Co-authored-by: marianunez <maria.nunez@mattermost.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-07-06 18:50:19 +03:00
Ibrahim Serdar Acikgoz
f69b28610a user service: add oauth and profile images (#17784)
* users: add cache to service

* reflect review comments

* add oauth

* move profile picture genaration

* reflect review comments

* move default bot image to users package

* add missing wraps and apply suggestions

* add comment for app.GetSession
2021-06-18 19:25:03 +03:00
Harshil Sharma
e4aa729a0c MM-34437 Preventing infinite trial requests (#17472)
* MM-34434: Added 'is_trial' boolean to all trial license requests and to the License struct.

* MM-34434: Generalized the concept of a license request.

* MM-34434: Verifies JSON field of license instance is set.

* MM-34434: Added missing client param.

* MM-34434: Added some tests of the request trial API endpoint.

* MM-34434: Removed comment.

* fix broken test (#17348)

* Add missing wrapped errors (#17339)

* Improve document extraction and including a document extraction command (#17183)

* Add extract documents content command

* Adding the extraction command and making the pure go pdf library as secondary option

* Improving the memory usage and docextractor interface

* Enable content extraction by default in all the instances

* Tiny improvement on archive indexing

* Adding App interface generation and the opentracing layer

* Fixing linter errors

* Addressing PR review comments

* Addressing PR review comments

* Update en.json (#17356)

Automatic Merge

* adding new feature flag (#17308)

Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>

* Bump no_output_timeout to 2 hours (#17358)

* log invalid username (#17345)

Automatic Merge

* MM-34434: Added missing client param.

MM-34434: Added some tests of the request trial API endpoint.

MM-34434: Removed comment.

* MM-34434: Switched to a hard-coded true value.

* MM-34434: Reverts test change.

* MM-34434: Removes unnecessary field.

* MM-34434: Tests that is_trial is hard-coded by TrialLicenseRequest.

* MM-34434: Removed accidental commit.

* MM-34434: Removes unnecessary is_trial key from JSON payload.

* MM-34434: Reverts to old pointer receiver variable name.

* MM-34434: Removes test.

* #MM-34437 Initialized license service

* ##MM-34437 Verified at all points if server is trial elligible

* WIp

* #MM-34437 removed unused commented code

* MM-34437 make a log less severe

* #MM-34437 generated einterface mocks

* #MM-34437 added license on new file

* #MM-34437 removed unused translation

* #MM-34437 some refactoring

* Update api4/license.go

* Update api4/license.go

* #MM-34437 made a variable name consistent

* #MM-34437 Added mocks for lince validator

* #M--34437 Added license validator test framework

* #MM-34437 Renamed isTrial method to isTrialLicense to avoid conflict with newlya dded field

* #M--34437 Allowed sales-sanctioned trials

* #MM-34437 fixed trial license API tests

* Added tests for add license API

* #MM-34437 fixed ValidateLicense test

* #MM-34437 Added util tests

* #MM-34437 using NoError for checking no error

* #MM-34437 using NoError for checking no error

* Added dummy piblic key for testing

* Fixed tests

* #MM-34437 udpaetd trial license URL for testing

* #MM-34437 adjusted times for licences generated through admin portal

* Reverted test-only changes

Co-authored-by: Martin Kraft <martin@upspin.org>
Co-authored-by: Hossein <hahmadia@users.noreply.github.com>
Co-authored-by: Ben Schumacher <ben.schumacher@mattermost.com>
Co-authored-by: Jesús Espino <jespinog@gmail.com>
Co-authored-by: Amy Blais <amy_blais@hotmail.com>
Co-authored-by: Ben Cooke <benkcooke@gmail.com>
Co-authored-by: Benjamin Cooke <benjamincooke@Benjamins-MacBook-Pro.local>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Co-authored-by: Max Erenberg <max.erenberg@mattermost.com>
2021-06-17 17:37:34 +05:30
darkLord19
b0bdd23d2c [MM-8497] Ability to set Do Not Disturb for a specified period of time (#17680)
* Revert "Revert "[MM-8497] Ability to set Do Not Disturb for a specified period of time (#16067)" (#17657)"

This reverts commit ff383990f8.

* add debug log for recurring function

* add feature flag for dnd timed status

* refactoring changes

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-06-16 14:38:26 -04:00
Ibrahim Serdar Acikgoz
24fb0033f4 users service: add sessions (#17744)
* users: add cache to service

* reflect review comments
2021-06-14 18:08:00 +03:00
Agniva De Sarker
723902f4cb MM-35030: Hoist GetDBVersion before creating store (#17764)
* MM-35030: Hoist GetDBVersion before creating store

The creation of the store included running the migrations
which means that SQL statements would be executed before
we could make the check for DB version.

We perform the DB check before running the migrations.

https://mattermost.atlassian.net/browse/MM-35030

```release-note
NONE
```

* fix lint error
2021-06-14 15:00:18 +05:30
Claudio Costa
39c3b8ebf9 [MM-29241] Image logic refactor (#17702)
* Image logic refactor

* Add missing translations

* Improve prepareImage

* Use iota

* Limit image encoder concurrency

* Unexport validation methods

* Avoid shortening on exported names

* Remove unnecessary complexity
2021-06-05 17:08:29 +08:00
Ibrahim Serdar Acikgoz
ac3bb2e811 user service: initial implementation (#17668)
* conceptual user service: initial commit

* reflect review comments

* fix i18n issues and some tests

* implement get user methods

* add license

* reflect review comments
2021-06-01 14:52:55 +03:00
Agniva De Sarker
a4f7df6f6e Remove all remaining occurences of fakeApp (#17661)
* Remove all remaining occurences of fakeApp

Migrated any remaining methods under server.

And for everything else, renamed fakeApp to app.
The word fakeApp is confusing and we should just call
it for what it is - an app.

https://focalboard-community.octo.mattermost.com/workspace/zyoahc9uapdn3xdptac6jb69ic?id=285b80a3-257d-41f6-8cf4-ed80ca9d92e5&v=495cdb4d-c13a-4992-8eb9-80cfee2819a4&c=639a0bc1-4401-43d5-81ec-0dd54e796d9a

```release-note
NONE
```

* fix tests
2021-05-24 10:24:51 +05:30
Jesse Hallam
ff383990f8 Revert "[MM-8497] Ability to set Do Not Disturb for a specified period of time (#16067)" (#17657)
This reverts commit 77d42568f9.
2021-05-21 09:38:27 -03:00
Doug Lauder
2b02b03497 MM-35133 trigger sync user immediately after change (#17579)
- ensure changes to user profile sync immediately
- refactor sync send
2021-05-20 12:07:40 -04:00
Claudio Costa
3681cd3688 [MM-32390] Config logic refactor (#17578)
* Replace config generator

* Cleanup

* Some renaming and docs additions to add clarity

* Cleanup logging related methods

* Cleanup emitter

* Fix TestDefaultsGenerator

* Move feature flags synchronization logic out of config package

* Remove unnecessary util functions

* Simplify load/set logic

* Refine semantics and add some test to cover them

* Remove unnecessary deep copies

* Improve logic further

* Fix license header

* Review file store tests

* Fix test

* Fix test

* Avoid additional write during initialization

* More consistent naming

* Update app/feature_flags.go

Co-authored-by: Christopher Speller <crspeller@gmail.com>

* Update config/store.go

Co-authored-by: Christopher Speller <crspeller@gmail.com>

* Update config/store.go

Co-authored-by: Christopher Speller <crspeller@gmail.com>

* Update config/store.go

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>

* Move FF synchronizer to its own package

* Remove unidiomatic use of sync.Once

* Add some comments

* Rename function

* More comment

Co-authored-by: Christopher Speller <crspeller@gmail.com>
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
2021-05-19 13:30:26 +02:00
Claudio Costa
7bb23323d1 [MM-35424] Fix job schedulers server from missing leader change event (#17602)
* Fix cluster leader change message potentially getting lost

* Start jobs and schedulers earlier in server initialization

* Fix possible deadlock

* Update jobs/schedulers.go

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
2021-05-18 20:33:09 +02:00
Ibrahim Serdar Acikgoz
5ea06e51d0 Migrate to stateless app.App (#17542)
* add request context

* move initialialization to server

* use app interface instead of global app functions

* remove app context from webconn

* cleanup

* remove duplicated services

* move context to separate package

* remove finalize init method and move content to NewServer function

* restart workers and schedulers after adding license for tests

* reflect review comments

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-05-11 13:00:44 +03:00
Jyoti Patel
d97daaa1de [GH-15906][MM-22844] Redesign message notification emails. (#17184)
* Redesign message notification emails.

* Fix tests and linter.

* Fix tests

* Fix tests

* gofmt.

* Fix date separator

* Update html files

* Remove date for message notification email.

* Modify subtitle for mentions and direct and group messages.

* Fix lint error

* Fix DM subtitle.

* Fixing translations

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Jesús Espino <jespinog@gmail.com>
2021-05-10 17:50:44 +02:00
darkLord19
77d42568f9 [MM-8497] Ability to set Do Not Disturb for a specified period of time (#16067)
* Add support for timed DND status

- accept a date time value in api query when dnd mode for user needs to be unset
- Create a new function to handle SetDNDStatus calls
- Create a scheduled task to unset dnd mode to wahtever mode was before setting it to DND

* update schema version

* Model changes to make fields more intuitive

- move dndendtime to status model
- add new field prev status in status to keep track of previous status of user
- update db migration function
- make use of prevstatus and dndendtime from status model

* set prev status and dndendtime appropriately after unsetting dnd mode

* add json tag for dndendtime

* unset dnd status only if not changed manually by user

* update dnd statuses after server restart

* make app-layers

* fix failing tests

* don't create sched task when setting status to DND

* get only expired statuses from db

- convert end time from any timezone to utc
- store dnd end time in unix format for usability reasons

* run update dnd status only on leader

* make mocks

* fix tests

* run UpdateDNDStatusOfUsers as recurring task

* save all statuses at once in db and update UpdateDNDStatusOfUsers logic

* add app method to get timezone of user

* store dnd end time in context.Params

* set max size of prevstatus

* update status model to take endtime input as string and store in db as unix time(int64)

* Add tests for SetStatusDoNotDisturbTimed

* if dnd_end_time is not passed the call old api to set dnd mode

* fix tests

* new plugin api to use new timed dnd mode

* get and update rows in a single db query

* dnd end time will be stored in request body and not route param

* exclude statuses which has dndendtimeunix < 0

* update and get the updated dnd statuses in single db query

* add updated status to cache

* DNDEndTimeUnix and PrevStatus need not to be visible to users

* update db schema version for migration

* Keep Status and PrevStatus varchar size same

* add test to verify status is restored after dnd end time expires

* expect endtime in utc from client

- remove store method GetTimezone as no longer needed
- add documentation for SetStatusDoNotDisturbTimed

* reduce sleep time for dnd timed restore test

* more appropriate name for new api to update user status

* update db migration function

* parse and validate time before potentially triggering db query to get status of user

* add migration changes in to existing upgrade function

* not supporting un-timed dnd status via api

* don't call Srv.Store directly, call via app layer

* rename dndendtime to statuscleartime to make it suitable for custom status usage as well

* Revert "rename dndendtime to statuscleartime to make it suitable for custom status usage as well"

This reverts commit fa69152d9a3db18f1c59b34c878fb7ce494440b5.

* mysql doesn't support RETURNING clause so add tx to get and update statuses

* add UpdateDNDStatusOfUsers mock in tests

* update store mock import path

* add mock in storelib

* Add status mocks to empty store

* Close the task during server shutdown

* Do not cancel a nil task

* update squirrel queries

* remove untimed dnd test

* start recurring task to unset statuses on leadership change

* set dndTask to nil after cancelling it upon server shutdown

* new recurring task which starts at nearest rounded time of the interval

* mock Get() call for status

* return updated statuses in case of mysql

* remove unneccessary code

* add Get() mock to empty store

* fix mocking for once and all

* address review comments

fix mysql updateStatus fn

protect dndTask with mutex

minor refactors

* move runDNDStatusExpireJob to server.go and pass App as arg instead of method receiver

* frontend will send endtime in unix epoch format so get rid of double representation

* scan for all fields and not just two

* add some tests and fix review comments

* remove extra sql query and create needed result in go

* add storetest for UpdateExpiredDNDStatuses

* add migrations to latest version

* update min supported version

* add comment to fix a bug in future

* update test to expect 1 status in return

* rename UpdateUserStatusWithDNDTimeout to SetUserStatusTimedDND

* rename DNDEndTimeUnix to DNDEndTime

* cast int to int64 for equality

* fix tests and error handling

* move updating values to retrieved statuses fields outside sql transaction

* move migrations to 5.36

Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-05-07 18:20:55 +02:00
Harshil Sharma
79dd9b13dc MM-18818 Prompt admins to set the Support Email (#17296)
* MM-18818 Set default support to empty and handled the same in email templates

* #MM-18818 Admin advisor now warns for inconfigured support email address

* Updated text

* #MM-18818 gofmt'ed files

* #MM-18818 updated text and removed unused i18n strings:

* #MM-18818 updated i18n string ordering

* #MM-18818 Added test for support email advisory

* MM-18818 gofmt'd the file

* MM-18818 separated contextual notifications from configuration notifications in Admin Advisor

* #MM-18818 prevented support email with whitepsaces from being detected as filled

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-04-28 12:05:38 +05:30
Ben Schumacher
f14b0097dd [MM-34798] Respect MM_SERVER_PATH when looking for templates (#17410) 2021-04-21 20:35:57 +02:00
Max Erenberg
f36f5c74b1 Mm 30807 granular data retention scaffold (#17464) 2021-04-20 13:16:40 -04:00
Harshil Sharma
a16fdb5001 #MM-34893 Completely disabled Admin Advisor (#17422)
* #MM-34893 Completely disabled Admin Advisor

* #MM-34893 added lint fixes
2021-04-20 05:43:01 +05:30
Agniva De Sarker
e0efdd708b Revert "Mm 30807 granular data retention scaffold (#16891)" (#17437)
This reverts commit 3ea75332e7.
2021-04-18 22:41:50 +05:30
Max Erenberg
3ea75332e7 Mm 30807 granular data retention scaffold (#16891)
create the necessary tables, models and APIs for the granular data retention policy feature
2021-04-16 11:32:09 -04:00
Max Erenberg
3a3ec001bf [MM-33826] add more detailed S3 error messages (#17182)
Automatic Merge
2021-04-09 18:16:30 +02:00
Doug Lauder
02196e04fa MM-27493 Shared channels (MVP) (#17301)
Remote Cluster Service
- provides ability for multiple Mattermost cluster instances to create a trusted connection with each other and exchange messages
- trusted connections are managed via slash commands (for now)
- facilitates features requiring inter-cluster communication, such as Shared Channels
Shared Channels Service
- provides ability to shared channels between one or more Mattermost cluster instances (using trusted connection)
- sharing/unsharing of channels is managed via slash commands (for now)
2021-04-01 13:44:56 -04:00
Ibrahim Serdar Acikgoz
ff980266ac app/server: make sure if metrics server goroutine executed (#17323) 2021-04-01 20:05:24 +03:00
Ibrahim Serdar Acikgoz
e13d85d8c7 [MM-34179] app: move cluster registeration to server creation (#17306)
* app: move cluster registeration to server creation

* initserver via fakeapp

* reflect review comments
2021-04-01 11:29:56 +03:00
Agniva De Sarker
f865703ce9 Remove server from WebsocketRouter (#17238)
App contains server.
Server contains WebsocketRouter.

There is no need for WebsocketRouter to contain
server too. As is evident because the code never used it.

```release-note
NONE
```
2021-03-26 15:19:04 +05:30
Claudio Costa
017e14d4b5 [MM-32822] Improve JobServer state handling (#16959)
Automatic Merge
2021-03-26 10:32:16 +01:00
Agniva De Sarker
aba6471512 revertws (#17216)
* Revert "MM-34000: Use non-epoll mode for TLS connections (#17172)"

This reverts commit 2743089b54.

* Revert "MM-33233: Fix double close of webconn pump (#17026)"

This reverts commit 0f98620b65.

* Revert "MM-33836: Detect and upgrade incorrect HTTP version for websocket handshakes (#17142)"

This reverts commit 4c5ea07aff.

* revert i18n

* Revert "MM-21012: Revamp websocket implementation (#16620)"

This reverts commit a246104d04.

* fix go.mod

* Trigger CI
2021-03-24 15:29:23 +05:30
Max Erenberg
4699845c0a Mm 29605 read permission s3 bucket (#16977)
Automatic Merge
2021-03-16 15:48:32 +01:00
Jesús Espino
8abb4184b5 Extracting mail service into shared libs (#17030)
* Extracting mail service into shared libs

* Fixing golangci-lint
2021-03-15 11:50:13 +01:00
Jesús Espino
95b0809850 Extracting html templates into a library (#16946)
* Extracting html templates into a library

* Moving tests to the right place

* Fixing tests

* Addressing PR review comments

* Addressing PR review comments

* Replacing attomic with RWMutex

* Returning errors as channel for Templates watcher

* Address PR review comments

* Other small fixes

* Simplifying NewWithWatcher

* Addressing PR review comments

* Making error handling on rendering templates more robust

* Fixing tests

* Changing how we return errors

* Fixing shadow variables

* Addressing PR review comments

* Logging errors from the outside of sendNotificationEmail

* Fixing lock in shutdown

* Fixing the resource copy for commands tests temporary directories

* Removing unused import

* A couple of tiny fixes
2021-03-12 18:46:43 +01:00
Ibrahim Serdar Acikgoz
f31a9ed1a8 [MM-31132] app/server: add pprof endpoint (#17001)
* app/server: add pprof endpoint

* reflect review comments

* make metrics link conditional

* app/server: add metrics server setup to licence listeners

* refactor a bit

* trigger CI

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-03-12 14:23:24 +03:00
Mario de Frutos Dieguez
5203fc8608 [MM-28694] Add MM User information in the CWS request headers (#15824)
* Include user data in the cloud endpoints

Those headers will include the user ID and Email so we can use them
in CWS

* Removed AppError from enterprise/cloud

We're removing the AppError from all the places that don't belong
to the app or api4 packages.

* Remove unused i18n strings

* Move it to the server init of enterprise

Also moved the initialization of the enterprise part in the server after the store is initialized

* Initialize after the store is set in NewServer

The ideal way to do it should be to move the initEnterprise call after
the store is set but that would lead to undesired side-effects so we
initialize the cloud part alone.

Signed-off-by: Mario de Frutos <mario@defrutos.org>
2021-03-05 09:23:39 +01:00
Jesús Espino
9cc5089af8 Moving mlog to corelibs (#16915)
* Moving mlog to corelibs

* Regenerating app layers

* Fix golangci-lint problem

* Fixing golangci-lint errors

* Renaming from corelibs to shared

* Renaming from corelibs to shared

* Fixing import

* Fixing merge problems

* Fixing build
2021-03-05 09:18:37 +01:00
Ibrahim Serdar Acikgoz
b3498a1edf [MM-32626] Disable notices for migrations test helper (#17058)
* app/server: make initial product notice fetch blocking

* disable notices on migrations_test helper
2021-03-05 10:57:23 +03:00
Jesús Espino
78355ae2a7 Moving filesstore services into shared folder (#16940)
* Moving filesstore services into shared folder

* Fixing app-layers generation

* Renaming from filesstore to filestore
2021-03-02 14:37:21 +01:00
Jesús Espino
5dd2e75c10 Extracting i18n functionality to i18n core library (#16914)
* extracting i18n functionality to i18n core library

* Removing utils.T

* Adding documentation and changing one function name for better explanation

* Changing other missing utils.T

* Adding license string

* Renaming corelibs to pkg

* Renaming corelibs to pkg (moving directory)

* Renaming from pkg to shared

* Fixing bodyPage.Html casing

* Fixing merges

* Fixing merge problem

* Fixing tests
2021-02-26 08:12:49 +01:00
Ibrahim Serdar Acikgoz
7277e23e04 [MM-32521] remove local socket file if it already exists (#16945)
* remove socket file if it exists

* reflect review comments
2021-02-23 08:19:46 +01:00
Jesús Espino
d06a62ce64 Reduce the coupling of the mailservice with the rest of the application (#16898)
* Reduce the coupling of the mailservice with the rest of the application

* Fixing tests in CI

* Simplifiying mailservice config

* Addressing PR review comments

* Fixing tests

* Removing unnecesary type definition

* Fixing ServerName usage
2021-02-16 12:42:03 +01:00