Граф коммитов

56 Коммитов

Автор SHA1 Сообщение Дата
Eli Yukelzon
f4ccc7061c MM-27199 - Add metrics to MM server for total enabled user count (#15116)
* Add metrics to MM server for total enabled user count and include installation ID from a new env var

* diagnostics context

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-07 09:26:25 -04:00
Scott Bishel
763203de9a MM-28043:NoRows error now being returned, handle in callers (#15331)
* NoRows error now being returned, handle in callers

* remove nested ifs
2020-08-24 08:42:40 -04:00
Agniva De Sarker
78766625df jobstore (#15250)
* Migration completed

* Modify test case

* Update en.json

* Fix layers

* Lint: remove unnecessary use of sprint

* trigger CI

* fix error

* Fixes

* fix test

Co-authored-by: Rodrigo Villablanca <villa061004@gmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-08-20 19:37:19 +05:30
Scott Bishel
0302e0f477 MM-27449: Compliance Jobs rerun after warning status (#15178)
* add warning as success result

* revert config changes

* add unit test, update unit test

* add a couple more tests

* update store layers

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-08-17 21:18:33 -06:00
Doug Lauder
b317ee5cf2 MM-25394 session expired push notifications (#14732)
* new job type created that checks for expired mobile sessions and pushes notifications.

* only send session expired notifications if ExtendSessionLengthWithActivity is enabled.

* includes schema change:  field added to Sessions table
2020-06-17 14:47:54 -04:00
Miguel de la Cruz
0154b8059b [MM-20979] Add first implementation of the Bleve search engine (#14562)
* [MM-20979] Add first implementation of the Bleve search engine

* Fix i18n

* Migrate searchengine utils tests

* Fix linter

* Don't add allTermsQ if both termQueries and notTermQueries are empty

* Fix test that should work if user is system admin

* Modify naming according to review comments

* Abstract getIndexDir function

* Extracting bleve engine name as a constant

* Merge both Indexer interfaces into one

* Add worker stopped message

* Allow worker to be started/stopped with config change

* Use constants for index names

* Modify test order

* Fix linter

* Trying to unlock the CI
2020-05-20 01:29:55 +02:00
Scott Bishel
1031e27fd8 MM-23038: Compliance Report Fixes (#14242)
* add warning count as return value

* add warning count as return value

* fix file name

* update mock

* add setting warning to db

* replace wrongly removed string

* add dummy function to see if it will build

* remove dummy function

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-05-07 14:12:47 -06:00
Jesús Espino
c66e182b08 Adding the new search engine abstraction (#13304)
* WIP

* Adding bleve to go modules

* WIP

* Adding missing files from searchengine implementation

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* User and channel indexing and searches implemented

* Make bleve tests run with in-memory indexes

* Implement post index and deletion tests

* Initial commits for the search layer

* Removing unnecesary indexing

* WIP

* WIP

* More fixes for tests

* Adding the search layer

* Finishing the migration of searchers to the layer

* Removing unnecesary code

* Allowing multiple engines active at the same time

* WIP

* Add simple post search

* Print information when using bleve

* Adding some debugging to understand better how the searches are working

* Making more dynamic config of search engines

* Add post search basics

* Adding the Purge API endpoint

* Fixing bleve config updates

* Adding missed file

* Regenerating search engine mocks

* Adding missed v5 to modules imports

* fixing i18n

* Fixing some test around search engine

* Removing all bleve traces

* Cleaning up the vendors directory and go.mod/go.sum files

* Regenerating timer layer

* Adding properly the license

* Fixing govet shadow error

* Fixing some tests

* Fixing TestSearchPostsFromUser

* Fixing another test

* Fixing more tests

* Fixing more tests

* Removing SearchEngine redundant text from searchengine module code

* Fixing some reindexing problems in members updates

* Fixing tests

* Addressing PR comments

* Reverting go.mod and go.sum

* Addressing PR comments

* Fixing tests compilation

* Fixing govet

* Adding search engine stop method

* Being more explicit on where we use includeDeleted

* Adding GetSqlSupplier test helper method

* Mocking elasticsearch start function

* Fixing tests

Co-authored-by: Miguel de la Cruz <miguel@mcrx.me>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-13 15:33:18 +01:00
Jesús Espino
a63684fcb5 Consistent license message for all the go files (#13235)
* Consistent license message for all the go files

* Fixing the last set of unconsistencies with the license headers

* Addressing PR review comments

* Fixing busy.go and busy_test.go license header
2019-11-29 12:59:40 +01:00
Miguel de la Cruz
2259b7f2a8 [MM-19948] Set version on module file and internal paths (#13186)
* [MM-19948] Set version on module file and internal paths

* Fixes after merge

* Fix i18n checker error
2019-11-28 14:39:38 +01:00
Scott Bishel
2bcb4d9913 MM-17508: Modifying config files causes compliance exports to run twice (#13053)
* Check if leader before setting schedule.

* add check to normal run as well
2019-11-13 12:25:25 -07:00
Alexander Morozov
9b9923ea42 Replace time.After with time.NewTimer (#12945)
time.After might leave stray timers if other channels in select triggered early

Signed-off-by: Alexander Morozov <lk4d4math@gmail.com>
2019-10-29 18:41:54 +01:00
Nikhil Ranjan
be25dcd6f3 Converting to structured logging the file jobs/jobs.go (#12120) 2019-09-11 14:19:13 -04:00
Nikhil Ranjan
8b969426f0 Converting to structured logging the file jobs/jobs_watcher.go (#12121) 2019-09-10 18:23:49 -04:00
Christopher Speller
3ea8db5dc6 Fix panic in jobs structured logging. (#12132) 2019-09-09 20:41:29 -04:00
Nikhil Ranjan
ec803ada0a Converting to structured logging the file jobs/schedulers.go (#12122)
* Converting to structured logging the file jobs/schedulers.go

* merge mlog warn and error in single error
2019-09-09 18:53:33 +02:00
Jesús Espino
f934502a56 Migrate Jobs store to sync by default (#11183)
* Migrate Jobs store to sync by default

* Fixing compilation

* Fixing compilation

* Fixing govet
2019-06-15 17:55:06 +02:00
Yusuke Nemoto
06f384df6d Enable PluginJobs (#10198) 2019-02-06 08:25:07 +01:00
Christopher Speller
8429add371 Cleanup related to context refactor (#9988) 2018-12-17 08:51:46 -08:00
Daniel Schalla
c36e85c912 DeleteAll for KV (#9431)
Expire K/V Values

Regenerate Code

pathfix

Update Expiry on Update

Check for Exit Signal

gofmt

Rewrote Go Routine

Remove tempoarily cleanup loop

fix expiretime

TEST: Expired Watchdog as GoRoutine

Check if Srv is nil

Use Scheduler/Worker for Expired Key CleanUp

add license

fix scheduler job type; DoJob Restructuring

Remove unused imports and constants

move db migration from 5.4 to 5.5
2018-10-10 10:55:12 -07:00
Harrison Healey
4e59a27293 Move HTTPService and ConfigService into services package (#9422)
* Move HTTPService and ConfigService into utils package

* Re-add StaticConfigService

* Move config and http services into their own packages
2018-09-26 12:42:51 -04:00
George Goldberg
8766690c81 MM-10502: Only cluster master should run job schedulers. (#9174)
* MM-10502: Only cluster master should run job schedulers.

* Use sync.Map for thread safety.

* Fix tests.
2018-07-31 07:40:23 -07:00
George Goldberg
60359b4bae MM-10269: Make jobserver work without restart. (#9005) 2018-06-26 13:30:33 -04:00
George Goldberg
51bd710ecd MM-9728: Online migration for advanced permissions phase 2 (#8744)
* MM-9728: Online migration for advanced permissions phase 2

* Add unit tests for new store functions.

* Move migration specific code to own file.

* Add migration state function test.

* Style fixes.

* Add i18n strings.

* Fix mocks.

* Add TestMain to migrations package tests.

* Fix typo.

* Fix review comments.

* Fix up the "Check if migration is done" check to actually work.
2018-05-14 15:59:04 +01:00
Christopher Speller
686c2fbab7 Structured logging (#8673)
* Implementing structured logging

* Changes to en.json to allow refactor to run.

* Fixing global logger

* Structured logger initalization.

* Add caller.

* Do some log redirection.

* Auto refactor

* Cleaning up l4g reference and removing dependancy.

* Removing junk.

* Copyright headers.

* Fixing tests

* Revert "Changes to en.json to allow refactor to run."

This reverts commit fd8249e99bcad0231e6ea65cd77c32aae9a54026.

* Fixing some auto refactor strangeness and typo.

* Making keys more human readable.
2018-04-27 12:49:45 -07:00
Martin Kraft
ffa24fbcb2 MM-8834: Changes separator between jobs message and details. (#8589) 2018-04-09 09:39:57 -04:00
Jesús Espino
014a3b6a60 Fixing misspell errors (#8544) 2018-03-29 10:04:54 -04:00
Chris
e8943936c5 general cleanup (#8387) 2018-03-07 12:36:40 -06:00
Pierre de La Morinerie
07fd7aeeb8 Add tests for the platform server command (#8231)
* Cleanup app state on initialization error

When returning an initialization error, the app state was not cleaned
up. This is especially visible during tests, as `appCount` is not
decremented, and makes the new app initialization fail.

* Test the `platform server` command

As the `platform server` command only exits when interrupted by
a signal, it is not possible to test it as the other cobra
commands. Instead we directly test the actual command function.

The internal command handler is slighly refactored to take
a channel in argument, and registers it as the signal handler.
Nothing very different—except than controlling this channel
from the outside allows the test to send the system signal
itself, thus preventing the server to run forever.
2018-02-12 08:46:32 -08:00
Chris
a6309aaf48 Remove license globals entirely (#8229)
* remove license globals entirely

* fix infinite recursion

* test fix
2018-02-09 10:04:48 -06:00
Chris
4e6cc846a6 Finally remove utils.Cfg (#8113)
* finally remove utils.Cfg

* fix compile error

* another test compilation fix
2018-01-17 13:38:37 -05:00
Chris
7e5ce97668 Remove global cfg vars (#8099)
* remove global cfg vars

* enterprise update
2018-01-12 09:02:11 -05:00
Jesús Espino
dd9ad10d70 [PLT-6936] Translate AppError.Message automatically by default (#8063) 2018-01-08 13:13:24 -05:00
Jonathan
375c0632fa PLT-7503: Create Message Export Scheduled Task and CLI Command (#7612)
* Created message export scheduled task

* Added CLI command to immediately kick off an export job

* Added email addresses for users joining and leaving the channel to the export

* Added support for both MySQL and PostgreSQL

* Fixing gofmt error

* Added a new ChannelMemberHistory store and associated tests

* Updating the ChannelMemberHistory channel as users create/join/leave channels

* Added user email to the message export object so it can be included in the actiance export xml

* Don't fail to log a leave event if a corresponding join event wasn't logged

* Adding copyright notices

* Adding message export settings to daily diagnostics report

* Added System Console integration for message export

* Cleaned up TODOs

* Made batch size configurable

* Added export from timestamp to CLI command

* Made ChannelMemberHistory table updates best effort

* Added a context-based timeout option to the message export CLI

* Minor PR updates/improvements

* Removed unnecessary fields from MessageExport object to reduce query overhead

* Removed JSON functions from the message export query in an effort to optimize performance

* Changed the way that channel member history queries and purges work to better account for edge cases

* Fixing a test I missed with the last refactor

* Added file copy functionality to file backend, improved config validation, added default config values

* Fixed file copy tests

* More concise use of the testing libraries

* Fixed context leak error

* Changed default export path to correctly place an 'export' directory under the 'data' directory

* Can't delete records from a read replica

* Fixed copy file tests

* Start job workers when license is applied, if configured to do so

* Suggestions from the PR

* Moar unit tests

* Fixed test imports
2017-11-30 09:07:04 -05:00
Chris
10c5a927cb more global config ref cleanup (#7802) 2017-11-09 14:46:20 -06:00
Corey Hulen
b3b46a01aa PLT-5813 support SAML sync via LDAP (#7668)
* PLT-5813 support SAML sync via LDAP

* Cleaning up based on review
2017-10-31 08:37:34 -07:00
Chris
c5e8cb25ca simplify things (#7735) 2017-10-30 11:57:24 -05:00
Chris
7ed011745a Remove more global refs / state (#7723)
* remove more global refs / state

* fix job enterprise initialization

* fix api4 test compilation

* saml api endpoints fix
2017-10-26 15:21:22 -04:00
Chris
6f6005c617 Store mocks (#7724)
* store mocks

* add example
2017-10-26 11:36:54 -05:00
George Goldberg
76bd1bb212 PLT-7705: API to get data retention policy. (#7539)
* PLT-7705: API to get data retention policy.

* Fix review comments.
2017-10-02 12:43:21 +01:00
Chris
4e79d2d4d0 remove jobs.Srv and other jobs-related globals (#7535) 2017-09-29 10:29:29 +01:00
George Goldberg
a06830b2f8 PLT-7644: Improve job scheduler architecture. (#7532) 2017-09-28 09:11:13 -07:00
Corey Hulen
81c18a01bd PLT-7542 Converting LDAP sync to the job server (#7452)
* PLT-7542 Converting LDAP sync to the job server

* Fixing minor issues

* Fixing build failure

* Translate error message

* Translate error message

* Translate error message

* Translate error message

* Fixing merge

* Fixing bad merge
2017-09-25 13:30:33 -07:00
George Goldberg
7243aa6751 PLT-6558: Basic data retention job scheduler/worker implementation. (#7449)
* PLT-7639: Batch delete methods for data retention.

* PLT-6558: Basic data retention job worker/scheduler implementation.
2017-09-15 09:53:48 -07:00
Christopher Speller
29fca51821 Renaming repo 2017-09-06 23:11:59 -07:00
Chris
1adfd0e9be app type transition (#7167) 2017-09-06 17:12:54 -05:00
George Goldberg
22459ee17a PLT-7302: Aggregate Elasticsearch indexes over a certain age. (#7224)
* PLT-7302: Aggregate Elasticsearch indexes over a certain age.

This is done by a scheduled daily job, in order to keep the shard count
to a sensible level in Elasticsearch.

* Use map[string]string instead of StringMap
2017-08-17 10:05:17 -04:00
Chris
ff0811e4a4 lint all the things (#7154)
* lint all the things

* remove extra echo
2017-08-09 13:35:49 -07:00
Christopher Speller
0331409488 Add jobserver command to platform binary (#7067)
* Add jobserver command to platform binary

* Fixing jobserver config. Adding jobserver flags.

* Fixing govet in makefile
2017-08-01 08:53:42 -04:00
George Goldberg
00e8b6621a PLT-6595 (Client): Elasticsearch indexing system console UI (#6991)
* PLT-6595: System Console for Elasticsearch Job Management.

* Fixing UI issues

* Fixing colors

* ESLint Fixes.

* Update test snapshots.

* Fixing cancel button

* Fix review comments.

* Config capitalisation.

* Review fixes.
2017-08-01 13:11:29 +01:00