Граф коммитов

1964 Коммитов

Автор SHA1 Сообщение Дата
Agniva De Sarker
c8b60c2d75 MM-22783: Fix scopelint issues (#13969)
* Fix scopelint issues

* Incorporating review comments

* Keeping the order of linters alphabetical

* Fix issues after merge

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-13 08:44:39 +05:30
Doug Lauder
4ac0619c90 MM-22273 New auditing system (phase 1) (#13967)
* New auditing API outputting to syslog via TLS

* New config section for specifying remote syslog server IP, port, and cert.

* Legacy audit API retained for access history feature
2020-03-12 15:50:21 -04:00
Jesús Espino
27d536b212 MM-21552: Adding SaveMultiple to posts (#13766)
* Adding SaveMultiple to posts

* Improving tests

* fixing i18n

* Fixing tests

* Improving testing on top of Save and SaveMultiple

* Fixing shadow variables

* Addressing some PR comments

* More clear update post test

* Addressing some PR comments

* Addressing some PR comments and simplifying the code

* Improting replies in bulk too

* Fixing reply count and processing last imported replies

* Adding OverwriteMultiple to posts aggregating everything in the same transaction

* Adding 2 pending tests to implement

* Adding tests for overwrite multiple posts

* Adding tests for TeamStore.GetByNames method

* Fixing shadow variables

* Addressing PR comments

* Extracting i18n strings

* Fixing tests

* Fixing tests

* Adding more test cases

* Using a variable instead of a fake timestamp
2020-03-11 14:29:32 +01:00
Alejandro García Montoro
2bec92a404 MM-21987 Resolve mentions in slash commands (#13762)
* Create infrastructure to manage mentions

Two new files have been added (along with their tests); namely:

- model/at_mentions.go: utilities to parse and manage mentions; for the moment,
it just contains a regex and a couple of functions to parse possible mentions
and to post-process them, but it can be extended in the future.
- model/mention_map.go: it contains two new types (UserMentionMap and
ChannelMentionMap) that both have FromURLValues and ToURLValues. These types
can be used when adding the mentions to the payload of the plugin slash
commands.

* Extend custom commands payload with mentions

Two couples of new fields are added to the payload; namely:

- user_mentions and user_mentions_ids: two aligned arrays of the same length
containing all the different @-mentions found in the command: the i-th element
of user_mentions_ids is the user identifier of the i-th element of
user_mentions.
- channel_mentions and channel_mentions_ids: two aligned arrays of the same
length containing all the different ~-mentions found in the command: the i-th
element of channel_mentions_ids is the channel identifier of the i-th element
of channel_mentions.

* Fix shadowing of variables and redundant return

* Fix shadowing of variable

* Address review comments (HT @lieut-data)

- Improvements in mentionsToTeamMembers and mentionsToPublicChannels:
	- Scope implementation details inside the functions.
	- Improve goroutines synchronization by using a sync.WaitGroup.
	- Retry lookup of username only if the returned error is http.StatusCode,
	  so we can return early if the error is more severe.
- Invert check in PossibleAtMentions to improve readability.
- Make user and channel mention keys private to the module.
- Allow the specification of an empty map of mentions in
(Channel|User)MentionsFromURLValues when both mentions keys are absent.
- Replace custom functions in tests with require.Equal on maps.

* Test functions to parse mentions from messages

* Extend plugin commands payload with mentions

* Add functions to CommandArgs to add mentions

The functions make sure that the maps are initialized before adding any value.

* Address review comments (HT @lieut-data)

- Adds a mlog.Warn to avoid burying the error when the user is not found.
- Improve readability in loop populating the mention map by moving the
initialization of the map closer to the loop and by iterating over the channel
itself, not over its length.

* File was not gofmt-ed with -s

* Close channel when all goroutines are finished

* Again, all code should be checked with gofmt -s

* Refactor code out of a goroutine

This change helps improve the readability of the code and does not affect its
overall performance. Less complexity is always better.

* Close channel and iterate over its range

Adapt mentionsToPublicChannels to have the same structure in the management
of the mentions channel as in mentionsToTeamMembers.

* Adapt mentionsToTeamMembers to new App

Commit 17523fa changed the App structure, making the *Server field
private, which is now accessed through the Srv() function.

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-11 11:50:12 +01:00
Shota Gvinepadze
5d928b4f94 [MM-18150] plugin panic trace should not be lost (#13559)
* Transit panic from debug to error

* Parse plugin's StdErr and output panic to the mlog.Error

* Add unit tests

* Change log test

* Remove buffer from logger

* Remove 'panic' string filter

* Change *Buffer to io.Writer

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-11 11:41:11 +04:00
Ibrahim Serdar Acikgoz
771574b652 [MM-17797] Add Timeout To SMTP Connection (#13251)
* model/config: add timeout setting to email settings

* services/mailservice: add timeout to connection

* app/server_app_adapter: show SMTP connection errors on log

* services/mailservice: add r/w deadline to smtp conn

* services/mailservice: add context timeout

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-10 18:34:31 +03:00
Mario de Frutos Dieguez
ee86413cdd Include websocket users metric (#14020)
This commit adds new metrics to know the number of registered users
we have in every WebSocket broadcasts hub
2020-03-10 10:15:56 +01:00
Jesús Espino
cd382412fd Migrating file info store cache to the cache layer (#13045)
* Migrating file info store cache to the cache layer

* Remove unrelated changes

* Updating timer layer store

* Fixing usage of invalidate cache

* Removed repeated tests and unnecesiary require

* Using doInvalidateCacheCluster instead of removing the cache directly

* Addressing PR comments

* Fixing imports

* Fixing tests

* Fixing license header
2020-03-09 18:34:25 +01:00
Hossein Ahmadian-Yazdi
34668511cf fix read only channel when join channel (#13946)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-06 15:58:14 -05:00
Ibrahim Serdar Acikgoz
d5df6a3fc4 [MM-22314] Remove read limit at app/import which cause bulk import to fail (#13863)
* app/import: increase buffer size to avoid failing in large inputs

* remove commented out lock

* app/import: make maxTokenSize constant

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-06 23:14:35 +03:00
Miguel de la Cruz
cdc916ff52 Add missing opentracing layer changes (#14013) 2020-03-06 12:18:02 +05:30
Farhan Munshi
79c786bc0c [MM-22206] Add PATCH channel moderations (PUT /moderations/patch) (#13845)
* MM-22205 Add get channel moderations endpoint

* MM-22206 Add patch channel moderations endpoint

Add api tests for patch channel moderations

* MM-22205 Ensure ordered permissions returned and create struct ChannelModeratedRoles

* MM-22206 Use structs instead of map

* MM-22206 Add test cases for GetChannelModeratedPermissions

* MM-22206 Add tests for ChannelModeratedPermissionsChangedByPatch

* MM-22206 Use NewBool instead of defining booleans

* MM-22206 Tie Channel Mentions to Create Posts when building Channel Moderations

* Revert "MM-22206 Tie Channel Mentions to Create Posts when building Channel Moderations"

This reverts commit a0bfc95f1732955c5ef5fc3e4b05ea8ab954acea.

* MM-22206 Review changes

Modify GetSchemeRolesForChannel to return named variables
Change calls to SessionHasPermissionToChannel to SessionHasPermissionTo
Add a CreateChannelScheme method
Add a DeleteChannelScheme method
Move GetChannelModeratedPermissions to Role model

* Fix lint

* Add ChannelModeration methods to App interface

* MM-22206 Rename method to GetTeamSchemeChannelRoles

* MM-22206 Check CHANNEL_MODERATED_PERMISSIONS_MAP for existing permission before iterating through it

* Modify wording to higherScoped to match #13813

* MM-22206 Delete channel scheme between tests

* MM-22206 Fix tests

* Actually patch role

* MM-22206 Shadow declaration of err
2020-03-05 10:04:34 -05:00
Miguel de la Cruz
182c29b456 MM-21898: Part 2. Add opentracing (#13904)
* initial implementation of opentracing

* app layer

* Revert Makefile

* .

* cleanup

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* .

* [ci]

* autogenerate interface

* .

* missed vendor files

* updated interfaces

* updated store layers

* lint fixes

* .

* finishing layer generators and nested spans

* added errors and b3 support

* code review

* .

* .

* fixed build error due to misplased flag.Parse()

* code review addressed
2020-03-05 14:46:08 +01:00
Ben Schumacher
5a34ec4793 Fix app error messages (#13852)
* Summary
Fixing inconsistencies of the app_error messages. Now all end with a 
period and there is the same user experience all over the system. 

We agreed on this change on the community server a while ago and I just 
came around finally doing it.


Ticket Link
N/A

Checklist
 Includes text changes and localization file (.../i18n/en.json and 
.../webapp/i18n/en.json) updates

* Update i18n/en.json

Approving changes

Co-Authored-By: Jason Blais <13119842+jasonblais@users.noreply.github.com>

* Update i18n/en.json

Approving changes

Co-Authored-By: Jason Blais <13119842+jasonblais@users.noreply.github.com>

* Update i18n/en.json

Approving changes

Co-Authored-By: Jason Blais <13119842+jasonblais@users.noreply.github.com>

* Fix test failures due string differences

Co-authored-by: Jason Blais <13119842+jasonblais@users.noreply.github.com>
Co-authored-by: Miguel de la Cruz <mgdelacroix@gmail.com>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-04 20:14:23 +01:00
Jesús Espino
54bb6d10f3 Making InvalidateCacheForUser public again (#13997) 2020-03-04 16:16:27 +01:00
Eli Yukelzon
73dae67c8e MM-22980 - Revert "server init order fixed" (#13994)
This reverts commit daebc93d90958aa2831618f64e0ab2f667a1aa01.
2020-03-04 16:25:45 +02:00
Mario de Frutos Dieguez
b162cf92cd MM-21976: Include cache layer to be tested (#13749) 2020-03-04 14:18:03 +01:00
Jesús Espino
c9a0418a32 Making private some methods that are not needed publicly (#13959)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-03 22:43:48 +01:00
Jesús Espino
ccc57e56c3 MM-22235: Fixing stop/start behavior on Elasticsearch engine (#13887)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-04 00:52:59 +08:00
Patryk Pomykalski
662eef6881 Cache test users to speed up testing (#13741)
* Cache test users to speed up testing

Users were recreated for many tests and hashing passwords takes a lot of
time. Testing /api4 takes around 120s less and /app 60s less.

* fix build

* simplifying a bit the approach

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Jesús Espino <jespinog@gmail.com>
2020-03-03 14:19:54 +01:00
Mario de Frutos Dieguez
89f4e9d7ef Metrics operations shouldn't be inside goroutine (#13950)
Metrics operations are done in memory so there is no need
to spawn a new goroutine
2020-03-03 16:23:32 +05:30
Farhan Munshi
aec606500f [MM-22622] Add Ephemeral response when using mentions without permissions (#13902)
* MM-22282 Add Ephemeral response when using mentions without permission and add new prop to disable mention highlights on client

* MM-22622 Make test name test actually what it does and fix comment style

* MM-22622 Check ephemeral post created when post create with mentions on API

* MM-22622 More tests for App>CreatePost

* MM-22622 Make DisableMentionHighlights more concise and rename ephemeral post

* MM-22622 Dont send ephemeral message for system message created by user

* Trigger build
2020-03-03 15:52:49 +05:30
Jason Frerich
4c7fee7ac8 [MM-22485] Sync bot message to message displayed in webapp (#13867)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-02 15:57:29 -06:00
Mario de Frutos Dieguez
a43439dd60 Include deleted channels on get user channels endpoint (#13877) 2020-03-02 22:15:15 +01:00
Mario de Frutos Dieguez
9742a0595d Include Mutex for Hubs property (#13917)
Hubs property has been detected by the race condition detector
as a possible one because we could set the Hubs variable
and also we're spawning goroutines that could ask for that property

This condition is not happening right now because we're setting the
Hubs variable in the server start and after that, we're spawning the
goroutines but the tests are failing because of this and in the future
this could be a problem
2020-03-02 19:37:57 +01:00
Agniva De Sarker
a981099652 Add error checks to TestUnlinkGroupTeam (#13956)
I could not figure out the actual root cause for this.

But a possible area of concern is that somewhere in the call stack,
the login failed partially. Therefore, the session did not have the teamMembers
populated. And the permission check failed therefore.

Adding the error checks should let us know in future if this is not the case.

Another possible case is the app.ClearTeamMembersCache which gets called in
the goroutine app.SyncRolesAndMembership. If this races with the session,
then it's possible that the teamMember is wiped off from the session. But this
happens after the permission check. So it's not very likely.

Also, while we are here, I found that (*app).ClearTeamMembersCache
does not log the internal error bubbled up. This prevents us from understanding
what actually happened. Added that logging.
2020-03-03 00:04:39 +05:30
Jesús Espino
7035e09fe9 Adding changes to separate unit tests and integration tests (#13670)
* Introducing unit (not integration) tests for the app layer

* Initial support for unit tests at the API

* Adding unit tests support to the store layer

* Add unit tests support in commands

* Adding last tests needed for run unit tests properly

* Fixing govet

* Removing some duplication

* Fixing tests

* Fixing tests

* Not compiling test helpers with the main module for api

* Revert "Not compiling test helpers with the main module for api"

This reverts commit 36a199bbe0f7503f665f5d6c7b41c53aabc2e54f.

* Fixing tests

* Fixing unit tests

* More consistency between api4/apiteslib.go and app/helper_test.go

* Renaming things to make more obvious the new Setup functions purpose

* Reverting change in go.sum

* Start with empty mock for app layer

* Start with empty mock for api layer

* Start with empty mock for web layer

* Renaming SetupWithStoreMockConfig to SetupConfigWithStoreMock

* Fixing tests on web package

* Removing unnecesary function
2020-03-02 08:13:39 -08:00
JtheBAB
cd36c9f041 MM-22355 - Fix typos detected by goreportcad.com (#13842)
* Update webhook_test.go

* Update user_test.go

* Update filesstore_test.go

* Update plugin_requests.go

* Update syncables.go

* Update helper.go

* Update html_entities.go

* Update user.go

* Update team.go

* Update notification.go

* Update notification_test.go

* Update plugin_api_test.go

* Update post_metadata.go

* Update channel_test.go

* Update database.go

* Update channel.go

* Update user_store.go

* Update team_test.go

* revert andd

* Revert back to infintie

Co-authored-by: Jesús Espino <jespinog@gmail.com>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-02 17:10:40 +01:00
Mario de Frutos Dieguez
75197d291a Added Websocket's broadcast queue load metrics (#13935)
Now we send metrics when the websocket's broadcast queue receive or
consume a message from the queue
2020-02-28 11:44:56 +01:00
Rajat Varyani
fdb339fbd7 [MM-20941] Parallelize plugin loading on startup (#13617)
* [MM-20941] Parallelize plugin loading on startup

* [MM-20941] Parallelize plugin SyncPlugins

* [MM-20941] Make activation and deactivation of plugin concurrent

* [MM-20941] Remove uncessary optimization

This reverts commit aad2680aec843125bf72d8d2406869f4948dba42.

[MM-20941] Remove uncessary wait in SyncPlugins

[MM-20941] Parallelize processing of prepackaged plugins

* [MM-20941] Fix lint issue

* [MM-20941] Fix bug for getting the number of plugins to process

* [MM-20941] Use buffered channel and fix return issue in prepackaged plugin processing

* [MM-20941] Implement feedback

* [MM-20941] Fix looping over channel for plugins

* [MM-20941] Retain signature of process plugin

* [MM-20941] Fix function documentation

* [MM-20941] Fix bug to pass the plugin path in anonymous function

* [MM-20941] Rename variable

* [MM-20941] Make minor changes in variable naming

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-02-17 18:30:48 -04:00
Eli Yukelzon
53e07a68f5 fixing a screwup (#13906)
Automatic Merge
2020-02-17 03:44:34 -05:00
Jesse Hallam
9a51c73f64 MM-22619: check for nil plugins environment (#13901)
* MM-22619: check for nil plugins environment

Check if plugins were disabled before attempting to collect metrics and emit telemetry for same.

Fixes: https://mattermost.atlassian.net/browse/MM-22619

* Update app/diagnostics_test.go
2020-02-15 17:46:26 -05:00
Ali Farooq
bd9d674006 Fixing master (#13900)
* updating api_timer_layer_generated.go

* Fixing GetFileInfos

* Fixing TestPluginAPIGetFileInfos

* Remove file.go.orig
2020-02-14 18:08:00 -05:00
Ali Farooq
e4fb5791b0 MM-21626, MM-21627 - Plugin API/Hooks Prometheus instrumentati… (#13825)
* MM-21626,MM-21627 - Plugin API/Hooks Prometheus instrumentation

* Updated einterface mocks

* Fixed supervisor tests

* ignoring golint errors for plugin metrics wrappers

* Making golangci happy

* Using variadic form when generating wrapper code

* Removed artificial delay

* Removed comments from tests

* Renaming plugin wrappers to api/hooks_timer_layer

* updating vendor dir and mod files

* Recording plugin api/hook responses in prometheus

* Updated einterfaces-mocks

* Updating go sum

* Updating go sum

* Fixing conflicts

* More conflicts fixing

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-02-14 15:47:43 -05:00
Ashish Bhate
3fc5287f6b MM-17912: Allow searching for files through plugin API (#13647)
* constants and options for getting files

* Method to get files with options

* Add i18n strings for en

* Add API methods for getting files with options

* gofmt -s file

* explicitly set create at in tests

* use greater than nanosecond time difference for tests

* use gte instead of gt for getting files by created time

* use created at time as default sort order for getting file infos

* use explicit inline strings instead of format strings

* join tables only when required

* use if as secondary sort, and update tests

* update field docs to reflect previous changes

* make page and perPage get options as required

* add json struct tags to GetFileOptions

* bump minimum server versioni

* remove sorting by username and channelname

* use bool for sort order type

* use FileInfo prefix instead of just File

* clearer comments

* use zero-based page numbering

* test filtering by user and channel

* remove unnecessary whitespace

* use int instead of uint for page and perPage arguments

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-02-14 16:21:54 -04:00
Miguel de la Cruz
84f45634a8 Remove remaining t.Fatal from the codebase (#13876)
* Remove remaining t.Fatal from the codebase

* Fix job_store test

* Address review comments

* Remove comments
2020-02-13 17:53:23 +01:00
Eli Yukelzon
17523fa5d9 MM-21898 - Part 1: Generate and use an interface instead of *A… (#13840)
* Generate and use an interface instead of *App
2020-02-13 13:26:58 +01:00
Jesús Espino
66fc096768 MM-22340: Fixing flaky tests around Team name validation (#13875)
* MM-22340: Fixing flaky tests around Team name validation

* Fixing other cases where can happend the flaky tests
2020-02-13 09:36:21 +01:00
Jason Blais
35ac2629e2 Add missing "AutomaticPrepackagedPlugins" to diagnostics.go (#13868) 2020-02-12 17:52:15 -04:00
Farhan Munshi
897715f883 [MM-21919] Add use_channel_mentions permission (#13781)
* MM-21919 Add channel_mention permission

* MM-21919 Fix emojis migration test

* Enforce Channel Mentions on the in the posts api

* MM-21919 Rename permission to use_channel_mentions

Allow posts with channel mentions to still be posted without the permission just don't send notifications to users

* MM-21919 Add tests for a post with @all and @here

* MM-21919 Add use channel mentions to all roles that have create post

* MM-21919 Update app_test to include use_channel_mentions permission in default permissions
2020-02-12 21:15:34 +05:30
Shota Gvinepadze
a7854f1b97 [MM-22037] Enable uppercase letters in the email (#13794)
* Enable uppercase letters in the email

* Lowercase email on every input

* Remove invalid test
2020-02-12 08:51:45 -05:00
Eli Yukelzon
fb4d27d596 MM-22265 - Restrict the WS "update_team" event broadcast (#13848)
Automatic Merge
2020-02-11 03:41:55 -05:00
Maria A Nunez
c6e7030e03 MM-19445 - Fix conditional for plugin version diagnostics (#13844)
* Merge upstream

* Adding plugin version data if no error

* linting

Co-authored-by: Jason Paul Deland <9366595+jaydeland@users.noreply.github.com>
2020-02-10 14:45:19 -05:00
Jesús Espino
1d1ab03c38 Passing t to all tests setup functions (#13841)
* Passing t to all tests setup functions

* Fixing build
2020-02-10 19:31:41 +01:00
Agniva De Sarker
73ce92400b MM-22057: Limit incoming request bodies (#13827)
* MM-22057: Limit incoming request bodies

Set the max request body size to be equal to MaxFileSize.
Ideally, non-file request bodies should be smaller than file request bodies,
but we don't have a clean way to identify all file upload handlers.

There shouldn't be any valid request which exceeds the max file upload size.
So this is a safe global limit to apply.

* Fix tests

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-02-07 14:59:59 -05:00
Martin Kraft
240e57d581 MM-21199: Adds groups diagnostics. (#13736) 2020-02-06 09:25:36 -05:00
Nick Malcev
35102b081e GH-13421 Autoresponder should not reply to bot DMs (#13583) 2020-02-06 12:49:58 +01:00
Andrei Avram
e965d21359 Log error of OnConfigurationChange hook. (#13426) 2020-02-06 07:53:14 +01:00
Eli Yukelzon
597a2b77cd MM-17468 - Improve thread fetching (#13653)
* Revert "Thread fetching revert (#13616)"

This reverts commit 8e0fe90897.

* renamed query param for clarity

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-02-05 13:27:35 +01:00
Agniva De Sarker
435d43049f MM-22049: Add idle timeout to the HTTP server (#13788)
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-02-05 11:32:04 +05:30