Граф коммитов

12877 Коммитов

Автор SHA1 Сообщение Дата
Farhan Munshi
2af1bb7579 [MM-28536] [MM-28535] sysconsole_write_authentication bug fixes (#15558)
* MM-28536 User with write-auth should be able to use all saml related endpoints

* MM-28535 allow users with write auth to invalidate pending email invites

* Update api4/team_test.go

* No need for init basic

* Address review comments

* Fix test ordering
2020-09-24 11:26:11 -04:00
Farhan Munshi
4ef44a3ba1 MM-28537 System role with write experimental should be able to purge indexes (#15557) 2020-09-24 11:26:00 -04:00
Harrison Healey
e8a18ab6a6 MM-28347 Enable custom emoji by default for new installs (#15560) 2020-09-24 10:28:27 -04:00
Jared Shields
e726b04268 Trimmed trailing slash in URL provided to create Client API (#15513) 2020-09-24 12:06:36 +02:00
Ashish Bhate
ee76ad435b [MM-28303]: check if mmctl exists before checking on Github and reduce requests to Github (#15387)
Summary:
Every make call would ping Github to find out the mmctl version to download. Each check made 4 requests to Github. So every make execution resulted in 4 requests to Github. This leads to frequent rate-limit errors from Github.
In this PR we check for the mmctl version only if mmctl doesn't already exist. We also print a more helpful error message.
Reduce the number the number of requests to Github from 4 to 2.

Ticket Link:
https://mattermost.atlassian.net/browse/MM-28303
2020-09-24 14:54:57 +05:30
Claudio Costa
605f96fbea Add multipart support for new resumable uploads API endpoint (#15510)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-24 09:33:53 +02:00
Rodrigo Villablanca
8118cac350 Poststore migration part3 (#15505)
* Migration completed

* Order in translations file

* Fix: lints

* Trigger CI

* Fix message key

* Change mlog.Error for mlog.Warn

* Fix imports

* Adding translations needed for EE

* Trigger CI

* Fix merge with master

Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-24 10:46:36 +05:30
Ashish Bhate
cd7d68effb [MM-25647][MM-25659] Add missed config settings to telemetry (#15506)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-24 10:23:23 +05:30
Ben Schumacher
d535e66638 [MM-28520] Update bundled version of nps to 1.1.0 (#15537) 2020-09-23 18:00:08 +02:00
Ibrahim Serdar Acikgoz
dc1b42390b [MM-28664] model/integration_action: fix panic for uncomparable types (#15514)
* model/integration_action: fix panic for uncomparable types

* model/integration_action: improve performance

* reflect review comments

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-23 18:32:47 +03:00
Claudio Costa
1e34bdac1f Fix possible nil dereference (#15435)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-23 08:51:12 +02:00
Claudio Costa
9a4774bcc2 [MM-28383] Fix nil pointer dereference in addTeamMember (#15422)
* Fix nil pointer dereference in addTeamMember

* Revert to simple nil check

* Add test case

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-23 08:50:38 +02:00
Rodrigo Villablanca
5c449f4202 Fix incorrect error handling (#15502) 2020-09-22 21:51:52 -03:00
Jaime Cepeda
f3cb2538a7 Refactor SQL queries in store/sqlstore/channel_member_history_store.go to use the squirrel builder (#15529) 2020-09-23 01:32:03 +02:00
Ben Schumacher
31e99920ef Fix GetPluginStatus in a non-cluster environment (#15363) 2020-09-22 10:57:08 +02:00
Ibrahim Serdar Acikgoz
b0216053a1 [MM-28195] storetest/user_store: fix flaky test (#15543)
* storetest/user_store: fix flaky test

* reflect review comments
2020-09-22 11:28:12 +03:00
Ashish Bhate
fb022461f4 [MM-28100]: fix getChannelByNameForTeamName permissions (#15525)
Summary:
fix getChannelByNameForTeamName to allow team members to join public channels. I'm not sure why, but the authorization logic of getChannelByName and getChannelByNameForTeamName was different. After this change they behave the same. This fixes an issue with the mobile app.

Ticket Link:
https://mattermost.atlassian.net/browse/MM-28100
Related PR: mattermost/mattermost-mobile#4810
2020-09-22 13:52:42 +05:30
Eli Yukelzon
4e9ddd4686 MM-27918 In-Product notices support (#15316) 2020-09-21 10:28:46 +03:00
Maria A Nunez
43ed6ad690 Restricted cloud settings (#15528)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-18 11:39:34 -04:00
Miguel de la Cruz
1d2220141e [MM-26420] Adds wildcard support to Bleve (#14885) 2020-09-18 10:49:56 +02:00
Claudio Costa
334cf9d84a [MM-28131] Fix desanitization of DataSourceReplicas and DataSourceSearchReplicas (#15389)
* Fix desanitization of DataSourceReplicas and DataSourceSearchReplicas

* Fix test case

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-17 20:16:59 +02:00
Miguel de la Cruz
42657fafa1 [MM-25656] Adds a function to allow to update assets subpath in a given directory (#15020) 2020-09-17 16:19:26 +02:00
Claudio Costa
4c64cec948 Fix failing test (#15522)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-17 08:53:04 +02:00
Rodrigo Villablanca
1d4a7c0552 TeamStore migration part 2 (#15396)
* Migration completed

* Fix tests

* Reduce to one line

* Fix: change to plain error

* Fix imports

* Trigger CI

* Fix i18

* Fix merge with master

* Trigger CI
2020-09-16 21:54:39 -03:00
Agniva De Sarker
b329635b23 MM-27169: Remove uneeded fields from App (#15408)
* MM-27169: Remove uneeded fields from App

https://mattermost.atlassian.net/browse/MM-27169

* run app-layers

* Bring back searchengine

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-16 21:46:44 +05:30
Rodrigo Villablanca
f0fc51244c TeamStore migration Part 3 (#15397)
* Migration completed

* TODO
2020-09-16 12:12:10 -03:00
Elisabeth Kulzer
b683a9020e Revert bulgarian language. (#15512) 2020-09-16 16:44:36 +02:00
Ibrahim Serdar Acikgoz
e69a2a41ca [MM-28210] sqlstore/user_store: filter deleted users for GetProfilesInChannel (#15390)
* sqlstore/user_store: filter deleted users for GetProfilesInChannel

* allow GetProfilesInChannel use userGetOptions

* sqlstore/user_store: add more test cases

* store/user_store: refine filter
2020-09-16 11:04:17 +03:00
Claudio Costa
9c272f0b20 [MM-26812] Add support for resumable file uploads (#15252)
* Implement AppendFile for FileBackend

* Split test into subtests

* [MM-26812] Add support for resumable file uploads (#15252)

* Implement UploadSession

* Implement UploadSessionStore

* Add error strings

* Implement resumable file uploads

* Add UploadType

* Fix retry layer tests

* Regenerate store layers

* Fix store error handling

* Use base for filename

* Prevent concurrent uploads on the same upload session

* Fix erroneus error string

* Improve error handling

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>

* Fix translations

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-15 21:28:25 +02:00
Farhan Munshi
6a58834f34 [MM-28475] Add server telemetry for the new system roles (#15437)
* MM-28475 Add server telemetry for the new system roles

* Move system roles to permissions_system_scheme table

* Add testing mocks
2020-09-15 14:21:32 -04:00
Rodrigo Villablanca
9ee9c78412 ChannelStore migration Part 2 (#15405)
* Migration completed

* Several corrections in tests

* Fix imports

* Fix some errors after testing

* Trigger CI

* Fix tests

* Suggestions

* Suggestions

* Add license

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-15 23:18:30 +05:30
Claudio Costa
7abc4f5383 [MM-28033] Perform localcachelayer tests for one DB type only when in CI (#15433)
* Perform localcachelayer tests for one DB type only if in CI

* Add newStoreType() for sqlstore case
2020-09-15 15:05:13 +02:00
Ibrahim Serdar Acikgoz
9c2d2180e6 store/channel_store_categories: fix index out of bounds error (#15423) 2020-09-15 15:25:12 +03:00
Jesús Espino
473e286feb Fix telemetry initialization problem on placeholder usage (#15434) 2020-09-15 08:38:19 +01:00
George Goldberg
4ff12c2aed MM-28115: Add env var for telemetry installation type. (#15431) 2020-09-15 08:27:35 +01:00
Scott Bishel
eba38625eb Implement LDAP Certificate (#15361)
* Implement LDAP Certificate

* add diagnostics and translations

* update from code review

* pass pointer to update pict function

* pass object to first function

* remove debug log messages

* update test to add localmode test

* update lint errors

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-14 12:53:42 -06:00
Ibrahim Serdar Acikgoz
6fd7cb2a80 app/post: continue to loop in case of an error (#15418) 2020-09-11 18:42:01 +03:00
Hossein Ahmadian-Yazdi
5be6da2e9a [MM-23651] No more than 256 members are allowed to be invited (#15414)
* Update constant to 256

* update test

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-11 11:02:59 -04:00
Agniva De Sarker
42eca5315b MM-27957: Fix flaky test TestRecycleDBConns (#15415)
https://mattermost.atlassian.net/browse/MM-27957
2020-09-11 18:33:09 +05:30
Jesse Hallam
1337fa919c MM-28103: prepackage incident-response (#15360)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-10 21:05:20 -03:00
Jesús Espino
1058090a24 Add support for rc versions in upgrader (#15424) 2020-09-10 16:30:03 +02:00
Ayan Banerjee
748526cc4c [MM-23003] Document sql store 'SqlTeamStore' methods (Task 8) fixes (#15318)
Automatic Merge
2020-09-10 07:29:57 -04:00
Claudio Costa
4e1824e143 [MM-28397] Fix data race in SqlSupplier (#15419)
* Fix data race in SqlSupplier

* Add comment

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-10 09:57:00 +02:00
Agniva De Sarker
96a3407b7a Optimize gzip compression of responses (#15416)
It looks like maintainers aren't very active on the repo
and https://github.com/nytimes/gziphandler/pull/107 has come to a standstill.

I don't want to wait forever for this to go in. Let's use a replace directive
to point to the fork. When the PR gets merged, we can just bump the dependency
and remove the replace directive.

Re: the PR, copying the text from the PR description

This gives 40-50% improvements in CPU with very minor increase
in memory, just as a drop-in replacement. I think that is a very reasonable tradeoff.

The library is mature and safe to be used in production.

```
name                 old time/op    new time/op    delta
GzipHandler_S2k-8      74.9µs ± 2%    34.4µs ± 2%  -54.07%  (p=0.000 n=10+9)
GzipHandler_S20k-8      379µs ± 1%     226µs ± 3%  -40.42%  (p=0.000 n=9+10)
GzipHandler_S100k-8    1.95ms ± 2%    1.15ms ± 1%  -41.27%  (p=0.000 n=9+9)
GzipHandler_P2k-8      24.3µs ±25%    10.7µs ±25%  -55.80%  (p=0.000 n=10+10)
GzipHandler_P20k-8      132µs ± 2%      75µs ± 1%  -42.95%  (p=0.000 n=9+10)
GzipHandler_P100k-8     658µs ± 2%     371µs ± 3%  -43.68%  (p=0.000 n=9+10)

name                 old alloc/op   new alloc/op   delta
GzipHandler_S2k-8      7.71kB ± 5%    9.13kB ± 7%  +18.33%  (p=0.000 n=10+10)
GzipHandler_S20k-8     65.1kB ± 3%    70.3kB ± 3%   +8.05%  (p=0.000 n=10+10)
GzipHandler_S100k-8     348kB ± 4%     382kB ± 2%   +9.85%  (p=0.000 n=10+10)
GzipHandler_P2k-8      7.60kB ± 1%    7.93kB ± 2%   +4.33%  (p=0.000 n=10+10)
GzipHandler_P20k-8     64.4kB ± 1%    66.3kB ± 2%   +2.92%  (p=0.000 n=10+10)
GzipHandler_P100k-8     304kB ± 1%     309kB ± 1%   +1.67%  (p=0.000 n=10+9)

name                 old allocs/op  new allocs/op  delta
GzipHandler_S2k-8        21.0 ± 0%      21.0 ± 0%     ~     (all equal)
GzipHandler_S20k-8       24.0 ± 0%      24.0 ± 0%     ~     (all equal)
GzipHandler_S100k-8      27.0 ± 0%      27.0 ± 0%     ~     (all equal)
GzipHandler_P2k-8        21.0 ± 0%      21.0 ± 0%     ~     (all equal)
GzipHandler_P20k-8       24.0 ± 0%      24.0 ± 0%     ~     (all equal)
GzipHandler_P100k-8      26.0 ± 0%      26.0 ± 0%     ~     (all equal)
```

https://mattermost.atlassian.net/browse/MM-28491
2020-09-10 09:02:20 +05:30
Doug Lauder
1b67322fb9 MM-27744 disable Zap for unit tests. (#15398)
MM-27744 disable Zap for unit tests.

Zap has no concept of shutdown or close. Zap is only shutdown when the app exits. Not a problem for console logging, but when creating a new Zap logger that outputs to files on every unit test, that leaves no easy way to clean up until process exit. Depending on what else is running this can exhaust all file handles and cause unit tests to fail.

Zap is now disabled unit tests and uses Logr instead, regardless of config settings. `make test-server` peak file handle usage dropped from ~5K to less than 100.
2020-09-09 15:25:55 -04:00
Agniva De Sarker
1fde5112b6 MM-28444: Optimize GetPostsSince in postgres (#15411)
GetPostsSince is used when loading posts for a channel.

An opportunity for optimization is that the primary SQL query
is repeated twice and then a UNION is constructed for the results.

```
SELECT
		       *
		FROM
		       Posts
		WHERE
		       UpdateAt > :Time AND ChannelId = :ChannelId
		       LIMIT 1000
```

But we can use a CTE for this which caches the results to be reused later.

This leads to the main query being executed once rather than twice. And from
Postgres 12 onwards, CTEs can be inlined which opens the door to further optimizations.

From the docs (https://www.postgresql.org/docs/10/queries-with.html)

> A useful property of WITH queries is that they are evaluated only once per
execution of the parent query, even if they are referred to more than once
by the parent query or sibling WITH queries. Thus, expensive calculations
that are needed in multiple places can be placed within a WITH query to avoid redundant work.
Another possible application is to prevent unwanted multiple evaluations of functions
with side-effects. However, the other side of this coin is that the optimizer is
less able to push restrictions from the parent query down into a WITH query than an ordinary subquery.

In our case, the caveat does not apply because we are only filtering columns and not rows,
so we can safely use it.

Following are the query plan comparisons:
Old: http://tatiyants.com/pev/#/plans/plan_1599394993105
New: http://tatiyants.com/pev/#/plans/plan_1599395970886

As we can see, in old bitmap index scan+heap scan happens twice, but in the new one,
it happens only once.

This has been load tested with a large dataset and confirmed to exhibit good improvements.
2020-09-09 21:22:54 +05:30
Claudio Costa
7612888052 Fix panic in api4.searchPosts (#15410) 2020-09-08 20:36:59 +02:00
Claudio Costa
4c518055f4 [MM-27952] Fix some logging related races (#15404)
* Fix some logging races

* Move initLogging after config store initialization
2020-09-08 20:36:42 +02:00
Jesús Espino
44079785eb Moving diagnostics into a service (#14832)
* Moving diagnostics into a service

* Fixing golint checks

* Fixing tests

* Renaming from diagnostics to telemetry

* Adding missing files

* Initializing telemetry earlier in the server startup

* Fixing tests

* Adding a log for the telemetryID initialization error

* Addressing PR review comments

* Fixing merge problem

* Removing some extra Diagnostics mentions

* Making tests pass
2020-09-08 20:30:54 +02:00
catalintomai
f0eb67fa0d MM-28226: Admin Advisor: add mitigations (#15379)
* Admin Advisor: add mitigation
2020-09-08 07:51:14 -07:00