Граф коммитов

53 Коммитов

Автор SHA1 Сообщение Дата
Tim Scheuermann
1738bd6e92 [MM-45991] Check and return JSON errors (#20735) 2022-08-10 10:56:58 +02:00
Tim Scheuermann
b4570afa90 Replace deprecated ioutil with io and os (#20776)
* Replace ioutil with io and os

* Replace ioutil in utils/file.go

* Minor fix to tests and excluded files

Co-authored-by: Tim Scheuermann <tim.scheuermann@mattermost.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2022-08-09 14:25:46 +03:00
Agniva De Sarker
6cc9933b58 Fix minio client breakage with bifrost round 2 (#20758)
* Revert "Revert "MM-43828: Pass object length for some image operations (#20711)" (#20740)"

This reverts commit c7ae090dad.

* Fix to work in bifrost mode

```release-note
NONE
```
2022-08-04 11:21:51 +05:30
Agniva De Sarker
c7ae090dad Revert "MM-43828: Pass object length for some image operations (#20711)" (#20740)
Automatic Merge
2022-07-29 17:53:14 +03:00
Tim Scheuermann
832736da37 [MM-45990] log JSON parsing errors in API4 (#20724) 2022-07-28 17:05:03 +02:00
Agniva De Sarker
072c859219 MM-43828: Pass object length for some image operations (#20711)
For user profile and plugin upload, we use a bytes.Buffer.
In that case, we know the object size and can find
it out from the length of the buffer.

This helps reduce multi-part uploads.

This approach can also be taken in thumbnail and preview
images. However, they use an io.Pipe to directly upload
the image as it is being encoded. We could make the whole
process in separate parts of writing the full image in the
buffer and then upload it. But taking a conservative approach
for now.

Also, while here, removed some unused code.

https://mattermost.atlassian.net/browse/MM-43828

```release-note
NONE
```
2022-07-27 19:02:24 +05:30
Tim Scheuermann
eba08cbb11 Replace ioutil.Discard with io.Discard (#20707) 2022-07-27 11:40:33 +02:00
Agniva De Sarker
b70a0f7880 MM-44542: Add timeout to S3 operations (#20648)
https://mattermost.atlassian.net/browse/MM-44542

```release-note
We add a new config setting AmazonS3RequestTimeoutMilliseconds under
FileSettings which sets a timeout for requests to AWS S3.

By default, the timeout is at 30 seconds.
```

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2022-07-19 10:03:24 +05:30
Ibrahim Serdar Acikgoz
717a4d04a9 Use any instead of interface{} (#20577)
* replace interface{} with any
2022-07-05 09:46:50 +03:00
Collin Eng
83b4e7285e [MM-36872] add threading headers to notification emails (#18232)
* [MM-36872] Adding headers for mail threading

* MM-36872 Add message id and mail threading headers

* MM-36872 Add message id and mail threading headers

* MM-36872 Add msgId, inreplyto, references headers

* Fixing comment issues

* Applying gofmt lint fixes

* Applying gofmt lint fixes

* MM-36872: Updates required from merge.

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Martin Kraft <martin@upspin.org>
2022-06-29 15:29:29 -04:00
Doug Lauder
7e4024665c Add KVStore service and Log service for Focalboard multi product architecture (#20563)
* add KVStore service, add Log service
2022-06-29 09:08:34 -04:00
Doug Lauder
f4dcffd8d3 Additional Product APIs for Focalboard multi-product architecture (#20527)
* bump focalboard version; add services to product API

* add API to fetch logger

* import boards in same fashion as enterprise
2022-06-23 07:55:50 -04:00
Agniva De Sarker
de50943d61 jake/gh 20356 (#20485)
* [MM-44667] Set InsecureSkipVerify for S3 when EnableInsecureOutgoingConnections is set

* Cleanup s3New, customTransport

* Lint fixes

* Cleanup s3store_test.go

* Using default transport

```release-note
NONE
```

Co-authored-by: Jake Gutierrez <jakegut0108@gmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2022-06-20 15:42:01 +05:30
Tim Scheuermann
19098e93fc [MM-42878] Allow + in the timestamp for mlog tests (#20360) 2022-06-03 15:39:03 +02:00
lkyuchukov
79a0d3dac4 Don't log error when SendEmailNotifications is set to false (#20007)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2022-04-29 14:45:24 +05:30
Nathaniel Allred
1198191a1c fix makefile mock generator typos (#19963)
* fix makefile mock generator typos
* rerun mocks
2022-04-13 08:05:57 -05:00
Agniva De Sarker
16b1cbd6e3 spelling (#19956)
* spelling: activated

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: attachments

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: categories

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: category

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: cellspacing

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: channel

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: compliance

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: constraint

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: counts

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: createat

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: deactivate

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: destination

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: exceeded

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: failed

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: foreign

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: hours

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: inactivity

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: inappropriate

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: initialization

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: initialized

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: management

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: mismatch

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: recipients

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: scheme

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: signature

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: subscription

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: suggestions

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: sync

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: telemetry

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* spelling: webhook

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>

* Trigger CI
```release-note
NONE
```

Co-authored-by: Josh Soref <jsoref@users.noreply.github.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2022-04-11 19:31:19 +05:30
Guillermo Vayá
4bf3d6a7f5 [MM-40179] Go into directories when trying to export attachments (#19249)
* MM-40179 traverse directories when trying to export

* fix test

* fix s3 to behabe the same way

* add test for paths way too deep

* test recursion only on localstorage

* make list directories non recursively the default approach

* fix linting error

* fix non-recursive case

Co-authored-by: = <=>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2022-03-15 12:57:29 +01:00
Doug Lauder
eff39010d8 include stdlog level in default mlog config (#19722) 2022-03-08 14:22:26 -05:00
Josh Soref
5d85417a8b spelling: strings (#19542)
Co-authored-by: Josh Soref <jsoref@users.noreply.github.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2022-02-28 15:01:00 +05:30
Agniva De Sarker
3f52bd197a GraphQL (part 2): Model changes and resolvers (#19486)
This PR adds the necessary resolvers to support
the graphQL front-end. Some additional methods
need to be added to the model structs to support
this. Comments have been made to clarify their purpose.

There is a slight duplication of code in the api layer.
But eventually that's going to go away when we move
away from the REST API entirely.

The schema is in api4/schema.graphqls and gets compiled
into the binary as an asset.

The GraphiQL editor url is at GET /api/v5/graphql.

Everything is behind the feature flag MM_FEATUREFLAGS_GRAPHQL.

```release-note
NONE
```
2022-02-11 12:37:05 +05:30
Josh Soref
294bd44971 [GH-19267] Spelling comments (#19268)
* spelling: comments

Signed-off-by: Josh Soref <jsoref@users.noreply.github.com>
Co-authored-by: Josh Soref <jsoref@users.noreply.github.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2022-01-20 10:07:27 +05:30
Pablo Andrés Vélez Vidal
f4809def78 MM-40752 - set encryption when copying files to minio honoring config defined value (#19251)
Co-authored-by: Pablo Velez Vidal <pablo.velez@mattermost.com>
2022-01-05 10:04:16 +01:00
Agniva De Sarker
e4ef39d68f Remove unnecessary test TestValidateToken (#19177)
The TestValidateToken was actually testing
the underlying library rather than testing our code.
So, in a way, it was ineffective.

The actual flaky test was that in a rare care,
the random string generated might actually lead
to 000000 being an actual code rather than invalid.

So using random strings is fundamentally incorrect.
Even the tests in the library use hardcoded strings
and not random strings.

To fix this properly would be to use hardcoded strings,
but then we would just be testing the library
and not our code. To keep things simple,
we just keep the test to verify the error message
and remove the others.

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4/cidjhgy1ag3yktn5eszae1489yr

```release-note
NONE
```
2021-12-15 23:01:32 +05:30
Agniva De Sarker
688ee34054 MM-38712: Flush logs while exiting from store upgrade (#18438)
Our store upgrade methods used mlog.Critical and then slept
for a second and triggered os.Exit.

First of all, with the new mlog refactor Critical wasn't a standard
log level so it was never to be logged.

And secondly, this completely missed the fact that there already
existed an API method Fatal which did exactly this.

We mark the Critical API as deprecated, and use Fatal for all
occurences.

The choice should be between either to use Error or Fatal.

While here, we also remove the non-standard exit codes which
just confused things further while debugging.

https://mattermost.atlassian.net/browse/MM-38712

```release-note
NONE
```
2021-09-22 20:12:57 +05:30
Doug Lauder
bb659d03fd Fix plugin logging missing context fields
Fix an issue where context fields logged by server on behalf of plugins didn't contain correct name/value pairs:
- bump Logr version to add key/value methods to sugar logger
- expose factory args when configuring logging with custom target types (needed for FocalBoard to create log target adapter that converts typed fields into slices of interface{} as per plugin logging API)
2021-09-13 10:12:18 -04:00
Doug Lauder
2560469bc7 MM-38016 Fix racy unit test (TestService_AddTopicListener) (#18207)
* remove CreateTestLogger API 
* add missing mockServer.Shutdown
2021-08-18 13:44:04 -04:00
Devin Binnie
0dbaa48741 Change inbucket default port from 10080 to 9001 (#18145) 2021-08-18 09:31:25 -04:00
Doug Lauder
328d96b76b MM-38004 fix racy unit test (#18189)
* fix racy unit test
* check for nil err
* fix stringer and namederr as well
2021-08-18 08:42:48 -04:00
Doug Lauder
a4507327a7 MM-36764 mlog refactor (#18118)
Refactor mlog
- simplify mlog by removing redundant code
- remove Zap dependency
- update unit test helpers
- update logging config
- update auditing
2021-08-17 16:08:04 -04:00
Ben Schumacher
757dc96461 [MM-37772] Idiomatic naming (URL, URI, API) (#18128)
* s/Url/URL/g & s/Uri/URI/g

* s/Api/API/g
2021-08-16 19:46:44 +02:00
Ben Schumacher
fd853e74a6 [MM-37755] Idiomatic naming (SMTP, TLS, TCP, XML, CSS, HTML, HTTP) (#18103) 2021-08-12 11:49:16 +02:00
Agniva De Sarker
c4b4e1bc38 MM-36271: Bump major version to 6.0 (#17973)
https://mattermost.atlassian.net/browse/MM-36271

```release-note
We bump the major version to 6.0
```
2021-07-22 12:21:47 +05:30
Agniva De Sarker
79d4e9e9a9 DB RPC driver: add master/replica support (#17792)
Automatic Merge
2021-06-17 17:40:22 +02:00
Agniva De Sarker
4b95d47923 DB driver implementation via RPC (#17779)
This PR builds up on the pass-through DB driver to a fully functioning DB driver implementation via our RPC layer.

To keep things separate from the plugin RPC API, and have the ability to move fast with changes, a separate field Driver is added to MattermostPlugin. Typically the field which is required to be compatible are the API and Helpers. It would be well-documented that Driver is purely for internal use by Mattermost plugins.

A new Driver interface was created which would have a client and server implementation. Every object (connection, statement, etc.) is created and added to a map on the server side. On the client side, the wrapper structs hold the object id, and communicate via the RPC API using this id.

When the server gets the object id, it picks up the appropriate object from its map and performs the operation, and sends back the data.

Some things that need to be handled are errors. Typical error types like pq.Error and mysql.MySQLError are registered with encoding/gob. But for error variables like sql.ErrNoRows, a special integer is encoded with the ErrorString struct. And on the cilent side, the integer is checked, and the appropriate error variable is returned.

Some pending things:

- Context support. This is tricky. Since context.Context is an interface, it's not possible to marshal it. We have to find a way to get the timeout value from the context and pass it.
- RowsColumnScanType(rowsID string, index int) reflect.Type API. Again, reflect.Type is an interface.
- Master/Replica API support.
2021-06-17 08:53:52 +05:30
Agniva De Sarker
32bc33942f DB driver: Add wrapper structs for low-level objects (#17709)
This is a necessary step for adding the RPC communication
because each method call of each object needs to go through
the RPC layer.

Hence doing this part in a separate PR rather than heaping
everything in a single PR.

https://focalboard-community.octo.mattermost.com/workspace/zyoahc9uapdn3xdptac6jb69ic?id=285b80a3-257d-41f6-8cf4-ed80ca9d92e5&v=495cdb4d-c13a-4992-8eb9-80cfee2819a4&c=c7386db7-65fd-469b-8bcf-8dc8f8e61e4f

```release-note
NONE
```
2021-06-04 14:24:22 +05:30
Agniva De Sarker
e35c78cac2 Pass-through DB Driver implementation (#17685)
* Pass-through DB Driver implementation

This is the first step in implementing a DB layer via RPC.

The plan is to migrate the mattermost-plugin-api to use
this DB connector so that all queries start to get
routed through this library.

And then we will add the DB query capability to the plugin RPC
API and route all queries via RPC. At that point, this will be
completely transparent to all plugins because they will already
be using the DB connector and everything will be behind the scenes
for them.

https://focalboard-community.octo.mattermost.com/workspace/zyoahc9uapdn3xdptac6jb69ic?id=285b80a3-257d-41f6-8cf4-ed80ca9d92e5&v=495cdb4d-c13a-4992-8eb9-80cfee2819a4&c=c7386db7-65fd-469b-8bcf-8dc8f8e61e4f

```release-note
NONE
```

* remove deprecated interfaces
2021-05-27 14:59:06 +05:30
Ben Schumacher
f14b0097dd [MM-34798] Respect MM_SERVER_PATH when looking for templates (#17410) 2021-04-21 20:35:57 +02:00
Agniva De Sarker
fa16ecf98a MM-34787: Add colored output for non JSON console logs (#17388)
* MM-34787: Add colored output for non JSON console logs

https://mattermost.atlassian.net/browse/MM-34787

```release-note
A new field EnableColor is added to LogSettings and NotificationLogSettings.
Non-JSON console logs will now be colored if that field is set to true.
```

* Trigger CI
2021-04-13 23:45:56 +05:30
Max Erenberg
3a3ec001bf [MM-33826] add more detailed S3 error messages (#17182)
Automatic Merge
2021-04-09 18:16:30 +02:00
Doug Lauder
02196e04fa MM-27493 Shared channels (MVP) (#17301)
Remote Cluster Service
- provides ability for multiple Mattermost cluster instances to create a trusted connection with each other and exchange messages
- trusted connections are managed via slash commands (for now)
- facilitates features requiring inter-cluster communication, such as Shared Channels
Shared Channels Service
- provides ability to shared channels between one or more Mattermost cluster instances (using trusted connection)
- sharing/unsharing of channels is managed via slash commands (for now)
2021-04-01 13:44:56 -04:00
Doug Lauder
1fb7f512ff fix race in unit test re logging (#17235) 2021-03-29 08:19:42 -04:00
Claudio Costa
cd0f92e2e7 [MM-34164] Fix flaky TestS3FileBackend (#17229)
* Fix flaky test

* Disable dumping data on error
2021-03-25 16:58:29 +01:00
Agniva De Sarker
94abb34821 MM-34271: Skip test TestLoggingAfterInitialized (#17226)
https://mattermost.atlassian.net/browse/MM-34271

```release-note
NONE
```

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-03-25 19:29:06 +05:30
Max Erenberg
4699845c0a Mm 29605 read permission s3 bucket (#16977)
Automatic Merge
2021-03-16 15:48:32 +01:00
Jesús Espino
2417fb265d Moving markdown to corelibs (#16916)
* Moving markdown to corelibs

* Fix golangci-lint problem

* Renaming corelibs to pkg

* Fixing golangci-lint

* Renaming from pkg to shared

* Fixing gofmt

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-03-16 11:18:41 +01:00
Mahmudul Haque
62fa7b9350 unparam lint (#16927)
* fix "always receives ..." lint err

* add unparam lint check

* fix failed test

* rm details param

* ignore unparam lint

* magic string replaced with model.NewRandomString

* rm unused enableComplianceFeatures param

* generate random message inside createPost

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-03-16 12:48:20 +05:30
Jesús Espino
8abb4184b5 Extracting mail service into shared libs (#17030)
* Extracting mail service into shared libs

* Fixing golangci-lint
2021-03-15 11:50:13 +01:00
Jesús Espino
95b0809850 Extracting html templates into a library (#16946)
* Extracting html templates into a library

* Moving tests to the right place

* Fixing tests

* Addressing PR review comments

* Addressing PR review comments

* Replacing attomic with RWMutex

* Returning errors as channel for Templates watcher

* Address PR review comments

* Other small fixes

* Simplifying NewWithWatcher

* Addressing PR review comments

* Making error handling on rendering templates more robust

* Fixing tests

* Changing how we return errors

* Fixing shadow variables

* Addressing PR review comments

* Logging errors from the outside of sendNotificationEmail

* Fixing lock in shutdown

* Fixing the resource copy for commands tests temporary directories

* Removing unused import

* A couple of tiny fixes
2021-03-12 18:46:43 +01:00
Jesús Espino
9cc5089af8 Moving mlog to corelibs (#16915)
* Moving mlog to corelibs

* Regenerating app layers

* Fix golangci-lint problem

* Fixing golangci-lint errors

* Renaming from corelibs to shared

* Renaming from corelibs to shared

* Fixing import

* Fixing merge problems

* Fixing build
2021-03-05 09:18:37 +01:00