Граф коммитов

56 Коммитов

Автор SHA1 Сообщение Дата
Guillermo Vayá
c6f80dfe0a [MM-44573] Sanitize options (#20380)
Automatic Merge
2022-06-09 00:36:28 +03:00
Agniva De Sarker
319a2b2428 MM-39690: Clean up some unnecessary logs (#20368)
These logs would appear at every request
and didn't have any useful info for the admin.
It was more of a developer logging thing
and therefore removing it.

https://mattermost.atlassian.net/browse/MM-39690
```release-note
NONE
```
2022-06-07 21:21:36 +05:30
Ibrahim Serdar Acikgoz
fe3816cc20 store: guests will only receive results of which channels they are in (#20295)
* store: guests will only receive results of which channels they are in

* api4/channel_test: add test case for guest accounts channel autocomplete

* apply to searchengine and also for AutocompleteInTeam

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2022-06-01 09:18:59 +03:00
Agniva De Sarker
c1f3827801 MM-38164: Paginate the GetPostThread API (#19485)
We implement a cursor based pagination model
to page through the posts in a given thread.

The cursor is a combination of the post.CreateAt+
post.Id to differentiate multiple posts in a given
timestamp.

Some additional parameters like direction, fromPost,
fromCreateAt and perPage were introduced to implement
this.

```release-note
NONE
```
2022-03-24 12:51:41 +05:30
Agniva De Sarker
a733fb840d GraphQL (part 1): Store layer (#19465)
We add the necessary store methods to use GraphQL

```release-note
NONE
```
2022-02-02 22:44:22 +05:30
Agniva De Sarker
5ad7fb5b6f Remove job server dependency searchengine (#18908)
* Remove job server dependency searchengine

The job server field was not being used. We remove it
as part of a cleanup of server initialization refactor.

https://community-daily.mattermost.com/boards/workspace/zyoahc9uapdn3xdptac6jb69ic/285b80a3-257d-41f6-8cf4-ed80ca9d92e5/495cdb4d-c13a-4992-8eb9-80cfee2819a4?c=8a171d5e-4398-4bc9-8fbf-8b9b14f37069

```release-note
NONE
```

* Fix missing field

```release-note
NONE
```
2021-11-03 18:44:57 +01:00
Agniva De Sarker
e24f22745e [MM-18946] [MM-26721] [MM-6842] Cross team search+private channel autocomplete (#18468)
* Show private channels in autocomplete

This is supported in all Engines:
MySQL, Postgres, Bleve, Elasticsearch.

https://mattermost.atlassian.net/browse/MM-18496

```release-note
Private channels will now appear in channel autocomplete.
If you are using Bleve or ElasticSearch, you will have to reindex
the channels again to populate them with the new attributes.
```

A large chunk of this work has been based on the earlier
effort at https://github.com/mattermost/mattermost-server/pull/17804.

Full credit goes to https://github.com/arvinDarmawan.

* Add comment

```release-note
NONE
```

* Adding more tests

```release-note
NONE
```

* fix more tests

```release-note
NONE
```

* tmp

```release-note
NONE
```

* more fixes

```release-note
NONE
```

* add tests

```release-note
NONE
```

* Add review comments from previous PR

```release-note
NONE
```

* Add API to return all channels from all team

```release-note
NONE
```

* Added support for bleve and ES

```release-note
NONE
```

* Streaming response for GetAllChannels

```release-note
NONE
```

* Fix tests

```release-note
NONE
```

* Trigger CI
```release-note
NONE
```

* fix tests

```release-note
NONE
```

* Addressing review comments

```release-note
NONE
```

* Fix lint

```release-note
NONE
```

* Removing flaky test

```release-note
NONE
```

* Address comments

```release-note
NONE
```

* Trigger CI
```release-note
NONE
```

* Added /users/<userid>/channel_members endpoint

```release-note
NONE
```

* Minor edit

```release-note
NONE
```

* Improve embedding

```release-note
NONE
```

* Fix lint error

```release-note
NONE
```

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-10-26 11:30:59 +05:30
Daniel Espino García
02a9ef3f82 [MM-38216] Add API endpoint and adapt search to allow multi-team search (#18371)
* Add API endpoint and adapt search to allow multi-team search

* Refactor handler, refactor sql query to use squirrel, rename app and store functions and add tests

* Fix lint

* Fix search engines and remove unneeded comments

* Fix test

* Remove user from channel after test
2021-09-23 14:43:09 +02:00
Claudio Costa
04b27ce93c [MM-28985] Remove pointers to slice (part 1) (#18034)
* Remove pointers to slice (part 1)

* Remove use of pointers to slice from model package (#18045)

* Fix after merge
2021-08-17 11:18:33 +02:00
Agniva De Sarker
c4b4e1bc38 MM-36271: Bump major version to 6.0 (#17973)
https://mattermost.atlassian.net/browse/MM-36271

```release-note
We bump the major version to 6.0
```
2021-07-22 12:21:47 +05:30
Ben Schumacher
97ccf0bdf6 [MM-16751] golint model (#17896) 2021-07-21 16:38:39 +02:00
Jesús Espino
368b642105 Fixing problem with read-replica on indexing files (#17460) 2021-04-20 12:30:43 +02:00
Doug Lauder
02196e04fa MM-27493 Shared channels (MVP) (#17301)
Remote Cluster Service
- provides ability for multiple Mattermost cluster instances to create a trusted connection with each other and exchange messages
- trusted connections are managed via slash commands (for now)
- facilitates features requiring inter-cluster communication, such as Shared Channels
Shared Channels Service
- provides ability to shared channels between one or more Mattermost cluster instances (using trusted connection)
- sharing/unsharing of channels is managed via slash commands (for now)
2021-04-01 13:44:56 -04:00
Eli Yukelzon
0cc72342de MM-33544 is_following prop in getPosts API methods (#17093)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-03-23 13:31:54 +02:00
Martin Raymond Kraft
58dce5930e [MM-31094] Replication Lag (#16888)
* MM-31094: Adds tooling to develop and test using a MySQL instance with replication lag. Adds some lazy lookups to fallback to master if results are not found.

* MM-31094: Removes mysql-read-replica from default docker services.

* MM-31094: Switches (store..SessionStore).Get and (store.TeamStore).GetMember to using context.Context.

* MM-31094: Updates (store.UsersStore).Get to use context.

* MM-31094: Updates (store.PostStore).Get to use context.

* MM-31094: Removes feature flag and config setting.

* MM-31094: Rolls back some master reads.

* MM-31094: Rolls a non-cache read.

* MM-31094: Removes feature flag from the store.

* MM-31094: Removes unused constant and struct field.

* MM-31094: Removes some old feature flag references.

* MM-31094: Fixes some tests.

* MM-31094: App layers fix.

* MM-31094: Fixes mocks.

* MM-31094: Don't reparse flag.

* MM-31094: No reparse.

* MM-31094: Removed unused FeatureFlags field.

* MM-31094: Removes unnecessary feature flags variable declarations.

* MM-31094: Fixes copy-paste error.

* MM-31094: Fixes logical error.

* MM-30194: Removes test method from store.

* Revert "MM-30194: Removes test method from store."

This reverts commit d5a6e8529bd5f4d993824c828e239d009b05e567.

* MM-31094: Conforming to make's strange syntax.

* MM-31094: Configures helper for read replica with option.

* MM-31094: Adds some missing ctx's.

* MM-31094: WIP

* MM-31094: Updates test names.

* MM-31094: WIP

* MM-31094: Removes unnecessary master reads.

* MM-31094: ID case changes out of scope.

* MM-31094: Removes unused context.

* MM-31094: Switches to a helper. Removes some var naming changes. Fixes a merge error.

* MM-31094: Removes SQLITE db driver ref.

* MM-31094: Layer generate fix.

* MM-31094: Removes unnecessary changes.

* MM-31094: Moves test method.

* MM-31094: Re-add previous fix.

* MM-31094: Removes make command for dev.

* MM-31094: Fix for login.

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-03-12 12:37:30 -05:00
Jesús Espino
9cc5089af8 Moving mlog to corelibs (#16915)
* Moving mlog to corelibs

* Regenerating app layers

* Fix golangci-lint problem

* Fixing golangci-lint errors

* Renaming from corelibs to shared

* Renaming from corelibs to shared

* Fixing import

* Fixing merge problems

* Fixing build
2021-03-05 09:18:37 +01:00
Agniva De Sarker
021c90f29f MM-30882: Fix read-after-write issue for demoting user (#16911)
* MM-30882: Fix read-after-write issue for demoting user

In (*App).DemoteUserToGuest, we would demote a user, and then immediately
read it back to do future operations from the user. This reading back
of the user had the effect of sticking the old value into the cache
after which it would never be updated.

There was another issue along with this, which was when the invalidation
message would broadcast across the cluster, it would hit the cache invalidation
problem where an unrelated store call would miss the cache because
it was invalidated, and then again read from replica and stick the old value.

To fix all these, we return the new value directly from the store method
to avoid having the app to read it again.

And we add a map in the localcache layer which tracks invalidations made,
and then switch to use master if it's true.

The core change is fairly limited, but due to changing the store method signatures,
a lot of code needed to be updated to pass "context.Background". Therefore the PR
just "appears" to be big, but the main changes are limited to app/user.go,
sqlstore/user_store.go and user_layer.go

https://mattermost.atlassian.net/browse/MM-30882

```release-note
Fix an issue where demoting a user to guest would not take effect in
an environment with read replicas.
```

* Fix concurrent map access

* Fixing mistakes

* fix tests
2021-02-12 19:04:05 +05:30
Eli Yukelzon
b0bddeb5e6 MM-29988 - Update exising Post APIs with collapsed threads (#16503) 2021-01-14 13:46:27 +02:00
Jesús Espino
2a63b5552a Add search engine support for files (#16190)
* Add search engine support for files

* Fixing i18n

* Fix golangci-lint

* Fix consistency problem in the Search receiver functio of the SqlFileStore

* Fixing some tests

* Fixing test

* Apply suggestions from code review

Co-authored-by: Mario de Frutos Dieguez <mario@defrutos.org>

* Addressing PR review comments

* Removing some empty lines

* Address PR review comments

* Fixing problem after merge master

* Fixing spelling problem

* Add missed translations

* Fixing certain global variable usages after merge master

* Fixing some constants usage

* Fixing goimports order

Co-authored-by: Mario de Frutos Dieguez <mario@defrutos.org>
2021-01-11 15:14:16 +01:00
Agniva De Sarker
e89b26e8f3 goimports (#16640)
* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w`

* added goimports lint check to .golangci.yml

* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w` for a corner case

* make app-layers, *-mocks and store-layers for ci check

Co-authored-by: Mahmudul Haque <mahmudulhaque@protonmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-01-07 22:42:43 +05:30
Ibrahim Serdar Acikgoz
037ce8bc31 downgrade some error logs for store package (#16632) 2021-01-06 16:37:36 +03:00
Agniva De Sarker
c1dd23a3c8 MM-31063: Change constants to use CamelCase (#16608)
* MM-31063: Change constants to use CamelCase

* store package

* change allcaps to camel case (#16615)

* New tools.mod

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
2021-01-04 11:32:29 +05:30
SimonSimonB
8b6ac5f5d2 Removed AppError from searchlayer/user_layer.go (#16532)
Automatic Merge
2021-01-04 05:45:17 +01:00
Ibrahim Serdar Acikgoz
40e16ba07f [MM-31359] remove duplicated error logs (#16583)
* remove duplicated error logs

* reflect review comments

* add context

* add error details
2020-12-29 09:48:36 +03:00
Mohammed Salman
6b7e261e06 Remove model.AppError from searchlayer/post_layer.go (#16452)
* Remove model.AppError from searchlayer/post_layer.go

* change nErr to err

* fix tautological condition

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-12-16 19:19:18 +05:30
Miguel de la Cruz
ec5c33cf14 [MM-31357] Do not return empty appError as part of users autocompletion (#16550)
Automatic Merge
2020-12-15 14:45:17 +01:00
Mohammed Salman
6e92b76932 Remove model.AppError from searchlayer/channel_layer.go (#16428)
Automatic Merge
2020-12-07 10:45:17 +01:00
Jesús Espino
a74fe05695 Removing supplier concept from the sql store (#16355)
* Removing supplier concept from the sql store

* Removing other metions to supplier

* Fixing gofmt

* Fixing gofmt

* Renaming NewSqlStore to New

* Fixing tests

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-12-02 12:06:23 +01:00
John Tzikas
b375037a42 Syncronize access when updating config on search layer (#16393)
* Syncronize access when updating config on search layer

* Simplify validity of race condition on tests

* Add License header on new file

* Use atomic.Value for config access on search layer

* Apply PR suggestions
2020-11-25 12:23:43 +02:00
Rodrigo Villablanca
96f1739f8f UserStore migration (#15563)
* Migration completed

* Fix tests

* Fix tests

* Fix tests

* Suggestions

* Trigger CI

* Suggestions

* Merge with master

* Trigger CI

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
2020-10-26 15:11:27 +05:30
Rodrigo Villablanca
bb4df5a68e ChannelStore migration Part 3 (#15504)
* Migration finished

* Change error var name

* Fix imports

* Fix tests

* Merge with master

* Doing some suggestions

* More suggestions

* Fix i18n

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
2020-10-04 10:12:29 +05:30
Rodrigo Villablanca
8118cac350 Poststore migration part3 (#15505)
* Migration completed

* Order in translations file

* Fix: lints

* Trigger CI

* Fix message key

* Change mlog.Error for mlog.Warn

* Fix imports

* Adding translations needed for EE

* Trigger CI

* Fix merge with master

Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-24 10:46:36 +05:30
Rodrigo Villablanca
1d4a7c0552 TeamStore migration part 2 (#15396)
* Migration completed

* Fix tests

* Reduce to one line

* Fix: change to plain error

* Fix imports

* Trigger CI

* Fix i18

* Fix merge with master

* Trigger CI
2020-09-16 21:54:39 -03:00
Rodrigo Villablanca
f0fc51244c TeamStore migration Part 3 (#15397)
* Migration completed

* TODO
2020-09-16 12:12:10 -03:00
Rodrigo Villablanca
9ee9c78412 ChannelStore migration Part 2 (#15405)
* Migration completed

* Several corrections in tests

* Fix imports

* Fix some errors after testing

* Trigger CI

* Fix tests

* Suggestions

* Suggestions

* Add license

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-15 23:18:30 +05:30
Rodrigo Villablanca
2cb655ed67 Migration of ChannelStore Part 1 (#15235)
* Lint: remove unnecessary use of sprintf

* Fix i18n

* Returning the right error (InternalServerError)

* Doing some suggestions

* Fix store layers

* Fix missed translation

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-09-03 11:24:46 +05:30
Rodrigo Villablanca
54f86e7fb1 Migration of TeamStore Part 1 (#15246)
Automatic Merge
2020-09-03 00:29:57 -04:00
Abdulkadir Poyraz
f12ca27bac [MM-24522] remove duplication in OR and IncludeDeletedChannels params for search (#14573) 2020-08-31 13:40:58 +02:00
Rodrigo Villablanca
6a50106cd9 Filter stop words when searching posts in mysql (#14509) 2020-08-21 17:58:17 +02:00
Jesús Espino
19173ea6ff Migrating Team.SaveMember/SaveMultipleMembers into errors (#14924)
* Migrating Team.SaveMember/SaveMultipleMembers into errors

* Removing copy/paste comments

* Fixing another small thing

* Fixing tests

* Addressing PR review comments

* Fixing some tests

* Fixing golangci-lint

* Fixing linter

* Regenerating store layers

* Fix lint error after merge

* Other lint error fixed

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-08-12 21:02:41 +02:00
Rodrigo Villablanca
5566395032 First part of PostStore migration (#15123)
Automatic Merge
2020-08-12 13:35:57 -04:00
Agniva De Sarker
77f7a97bee MM-25563: Add endpoint to fetch only archived channels (#15031)
Automatic Merge
2020-07-27 13:31:39 +02:00
Mario de Frutos Dieguez
4c33b7a35d [MM-25767] Quick switch users search is always falling back to the database (#14762)
* Refactor of getListOfAllowedChannelsForTeam

Also, I've fixed some problematic scenarios:

- The quick search doesn't provide team id so it was always failing
- When the teamId was empty and view restrictions too we always
  return all the channels because if we do "strings.Contains("foo", "")
  it always returns true
- There was a case, in quick search with a guest account, where you
  get an empty result because teamId is not provided

* Error if team id is not passed when searching for the channel

If we search users passing the channel id, we must pass the team id
too so we avoid returning all the channels if we remove the empty
team id restriction we have in the getListOfAllowedChannelsForTeam

There is no known reason to search for a channel but not filtering
using the team id. Even guest accounts belong to a team
2020-06-26 20:37:35 +02:00
Mario de Frutos Dieguez
05ec3733c0 [MM-25406] Include missing methods in the search layer (#14799)
* Two missing methods to add in the channel layer

* Added delete user/channel posts methods

- Created in both search engines but only implemented in ES
- Add those methods in the search layer
- Included the PermanentDeleteByUser/Channel methods

* Two new delete documents are included in the bleve code with this
change:

- DeleteChannelPosts
- DeleteUserPosts

These two new functions delete post documents from the index-based
in the filed value provided
2020-06-25 13:45:39 +02:00
Rodrigo Villablanca
41d9c673cf Migrate GetChannels method from ChannelStore to return error interface (#14711)
* Migrate GetChannels method from ChannelStore to return error interface

* Fix testing

* Changed error type: ErrInvalidInput -> ErrNotFound

* Added note about error migrations

* Fix en.json

* Fix i18n

Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
2020-06-11 14:02:03 +05:30
Rodrigo Villablanca
6f28f3526d Migrate PermanentDelete method from ChannelStore to return error inte… (#14706)
Automatic Merge
2020-06-06 11:47:18 +02:00
Rodrigo Villablanca
60cc775cf6 Migrate Update method from ChannelStore to return idiomatic plain error (#14687)
* Partial advances

* Update migrated

* Fix imports

* Suggestions

* Suggestions

* Updating i18n

Co-authored-by: Jesús Espino <jespinog@gmail.com>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-06-02 20:50:34 +05:30
Mario de Frutos Dieguez
f135e6c1bf [MM-24453] Sanitize user term search (#14494)
* Sanitize user term search

Remove the @ at the beginning of the search term in order to
make it possible for ES to return results
2020-05-24 10:50:07 +02:00
Agniva De Sarker
090743de86 MM-24132: Migrate AppError from SaveDirectChannel/channel_store.go (#14318)
* MM-24135: Migrate AppError from SaveChannel/channel_store.go

This is the first POC of migration of store app errors to plain error.

We create a few basic error types in the store package and use
them to return the errors from store methods. In the app layer,
we inspect the error and re-create the exact app errors. This lets
us preserve the same error content, but yet move to plain errors.

Since this is a gradual migration, this means that the error inspection
code will be duplicated across the app layer whenever a store method
is invoked. But all of that should go away once we start propagating
the errors higher up the hierarchy.

There have been a significant amount of changes in the storetest and searchtest
layer, primarily because we have to rename the err variable now that it is of
a different type.

* Addressed review comments

* MM-24132: Migrate AppError from SaveDirectChannel/channel_store.go

This PR migrates 2 new methods SaveDirectChannel and CreateDirectChannel
to return error instead of AppError.

We also need to handle the error internally in SaveMultipleMember for now
until that is migrated too.

* Fix layers

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-05-14 23:41:05 +05:30
Miguel de la Cruz
4fde004a5b [MM-23535] Add disable database search flag and return empty posts results if set (#14245)
* [MM-23535] Add disable database search flag and return empty posts results if set

* Add UpdateConfig function for the SearchStore and hook it into the app lifecycle

* Add the config listener in the server instance instead of using FakeApp

* Instantiate searchlayer as a pointer to avoid passing around copies of it
2020-05-13 14:00:57 +02:00