Граф коммитов

13 Коммитов

Автор SHA1 Сообщение Дата
Mario de Frutos Dieguez
05ec3733c0 [MM-25406] Include missing methods in the search layer (#14799)
* Two missing methods to add in the channel layer

* Added delete user/channel posts methods

- Created in both search engines but only implemented in ES
- Add those methods in the search layer
- Included the PermanentDeleteByUser/Channel methods

* Two new delete documents are included in the bleve code with this
change:

- DeleteChannelPosts
- DeleteUserPosts

These two new functions delete post documents from the index-based
in the filed value provided
2020-06-25 13:45:39 +02:00
Rodrigo Villablanca
41d9c673cf Migrate GetChannels method from ChannelStore to return error interface (#14711)
* Migrate GetChannels method from ChannelStore to return error interface

* Fix testing

* Changed error type: ErrInvalidInput -> ErrNotFound

* Added note about error migrations

* Fix en.json

* Fix i18n

Co-authored-by: Agniva De Sarker <agnivade@yahoo.co.in>
2020-06-11 14:02:03 +05:30
Rodrigo Villablanca
6f28f3526d Migrate PermanentDelete method from ChannelStore to return error inte… (#14706)
Automatic Merge
2020-06-06 11:47:18 +02:00
Rodrigo Villablanca
60cc775cf6 Migrate Update method from ChannelStore to return idiomatic plain error (#14687)
* Partial advances

* Update migrated

* Fix imports

* Suggestions

* Suggestions

* Updating i18n

Co-authored-by: Jesús Espino <jespinog@gmail.com>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-06-02 20:50:34 +05:30
Mario de Frutos Dieguez
f135e6c1bf [MM-24453] Sanitize user term search (#14494)
* Sanitize user term search

Remove the @ at the beginning of the search term in order to
make it possible for ES to return results
2020-05-24 10:50:07 +02:00
Agniva De Sarker
090743de86 MM-24132: Migrate AppError from SaveDirectChannel/channel_store.go (#14318)
* MM-24135: Migrate AppError from SaveChannel/channel_store.go

This is the first POC of migration of store app errors to plain error.

We create a few basic error types in the store package and use
them to return the errors from store methods. In the app layer,
we inspect the error and re-create the exact app errors. This lets
us preserve the same error content, but yet move to plain errors.

Since this is a gradual migration, this means that the error inspection
code will be duplicated across the app layer whenever a store method
is invoked. But all of that should go away once we start propagating
the errors higher up the hierarchy.

There have been a significant amount of changes in the storetest and searchtest
layer, primarily because we have to rename the err variable now that it is of
a different type.

* Addressed review comments

* MM-24132: Migrate AppError from SaveDirectChannel/channel_store.go

This PR migrates 2 new methods SaveDirectChannel and CreateDirectChannel
to return error instead of AppError.

We also need to handle the error internally in SaveMultipleMember for now
until that is migrated too.

* Fix layers

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-05-14 23:41:05 +05:30
Miguel de la Cruz
4fde004a5b [MM-23535] Add disable database search flag and return empty posts results if set (#14245)
* [MM-23535] Add disable database search flag and return empty posts results if set

* Add UpdateConfig function for the SearchStore and hook it into the app lifecycle

* Add the config listener in the server instance instead of using FakeApp

* Instantiate searchlayer as a pointer to avoid passing around copies of it
2020-05-13 14:00:57 +02:00
Agniva De Sarker
729a84a3e6 MM-24135: Migrate AppError from SaveChannel/channel_store.go (#14299)
* MM-24135: Migrate AppError from SaveChannel/channel_store.go

This is the first POC of migration of store app errors to plain error.

We create a few basic error types in the store package and use
them to return the errors from store methods. In the app layer,
we inspect the error and re-create the exact app errors. This lets
us preserve the same error content, but yet move to plain errors.

Since this is a gradual migration, this means that the error inspection
code will be duplicated across the app layer whenever a store method
is invoked. But all of that should go away once we start propagating
the errors higher up the hierarchy.

There have been a significant amount of changes in the storetest and searchtest
layer, primarily because we have to rename the err variable now that it is of
a different type.

* Addressed review comments

* Made all appError origins to be CreateChannel

* Remove typed internal error

* Fix translations

* fix layer generation

Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-05-12 21:23:41 +05:30
Jesús Espino
1e89ef7314 Fixing searching users without team (#14182) 2020-04-02 17:22:08 +02:00
Jesse Hallam
e276a2bf57 avoid panic in search channel store (#14209)
Seen in Kibana logs:
```
http: panic serving 10.128.148.31:47536: runtime error: invalid memory address or nil pointer dereference
goroutine 1063732 [running]:
net/http.(*conn).serve.func1(0xc0042995e0)
	net/http/server.go:1767 +0x139
panic(0x1a4e080, 0x2edab40)
	runtime/panic.go:679 +0x1b2
github.com/mattermost/mattermost-server/v5/model.(*AppError).Error(0x0, 0xc000032000, 0x1a618c0)
	github.com/mattermost/mattermost-server/v5@/model/utils.go:73 +0x26
go.uber.org/zap/zapcore.encodeError(0x1c9c1d0, 0x5, 0x1fd6cc0, 0x0, 0x2028c60, 0xc0089fdfb0, 0xc0089fdfb0, 0x34)
	go.uber.org/zap@v1.13.0/zapcore/error.go:46 +0x3b
go.uber.org/zap/zapcore.Field.AddTo(0x1c9c1d0, 0x5, 0x19, 0x0, 0x0, 0x0, 0x1aa8c00, 0x0, 0x2028c60, 0xc0089fdfb0)
	go.uber.org/zap@v1.13.0/zapcore/field.go:165 +0xc8c
go.uber.org/zap/zapcore.addFields(0x2028c60, 0xc0089fdfb0, 0xc001df0800, 0x2, 0x2)
	go.uber.org/zap@v1.13.0/zapcore/field.go:199 +0xcf
go.uber.org/zap/zapcore.(*jsonEncoder).EncodeEntry(0xc00167d200, 0x2, 0xbf9678532a114ede, 0x3d02c134ae83, 0x2f284e0, 0x0, 0x0, 0x1d0d926, 0x2a, 0x1, ...)
	go.uber.org/zap@v1.13.0/zapcore/json_encoder.go:376 +0x1e4
go.uber.org/zap/zapcore.(*ioCore).Write(0xc00167d230, 0x2, 0xbf9678532a114ede, 0x3d02c134ae83, 0x2f284e0, 0x0, 0x0, 0x1d0d926, 0x2a, 0x1, ...)
	go.uber.org/zap@v1.13.0/zapcore/core.go:86 +0xa9
go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc004b2e8f0, 0xc001df0800, 0x2, 0x2)
	go.uber.org/zap@v1.13.0/zapcore/entry.go:216 +0x117
go.uber.org/zap.(*Logger).Error(0xc001fc4f00, 0x1d0d926, 0x2a, 0xc001df0800, 0x2, 0x2)
	go.uber.org/zap@v1.13.0/logger.go:203 +0x7f
github.com/mattermost/mattermost-server/v5/mlog.(*Logger).Error(...)
	github.com/mattermost/mattermost-server/v5@/mlog/log.go:184
github.com/mattermost/mattermost-server/v5/store/searchlayer.(*SearchChannelStore).SaveMember(0xc0009749c0, 0xc003e69a40, 0x2f284e0, 0x0)
	github.com/mattermost/mattermost-server/v5@/store/searchlayer/channel_layer.go:84 +0x2ed
github.com/mattermost/mattermost-server/v5/store.(*TimerLayerChannelStore).SaveMember(0xc000974ae0, 0xc003e69a40, 0xc000974ae0, 0xc)
	github.com/mattermost/mattermost-server/v5@/store/timer_layer.go:1559 +0x66
github.com/mattermost/mattermost-server/v5/app.(*App).createGroupChannel(0xc006d54f00, 0xc0085d1740, 0x3, 0x4, 0xc00a7396a0, 0x1a, 0xc008a319a0, 0xc0089fd2c0)
	github.com/mattermost/mattermost-server/v5@/app/channel.go:431 +0x6de
github.com/mattermost/mattermost-server/v5/app.(*App).CreateGroupChannel(0xc006d54f00, 0xc0085d1740, 0x3, 0x4, 0xc00a7396a0, 0x1a, 0x0, 0x1710a4516c7)
	github.com/mattermost/mattermost-server/v5@/app/channel.go:371 +0x81
github.com/mattermost/mattermost-server/v5/api4.createGroupChannel(0xc0085d1640, 0x1ff94a0, 0xc003e698f0, 0xc004dfae00)
	github.com/mattermost/mattermost-server/v5@/api4/channel.go:551 +0x437
github.com/mattermost/mattermost-server/v5/web.Handler.ServeHTTP(0xc0050cb6f0, 0x1da88d8, 0x2b0cb00, 0x12, 0x10001, 0x0, 0x0, 0x1ff94a0, 0xc003e698f0, 0xc004dfae00)
	github.com/mattermost/mattermost-server/v5@/web/handlers.go:212 +0x1e9a
github.com/NYTimes/gziphandler.GzipHandlerWithOpts.func1.1(0x1ff8f20, 0xc0060882a0, 0xc004dfae00)
	github.com/NYTimes/gziphandler@v1.1.1/gzip.go:336 +0x23f
net/http.HandlerFunc.ServeHTTP(0xc004024120, 0x1ff8f20, 0xc0060882a0, 0xc004dfae00)
	net/http/server.go:2007 +0x44
github.com/gorilla/mux.(*Router).ServeHTTP(0xc001680180, 0x1ff8f20, 0xc0060882a0, 0xc004dfac00)
	github.com/gorilla/mux@v1.7.3/mux.go:212 +0xe2
net/http.serverHandler.ServeHTTP(0xc00140c540, 0x1ff8f20, 0xc0060882a0, 0xc004dfac00)
	net/http/server.go:2802 +0xa4
net/http.(*conn).serve(0xc0042995e0, 0x1ffeee0, 0xc0063ee780)
	net/http/server.go:1890 +0x875
created by net/http.(*Server).Serve
	net/http/server.go:2927 +0x38e
```
2020-04-01 16:40:12 -03:00
Miguel de la Cruz
4fe25b1cdd [MM-21551] Add search tests structure to test the search engines (#14031)
* WIP

* Adding bleve to go modules

* WIP

* Adding missing files from searchengine implementation

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* User and channel indexing and searches implemented

* Make bleve tests run with in-memory indexes

* Implement post index and deletion tests

* Initial commits for the search layer

* Removing unnecesary indexing

* WIP

* WIP

* More fixes for tests

* Adding the search layer

* Finishing the migration of searchers to the layer

* Removing unnecesary code

* Allowing multiple engines active at the same time

* WIP

* Add simple post search

* Print information when using bleve

* Adding some debugging to understand better how the searches are working

* Making more dynamic config of search engines

* Add post search basics

* Adding the Purge API endpoint

* Fixing bleve config updates

* Adding missed file

* Regenerating search engine mocks

* Adding missed v5 to modules imports

* fixing i18n

* Fixing some test around search engine

* Removing all bleve traces

* Cleaning up the vendors directory and go.mod/go.sum files

* Regenerating timer layer

* Adding properly the license

* Fixing govet shadow error

* Fixing some tests

* Fixing TestSearchPostsFromUser

* Fixing another test

* Fixing more tests

* Fixing more tests

* Removing SearchEngine redundant text from searchengine module code

* Fixing some reindexing problems in members updates

* Fixing tests

* Addressing PR comments

* Reverting go.mod and go.sum

* Addressing PR comments

* Fixing tests compilation

* Fixing govet

* Adding search engine stop method

* Being more explicit on where we use includeDeleted

* Adding GetSqlSupplier test helper method

* Mocking elasticsearch start function

* Fixing tests

* Search tests

* Fix tests

* Fix mod

* Fixing searchEngine for test helpers with store mocks

* Remove loglines

* Fix i18n strings

* Migrate search posts tests

* Fix linter

* Do not run search tests if -short flag is enabled

* Migrate back store tests that didn't belong to the searchlayer

* Fix scopelint issues

Co-authored-by: Jesús Espino <jespinog@gmail.com>
2020-03-30 19:17:40 +02:00
Miguel de la Cruz
11debb55a6 [MM-23510] Avoid race condition searching post's channel by channel id instead of post id (#14154) 2020-03-25 23:50:11 +01:00
Jesús Espino
c66e182b08 Adding the new search engine abstraction (#13304)
* WIP

* Adding bleve to go modules

* WIP

* Adding missing files from searchengine implementation

* WIP

* WIP

* WIP

* WIP

* WIP

* WIP

* User and channel indexing and searches implemented

* Make bleve tests run with in-memory indexes

* Implement post index and deletion tests

* Initial commits for the search layer

* Removing unnecesary indexing

* WIP

* WIP

* More fixes for tests

* Adding the search layer

* Finishing the migration of searchers to the layer

* Removing unnecesary code

* Allowing multiple engines active at the same time

* WIP

* Add simple post search

* Print information when using bleve

* Adding some debugging to understand better how the searches are working

* Making more dynamic config of search engines

* Add post search basics

* Adding the Purge API endpoint

* Fixing bleve config updates

* Adding missed file

* Regenerating search engine mocks

* Adding missed v5 to modules imports

* fixing i18n

* Fixing some test around search engine

* Removing all bleve traces

* Cleaning up the vendors directory and go.mod/go.sum files

* Regenerating timer layer

* Adding properly the license

* Fixing govet shadow error

* Fixing some tests

* Fixing TestSearchPostsFromUser

* Fixing another test

* Fixing more tests

* Fixing more tests

* Removing SearchEngine redundant text from searchengine module code

* Fixing some reindexing problems in members updates

* Fixing tests

* Addressing PR comments

* Reverting go.mod and go.sum

* Addressing PR comments

* Fixing tests compilation

* Fixing govet

* Adding search engine stop method

* Being more explicit on where we use includeDeleted

* Adding GetSqlSupplier test helper method

* Mocking elasticsearch start function

* Fixing tests

Co-authored-by: Miguel de la Cruz <miguel@mcrx.me>
Co-authored-by: mattermod <mattermod@users.noreply.github.com>
2020-03-13 15:33:18 +01:00