Граф коммитов

999 Коммитов

Автор SHA1 Сообщение Дата
jingkai
22853628b2 Refactor term_of_service_store to use squirrel (#16923)
* refactor: replace select statement with squirell

* fix: fixed sqlquery for GetLatest

* fix: gofmt

* fix: replaced old AppError with errors.Wrap

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-03-01 15:43:56 +01:00
Claudio Costa
cc86ec9f50 [MM-33333] Fix GetSidebarCategories() failing on db replicas (#17031)
* Fix GetSidebarCategories failing on replicas

* Simplify further

* Add comment
2021-02-27 07:54:13 -05:00
Agniva De Sarker
6dcbad049c MM-32693: Use master when emoji_layer is invalidated (#16951)
We add a map which tracks invalidations to the emoji caches
and use master whenever a fresh item is fetched from DB.

https://mattermost.atlassian.net/browse/MM-32693

```release-note
NONE
```

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-02-26 17:47:51 +01:00
Claudio Costa
33c8b0ad73 Remove waitForSidebarCategories (#17004) 2021-02-26 09:42:19 +01:00
Jesús Espino
5dd2e75c10 Extracting i18n functionality to i18n core library (#16914)
* extracting i18n functionality to i18n core library

* Removing utils.T

* Adding documentation and changing one function name for better explanation

* Changing other missing utils.T

* Adding license string

* Renaming corelibs to pkg

* Renaming corelibs to pkg (moving directory)

* Renaming from pkg to shared

* Fixing bodyPage.Html casing

* Fixing merges

* Fixing merge problem

* Fixing tests
2021-02-26 08:12:49 +01:00
John Tzikas
809affd6d1 Create basic make commands for configuring golang-migrate (#16938)
* Create basic make commands for configuring golang-migrate

* Showcase full flow with new migrations

* Apply PR suggestions

* Migrate over team members

* Update mocks

* Fix specs

* Move columns that added after table creation onto separate stmts

* Put back gorp table definitions

* Fix issues with golang-migrate that not tracks underlying db driver

* Help prompt after new migration and consistent checksum for bindata

* Put gorp mapping back

* Apply PR suggestiong

* Close migrations after they run

* Add migration file to bindata check

* Updates needed

* Reset store_test

* Add copyright

* Apply PR suggestions

* Fix new circleci check

* Put back upgrade step for backwards comp

* Add store test to test migration directions

* Apply PR suggestions

* Add go-bindata to tools

* Apply PR suggestios

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-02-23 16:22:31 +02:00
Jesús Espino
2b6c0e9746 Adding bulk-indexing and improving a bit the name indexing for bleve and elasticsearch (#16704)
* Adding bulk-indexing and improving a bit the name indexing for bleve and elasticsearch

* Update services/searchengine/bleveengine/bleve.go

Co-authored-by: Mario de Frutos Dieguez <mario@defrutos.org>

* Update store/sqlstore/file_info_store.go

Co-authored-by: Mario de Frutos Dieguez <mario@defrutos.org>

* Update store/sqlstore/file_info_store.go

Co-authored-by: Mario de Frutos Dieguez <mario@defrutos.org>

* Adding tests requested in the PR review

* fixing tests

* Adding a feature flag to avoid indexing files before the feature is released

* Fixing i18n

Co-authored-by: Mario de Frutos Dieguez <mario@defrutos.org>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-02-22 14:28:52 +01:00
John Tzikas
813f60649f Close search replicas on store close (#16985) 2021-02-19 16:30:23 +02:00
John Tzikas
8d6dd195e1 Remote any references to sqlite (#16980)
* Remote any references to sqlite

* Fix specs
2021-02-19 11:09:52 +02:00
Agniva De Sarker
9992a27a90 Revert "MM-24530-Add support for search terms with underscore using postgresql engine (#16618)" (#16950)
This reverts commit ba3b788e46.
2021-02-17 13:52:15 +05:30
Agniva De Sarker
2596a9e630 MM-32893: Refactor the getting of DB handle (#16943)
* MM-32893: Refactor the getting of DB handle

We add a method in *SqlStore that returns the correct DB handle
depending on the context.

https://mattermost.atlassian.net/browse/MM-32893

```release-note
NONE
```

* fix receiver name
2021-02-16 14:00:03 +05:30
Jesús Espino
59d33b9002 Migrate channel to waitgroup for access the store in parallel (#16926)
* Migrate channel to waitgroup for access the store in parallel

* Addressing PR review comments

* Using a cleanest way of group errors

* Reverting go.mod change
2021-02-15 19:32:45 +01:00
Eli Yukelzon
830594004b MM-32652 /threads?unread=true returns null threads and no counts if there are no unread threads (#16885)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-02-15 15:42:05 +02:00
Agniva De Sarker
021c90f29f MM-30882: Fix read-after-write issue for demoting user (#16911)
* MM-30882: Fix read-after-write issue for demoting user

In (*App).DemoteUserToGuest, we would demote a user, and then immediately
read it back to do future operations from the user. This reading back
of the user had the effect of sticking the old value into the cache
after which it would never be updated.

There was another issue along with this, which was when the invalidation
message would broadcast across the cluster, it would hit the cache invalidation
problem where an unrelated store call would miss the cache because
it was invalidated, and then again read from replica and stick the old value.

To fix all these, we return the new value directly from the store method
to avoid having the app to read it again.

And we add a map in the localcache layer which tracks invalidations made,
and then switch to use master if it's true.

The core change is fairly limited, but due to changing the store method signatures,
a lot of code needed to be updated to pass "context.Background". Therefore the PR
just "appears" to be big, but the main changes are limited to app/user.go,
sqlstore/user_store.go and user_layer.go

https://mattermost.atlassian.net/browse/MM-30882

```release-note
Fix an issue where demoting a user to guest would not take effect in
an environment with read replicas.
```

* Fix concurrent map access

* Fixing mistakes

* fix tests
2021-02-12 19:04:05 +05:30
Claudio Costa
6b35b933a8 Revert "MM-27312 Allow uploading 10 files per post (#16235)" (#16908)
This reverts commit 8e2986e5ed.
2021-02-10 20:19:05 +01:00
John Tzikas
1e3b6b56a6 Optimise creation of DM (#16819)
* Optimise creation of dm

* Handle direct channels with the same user

* Cover GetMany with specs and add it on tha cache layer as well

* Fix specs by handling user dming themselves

* Apply PR suggestions

* Apply PR suggestions

* Use require.NoError instead of require.Nil on userstore test

* Improve readability of GetOrCreateDirectChannel

* Apply PR suggestions

* Update layers

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-02-10 15:51:04 +02:00
Jesús Espino
0e9ad9f7f8 Replacing {require,assert}.{Nil,NotNil} with {require,assert}.{NoError,Error} in the store tests (when comparing errors) (#16886)
* Replacing require.Nil with require.NoError

* More replacements

* More Nil/NotNill to NoError/Error

* Other detected errors

* renaming apperr to err

* Removed not needed line

* Rename old appErr variables that are no longer model.AppError values

* Fixing tiny typo

* Reverting changes outside the store (accidentally added)

* Apply suggestions from code review

Co-authored-by: Doug Lauder <wiggin77@warpmail.net>

Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
2021-02-10 11:34:41 +01:00
Eli Yukelzon
9a33c3706a MM-32525 Incorrect number of mentions for channels when threads are enabled (#16853) 2021-02-09 12:03:32 +02:00
Mahmudul Haque
a63dea6c55 unparam lint (#16771)
* fixed: `identifier` is unused lint error

* make saveMultipleMembersT method saveMultipleMembers

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-02-04 11:08:05 +05:30
Daniel Espino García
745d61f388 Fix cleanup threads to properly handle not found errors (#16797)
Automatic Merge
2021-02-01 15:52:27 +01:00
Eli Yukelzon
13616cac0f MM-31711 - Implement cursor paging for threads (#16748)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-01-31 12:28:14 +02:00
Eli Yukelzon
bb7e5b6e9d MM-31712 Server/API: Unreads-only filter for GetUserThreads (#16660)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-01-31 11:54:35 +02:00
Eli Yukelzon
77da23e84b MM-31713 Server/API: GetUserThread method (#16659)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-01-28 18:07:39 +02:00
Agniva De Sarker
2846ad9f93 MM-31391: Add max connection idle timeout to config (#16792)
* Update config.go

* Update telemetry.go

* Update store and store test

* Update settings.go

* use new error code

* Trigger CI

Co-authored-by: Haardik Dharma <dharmahaardik08@gmail.com>
2021-01-28 09:37:15 +05:30
Madhav Hugar
94c24eea20 Fix empty string comparison issues in the codebase (#16686)
Automatic Merge
2021-01-25 11:15:17 +01:00
Eli Yukelzon
c2316c15f3 MM-32027 - Marking a thread as unread not possible or marks all threads as unread. (#16757)
* mark thread as read query fix

* test fix

* lint
2021-01-24 11:37:09 +02:00
Doug Lauder
dbbf985e61 MM-31875 soft delete reactions (#16654)
- soft delete reaction by setting new field Reactions.DeleteAt to non-zero.
- include new field Reactions.UpdateAt
2021-01-20 10:09:23 -05:00
Antwi Gambrah
ba3b788e46 MM-24530-Add support for search terms with underscore using postgresql engine (#16618)
* add support for search terms with underscore using postgresql

* fix failing tests

Initially issue fix applied to all search terms instead of the ones that contains _

* direct cast tsquery

* fix test issues

* refactor underscored quoted term search

* support search term case-insensitive

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-01-20 00:10:10 +05:30
Eli Yukelzon
911d1f070e MM-31710 Server/API: change total_unread_replies to total_unread_threads (#16663)
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-01-19 13:33:57 +02:00
Harrison Healey
90952a1bd5 MM-30314 Don't delete all SidebarChannels when removed from team (#16692)
* MM-30314 Don't delete all SidebarChannels when removed from team

* Revert some changes to minimize diff
2021-01-18 12:42:55 -05:00
Eli Yukelzon
7de4864c66 migration hotfix (#16697) 2021-01-14 17:21:33 +02:00
Eli Yukelzon
b0bddeb5e6 MM-29988 - Update exising Post APIs with collapsed threads (#16503) 2021-01-14 13:46:27 +02:00
Eli Yukelzon
502786580b hotfix-threads-migrate (#16687) 2021-01-13 17:32:25 +02:00
Agniva De Sarker
d356ef349b MM-31876: Fix racy test GetReplica (#16656)
Automatic Merge
2021-01-12 15:15:17 +01:00
Jesse Hallam
9667c96f97 MM-30892: Avoid unnecessary table scan in AnalyticsPostCount. (#16643)
* MM-30892: Avoid unnecessary table scan in AnalyticsPostCount.

[AnalyticsPostCount](d9e8402dc2/store/sqlstore/post_store.go (L1513-L1541)) unconditionally joins on the `Channels` table even when not filtering to a team.

This is a proposal to stop doing this, reducing this to a simple index query when no team is specified. Note that this is technically a subtle semantic difference, since orphaned posts (from deleted or invalid channel ids) are currently excluded. But I submit the benefits outweigh the technical differences here.

Fixes: https://mattermost.atlassian.net/browse/MM-30892

* Update store/sqlstore/post_store.go

Co-authored-by: Claudio Costa <cstcld91@gmail.com>

* Update store/sqlstore/post_store.go

Co-authored-by: Claudio Costa <cstcld91@gmail.com>

Co-authored-by: Claudio Costa <cstcld91@gmail.com>
2021-01-11 14:01:47 -04:00
Jesús Espino
2a63b5552a Add search engine support for files (#16190)
* Add search engine support for files

* Fixing i18n

* Fix golangci-lint

* Fix consistency problem in the Search receiver functio of the SqlFileStore

* Fixing some tests

* Fixing test

* Apply suggestions from code review

Co-authored-by: Mario de Frutos Dieguez <mario@defrutos.org>

* Addressing PR review comments

* Removing some empty lines

* Address PR review comments

* Fixing problem after merge master

* Fixing spelling problem

* Add missed translations

* Fixing certain global variable usages after merge master

* Fixing some constants usage

* Fixing goimports order

Co-authored-by: Mario de Frutos Dieguez <mario@defrutos.org>
2021-01-11 15:14:16 +01:00
Agniva De Sarker
e89b26e8f3 goimports (#16640)
* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w`

* added goimports lint check to .golangci.yml

* format using `goimports -local github.com/mattermost/mattermost-server/v5 -w` for a corner case

* make app-layers, *-mocks and store-layers for ci check

Co-authored-by: Mahmudul Haque <mahmudulhaque@protonmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2021-01-07 22:42:43 +05:30
Ibrahim Serdar Acikgoz
037ce8bc31 downgrade some error logs for store package (#16632) 2021-01-06 16:37:36 +03:00
Agniva De Sarker
c1dd23a3c8 MM-31063: Change constants to use CamelCase (#16608)
* MM-31063: Change constants to use CamelCase

* store package

* change allcaps to camel case (#16615)

* New tools.mod

Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
2021-01-04 11:32:29 +05:30
Agniva De Sarker
c54b262351 MM-31356: Add a minimum required version check for Postgres (#16597)
* MM-31356: Add a minimum required version check for Postgres

To keep conformance with our failing fast and obvious philosophy,
we add a check to prevent Mattermost server from starting
if the postgres version is below 10.0.

This gives customers a chance to upgrade their database before upgrading
their Mattermost version, than to run into weird compatibility issues
after they have finished the upgrade.

https://mattermost.atlassian.net/browse/MM-31356

```release-note
NONE
```

* fix lint errors

* Use a function to pretty-print version string

* rectify comment
2020-12-23 22:06:13 +05:30
Mattermost Build
83afd756b6 Bump version (#16592)
* Bump version

* Uncomment shouldPerformUpgrade

Co-authored-by: Mmbot <mmbot@mattermost>
Co-authored-by: Elisabeth Kulzer <elikul@elikul.de>
2020-12-23 16:33:07 +01:00
Ibrahim Serdar Acikgoz
8036ba86e7 [MM-31398] upgrade error level to critical in case of panic or os.Exit (#16584)
* upgrade error level to critical in case of panic or os.Exit

* reflect revivew comments

* fix error capitalization
2020-12-23 13:11:35 +03:00
Agniva De Sarker
6487d0ca91 MM-31062: Rewrite empty string checks to be more idiomatic (#16587)
https://mattermost.atlassian.net/browse/MM-31062

```release-note
NONE
```
2020-12-22 19:20:59 +05:30
Agniva De Sarker
1a131b54af MM-31064: Simplify indentation (#16565)
* MM-31064: Simplify indentation

Reduce indentation where possible.

```release-note
NONE
```

Command ran to verify:
golangci-lint run --disable-all --enable golint --max-issues-per-linter=10000 --max-same-issues=100000 ./... | grep "block ends with a return state"

https://mattermost.atlassian.net/browse/MM-31064

* incorporate review comments

* enable golint for outdent rule

* fix remaining issues

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-12-21 18:50:47 +03:00
Claudio Costa
10be00f005 Update db schema version for MM-31401 and MM-27312 (#16580) 2020-12-18 22:25:52 +01:00
Doug Lauder
6a3bc50f13 MM-31401 Add Shared flag to Channels table (#16568)
* add Shared flag to Channels table
2020-12-16 14:02:16 -05:00
Diogo Nicolau
8e2986e5ed MM-27312 Allow uploading 10 files per post (#16235)
* Allow uploading 10 files per post

* Increase Post.Fileids column size to fit 10 files

* Add Post.Fileids column changes to DB v531

* Fix current_schema_version

Co-authored-by: Claudio Costa <cstcld91@gmail.com>

* Update store/sqlstore/upgrade.go

Co-authored-by: Claudio Costa <cstcld91@gmail.com>

* Fix schema version

Co-authored-by: Diogo Lima Nicolau <diogo.lima.nicolau@cern.ch>
Co-authored-by: Harrison Healey <harrisonmhealey@gmail.com>
Co-authored-by: Claudio Costa <cstcld91@gmail.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-12-14 09:30:02 +01:00
Florent Peterschmitt
349b83f23a implement and use striped LRU cache to lower mutex contention (#15764)
* Implement Striped LRU cache

* ci

* fix

Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
2020-12-09 19:25:14 +05:30
Carlos Tadeu Panato Junior
f27c55a884 db: upgrade db to 5.30 (#16453)
* db: upgrade db to 5.30
2020-12-07 10:20:22 +01:00
Eli Yukelzon
86e228b6c6 MM-30558 - Add unreadReplies and unreadMentions to thread membership (#16304) 2020-12-06 10:02:53 +02:00