* MM-31094: Adds tooling to develop and test using a MySQL instance with replication lag. Adds some lazy lookups to fallback to master if results are not found.
* MM-31094: Removes mysql-read-replica from default docker services.
* MM-31094: Switches (store..SessionStore).Get and (store.TeamStore).GetMember to using context.Context.
* MM-31094: Updates (store.UsersStore).Get to use context.
* MM-31094: Updates (store.PostStore).Get to use context.
* MM-31094: Removes feature flag and config setting.
* MM-31094: Rolls back some master reads.
* MM-31094: Rolls a non-cache read.
* MM-31094: Removes feature flag from the store.
* MM-31094: Removes unused constant and struct field.
* MM-31094: Removes some old feature flag references.
* MM-31094: Fixes some tests.
* MM-31094: App layers fix.
* MM-31094: Fixes mocks.
* MM-31094: Don't reparse flag.
* MM-31094: No reparse.
* MM-31094: Removed unused FeatureFlags field.
* MM-31094: Removes unnecessary feature flags variable declarations.
* MM-31094: Fixes copy-paste error.
* MM-31094: Fixes logical error.
* MM-30194: Removes test method from store.
* Revert "MM-30194: Removes test method from store."
This reverts commit d5a6e8529bd5f4d993824c828e239d009b05e567.
* MM-31094: Conforming to make's strange syntax.
* MM-31094: Configures helper for read replica with option.
* MM-31094: Adds some missing ctx's.
* MM-31094: WIP
* MM-31094: Updates test names.
* MM-31094: WIP
* MM-31094: Removes unnecessary master reads.
* MM-31094: ID case changes out of scope.
* MM-31094: Removes unused context.
* MM-31094: Switches to a helper. Removes some var naming changes. Fixes a merge error.
* MM-31094: Removes SQLITE db driver ref.
* MM-31094: Layer generate fix.
* MM-31094: Removes unnecessary changes.
* MM-31094: Moves test method.
* MM-31094: Re-add previous fix.
* MM-31094: Removes make command for dev.
* MM-31094: Fix for login.
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-30882: Fix read-after-write issue for demoting user
In (*App).DemoteUserToGuest, we would demote a user, and then immediately
read it back to do future operations from the user. This reading back
of the user had the effect of sticking the old value into the cache
after which it would never be updated.
There was another issue along with this, which was when the invalidation
message would broadcast across the cluster, it would hit the cache invalidation
problem where an unrelated store call would miss the cache because
it was invalidated, and then again read from replica and stick the old value.
To fix all these, we return the new value directly from the store method
to avoid having the app to read it again.
And we add a map in the localcache layer which tracks invalidations made,
and then switch to use master if it's true.
The core change is fairly limited, but due to changing the store method signatures,
a lot of code needed to be updated to pass "context.Background". Therefore the PR
just "appears" to be big, but the main changes are limited to app/user.go,
sqlstore/user_store.go and user_layer.go
https://mattermost.atlassian.net/browse/MM-30882
```release-note
Fix an issue where demoting a user to guest would not take effect in
an environment with read replicas.
```
* Fix concurrent map access
* Fixing mistakes
* fix tests
* use sync.pool for session
* added back to sync pool
* reverted change
* added a new line
* added back session object
* added back session object
* added back session object
* revert
* refactored into function
* added the session object back into the pool
* work in progress
* work in progress
* work in progress
* code review comments
Co-authored-by: Arjuna Marambe <arjunam@buildxact.com>
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-31063: Change constants to use CamelCase
* store package
* change allcaps to camel case (#16615)
* New tools.mod
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* Add a new handler to allow authentication via CWS API Key
* Make error better
* Add tests and cases for new handler functions
* Move some code around
* Add test for GetCloudSession function
* unset the env after test completion
* Remove white space
* Change Info to Warn
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-27916: Improve logging when session is not found
The error handling in this code is pretty bad and the same error message happens
for multiple conditions, making it difficult to diagnose the real issue.
Most of the times, we get a log like:
```
"Invalid session {"error": "GetSession: Invalid session token=jodb6sau47rnugaqj1fy7khmpr, err=<no value>, "}"
```
And it could have happened from multiple places. So this log turns out to be not that useful.
We improve this by populating the Error field to fix the "<no value>" issue and also add a separate detailed error
field for each log line to uniquely identify each error.
* fix
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-27648: Fix a hub deadlock while revoking session
This is a bug which has always been there in the codebase.
And it can only occur in the extreme of edge-cases.
Following is the call trace due to which this happens:
```
0 0x0000000001dfea68 in github.com/mattermost/mattermost-server/v5/app.(*Hub).InvalidateUser // deadlock
at ./app/web_hub.go:369
1 0x0000000001dfc0bd in github.com/mattermost/mattermost-server/v5/app.(*App).InvalidateWebConnSessionCacheForUser
at ./app/web_hub.go:109
2 0x0000000001db1be5 in github.com/mattermost/mattermost-server/v5/app.(*App).ClearSessionCacheForUserSkipClusterSend
at ./app/session.go:209
3 0x0000000001db1763 in github.com/mattermost/mattermost-server/v5/app.(*App).ClearSessionCacheForUser
at ./app/session.go:170
4 0x0000000001db2d2f in github.com/mattermost/mattermost-server/v5/app.(*App).RevokeSession
at ./app/session.go:275
5 0x0000000001db2c09 in github.com/mattermost/mattermost-server/v5/app.(*App).RevokeSessionById
at ./app/session.go:260
6 0x0000000001daf442 in github.com/mattermost/mattermost-server/v5/app.(*App).GetSession
at ./app/session.go:93
7 0x0000000001df93f4 in github.com/mattermost/mattermost-server/v5/app.(*WebConn).IsAuthenticated
at ./app/web_conn.go:271
8 0x0000000001dfa29b in github.com/mattermost/mattermost-server/v5/app.(*WebConn).shouldSendEvent
at ./app/web_conn.go:323
9 0x0000000001e2667e in github.com/mattermost/mattermost-server/v5/app.(*Hub).Start.func1.3 // starting from hub
at ./app/web_hub.go:491
10 0x0000000001e27c01 in github.com/mattermost/mattermost-server/v5/app.(*Hub).Start.func1
at ./app/web_hub.go:504
11 0x0000000001e27ee2 in github.com/mattermost/mattermost-server/v5/app.(*Hub).Start.func2
at ./app/web_hub.go:528
12 0x0000000000473811 in runtime.goexit
at /usr/local/go/src/runtime/asm_amd64.s:1373
```
The stars have to align in such a way that the session idle timeout _has_ to happen
_exactly_ when a broadcast is happening for that user. Only then, this code path gets
triggered.
Since this is an extreme rabbit hole of calls, I have not attempted any big
refactors and went with the most sensible approach which is to make the RevokeSessionById
call asynchronous.
There are 2 main reasons:
- It was already treated as an asynchronous call because it happened during an error condition
and we were not checking for the return value anyways.
- Session idle timeout is a relatively infrequent event, so creating unbounded goroutines is not a concern.
As a bonus, we also get to check the error return and log it.
https://mattermost.atlassian.net/browse/MM-27648
* Add a test case
* Fix an incorrect comment
Mobile users were having their sessions unexpectedly expired, despite having ServiceSettings.ExtendSessionLengthWithActivity enabled.
Every time a mobile app is opened it called `/api/v4/sessions/device` which calls attachDeviceId which calls `(*Session)SetExpireInDays`. This code above assumed the expiry should be relative to CreateAt which is incorrect when ExtendSessionLengthWithActivity is enabled. Therefore, every time the mobile app was opened, the maximum expiry was set in memory to CreateAt + session_length, even if the session was extended.
(*Session)SetExpireInDays is now deprecated and replaced with (*App)SetSessionExpireInDays which takes into account the ExtendSessionLengthWithActivity setting.
* Extracting email functions into a service
* Fixing two shadowing errors
* Address PR review comments
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* Pass device ID
* dont use device id as way of detecting
* fix spelling mistake
* update layers
* fix test
* fix linting
* save schema
* put columns in correct place
* fix linting
* update
* upgrade go change
* use props
* fix stuff
* update session tests
* address PR comments
* address PR comments
* MM-23935 extend session expiry on user activity
- if user types anything before a session expires the session will be extended to now + session length
- ensures new session expiries are not written to DB too frequently
- new session store func for updating session ExpiresAt
- session length defaults for mobile and web/ldap changed from 180 days to 30 days
* Consistent license message for all the go files
* Fixing the last set of unconsistencies with the license headers
* Addressing PR review comments
* Fixing busy.go and busy_test.go license header
* [MM-15797] Migrate Session.Remove to Sync by default
* Remove unnecessary channel logic and simplify
* Refactor code to be more consistent with code base
* Rename variable to avoid shadow declaration
* Add missing checks in tests
* Regenerate the mocks
* Run gofmt over code
* Permit Bot accounts to create user access tokens if
EnableUserAccessTokens config setting is set to false
* User is now outside of the go routine because we need user.IsBot for
this ticket. This renders the goroutine and channel unnecessary
* move err check following user def
* Migrating audit store
* Final migration example for the audit store
* async example
* Ending migration
* Removing Async helper
* Fixing tests
* Fixing govet problems with the StoreResult instanstiation
* Moving goroutine pool
* Auto refactor
* Moving plugins.
* Auto refactor
* Moving fields to server
* Auto refactor
* Removing siteurl duplication.
* Moving reset of app fields
* Auto refactor
* Formatting
* Moving niling of Server to after last use
* Fixing unit tests.
