* MM-31094: Adds tooling to develop and test using a MySQL instance with replication lag. Adds some lazy lookups to fallback to master if results are not found.
* MM-31094: Removes mysql-read-replica from default docker services.
* MM-31094: Switches (store..SessionStore).Get and (store.TeamStore).GetMember to using context.Context.
* MM-31094: Updates (store.UsersStore).Get to use context.
* MM-31094: Updates (store.PostStore).Get to use context.
* MM-31094: Removes feature flag and config setting.
* MM-31094: Rolls back some master reads.
* MM-31094: Rolls a non-cache read.
* MM-31094: Removes feature flag from the store.
* MM-31094: Removes unused constant and struct field.
* MM-31094: Removes some old feature flag references.
* MM-31094: Fixes some tests.
* MM-31094: App layers fix.
* MM-31094: Fixes mocks.
* MM-31094: Don't reparse flag.
* MM-31094: No reparse.
* MM-31094: Removed unused FeatureFlags field.
* MM-31094: Removes unnecessary feature flags variable declarations.
* MM-31094: Fixes copy-paste error.
* MM-31094: Fixes logical error.
* MM-30194: Removes test method from store.
* Revert "MM-30194: Removes test method from store."
This reverts commit d5a6e8529bd5f4d993824c828e239d009b05e567.
* MM-31094: Conforming to make's strange syntax.
* MM-31094: Configures helper for read replica with option.
* MM-31094: Adds some missing ctx's.
* MM-31094: WIP
* MM-31094: Updates test names.
* MM-31094: WIP
* MM-31094: Removes unnecessary master reads.
* MM-31094: ID case changes out of scope.
* MM-31094: Removes unused context.
* MM-31094: Switches to a helper. Removes some var naming changes. Fixes a merge error.
* MM-31094: Removes SQLITE db driver ref.
* MM-31094: Layer generate fix.
* MM-31094: Removes unnecessary changes.
* MM-31094: Moves test method.
* MM-31094: Re-add previous fix.
* MM-31094: Removes make command for dev.
* MM-31094: Fix for login.
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-30882: Fix read-after-write issue for demoting user
In (*App).DemoteUserToGuest, we would demote a user, and then immediately
read it back to do future operations from the user. This reading back
of the user had the effect of sticking the old value into the cache
after which it would never be updated.
There was another issue along with this, which was when the invalidation
message would broadcast across the cluster, it would hit the cache invalidation
problem where an unrelated store call would miss the cache because
it was invalidated, and then again read from replica and stick the old value.
To fix all these, we return the new value directly from the store method
to avoid having the app to read it again.
And we add a map in the localcache layer which tracks invalidations made,
and then switch to use master if it's true.
The core change is fairly limited, but due to changing the store method signatures,
a lot of code needed to be updated to pass "context.Background". Therefore the PR
just "appears" to be big, but the main changes are limited to app/user.go,
sqlstore/user_store.go and user_layer.go
https://mattermost.atlassian.net/browse/MM-30882
```release-note
Fix an issue where demoting a user to guest would not take effect in
an environment with read replicas.
```
* Fix concurrent map access
* Fixing mistakes
* fix tests
* Replacing require.Nil with require.NoError
* More replacements
* More Nil/NotNill to NoError/Error
* Other detected errors
* renaming apperr to err
* Removed not needed line
* Rename old appErr variables that are no longer model.AppError values
* Fixing tiny typo
* Reverting changes outside the store (accidentally added)
* Apply suggestions from code review
Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
* MM-31063: Change constants to use CamelCase
* store package
* change allcaps to camel case (#16615)
* New tools.mod
Co-authored-by: Ibrahim Serdar Acikgoz <serdaracikgoz86@gmail.com>
* Removing supplier concept from the sql store
* Removing other metions to supplier
* Fixing gofmt
* Fixing gofmt
* Renaming NewSqlStore to New
* Fixing tests
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* MM-30026: Use DB master when getting team members from a session
A race condition happens when the read-replica isn't updated yet
by the time a session expiry message reaches another node in the cluster.
Here is the sequence of events that can cause it:
- Server1 gets any request which has to wipe session cache.
- The SQL query is written to DB master, and a cluster message is propagated
to clear the session cache for that user.
- Now before the read-replica is updated with the master’s update,
the cluster message reaches Server2. The session cache is wiped out for that user.
- _Any random_ request for that user hits Server2. Does NOT have to be
the update team name request. The request does not find the value
in session cache, because it’s wiped off, and picks it up from the DB.
Surprise surprise, it gets the stale value. Sticks it into the cache.
By now, the read-replica is updated. But guess what, we aren’t going to
ask the DB anymore, because we have it in the cache. And the cache has the stale value.
We use a temporary approach for now by introducing a context in the DB calls so that
the useMaster information can be easily passed. And this has the added advantage of
reusing the same context for future DB calls in case it happens. And we can also
add more context keys as needed.
A proper approach needs some architectural changes. See the issue for more details.
```release-note
Fixed a bug where a session will hold on to a cached value
in an HA setup with read-replicas configured.
```
* incorporate review comments
Co-authored-by: Mattermod <mattermod@users.noreply.github.com>
* new job type created that checks for expired mobile sessions and pushes notifications.
* only send session expired notifications if ExtendSessionLengthWithActivity is enabled.
* includes schema change: field added to Sessions table
* MM-23935 extend session expiry on user activity
- if user types anything before a session expires the session will be extended to now + session length
- ensures new session expiries are not written to DB too frequently
- new session store func for updating session ExpiresAt
- session length defaults for mobile and web/ldap changed from 180 days to 30 days
* Remove goroutines from session_store.go
There isn't much benefit of running these 2 queries concurrently
versus just serially. These queries are not in the hot path anyways.
Maybe just simpler to keep it serial.
* Incorporate review comments
* Consistent license message for all the go files
* Fixing the last set of unconsistencies with the license headers
* Addressing PR review comments
* Fixing busy.go and busy_test.go license header
* [MM-15797] Migrate Session.Remove to Sync by default
* Remove unnecessary channel logic and simplify
* Refactor code to be more consistent with code base
* Rename variable to avoid shadow declaration
* Add missing checks in tests
* Regenerate the mocks
* Run gofmt over code
* Implementing structured logging
* Changes to en.json to allow refactor to run.
* Fixing global logger
* Structured logger initalization.
* Add caller.
* Do some log redirection.
* Auto refactor
* Cleaning up l4g reference and removing dependancy.
* Removing junk.
* Copyright headers.
* Fixing tests
* Revert "Changes to en.json to allow refactor to run."
This reverts commit fd8249e99bcad0231e6ea65cd77c32aae9a54026.
* Fixing some auto refactor strangeness and typo.
* Making keys more human readable.
