While investigating code paths during
Redis downtime, I found out that the library
by default retries indefinitely for read commmands.
This leads to it being stuck in a loop
while trying to read anything from Redis.
We fix that by disabling retry.
Secondly, we adjust the ConnWriteTimeout
value slightly from its default of 10s
which is a bit too high for normal working
conditions.
Ref: https://github.com/redis/rueidis/issues/680
```release-note
NONE
```
* Added support for S3 storage classes
* Added missing translations for S3 storage class
* Changed default storage class values to preserve original behaviour
* Changed storage class description and example
* Fix translations ordering
* Change configuration defaults to empty strings
* Remove redundant empty string check
* Validate storage class variable against constants from S3 API docs
* Validate export storage class against constants
* Use slices for config validation
* Applied patch for translation ordering
* shared channel event tracking
* Update notification.go
Co-authored-by: Miguel de la Cruz <miguel@mcrx.me>
---------
Co-authored-by: Miguel de la Cruz <miguel@mcrx.me>
* Forbid users from creating DMs and GMs with remote users
This change prevents the application from creating new DMs and GMs
when remote users are involved, and prevents as well new posts from
being created into preexisting DMs and GMs with remote users created
in previous versions.
* Adds i18n
* Skip preexisting DM tests and create a new one for the new behavior
We remove the remaining special casing for **model.User
and add unit tests to lock in the behavior.
Additional load tests were done locally to confirm
there are no hidden code paths left out.
https://mattermost.atlassian.net/browse/MM-59947
```release-note
NONE
```
* add known plugins to the list of possible plugins to be located
* improve testing and fix config data collection
* Improve testing
* remove empty lines
* make linter happy
* go through all plugins
While using Redis, there is no need to invalidate
and then update the new count from the DB when
we know the exact number it is going to be incremented
or decremented by.
In that case, we can directly use Redis primitives
to update the cache and prevent yet another DB query.
To achieve this, we modify the LRU cache get/set
paths slightly to not marshal into byte slices
for *int64 values. This is needed for Redis to operate
the INCR/DECR commands.
https://mattermost.atlassian.net/browse/MM-59933
```release-note
NONE
```
Co-authored-by: Mattermost Build <build@mattermost.com>
- Update library version.
- Added MaxFlush delay to help reduce CPU usage.
- Fall back to LRU cache for the caches which use SCAN.
- Added mattermost-redis and running for all api layer
tests in Postgres.
https://mattermost.atlassian.net/browse/MM-59934
```release-note
NONE
```
We were using a mock cache provider which mixed up LRU caches
with each other. This led to incorrect unmarshalling method calls.
We fix this by using the real cache provider.
https://mattermost.atlassian.net/browse/MM-60413
```release-note
NONE
```
* Changes the event to send when a channel is shared or unshared
Before we were sending the CHANNEL_CONVERTED websockets event, which
is intended to notify of a public channel being converted into a
private one, so the interface was showing the channel as private until
the client was refreshed.
Now a full channel update event is sent when the channel is shared or
unshared, so the interface gets the new information correctly.
* Fix message type on command tests
* Allows invites to be sent to offline remotes
Invites sent to remotes marked as offline will be stored as pending,
and when the remote comes back online, it will process the invites as
part of the synchronization process.
* Update condition name for excluding confirmed invites
* Adds logical deletes to shared channel remotes and remote clusters
Instead of physically deleting the shared channel remote and remote
clusters records when a channel is unshared, a remote uninvited or a
remote cluster is deleted, now those have a logical `DeleteAt` field
that is set.
This allows us to safely restore shared channels between two remote
clusters (as of now resetting the cursor without backfilling their
contents) and to know which connections were established in the past
and now are severed.
* Delete the index in remoteclusters before adding the new column
* Fix bad error check
* Adds status sync to shared channels
To allow for status to be synced, this changes add a new type of
shared channel internal task. This task has its contents pre-fetched
and stored in the `existingMsg` property, and it is keyed with a user
ID besides a channel ID, so it doesn't conflict with channel-driven
synchronization tasks.
All status synchronizations are triggered from the app layer, so there
is no need of watching for new WebSocket events. Although right now
we're only syncing one user status per message, the changes account
for a list of statuses in case we want to batch them in the future.
The feature is gated by a configuration property and can be disabled
independently of the rest of Shared Channels if it's necessary. It is
backwards compatible as well, and should cause no problems with
servers running older Mattermost versions.
* Adds status sync error management and retry
* Adds DisableSharedChannelsStatusSync to the telemetry report
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* Adds ServerFederationSettings configuration block
The Server Federation related configuration properties were located in
the `ExperimentalSettings` config block. Now that the feature is going
to get out of beta, and foreseeing more configuration properties being
added to it, we're moving them to a block of its own.
If the properties were set in the old location, their values should be
carried over to the new ones. The old properties are left in place and
marked as deprecated not to cause issues with the model.
* Update System Console to include Server Federation section
* Remove system console section
* Move the configuration properties to ConnectedWorkspaces
* Add ConnectedWorkspacesSettings to the telemetry
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
* Adds Shared Channel management API endpoints
New endpoints for the following routes are added:
- Get Shared Channel Remotes by Remote Cluster at `GET
/api/v4/remotecluster/{remote_id}/sharedchannelremotes`
- Invite Remote Cluster to Channel at `POST
/api/v4/remotecluster/{remote_id}/channels/invite`
- Uninvite Remote Cluster to Channel at `POST
/api/v4/remotecluster/{remote_id}/channels/uninvite`
These endpoints are planned to be used from the system console, and
gated through the `manage_secure_connections` permission.
* Adds i18n messages for API errors
* Fix pagination flaky test
* Fix linter
* Adds the posibility of filtering shared channel remotes by home
---------
Co-authored-by: Mattermost Build <build@mattermost.com>
- We introduce 2 new APIs:
1. Scan: this allows incremental iteration
without blocking the Redis server and is the
recommended way to iterate over keys. With this,
we have entirely removed the need for Keys.
2. RemoveMulti: this allows deletion of multiple
keys in a single operation which optimizes
network round trips.
- While here, we make a small improvement to
GetStatusFromCache, where we remove the shallow
copy which wasn't necessary because we always
serialize the data from the cache.
- We do not use Redis for session cache because of
frequent requests to iterate the entire cache which leads
to a lot of `SCAN` calls.
- Avoid broadcasting status update messages for Redis case.
- Setting cache expiry for status cache
- Removing .Set method altogether to prevent
any chances of setting an item with no expiry.
https://mattermost.atlassian.net/browse/MM-59932
```release-note
NONE
```
* Adds default team to the remote cluster entity
A new DefaultTeamId field is added to the RemoteCluster entity and its
endpoints, and used when receiving channel invites to choose in which
team to create a new channel.
This will be later extended with the ability for the system admin to
manually accept invites, choosing which team to create the channel on
each. This use case will be triggered when the DefaultTeamId field is
empty, which now simply chooses the first team it finds in the
database as a fallback.
* Fix migrations list
* Fixes channelinvite test case
* Fix i18n
* Fix migration list
* Fixed the issue for DB layer, ES to go
* Handled channel bookmarks
* Handled Bleve
* Lint fix
* Added channel bookmark test
* Skip bleve test
* Used common function
* SKipping ES as indexing logic in unavailable in test
* Adds Remote Cluster related API endpoints
New endpoints for the following routes are added:
- Get Remote Clusters at `GET /api/v4/remotecluster`
- Create Remote Cluster at `POST /api/v4/remotecluster`
- Accept Remote Cluster invite at `POST
/api/v4/remotecluster/accept_invite`
- Generate Remote Cluster invite at `POST
/api/v4/remotecluster/{remote_id}/generate_invite`
- Get Remote Cluster at `GET /api/v4/remotecluster/{remote_id}`
- Patch Remote Cluster at `PATCH /api/v4/remotecluster/{remote_id}`
- Delete Remote Cluster at `DELETE /api/v4/remotecluster/{remote_id}`
These endpoints are planned to be used from the system console, and
gated through the `manage_secure_connections` permission.
* Update server/channels/api4/remote_cluster_test.go
Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
* Fix AppError names
---------
Co-authored-by: Doug Lauder <wiggin77@warpmail.net>
Co-authored-by: Mattermost Build <build@mattermost.com>
