MM-61904: Make reliable websockets work in HA (#29489)
We do a cluster request to get the active and dead queues from other nodes in the cluster to sync any missing information. We check the dead queue in the other nodes to see if there's been any message loss or not. Accordingly, we send just the active queue or both active and dead queues. There's still an edge case that is left out where a client could have potentially connected and reconnected to multiple nodes leaving multiple active queues in multiple nodes. We don't handle this scenario because then potentially we need to create a slice of sendQueueSize * number_of_nodes. And then this can happen again, leading to an infinite increase in sendQueueSize. We leave this edge-case to Redis, acknowledging a limitation in our architecture. In this PR, when there's no message loss, we just take the active queue from the last node it connected to. And if there's message loss where the client's seqNum is within the last node's dead queue, we also handle that. But if there's severe message loss where the client's seqNum falls within the dead queue of another node, then we just send the data from that node to reconstruct the data as much as possible. It could be possible to set a new connection ID in this case, but this involves more data transfer always from all nodes and recomputing the state in the requestor node. https://mattermost.atlassian.net/browse/MM-61904 ```release-note NONE ``` Co-authored-by: Mattermost Build <build@mattermost.com>
Этот коммит содержится в:
коммит произвёл
GitHub
родитель
921604cf39
Коммит
cb75a20c54
@@ -222,6 +222,36 @@ func (_m *ClusterInterface) GetPluginStatuses() (model.PluginStatuses, *model.Ap
|
||||
return r0, r1
|
||||
}
|
||||
|
||||
// GetWSQueues provides a mock function with given fields: userID, connectionID, seqNum
|
||||
func (_m *ClusterInterface) GetWSQueues(userID string, connectionID string, seqNum int64) (map[string]*model.WSQueues, error) {
|
||||
ret := _m.Called(userID, connectionID, seqNum)
|
||||
|
||||
if len(ret) == 0 {
|
||||
panic("no return value specified for GetWSQueues")
|
||||
}
|
||||
|
||||
var r0 map[string]*model.WSQueues
|
||||
var r1 error
|
||||
if rf, ok := ret.Get(0).(func(string, string, int64) (map[string]*model.WSQueues, error)); ok {
|
||||
return rf(userID, connectionID, seqNum)
|
||||
}
|
||||
if rf, ok := ret.Get(0).(func(string, string, int64) map[string]*model.WSQueues); ok {
|
||||
r0 = rf(userID, connectionID, seqNum)
|
||||
} else {
|
||||
if ret.Get(0) != nil {
|
||||
r0 = ret.Get(0).(map[string]*model.WSQueues)
|
||||
}
|
||||
}
|
||||
|
||||
if rf, ok := ret.Get(1).(func(string, string, int64) error); ok {
|
||||
r1 = rf(userID, connectionID, seqNum)
|
||||
} else {
|
||||
r1 = ret.Error(1)
|
||||
}
|
||||
|
||||
return r0, r1
|
||||
}
|
||||
|
||||
// HealthScore provides a mock function with given fields:
|
||||
func (_m *ClusterInterface) HealthScore() int {
|
||||
ret := _m.Called()
|
||||
|
||||
Ссылка в новой задаче
Block a user