MM-25154: Fix data race in InvokeClusterLeaderChangedListener (#14610)
The order of booting up a server should be Job scheduler -> Cluster. And shutdown should be the opposite. This is needed because the job scheduler initializes certain data structures that are later called by ClusterLeaderChanged event handlers. And the event handlers run in a separate goroutine. Therefore, if the cluster initialization happens before the job scheduler, then a race happens between accessing the jobs variable and setting it. To prevent this race, we fix the order of startup, and add comments in both places to prevent things from regressing again.
Этот коммит содержится в:
коммит произвёл
GitHub
родитель
c1d0faa2e5
Коммит
d317dd2dde
@@ -350,6 +350,9 @@ func NewServer(options ...Option) (*Server, error) {
|
||||
mlog.Error("Error to reset the server status.", mlog.Err(err))
|
||||
}
|
||||
|
||||
// Scheduler must be started before cluster.
|
||||
s.initJobs()
|
||||
|
||||
if s.joinCluster && s.Cluster != nil {
|
||||
s.FakeApp().registerAllClusterMessageHandlers()
|
||||
s.Cluster.StartInterNodeCommunication()
|
||||
@@ -374,8 +377,6 @@ func NewServer(options ...Option) (*Server, error) {
|
||||
}
|
||||
}
|
||||
|
||||
s.initJobs()
|
||||
|
||||
if s.runjobs {
|
||||
s.Go(func() {
|
||||
runSecurityJob(s)
|
||||
@@ -490,6 +491,7 @@ func (s *Server) Shutdown() error {
|
||||
s.Metrics.StopServer()
|
||||
}
|
||||
|
||||
// This must be done after the cluster is stopped.
|
||||
if s.Jobs != nil && s.runjobs {
|
||||
s.Jobs.StopWorkers()
|
||||
s.Jobs.StopSchedulers()
|
||||
|
||||
Ссылка в новой задаче
Block a user