Stopping services which contain sub-routines is a hassle #2041
Comments
3 tasks
4 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This issue was motivated by the PR #2010, that addresses #646 and #652. The problem, however, is more general and possibly causes problems when stopping multiple services/reactors.
In short, when we call
Stop()for an implementation ofservice.Service, theservice.BaseServiceimplementation of this method on service/service.go does the following:IsRunning()to return falseOnStop(), which is supposed to be overridden by service implementationsQuit()channel, which also releases blocking calls toWait()Implementations of the
OnStop()method are supposed, in particular, to stop sub-routines, typically started uponOnStart(). The problem is how the routines know they have to leave, when the service was or is being stopped.The typical approach is to use the
Quit()channel for that, since once it is closed the service has been stopped. But this channel is only closed once closed onceOnStop()returns (see above). WhileOnStop()implementations Are supposed to stop all sub-routines. We thus have short circuit here.The solution adopted in #2010 consists in overriding the
Stop()method, not theOnStop()method as expected by the service definition. The overridden method callsBaseService.Stop(), so that to, in particular, close theQuit()channel. The sub-routines should eventually, once realizing that this channel is closed, return and notify this to a synchronized abstraction (typically, async.WaitGroup). The overriddenStop()method will then block on the synchronized abstraction before returning.While this solution, despite the added complexity, mostly works, we have an additional problem here. The
Wait()call returns once theQuit()channel is closed. This channel is closed byBaseService.Stop(), once theOnStop()method returns. This is the right way to implement this functionality. But when we override theStop()method, so that it waits for all sub-routines to quit, we first callBaseService.Stop(), as above described. Which means that theWait()method is released and returns. While the service is not yet completely stopped, as the overridden version ofStop()is still waiting for sub-routines to quit. Moreover, the same problem arises if another component or service is usingQuit()to identify whether this service has been stopped.In summary, we can get rid of the short circuit created by
OnStop()unblockingQuit(), whileQuit()being unblocked is a condition for sub-routines to quit, which is whatOnStop()is supposed to implement. But with this work-around we break the definition of theWait()andQuit()methods, which do not actually wait until the service is fully stopped.The text was updated successfully, but these errors were encountered: