When a shim process is unexpectedly killed in a way that was not initiated through containerd - containerd reports the pod as not ready but the containers as running. This results in kubelet repeatedly sending container kill requests that fail since containerd cannot connect to the shim.
Changes:
- In the container exit handler, treat `err: Unavailable` as if the container has already exited out
- When attempting to get a connection to the shim, if the controller isn't available assume that the shim has been killed (needs to be done since we have a separate exit handler that cleans up the reference to the shim controller - before kubelet has the chance to call StopPodSandbox)
Signed-off-by: Aditya Ramani <a_ramani@apple.com>
(cherry picked from commit 729c97cf39)
Signed-off-by: Danny Canter <danny@dcantah.dev>
CreatedAt was being used as the ExitedAt timestamp, and Info from the
response wasn't being set on the response.
Signed-off-by: Danny Canter <danny@dcantah.dev>
(cherry picked from commit d835fd2a3a)
Signed-off-by: Danny Canter <danny@dcantah.dev>
Made a change a bit ago to cleanup the shim on CreateSandbox failures and
noted that we should probably do the same on Start as well as nothing gets
cleaned up otherwise, and nothing states that a sandbox server/shim should
exit itself if Create/Start fail. Ideally this could be hooked up to
some subsystem in containerd that'd do it for us, but for now to allow
developing prototyping sandbox shims this makes things much friendlier.
Signed-off-by: Danny Canter <danny@dcantah.dev>
(cherry picked from commit 70da5c7830)
Signed-off-by: Danny Canter <danny@dcantah.dev>
Currently if create fails the shim will just be kind of stuck in limbo.
This calls shutdown to allow the shim to exit, and then invokes shim
delete.
Signed-off-by: Danny Canter <danny@dcantah.dev>
CreateSandbox states "failed to start sandbox". This is quite confusing
when you're perusing logs and trying to pinpoint how far a shim got in
its execution.
Signed-off-by: Danny Canter <danny@dcantah.dev>
Aditya Ramani