If Sysadmin’s token device is lost, and his recovery codes are lost, and no console-like access is available… Well Sysadmin is locked out of the server!
…and to make a little more explicit what was implicit in Davide’s post, you really should have something simulating local console access to your server–either as the actual physical console, IPMI, or VNC into your VPS provider. Logging in there shouldn’t require 2FA.
Failing that, if you have another user that can log in via SSH, and has the ability to sudo, you may be able to remove 2FA that way.