Sometimes you just have to let go. Separation of duties is very real, and can actually help keep your sys admin out of unexpected scrutiny, questioning or accusations.
Just because you’re an admin on a server doesn’t mean you have to have access to the box. I have a few servers that I built, then moved to a secure location (like an HR area), and then had them change the password so I would no longer have an account to get in on.
When work or maintenance needs to be done they log in for me and get to sit and wait until I am done.