I'd move towards bringing the IT department back into the company. But I might think differently if I had more in-depth details. Most IT departments are ourtsorced because the in-house people have done a bad job. It is almost always more expensive to outsource.
But you are a new hire. You were hired to manage people. You feel you have the skills to get in the systems at an admin level. You should be running the IT departmetn.
In relation to the contract firm, you are their customer. It is their job to satisfy their customers, or they will lose their job. You are in the drivers seat, not them.
Don't resort to threats. Don't let them know that you might even consider bringing the IT department back in house. And check the contract.
If the contract allows it, by all means demand that you get admin access. But watch out for contractual perfomance issues. Once you have admin access you may not be able to hold them accountable for performance.