Hi NateH,
While this is a fairly old post/request by now, hopefully this will help others who may come across this in their searches. I'm a product manager with StarTech.com, and we've developed a portable USB 2.0 Laptop KVM Adapter that does exactly what you?re looking for. The product is called the NOTECONS01 (http://www.startech.com/notecons01) it's roughly the size of an iPhone, and plugs into the VGA and USB (or PS2 for legacy PCs) port on the server side, which is converted to USB only to plug into your laptop.
Your laptop becomes a portable console exactly like you're looking for, gaining full visibility of the connected server, and full control of mouse and keyboard functions on the server. Obviously as some others have stated, your simplest connection is via RDP, but for situations where the server is down, or when you need to be onsite, this guy will allow you to do everything including access the system BIOS, and whatever operating system you're running regardless of network status or driver issues.
If you're looking to try one, we do offer try and buy, even through our resellers and distributors where they sell for much less than our MSRP. If you try it, and don't love it, send it back for a full refund.