OK to have off-site Domain Controller?

I have 20 different locations to manage. All of them are currently on separate domains, which obviously creates an administrative hassle. They are all connected back to the corporate office via a DSL or cable connection.

If joined all of the computers at these locations to the same domain, hosted here at the corporate office, would that be the end of the world? Or would things still work OK?

I would enable cached credentials, for when the DSL has problems and whatnot.

Am I asking for trouble by having no domain controller physically on-site?