OWA and Account Log On To… <server name>

We have a group of users (Single OU) that cannot access the OWA. Each of the users in the OU are restricted to a particular computer the can log on to which is our terminal server. Even if we include the exchange server and AD DC’s we get no joy. If we remove the restrictions and allow the users to Log On To “All computers” they are able to access the OWA.

Any way we can enforce Log On To the terminal server and still have OWA access?