Blocking Remote Terminal Server and allowing Local Terminal Server

Within a windows domain, I have 2 group policies, one for local domain users and one for Terminal Server users. There are multiple remote terminal server users.

There is a specific user accessing the terminal server locally and remotely, but the remote access needs to be blocked.

Is there a way I can block the remote sessions for this 1 specific user so they can not access the terminal server from home but also allow this user access to terminal server only when this user is in the office so they can work?