Group Policy for Remote Users

I am trying to apply a GPO to a “Group” of Remote desktop users in Win2003 SBS. I specifically do not want them accessing anything but a shared public folder. When I link the GPO to the group, the remote users are able to logon, but have no desktop. Can anyone assist?