Moving pc from network to workgroup

I’ve just brought an older xp pro pc from work (SBS 2003 server-based lan) to use in a home workgroup, specifically for me to vpn into one application at work. Unfortunately it seems the desktop has been locked (by GPO?) so although I can install new software and have it show on the desktop, older icons and the taskbar cannot be modified as they always revert back after a reboot. Also startup programs can be removed but they come back on rebooting. What am I missing?
Thanks, Mark