IIS virtual directory delete recreate

When I delete virtual directory and re-do it says name already exists and adds # to end of name. How do I really delete these virtual directories?