Can I change my WordPress website design into custom coding site.

Hello, i have a website in WordPress (CMS). Based on user experience and in terms of SEO, it doesn’t seems to me up to the mark. I just want to ask can I change the website design with custom coding. If yes, then will it be affected my site stats?