veiwing html files

Hello
I just installed SP1 for XP Pro.
Before when I would look at a folder that had web pages the thumbprints would look like the html page, just a smaller version. I could tell what it looked like by the thumbprint.
Now that SP1 is installed all I get is a file with the blue E representing Internet Expoler.
I can still tell what my gifs and jpegs are without opening the file just the html file are affected.
Any Ideas on how to change it back so I can tell what the html is without opening it?
Any help will be appreciated.
Thanks