User experience
I agree. Chrome was the fastest when it first came out and people won't let go of that ideal. I read an article here on TR that stated IE9 is now the fastest browser beating Chrome by some small margin. Firefox 4 is due to come out very soon... My point is that the "Fastest browser" will change every week and everyone will sound the horns heralding the new king.
It all comes down to user experience. Pick the one that you like. I use multiple browsers. I like to set up an environment for a task that may not be ideal for my daily web browsing and vice versa. Let's end this pissing contest and focus on what we can control. (Which also may change rapidly, I know)