does user know if your site fails?
Yes. As an Opera user, I know I have to test my sites on the more popular browsers. (I'm not sure about the less popular ones!)
Another point to consider: Does an end user know when your site fails in the browser they are using? I may be working a site and think that something is missing and find it rendered when running another browser. Worse, perhaps, is when using some protection software such as NoScript which breaks a web site; not a browser problem, although added security in browsers (even IE 8 and 9) can cause problems; notable when including content from different servers on one page.