Before answering this question, let explore how VB Projects come to live:
So, you starting a new .NET project... As many said, C# and VB.NET produce the same result, but you still have to decide which language to go with for your development. If you are VB.NET developer or C# developer then it's easy, you just go with your expertise. But lets say you impartial, i.e. you never programmed in C# or VB.Net, how do you pick the language then? You have to choose based on your previous experience (you are professional developer after all), so if you had experience with VB6 you probably will pick VB.NET and if you had experience with C, C++ or Java you probably will go with C#. Once you pick the language you become C# or VB developer on this project.
It is also more likely(given number of developers out there) that you didn't start the project, you joined the group that works on existing products and you enhance them using whatever language was picked for development. So who picked that language? - most likely senior developers in the group. So your Senior developers have many years of experience with background that either goes back to VB or C, C++, Java.
Which brings us to my point: it doesn't matter what language YOU using, it matters what background people that make decisions have. You more likely to have better designed and structured project with people that come from solid OO languages.
So even if you are better than average C# developer but working on VB project your senior members probably weaker than senior members on C# project and they make you look bad as well. And that's why C# developers look down on ALL VB developers.
If you were a car dealer it would make a huge difference whether you selling Porsches or Hondas although you essentially doing the same job.