I can somewhat relate... 99.99% of my friends think all anime is the same, and all anime fans are the same type of people. Not to offend any anime fan, but realisticly how do most people view anime fans? Mostly they see us as sci-fi loving people who play D&D and have wear grungy clothing, watch Lord of the Rings and such... Well I don't resemble anything close to what many may think anime fans are like
I'm flat out a jock. I play a bunch of sports, like to watch boxing, dress decently, and haven't seen a LoR movie.
Like the point I made about not all fans are/act/look the same, the same applies to genres. Not every anime is a DBZ or sailormoon type. Gosh no. I would have given up on anime if it were.
I guess the best way to deal with it is much like Mangafanatic stated. Act like a mature adult and be calm and collective about how you do things. THey may not understand, but truth is the wold is sterotypical. Sucks but that's how it is.
If all else fails, just poke fun and mock something they enjoy
![Tongue :P](./images/smilies/tongue.gif)