Is Being a Cartoonist a Turn Off?

You know that meta-language that you pick up from your dates when you've said the wrong thing?
There's that sort of deflation in their countenance. They zone out suddenly when you talk. They start to seem like they're enduring your presence after your seemingly benign statement.
I've gotten that suprisingly frequently when I've mentioned that I draw cartoons.

(A little reference to Pixar's new film "Inside Out") (The award was just Best Editorial Cartoon for State High School Newspaper Competition)

I'm left to wonder what impression they have of cartoonists? Are cartoonists reputed for being weirdos or something? If that's the case, then I'm definitely a proud weirdo but no one who knows me has complained.