I get the concept that just because you find someone physically attractive doesn't mean you want to fuck them. So I know why ya'll are bucking at calling that "gay." The "a little gay" term is basically used for lack of anything different.
However, I'm going to flat out disagree that there are people that are universally attractive to men and women. What does that even mean? Physically, men and women are completely different, so you have to have a different set of standards of attractiveness for each. If I'm only interested in women, why would I even have a set of standards for what an attractive man is. Your last couple sentences are exactly what I was talking about in my post. Women are so used to judging the looks of both men and women that ya'll can't even conceive of the fact that some people simply can't and don't do that.