I think it’s so interesting that people make this error when talking about sex differences pretty consistently. If you are talking about differences on a group level (where the two groups = men and women), then citing individual cases of men and women who are exceptions doesn’t mean there are no group differences!
Take height, for example. Men are, on average, taller than women. This doesn’t mean that there are no women who are taller than any men. Likewise, if some men are good listeners, that doesn’t mean that as a group, men aren’t in fact poor listeners.
I am not a man, and I am guilty of being a poor listener. So this is something I pay close attention to, because I would like to learn how to be a better listener by seeing how good listeners do it. And my experience agrees with yours: women do, on average, seem to be better listeners.
But I am not a fan of anecdotal evidence, so I tried to find some research on this. One study about doctor-patient interactions found, “Female physicians engage in significantly more active partnership behaviors, positive talk, psychosocial counseling, psychosocial question asking, and emotionally focused talk.” (JAMA).
A review of many studies concluded, “Men were found generally to be more task-oriented, more active and aggressive verbally, more skillful in problem-solving, readier to take risks, less suggestible, more competitive, more likely to assume leadership, less expressive of emotions, less perceptive of the emotional states of others, and more reticent in disclosing self-information.” (Quarterly Journal of Speech).
I don’t know if any of that qualifies men as worse listeners, but some of the things mentioned above seem not unrelated.