Can wearing head coverings frequently lead to hair loss?

This is a thought I’ve had for a little while. It seemed to me that people that often wore ball caps, hard hats, welding helmets, etc. seemed to exhibit hair loss earlier than their peers that didn’t wear them.

My thought is that wearing hats helped to build up sweat and oils that clogged the pores, killing the hair follicles. Is this just in my imagination or could there be some truth to it?