Would it be erroneous to call the morning or afternoon busy drive time "rush hour" in a small town?

Not necessarily. The small town may be between two larger cities and be covered up with traffic both morning and afternoon.

The rural roads around my area turn into almost-highways between around 4 and 6, thanks to Waze.

If it takes twice the time to get somewhere because of traffic before and after typical work hours then it’s rush hour traffic. Most cities it’s 7–9am and 4–6pm more or less.

Where I live I usually just say there is heavier traffic certain times of day, I don’t use the term rush hour too often, because the traffic is on a few very specific roads, and it’s not specifically related to a typical work schedule. During rush hour in larger cities, everywhere you turn is a ton of traffic.

I’d call it rush hour – I mean, it’s busier, with more traffic, and it’s just easier to tell someone, “I’m running late cuz of rush hour.”

Happy Hour isn’t always happy, but people know what you’re talking about.

Lol! We saw 3 whole cars in a row, once, on a road coming into town. We actually had to wait to pull out. Inconceivable!
I said “It must be rush hour!”
More like Rush Second here.

@smudges Said it best. ”...people know what you’re talking about.” when you say “rush hour.”
If one was to call it “chicken killin’ with the pick-up truck time,” people who didn’t understand would look at you, like, “What??”