What would cause my iPhone4's sound to stop working unless it is on loud speaker or headphones?

Do you use the headphone jack and it stops working? It maybe the jack and you will need a new phone. Happened to me on another smartphone, new phone was only answer.

The sound works through headphones so it’s like the phone always thinks the headphones are plugged in.

Broken jack is shorting out, you will need a replacement phone. Take it to your local phone store.

Clean out the headphone jack gently with a paper clip and canned air. Look into it with a light and make sure you can see the white bottom.

Also if your bottom is red that is why your phone is broken. ;)

It fixed itself!

Good news! Make sure you plug it into iTunes often to make those backups