Does my "freedom" end where your "morality" begins?

This question was sparked by another question on California’s gun control and is something I’ve been pondering. People shout about freedom and their freedoms being taken away. I would happily curtail the use of guns but that is an infringement of your freedom. You, on the other hand, might want to take away my right to marry or even sleep with someone or to control what happens with my body. How do personal freedom and personal morality co-exist and how should government deal with them? I am not on a soapbox here or trying to pick a fight; it is a complex issue.