Is it safe for me to be a passenger with him driving ?

My husband’s legs/hips are growing progressively weaker after a hip replacement two years ago. He sometimes wants me to come with him into town. He fell down twice , two nights ago and , if there Is anything in his path in the house he wobbles and looks frightened. Would you hop in the car with him? He has a walker but won’t use it Or his cane. His mind is sharp but his hearing is just terrible. I don’t want to hurt his feelings but I get really scared in the car with him. TIA