How can we know when it's dangerous to let certain people drive us?

I admit I’m the world’s biggest coward. But when my s/o drives , for some reason, he and I act and speak like we’re really on edge. This is getting worse. If I drive he is the worst back-seat driver! And, he’s insulting. I know I’m on edge; I don’t feel safe. So, how would you personally come to the conclusion, “ok; I do not like being a passenger when he’s driving.” He’s diabetic.
What happens is he sometimes gets honked at; I don’t. He always says, “I can’t wait any longer; I have to BULL my way through.” “I have to pass these idiots; some moron is up there going way too slow” (as he passes very fast near the double yellow line). He turned left on a yellow light and someone ran the light and rammed into him and he fractured his neck. His mother said, “Im actually surprised it hasn’t happened before this.” He came out of that one fine after a couple hours of hallucinating. The car was totaled. His ex said she’d “never drive with him again.” (she did). So how does one know it just isn’t safe? Something else: when he’s talking he misses his turn off. He can’t talk and drive that well.