Why is there a problem with him changing schools? Did your Mom provide any reasons for stopping him?
Sounds to me like there’s a lot of disrespect in your family, and it isn’t all coming from your brother. Also, since you say that money is tight, that can put a lot of stress on people, and make them act more impulsively than they otherwise would. Do you guys get enough sleep? How is your nutrition? Is your Mom around much? What happened to Dad?
I’ll bet you have enough issues there to make folks put their anger on a very short fuse. It sounds like you are skeptical that you can get family counseling. You won’t know until you try, though.
Have you spoken to your mother about this? What does she think about your brother’s behavior? Does she admit to being afraid of him? Has violence been a standard mode of dealing with things in your family? Did you all get spanked a lot? What happened to Dad?
Do you ever have family meetings? I doesn’t sound likely that people would listen to each other if you did, but I was just wondering. I could help if you start talking directly to each other about your feelings. Like you could say to your brother, “I get scared when you push Mom. I’m afraid you’ll hurt her. I get scared when you say you want to kill Sis. I’m afraid you’ll hurt her. I’m afraid you’ll hurt me. I want you to stop threatening everyone. Why do you think you are always so angry?”
Who knows what he’ll say. If he is at all introspective, he might say that he’s angry because no one listens to him, or no one cares about him. You might tell him that you love him, but you are still scared and want him to stop being violent. You just say this over and over, and try not to resort to violence, but “use your words.” There are other problem solving mechanisms, and, as @DominicX says, violence generally only leads to more violence. Stopping violence by using fear is not the solution you want. It only drives you further apart. You’ve got to find out what is underneath all this, and address the real problems, not the symptoms.
This will be difficult for you. You’re only 16. You probably don’t know much about theories about abusers, nor are you skilled at problem solving. It sounds like you only know one problem-solving skill—violence. It’s not an effective way to deal with things. It may work in the short term, but it does nothing to deal with underlying causes, so it won’t work in the long term. But if it’s all you know, it’s what you’ll do.
At the very least, you need to talk to someone outside the family who may be able to help you develop other means for dealing with this. It sounds like your mother is unable to cope. It’s not supposed to be your job to deal with this, but you seem to have the job whether or not you should have it. Is there any adult outside the family who you trust? Who is experienced in things other than violence? Try to find one. Try to develop more problem-solving tools.
Really, talking about it openly and honestly is the best approach. Whether you can get to that place, I don’t know. It’ll probably be very difficult. At the very least, you can try it, and keep on trying it. All the while you still have to defend yourself and your sister, and now your Mom. You’re growing up faster than you might like, but you gotta deal with what life gives you. It’s good that you asked this question. I wish you luck. Please ask more specific questions as things go along. Tell us what is happening. We may be able to offer better advice.