You need a spectrum analyzer (plug in) to see exactly what’s happening; almost all instruments have fundamental and harmonic frequencies, that are in several octaves. Sometimes, clarity for instruments like bass involves giving them a clearer resonance on the upper harmonics (1KHz – 4KHz range).
Isolate the bass, and with a spectrum analyzer you may be able to see what’s happening there. Believe it or not, sometimes cutting down the bottom octaves improves clarity for bass instruments.
On vocals, placement is equally important. Compression for Vocals yields the impression of “presence” or even proximity to the listener on the “soundstage.” Of course, you want to isolate the vocals and get them to sound natural, as in, the way they would sound naturally under ideal acoustic environments.
Lastly, get a text and study some of the techniques of the pros-there’s never a right or wrong way to master, but in terms of getting the mix right, there are some tried and true techniques to apply.
http://www.focalpress.com/Book.aspx?id=1094&cat=166&sub=176
This text may be a bit heady, but Focal Press as a publisher has a pretty decent array of mixing and audio engineering texts; Modern Recording Techniques (Huber) is a standard in most post-secondary audio programs.
Some other recommendations:
-Listen to the mix in mono, before applying panning and compression; get the levels right with faders only.
-Also, try to use EQ sparingly, and rather than boosting everything, cutting tends to be more effective.
-on channel mixing, EQ before dynamics
-Try to not duplicate plug-ins across input channels; rather, route to busses, and apply the plug-ins (in common) at the bus insert.
For starters…and use compression very conservatively…don’t smack the sh*t out of everything, try and create depth without compression if possible.