1984 – 1991 BMW 325is or 318is:
Both handle extremely well for their age, have well-engineered (if outdated) engines, well-appointed interiors, and are generally reliable as long as they’ve been taken care of. 325is is a little faster, but the 318 has a chain-driven engine which is not quite as 70s and gets better mileage. Not hard to find (a nice one) for $5000ish.
1990 – 1992 Golf GTI
16v, especially, is a really strong little car that’s actually very, very good on gas. Handles well, strong aftermarket, readily-available parts, super-easy to work on, no big problem areas. 8v models are completely bulletproof. Easy to get one for under $4000.
1989 – 1997 Mazda Miata
Very popular autocrosser, convertibles are fun, and not bad on insurance / fuel. Personally I think they’re a little expensive for what they are, and they’re very small. But they are fun.
1989 – 1994 Nissan 240SX
Getting difficult to find in decent, unmolested condition because of kids sliding them around, but a solid platform with extremely strong aftermarket support. You can really spend about as much or as little as you want on one of these, but you get what you pay for. They’re in high demand amongst the growing crowd that likes them.
1988 – 1991 Honda Prelude Si
Has some aftermarket left, but OE parts are still attainable. Watch out for the S model (carbureted) and the 90/91 Si, which has FRM cylinder walls that tear rings up and burn oil. Also have a tendency to be neglected and develop classic rear-quarter rust spots Hondas are known for, but are still very reliable and particularly well-made cars. Still the only production car ever to offer fully-mechanical four-wheel-steering (which is a real blast). Low-mileage, cherry 4WS models don’t go over $3000.
Got to agree with girlofscience on the Celica front as well. If you can find one, the first-generation MR-2 is a fairly cheap sports car, but they’re very expensive to insure and quite difficult to drive for anyone not accustomed to mid-engined cars.
My vote’s for the BMW if you don’t mind fixing a couple things at the outset.