Does decaffeinating coffee really bring down the quality of taste when it's finally brewed?

I just had a cup of joe and realized after the matter that it was decaf. I don’t think I’ve ever had decaf before, but I always had the impression from others that there was a very comparable taste, a big difference in coffees that are and aren’t.

Are my taste buds dull or does it just depend on how well the coffee beans are processed?