I used to wonder that a lot when I was younger but I thought about it more thoroughly last night, and I think I have found a logical reasoning that shows we must be seeing the colors the same way (provided we don’t suffer from color blindness or other disabilities). If there are differences, they can only be slight.
We want to know if you and I see colors the same way, so let’s make an experiment with you being the reference and me being the test subject. This means that when I say blue, I refer to the color you see as blue, when I say red I refer to the color you see as red and so on…
Now let’s suppose we don’t see the colors the same way at all, and let’s say you taught me that the sky is blue, the sun is yellow and that the grass is green. We don’t see the colors the same way, so for all we know I could be seeing the sky as your green (which means your green is my blue), the sun as your red (your red is my yellow) and I could be seeing the grass as your purple but you would tell me that what I’m seeing is green.
The thing is, we know that when we mix blue paint and yellow paint we get green paint. Or at least that’s what you could have taught me. You’re the reference there, so when you mix blue and yellow you will see green. But that’s different for me : in your eyes, mixing blue and yellow is equivalent in my eyes to mixing green and red (which yields brown), because remember that when you see blue I see green and when you see yellow I see red. This means that when you see me mix blue paint and yellow paint you will see green, but I will see some other color. Not brown, brown is what you would see if you were to mix green and red, but remember I don’t see colors like you. Let’s say I’m seeing orange (once again it could be anything for all we know).
Then you could teach me that this color I’m seeing – orange – is green, but you also taught me that the grass I am seeing purple is green! And therein lies the contradiction : if you show me two different colors and tell me both are named the same, then knowing none of us have vision disabilities I would understand there is a parameter we didn’t take into account somewhere, and with some thinking I might eventually realize that we are in fact not seeing the same colours.
Here we only had to mix two colours together to realize that our apprehension of the world was different, so given the countless numbers of situations involving colors and other individuals we participate in in our everyday’s life, we surely would have been faced with such a contradiction if we weren’t seeing colours the same way.
Now you may be thinking, what if the grass looked orange to me? Well this would have been a singular case, and even if we were seeing colours differently we wouldn’t have realized it at this point. But eventually we would have, as I stated above our existence is prone to that.
You may also be wondering how it’s possible that while you are seeing green the whole time, I see the grass purple and the mix of the 2 paints orange. Even though it’s counterintuitive, you should convince yourself there is no magic in that. The most logical thing to say here is it’s impossible, because our hypothesis stating “we don’t see colors the same way at all” was wrong.
If you want something more intuitive, you could say it has to do with how colours are naturally ordered on the visible spectrum of light ( http://an.hitchcock.org/repairfaq/sam/spctrm2.jpg ). Looking at it you realize how it all flows together, every next colour being the continuation of the previous one, and you may very well convince yourself why everyone sees colours the same way simply by looking at it : if we saw colours differently it would feel like colours are put randomly on the spectrum for each one of us, with green being next to purple and red next to blue for example, and without getting into details you know that wouldn’t feel right.
All of this shows that we can’t be seeing colours completly differently, as long as we don’t have vision-related disabilities. The explanation above works provided I perceive a completly different spectrum than yours : constitued of the same colors, but randomly ordered. But what if my spectrum was the same than yours, but shifted on the “right” (towards infrared) or on the “left” (towards ultraviolet) ? We know for a fact that humans can’t see infrared or ultraviolet lights, so if my spectrum is shifted it can only be slightly so. But if I had a slightly shifted spectrum (or even not slightly for that matter), the experiment above wouldn’t work. Not because I wouldn’t notice the difference between the color of the grass and the color I see from mixing blue paint and yellow paint, but because there would be no difference! And it is crucial to understand that point, the color would be the same. Because my spectrum is shifted it wouldn’t be the same green as you, my green might be more bluish or more yellowish depending on which direction it’s shifted, but the green I see would remain the same.
If you have a hard time getting what I’m saying look again at the visible spectrum : http://an.hitchcock.org/repairfaq/sam/spctrm2.jpg . In your case when you mix blue and yellow you get the color equally distant to both, that’s to say green. Now try to imagine the same spectrum but slightly shifted, on the left for example (the spectrum I perceive). This means I won’t be seeing grass the same green as you : it would look more blueish to me. This also means that I wouldn’t see the sky blue like you : mine would look more dark blue, and same goes with the sun that I would be seeing more greenish. Now, when I mix your blue paint and yellow paint, to my eyes it will seem like I’m mixing dark blue and yellow-green. And what am I seeing this time? That’s right, bluish green, the same color I see the grass. It works this time because my spectrum is ordered like yours, while earlier it was completly random and I could be getting anything (you can try to imagine a messy spectrum and see what happens when you’re mixing two colours).
And what this means is that, in theory (or at least considering what I’ve just said), it’s perfectly possible that we all perceive the same spectrum but that some of us perceive it more or less shifted, one way or the other. And this would explain why sometimes people disagree between yellow and orange, orange and red, green and blue..Sometimes you feel like something look more blue than green, while someone else will feel it looks more green than blue. I can’t prove to you that the reason behind this is we don’t perceive the visible spectrum on exactly the same frequencies, but I believe this sounds somewhat coherent.
There is one last thing I haven’t spoken of : what if my spectrum is the same than yours, but it’s all reversed? (meaning I see the world in negative colors). It might sound completly different but it’s not that different actually, in the sense that there would be no randomness in the way I perceive colours : there would still be the progression in color you see on the usual visible spectrum, except it’s the other way around. And this implies that the method I used to discard the possibility that we don’t see colors the same way at all, doesn’t allow to rule out the possibility that I’m seeing in negative colors (try to imagine a reversed spectrum this time, you will see that the colour I get from mixing your blue and your yellow will be the same than the colour I see the grass).
We are not getting a contradiction there, but while the fact we are perceiving slightly shifted spectrums sounded plausible, this sounds much more unlikely.