Can this paradox be resolved?

There are two envelopes on a table, identical plain brown manila envelopes. One contains twice as much money as the other. You lift one of the envelopes from the table and you are then given a choice; to keep the contents of this envelope or to swap for the other envelope. What should you do?

Solution: If the envelope you chose contains £Y the other envelope will contain either £2Y or £0.5Y. You therefore stand to gain £Y by swapping but you can only lose half that or £0.5Y so clearly it is to your benefit to swap.

Paradox: What if we had chosen the other envelope. Wouldn’t the same logic apply? How can it always be beneficial to swap?