Gaining time when flying west (why not always fly west when practical?)

If you effectively ‘gain’ time when flying west, why don’t more people do their roundtrips between roughly ‘opposite side of the world’ pairs of cities as round the world trips (flying west for both their journey to the other city and their journey home), rather than flying in one direction then home in the other direction?

Is cost the primary reason (mirrored roundtrip flights being generally much cheaper than sets of two one-way flights), or am I missing something basic here?