What's the mathematical relationship between the system volume and the master volume control on my speaker?

I don’t think it’s possible to answer, since you’re measuring apples and oranges. Software volume is controlled differently than hardware volume. (This is why you should always control volume with an analog dial rather than with software; lower software volume degrades signal and causes downsampling.)

Potentiometers (the hardware that volume knobs are connected to) can be linear taper or logarithmic taper. The latter are usually used for audio volume controls, as they more closely mimic the human perception of loudness. Anything is possible however, and if it’s a software control then (as @SmashTheState said) all bets are off.

Full system volume + full speaker volume gets the loudest output, but either of them can reduce sound output to nothing on their own, with different combinations possible. It’s the relationship between these that I’m looking at, not so much the differences between hardware and software volume control.

The system = the signal being produced. e.g. The initial source of the sound. Therefor the signal is able to be volume controlled. The dial on your speaker is just merely that. Volume of your speakers.

@Axemusica In terms of output power.
(I could crank the speakers and mute the system volume and have no sound. I could switch off the speakers and max out the system volume and have no sound. I could turn off the system and speaker volumes and have no sound. I could raise the system and speaker volumes to their maximums and have the loudest sound. Or I could have a combination of the two, for variable sound.) Only that’s dang cumbersome.