What strategies do you think standardized test makers use in selecting which multiple choice answer should be the correct one?

If the test is constructed well, you won’t find a methodology to the answers. There are courses given for “tests and measurements” and in those courses they teach scientific ways of avoiding such a thing when you create a test.

It has to be random and a somewhat equal distribution of all answers. If it’s not, the entire integrity of the test is ruined. If I were personally designing a multiple choice A-E test, I would use a 10-sided Gamescience die (where 1 and 5 equal A, 2 and 6 equal B, etc) rolled three times consecutively, using the last roll as the final answer designation. I specifically mention the brand of die because Gamescience dice are proven to be more accurate in their distribution of numbers due to their construction method. Even some computer programs that provide “randomization” are not actually random at all, which is why I prefer a hands-on method; though logically an authenticated randomizer program and a flawlessly balanced die will both provide equally random and balanced results.

An equal distribution of all answers can cue the test taker just as much as all “A”.

But does it matter that the test taker is cued? They still only have a 1/5 chance of randomly selecting the correct answer.

A properly designed test shouldn’t cue the test taker at all as far as an answer pattern. Admittedly, with an equal distribution of answers the odds are good the test taker won’t determine the answer just based upon the answer pattern.

You haven’t actually responded to my question though Mr_M. Does it have any effect on the result of the test if the student believes they have discovered the correct distribution of answers and chooses to respond based on it? If anything I believe this would result in a lower score – the student may override the answer they come up with because they think they’ve figured out the pattern and the pattern says a different answer!

We said there WAS an equal distribution of answers. Assuming that’s a given, if the student is puzzled between choice A and choice C and there are already five A answers, B answers, D answers and E answers out of 25 and 4 “C” answers, the answer WILL HAVE TO BE choice C. If the student is fairly sure the answer is DEFINITELY either A or C and knows there are 5 “A” answers already, he should pick “C”. He gets some cues, albeit a small amount.

Your situation has one fatal assumption, and that is that the student knows they have answered all the previous questions correctly, as well as having figured out the fact that the test will have an equal division of responses. If the student knew they had answered the other questions correctly, they would be smart enough to answer the last question correctly on their own merits of intelligence! And if the student chose to answer C in this case, that answer may not be correct if ANY of their previous answers were incorrect, of which the odds are high because that’s just the nature of multiple choice tests.

But the above analysis is a bit moot because I never said there was an equal distribution of answers. In my first reply in this thread I took care to NOT say a precisely equal distribution, but a somewhat equal distribution. Just like when you flip a coin and it comes up heads for a string of flips, my method of rolling dice would come up with strings of answers that may leave one letter value being more answers than another. This is simply the nature of randomness. Having an exactly equal distribution of answers is not random by definition.

Valid points.

I think you should fight the man and draw pictures in the bubbles! Standardized testing is ruining education!

Forget about what should be. Most stats show that answer B is the most frequently the correct answer. So, if you’re running out of time taking a test, fill all the undone ones with B (if doing so doesn’t penalize you).

@Carol, can your provide a link to support that claim? I have been under the impression that “C” was the best default answer.

Nope! No link.
I was told this when in gradual school in a tests and measurements class.