What is the correct way to write a plural of a letter of the alphabet?

Oh goody! Jeruba’s on it!

It’s a matter of style, not rule. Most of the publishers I’ve worked for follow this model:

I got all A’s.

Although we don’t mark plurals of words with apostrophe-s (a common mistake I see: “potato’s and tomato’s for sale!”), in the case of individual letters, not using an apostrophe can make it confusing (like you wrote, is it the word “us” capitalized or multiples of the letter U?), so an apostrophe is often used. The apostrophe here is to aid comprehension. It may be okay to not use an apostrophe if the intended meaning is clear.

It’s the Oakland A’s. I have a hat to prove it.

(Let’s go Oakland! clap, clap, clap-clap clap)

I use the spelling of the letter. Two ayes three bees,, four aitches.five wyes.

I agree with @Jeruba —this is the way its taught in journalism and creative writing,

I write “A’s”, “B’s”, etc.