What's the right pronoun?

It’s correct. Object of the verb.

It’s right; it should be “us”. It probably sounds weird because people often make the mistake of saying things like “us guys need to stick together”. In that case “us guys” is the subject, so it should be “we”.

Somehow I think @Jeruba and @DominicX did now attend school in California. I am always amazed when people know these things.

Do you mean “did not”?

I attended school in the Boston area. But I think it’s not so much a matter of where as of how and, to some extent, when. I had a tremendous enthusiasm for all aspects of the study of English and not so great an appetite for other subjects. It’s also my impression that language is taught less rigorously than it used to be.

Yes, that was a typo. Somehow I managed to avoid traditional English classes in favor of Creative Writing, Modern Conversation and Short Story – those were the titles of the 3 English classes I took in high school.

…and a heck of a lot less rigorously than it ought to be, to add to @Jeruba‘s accurate assessment.

If the sentence omitted the ‘new arrivals’ description, which would sound more grammatically correct?

“A few dozen cows looked up to evaluate us and then went back to foraging.”
“A few dozen cows looked up to evaluate we and then went back to foraging.”

GA, @Pied_Pfeffer! That is exactly how you determine the correct pronoun. That is, you delete the distracting element (in this case, “new arrivals”), and see what sounds correct.

That’s why I hate it when people say things like, “to you and I”. Would you say, “to I”? Of course not. So, it’s “to you and me”.