There are several factors relating to when something falls under the Fair Use doctrine (taken from here):
1. the purpose and character of the use, including whether such use is of a commercial nature or is for nonprofit educational purposes;
2. the nature of the copyrighted work;
3. the amount and substantiality of the portion used in relation to the copyrighted work as a whole; and
4. the effect of the use upon the potential market for or value of the copyrighted work.
As determined in Campbell v. Acuff-Rose Music (1994), however, none of these factors is dispositive. That is to say, all four are supposed to be balanced. That a work is reused for commercial purposes does not immediately mean it is not a fair use. In the Campbell case, for instance, it was noted that to say otherwise is to outlaw virtually all commercial parodies. Similarly, you can often use the entirety of a work if it’s nature is such that you can only use all or nothing (e.g., Marcel Duchamp’s L.H.O.O.Q. uses all of Leonardo da Vinci’s Mona Lisa).
It’s a delicate balance, though, which is why transformativity weighs so heavily in most decisions. Transformation requires personal creative work—something typically avoided by people who are being dishonest and trying to get away with something. Not all fan fiction is transformative, but most probably is. Video game fan fiction, especially when the original game has a rather minimal plot, has a particularly strong claim to transformativity.
There are an awful lot of myths in this area, and fan fiction writers are not the only ones who believe them. The list provided by @Aethelflaed—which is very useful for the purpose it was provided—is enlightening. Several of the authors quoted have based their positions on falsehoods or exaggerations. A lot of publishers are also under the impression that commercial use is the main factor (possibly because it is the main factor in whether or not they care).
One thing to take away is that a disclaimer does not get one off the hook. YouTube provides an excellent case study here. Almost all videos that use commercial content have some sort of “no infringement intended” disclaimer. This has not prevented anything from being taken down, however, and the site is littered with videos that no longer have sound thanks to the RIAA.
One advantage of the fact that most fan fiction is published online, though, is that most companies are mollified by simply taking down infringing material (or material they think infringes). When one is willing to do that, further action is rarely taken. Unfortunately, this has led to people issuing removal requests en masse even when a particular use is legal (and host sites typically just bow to the pressure). But again, the fortunate part is that it means one is less likely to actually be sued over fan fiction.