What is the operation above exponents?

There is no real notation nor name for that (yet, as far as I know); however, new ideas are recognized everyday, especially in the field of mathematics ;).

Although your example doesn’t show it, the expression a^a^a is ambiguous. Consider 3^3^3: do you mean (3^3)^3 = 27^3 or do you mean 3^(3^3) = 3^27, which is much larger?

Nobody can say it’s “wrong” to apply some creative induction as you are, but I’ve never seen this kind of extension of operations for real or complex variables developed mathematically.

I’ve looked at the function x^x in years past, which could be seen as a “higher” unary operation along similar lines. It does strange things for values below 1, then grows quickly for values above 1. You can find its derivative but it can’t be integrated in closed form.

@gasman I always went with the idea that multiple powers are evaluated from the top down (makes more sense when they are written by hand), unless parenthesis say differently. If I saw 3^3^3, I always go with 3^(3^3). This seems to be the way Wolfram Alpha does it.

Why stop with one operation after exponentiation? Considering that a mere googol, 10^100, is larger than the estimated number of particles in the visible universe, there does not seem to be much practical gain in describing huge numbers, but it is fun to think about.

@LostInParadise Stupendous. I know of Donald Knuth but I never heard of tetration or other hyper-operations. You seem to have hit the nail on the head!

anti derivatives. calculus functions begin to go higher.

When the numbers in question are the natural numbers {0,1,2,3,...} the operations in question are sometimes called “superexponentiation.” Edward Nelson has an amusing discussion of them here.