中科普之alternating group is simple for n>=5

It's a well-known fact that the alternating groups, A_n, are simple for n\neq 4, with small cases like A_1, A_2, A_3 very easy to verify. Yet most proofs of n\geq 5, including Hungerford's, relies heavily on 'magical' permutation composition, which is both frustrating to read and hardly intuitive. Serre's new book on finite groups, however, gave an elegant proof (as exercises) whose only 'large' computation is merely 10 additions. The only flaw might be requiring a little bit of preliminaries which are kind of useless for other topics. In this article, we will go through the proof thoroughly, and again, paying tribute to the algebra master, Pierre Serre.

Preliminary: 2-transitivity and primitivity

Definition A group action of G on a set X is called 2-transitive if |X|\ge2, G is acting on X transitively, and G is acting on X^2_{\text{Dist}} transitively, where X^2_{\text{Dist}}=\lbrace(x,y)\in X^2:x\ne y\rbrace.

It's readily verified the action of G on X^2 by g.(x,y)=(g.x,g.y) must have \lbrace(x,x)\in X^2\rbrace as an orbit, so the action on X^2_{\text{Dist}} is naturally well-defined.

Definition A group action of G on X is called primitive if the action is transitive and for some x\in X, \text{Stab}(x) is a maximal proper subgroup of G (under partial order of inclusion).

As in a transitive action every stabilizer is conjugate to each other, characterising one arbitary stabilizer as maximal is amount to saying every stabilizer is maximal.
Now we just need three little lemmas for proving our final statement.

Lemma 1 G acting on X is primitive iff \forall f:X\rightarrow Y with G acting on Y s.t. f(g.x)=g.f(x), f is either injective or trivial (collapsing to one point).

Proof If the action is primitive, assume we have a f:X\rightarrow Y which is neither injective nor trivial. Then \exists y\ne z\in X s.t. f(y)=f(z). As the action is transitive, \exists g\in G with g.y=z. Clearly g\notin\text{Stab}(y). Now consider \text{Stab}(f(y)). It's not G: if it is, \forall x\in X\exists h\in G s.t. x=h.y, and then f(x)=f(h.y)=h.f(y)=f(y), namely f is trivial. Yet it's containing \text{Stab}(y) and g: \forall h\in\text{Stab}(y), h.f(y)=f(h.y)=f(y), and g.f(y)=f(g.y)=f(z)=f(y). Therefore \text{Stab}(y)\subsetneq \text{Stab}(f(y))\subsetneq G, contradicting with \text{Stab}(y) being maximal.
The converse direction is left to reader as an exercise. (Hint: once shown it's transitive, if there exists some x\in X with \text{Stab}(x)\subsetneq H\subsetneq G, try construct a f:X\rightarrow G/H)

Lemma 2 If G acts on X 2-transitively, the action is primitive.

Proof Using the fact above, for f: X\rightarrow Y not injective, it induces a map \bar{f}:X^2_{\text{Dist}}\rightarrow Y:(x,y)\mapsto(f(x),f(y)). It's readily verified that this map is compatible with group action. As f is not injective, there exists a\ne b with f(a)=f(b). Now for any c\ne b, \exists g\in G s.t. g.(a,b)=(b,c). Applying \bar f on this pair, (f(b),f(c))=\bar f(b,c)=\bar f(g.(a,b))=g.\bar f(a,b)=(g.f(a),g.f(b)). Thus f(c)=g.f(b)=g.f(a)=f(b), namely f is trivial.

Lemma 3 If G is a finite group and \text{Aut}(G) acts on G-\lbrace1\rbrace transitively, then all non-identity elements in G have the same order p, which is a prime number. If p=2, G=(C_2)^n for some n. If p>2 and this action is 2-transitive, |G|=3.

Proof The proof of this simple lemma is basically left to the reader. When p=2, it's readily verified that G is Abelian and is a vector space over \mathbb F_2, so taking a basis solves the problem. When p>2, |G|\ge 3. If |G|>3, take y\ne x,x^{-1} in G-\lbrace1\rbrace and consider (x,y) and (x,x^{-1}).

Stupid Intermezzo

In this section, we have a little break by proving the base case: A_5 is simple.

Proof A_5 has 5 conjugacy classes: identity, 3-cycle, two disjoint 2-cycle, and 2 distinct classes of 5-cycle. Each has a size of 1, 2(^5_3)=20, {1\over 2}(^5_2)(^3_2)=15, {4!\over 2}=12 and 12. If there's some nontrivial proper N\triangleleft G, N must fully contain 1 and conjugacy classes. Yet sum of conjugacy classes with 1 are 13, 16, 21, 25, 28, 33, 36, 40, 45, 48, none of which divides 60, so A_5 is simple.

Now you may wonder why only 5-cycle are divided into two conjugacy classes. This is a quite easy constructive result: try to prove some cycle type divides into two conjugacy classes in A_n iff none of the element commutes with an odd permutation in S_n; then prove a permutation doesn't commute with odd permutation iff it consists of disjoint odd length cycles with distinct lengths. (Note that 3-cycle in A_5 does not satisfy this since it's composition of 3-cycle and two 1-cycle!)

Final proof

Assume n\ge 6. We prove the result inductively. Let A_n act on X=\lbrace1,2,...,n\rbrace naturally. This action is obviously 2-transitive as we can add irrelevant transposes with the fact n-4\ge 2. Hence A_{n-1}=\text{Stab}(n) is maximal by lemma 2.
If there is a nontrivial proper N\triangleleft G, N\cap A_{n-1}\triangleleft A_{n-1}. As A_{n-1} is simple, the intersection is either \lbrace1\rbrace or A_{n-1}. Yet the latter case fails: as A_{n-1} is maximal, N=A_{n-1}, namely A_{n-1}\triangleleft A_n. But (1\ 2)(3\ 4)\in A_{n-1}, and (2\ 5)(4\ n)\circ(1\ 2)(3\ 4)\circ(2\ 5)(4\ n)=(1\ 5)(3\ n)\notin A_{n-1}. As N\ne 1, A_{n-1}\subsetneq NA_{n-1}, so NA_{n-1}=A_{n}. Thus |N|=[A_n:A_{n-1}]=n.
Now we want to obtain a contradiction by letting \text{Aut}(N) act on N-\lbrace1\rbrace 2-transitively. We do this by embedding A_{n-1} into \text{Aut}(N). Consider \varphi:N\rightarrow X:g\mapsto g.n. If \varphi(g)=\varphi(h), h^{-1}g.n=n, namely h^{-1}g\in\text{Stab}(n)=A_{n-1}. Clearly h^{-1}g\in N, so h^{-1}g=1, namely h=g. Hence \varphi is injective. As |N|=|X|, \varphi is bijective, meaning we can identify elements of X with elements in N!
For any even permutation \psi on N-\lbrace1\rbrace, h:m\mapsto\varphi\circ\psi\circ\varphi^{-1}(m) is a even permutation on X-\lbrace n\rbrace (why?), i.e. h\in A_{n-1}. Also \varphi(hgh^{-1})=hgh^{-1}.n=hg.n=h\varphi(g)=\varphi(\psi(g)), so hgh^{-1}=\psi(g), namely every even permutation on N-\lbrace1\rbrace corresponds to a conjugation by h, which is an automorphism. Hence our goal is furfilled.
Now if every g\in N-\lbrace1\rbrace has an order p\ge 3, n=|N|=3, contradiction. So p=2, and N contains all permutation which is of form of 2k disjoint transposes for a certain k. Note that this is where A_4 fails: A_4 actually has (C_2)^2 embedded! We do a case distinction here: if k=1, (1\ 2)(3\ 4),(1\ 5)(3\ 4)\in N but their product (1\ 5\ 2)\notin N. If k\ge 2, (1\ 2)(3\ 4)(5\ 6)\pi,(1\ 5)(2\ 3)(4\ 6)\pi\in N, but their product contains (1\ 6\ 3) as cycle, not in N!