JOURNAL OF COMBINATORIAL THEORY, Series A 32, 132-161(1982)

Some Aspects of Groups Acting on Finite Posets

RICHARD P. STANLEY*

Department of Mathematics,
Massachusetts Institute of Technology.
Cambridge, Massachusetts 02139
Received February 25, 1981

DEDICATED TO PROFESSOR MARSHALL HALL, JR., ON

THE OCCASION OF HIS RETIREMENT

Let P be a finite poset and G a group of automorphisms of P. The action of G on

P can be used to define various linear representations of G, and we investigate how
these representations are related to one another and to the structure of P. Several
examples are analyzed in detail, viz., the symmetric group G, acting on a boolean
algebra, CL,(q) acting on subspaces of an n-dimensional vector space over GF(q),
the hyperoctahedral group B, acting on the lattice of faces of a cross-polytope, and
G, acting on the lattice II, of partitions of an n-set. Several results of a general
nature are also proved. These include a duality theorem related to Alexander
duality, a special property of geometric lattices, the behavior of barycentric
subdivision, and a method for showing that certain sequences are unimodal. In
particular, we give what seems to be the simplest proof to date that the q-binomial
coefftcient [ ‘:‘I has unimodal coefficients.

Contents. I. Generalities. 2. Duality. 3. Geometric lattices. 4. Boolean algebras.

5. The subspace lattice S”(q). 6. The lattice of faces of a cross-polytope. 7. The
lattice of partitions of an n-set. 8. Barycentric subdivision. 9. Unimodality.

1. GENERALITIES

Let P be a finite poset (= partially ordered set) and G a (finite) group of

automorphisms of P. Thus G is a subgroup of the full group Aut P of
automorphisms of P. By definition each element of G permutes the elements
of P, so that the action of G on P defines a certain permutation represen-
tation of the abstract group G. There are many other actions or represen-
tations of G which can be defined in terms of its original action on P, and
our main purpose here is to see how these representations interact with each

* Partially supported by NSF Grant MCS78-02743.

132
0097.3165/82/020132-30$02.00/O
Copyright 0 1982 by Academic Press. Inc.
All rights of reproduction in any form reserved.
 GROUPS ACTING ON FINITE POSETS 133

other and with the combinatorial structure of P. There are several published
instances of this kind of interaction in the literature (which we will say more
about later). In most cases the purpose has been to study the representation
theory of G. We, on the other hand, will be primarily concerned with the
interplay between combinatorics and group theory, with no particular
concern for obtaining purely group-theoretic results. In a few cases, however,
we will be able to say something new about the representations of G
independent of P.
The following notation will be used throughout. The set { 1,2,..., n) is
denoted [n], so in particular [0] = 0. Also let R\l= (0, 1,2,...,}. The notation
s= {s , ,..., ski < means that S is the set {s L,..., sk) of real numbers and that
Sl ( -** < Sk. If S is a set and T c S, then S\T denotes the complement
{xES:xG T}.
From the outset we assume that the poset P has a unique minimal element
6, a unique maximal element 1, and that every maximal chain of P has the
same length n. Define the rankfunction r: P+ (0, I,..., n) of P by setting r(x)
equal to the length of any saturated chain in the interval
[a, X] = { y: 6 < y ,< x). If x < y in P, then set r(x, y) = r(y) - r(x). If
SC [n- 11, th en d ef me the rank-selected subposet Ps of P by

P,= (xEP:x=O,x=i-, or r(x)ES}.

If Q is any poset with 6 and 1, then define the order complex A(Q) to be
the abstract simplicial complex whose vertices are the elements of Q\{O, I},
and whose faces (or simplices) are the chains x,, < x, < a.. < xk in Q\{ O,T }.
Denote by Bi(Q) the reduced simplicial homology group I?i(d(Q), c).
(Recall that for any simplicial complex A, A- l(A, C) = 0 unless A = 0,
while g?_ i(0, C) z C and 1?,(0, C) = 0 for i > 0.)
Now suppose G is a group of automorphisms of P. For any S c [n - 11,
G permutes the maximal chains of P,. Let a: (or just a, if no confusion will
result) denote this permutation representation of G. The character of a
(complex, linear) representation a of G evaluated at g E G is denoted (a, g).
Thus (a,, g) is the number of maximal chains of P, fixed by g. In particular,
(as, 1) is just the number of maximal chains of P,. Since every element g of
G is order-preserving, G also acts on _each red_uced homology group H,(P,),
- 1 < i < ( S I-- 1. Let ys,i: G + Hom(Hi(P,), H,(Ps)) denote this represen-
tation of G. Now define a virtual representation /Is = ,iIg of G by

p, =I (-l)‘s’-‘-‘y,,i.

In particular, when S = 0 then pb is the trivial representation, i.e., p,(g) = 1

for all g E G.
134 RICHARD P. STANLEY

1.1. THEOREM. The representations a, and virtual representations ps are

related by
as= C PT, (2)
TcS

& = ,?I, (-l)lSWT1 aT. (3)

c
Proof. Let JS( g) be the reduced Lefschetz number of the map g: d(P,) -+
d(P,), i.e.,

ii,(g)= 1 (-l)“trgS*.“,
!I>-1

where g’**” denotes the induced action of g on I?JPs). Thus /i,(g) =

(-l)‘Sl-’ @, , g). Let fS( g) = f(Pi), the reduced Euler characteristic of the
subcomplex of d(P,) fixed by g. A version of the Hopf-Lefschetz fixed point
formula [4, Theorem 1.11 states that

A( g> = m>* (4)

(The result [4, Theorem 1.1) is stated for ordinary simplicial homology, but
the proof works just as well for reduced simplicial homology. The basic prin-
ciple behind (4) is well known 128, p. 389; 21; 121, but the first general
statement seems to be [4].) By definition of the Euler characteristic,

f(Pi) = C (-l)lT1-’ (a,, g).

TcS
Hence
(-I)“‘-’ @,, g) = C (-1)“‘-‘(a,, g)
TcS

for all g E G, and (3) follows. The equivalence of (2) and (3) is just the Prin-
ciple of Inclusion-Exclusion. u
Theorem 1.1 may be viewed combinatorially as follows. (a,, 1) is just the
number of maximal chains in P,, and (/3,, 1) is an invariant of P which has
been subjected to considerable scrutiny (e.g., [6,30,3 1] and the references
therein). Frequently Cgs, 1) counts some natural subset of the maximal
chains of P,, and we have a combinatorial interpretation of the formula
(as7 l)=CT~S@~y l>. N ow in general if X is the set of inequivalent
irreducible representations of G, then we have the decompositions

as = ZI, a,@) . p,
PEX
Ps = x bs@) . P,
PEX
 GROUPS ACTING ON FINITE POSETS 135

where a&) and b,@) are the multiplicities of p in a, and & , respectively. In
particular,

(Ps,l>= 1 w.Pm 1). (6)

PEX

Thus we can ask whether there are natural, pairwise disjoint subsets of
maximal chains of Ps of cardinalities a,@)@, 1) yielding a combinatorial
interpretation of (5), and furthermore decompositions of these subsets
yielding a combinatorial interpretation of the formula

as@)
= x bT@). TcS
(7)

Thus the group G gives us the opportunity (provided it has more than one
element) to try to “refine” the formula (a,, 1) = CTcSQT, 1) in a
combinatorial fashion. We will give several examples of this phenomenon in
subsequent sections.
In order for (7) to be interpreted combinatorially in the fashion suggested
above, it is certainly necessary that each b,@) > 0, i.e., that the virtual
representation & is a “genuine” representation. There is a very natural
condition on P which insures that this is the case. Define an arbitrary finite
poset P with 6 and T to be C~lten-MacauZay (over C) if for every interval
Z = [x, y] of P, we have I?,(Z) = 0 whenever i # dim d(Z). It follows that the
question of whether P is Cohen-Macaulay depends only on the geometric
realization Jd(P)J of d(P) [2, Proposition 3.4; 221. It is easily seen that if P is
Cohen-Macaulay, then every maximal chain of P has the same length n. It is
known [2, Theorem 6.4; 22, Sect. 6; 3 1, Theorem 4.31, moreover, that Ps is
also Cohen-Macaulay for any S c [n - I], It follows from (1) that
Ps = Ys.s-13 where s = )S ]. In other words:

1.2. THEOREM. Zf P is Cohen-Macaulay then & is isomorphic to the

natural representation of G on the reduced homology group E?,_, (Ps). 1
In particular, the numbers bT@), where Tc S and p E X, form a
refinement of a,@) into non-negative integers and thus could possibly count
something of combinatorial interest.

2. DUALITY

Given S c [n - I], let S= [n - 1]\S. Under certain circumstances the

representations /3s and ps will be closely related. We first give conditions
136 RICHARD p. STANLEY

under which the degrees of /Is and & are related, and then give stronger
conditions for the representations themselves to be related. Our first results
will be stated in terms of the Mobius function p = ,u,, of P [25]. All that
needs to be known about p is that if we have an interval Z = [x, y] in P, then

Pt.%Y>= 40

If Q is any poset with 6 and 1, then we write p(Q) =&a, ‘i). Now note that
if S c [n - I], then from (1) we have (putting g = 1)

(&, 1) = (-1)‘s’~‘p(P,). (9)

The following result appears in 13, Lemma 4.61.

2.1. LEMMA. Let Q be any subposetof P with 6 and 1. Then

P(Q) = +l>k~A~~xd~ptx,, -4 ..a cl,&, -i),

where the sum ranges over all chains 6 < x, < .a* < xk < ^i in P such that
x,&Qforalli. 1
We now consider a special case of the previous lemma. The poset P (as
usual, graded of rank n with 0 and 1, and with rank function r) is said to be
semi-Eulerian if 1(x, y) = (-l)r(xqy), whenever (x, y) # (0, I). If in addition
p(tj, ‘I) = (-1)“0.1’ = (-1)“, then P is called Eulerian. If the geometric
realization of A(P) is a manifold M, then P is semi-Eulerian by elementary
topological reasoning. If M is in fact a sphere (or more generally just a
homology sphere), then P is Eulerian.

2.2. PROPOSITION (The combinatorial Alexander duality theorem).

Suppose P is semi-Eulerian of rank n, and let Q be any subposet of P
containing 0 and 1. Set 0 = (P\Q) U {&I). Then

P(Q) = t-l)“-‘/@> +N’) - t-1)“. (10)

In particular, for all S c [n - 1 ] we have

@s,1)= l,&,1)+ (-1P ((-1)”-P(P)), (11)

where s = [n - 1 ]\S. Zf in fact P is Eulerian, then

P(Q) = t-l>“-‘P@>,
(12)
cg,, 1) = CBS91).
 GROUPS ACTING ON FINITE POSETS 137

Proof. If P is semi-Eulerian, then

Pp(b&&J*) ‘*‘P&k, 1) = C-l)“, k>l

= P(P), k=O

for all chains 6 < xi < *.a < xk < 1 in P. Hence from Lemma 2.1,

/i(Q) =.?Y(-l)k+n +p(P) - (-l)“,

where the sum ranges over all chains 6 < xi < ... < xk < 1 in Q. Thus

P(Q) = (-I)“-‘ik@N +i@-‘) - t-1)”

= (-1)=-‘/L(Q) +/l(P) - (-I)“,

which is (10). Picking Q = P,, Q = Ps, and using (8) yields (11). Since
,u(P) = (- 1)” when P is Eulerian, (12) follows. 1

2.3. COROLLARY. A semi-Eulerian poset P of odd rank n is Eulerian.

Proof. Substitute Q for Q in (10) and rearrange to obtain

P(Q) = WY-‘~(~1 + t-1)” W’) - C-1)“).

Comparing with (10) yields ,u(P) - (-1)” = (-1)” (a(P) - (-l)“), so

p(P) = (-1)” if n is odd. (There are many other ways to prove this
corollary.) I
According to Corollary 2.2, when P is Eulerian the virtual representations
ps and /3$ have the same degree, and we can ask what is the precise
relationship between them. For this purpose we need a stronger condition on
P so that the actual Alexander duality theorem can be invoked. (This
explains why we have called Corollary 2.2 “the combinatorial Alexander
duality theorem”-a special case of it follows directly from the usual Alex-
ander duality theorem.) Suppose therefore that the geometric realization
/A(P)1 of A(P) is the sphere $‘-* (or more generally a homology (n - 2)
sphere). Thus Hn-,(P) g C. If g E Aut P, then denote by sgn g the degree of
g, i.e., the induced map I’?~-,(P)-+~* I? “-*(P) is given by multiplication by
sgng. Since g is an automorphism, sgn g = kl, depending on whether g is
orientation-preserving or reversing (with respect to some given orientation of
A(P)). If g can be identified with some element of the orthogonal group
O(n - 1, IR) (i.e., if one has a triangulation (A(P)( -+S”-‘=
(x E w-‘:l(xlJ = 1) such that the automorphism g of A(P) is induced by
some element of O(n - 1, R)), then sgn g = det g. If G is a group of
automorphisms of A(P), then the function sgn: G -+ ( f 1 } is a character of G
138 RICHARD P. STANLEY

of degree one. A special case of the next result (which does not require Alex-
ander duality to prove) appears in [29, Lemma 71.

2.4. THEOREM. Suppose that the geometric realization of A(P) is the

spheresn-’ (or more generally a homology (n - 2)-sphere), so in particular
P is Cohen-Macaulay. Let G c Aut P. If S c [n - 11, then the represen-
tations & and & of G are related by

P, = (w) Ps.

ProoJ Let g E G, and set d = sgn g. By the Alexander duality theorem

for simplicial complexes contained in a sphere (or homology sphere), we
have an isomorphism

By the standard naturality properties of $, the following diagram commutes:

p-k(pd -+- I?,-,(P,)

G,, -II S.&I
* .
I I?
p-k(Ps) 5 &-I(&)

By the universal coefficient theorem, we may identify I?“-k(Ps) with

Homc(H,-k(Ps), C). Hence we have a pairing

satisfying a( g, a, g, b) = d wa(a, b). It follows that (ps,g> =

~wd(B~~ g-‘>. But BS is a linear combination of the permutation represen-
tations czT, so (&, g-i) = &, g), and the proof follows. 1

3. GEOMETRIC LATTICES

Before turning to examples, we wish to mention one additional result of a

general nature. Recall that a lattice is a poset for which any two elements x
and y have a least upper bound (join) x V y and greatest lower bound (meet)
x A y. A finite lattice is semimodular if whenever x and y cover x A y (i.e.,
no z. satisfies x A y < z < x or x A y < z < y), then XV y covers both x and y.
A finite lattice is geometric if it is semimodular and every element is a join of
 GROUPS ACTING ON FINITE POSETS 139

atoms (elements covering a). It is well known [7, IO] that a finite geometric
lattice is Cohen-Macaulay (and therefore graded).

3.1. THEOREM. Let L be a finite geometric lattice of rank n, and let

G c Aut L. Then there exists a basis B for fin-,(L) such that for all g E G,
the matrix of g, : r?,_*(L) -+ c?,_,(L) with respect to B has all entries equal
to 0, 1, or -1.

Proof. Let A be a base of L, i.e., a set of n atoms whose join is I. Bjorner

[ 7, Sect. 41 shows how to associate with A a cycle pA E fin-,(L). He
constructs a family I3 of bases of L such that B = (pa IA E 5) is a basis for
Hne2(L). He shows (Eq. (4.3)) that for any base A, if pA is written as a
linear combination of elements of B, then every coefficient is 0 or f 1. Since
any g E G permutes the bases of L, the proof follows. i

4. BOOLEANALGEBRAS

In this section and the next we present known examples which incorporate
the theory of the previous sections. They will prove useful as an aid to
understanding more complicated examples in subsequent sections.
Define 2,, to be the set of all subsets of [n], ordered by inclusion. 28” is a
graded poset with d and 1 (actually, a distributive lattice) of rank n, which
we call a boolean algebra of rank n. The symmetric group 6, of all
permutations of [n] acts on ~8’~ in an obvious way. In order to describe the
representations a, and ps, we first must review some facts concerning the
representations of 6,) such as may be found in [36, 371. The (complex)
irreducible representations {A} of 6, are in one-to-one correspondence with
the partitions I = (A,, A2,..., A,) of n. Here 1, > 1, > ... > il, > 0 and
ZEi = n, and we write 1 + n or (A]= n. If p = @,,...,,Q is any sequence of
positive integers summing to n, then set 6, = 6,, x 6,,* x . . . x 6,k c 6,
(the obvious imbedding). The action of 6, on the cosets of 6, (equivalently,
the induction of the trivial representation of 6, to G,) affords a certain
permutation representation d@) of 6,, called a Frobenius representation.
For each A k n, the multiplicity K,, of {A} in A(U) depends only on the
multiset (= set with repeated elements allowed) of entries of ,u, not on their
order. The number K,, (especially when ,u, > . . . >pk, i.e., when p b n) is
called a Kostka number, and is equal to the number of reverse column-strict
plane partitions (RCSPP) K of shape 1 and type ,u. This means that z = (xii)
is a planar array of positive integers rrij, with )L,.left-justified entries in the ith
row, subject to the conditions that the numbers weakly increase along each
row, strictly increase down each column, and exactly ,u~ of them are equal to

582a/32/2-2
140 RICHARD p. STANLEY

i. For instance, if L=(4,2,1) and ,~=(3,2,1,1), then K,,=4,

corresponding to
1112 1112 1113 1114
23 24 22 22 (13)
4 3 4 3

There is alternative description of K,, which is more useful for our

purposes. A standard Young tableau (SYT) of shape 1 I- n is an RCSPP of
type (1, I,..., 1), i.e., with the distinct entries 1, 2 ,..., n. The number f’ of
SYT of shape ,l is the degree of the irreducible representation {A} of 6,.
Given any RCSPP II of shape I and type ,u, replace the l’s in z with 1,
2 ,..., p, from left-to-right. Then replace the 2’s with p, + 1, ,~i + 2,..., p, + ,u~
from left-to-right, etc. We get a SYT 7 of shape A such that if i t 1 appears
in a row below i, then i=pu, +p2 + *** tpj for some J We call an i for
which i + 1 appears in a lower row a descent of 7, and we call the set of all
descents of 7 the descent set D(7). For instance, the four RCSPP of (13) are
converted to the SYTs

1235 1235 1236 1237

46 47 45 45
7 6 7 6

with descent sets (3, 5, 6}, (3, S}, (3,6}, (3, 5), respectively. Conversely,
suppose we are given an SYT 7 of shape A t n and a sequence
P = cuI,Pl,“‘, pk) of non-negative integers with Zli = n, such that D(7) c
{P,,P, tru,,...9 Pl t *a* + y,- ,}. If we replace 1, 2 ,..., p, in 7 with l’s, then
PI t 1, PI + L.., pi t p2 with 2’s, etc., we get a RCSPP n of shape A and
type ,u. Thus we have the following result, which is certainly implicit in [29,
Sect. 61 and seems fairly well known, but does not seem to have been
explicitly stated before.

4.1. PROPOSITION. Let S = {n, ,..., nJ, c [n - 11, and set ,u = (n,,
nl - n, ,..., nk - nk- I) n - nk). Then the number of SYT 7 of shape A
satisfying D(7) c S is the Kostka number K,,. a
Proposition 4.1 is closely related to Schensted’scorrespondence.This is a
well-known one-to-one correspondence between the group 6, of
permutations 71of [n] and the set of all pairs (a, 7) of SYT of the same shape
and with n entries each. We are only interested here in 7 and write Z,(n) = 7.
The precise description of this correspondence appears, e.g., in (26; 19,
 GROUPS ACTING ON FINITE POSETS 141

Sect. 5.1.41. If K = a, uz +.. Q, E 6, (where Ui = n(i)), then the descent set

D(Z) of K is defined by:

D(n)= {ila, > Ui+l} C [n- 11.

Schutzenberger [27] has shown, in our notation, that if Z,(n) = r, then

D(X) = D(r). There follows from Proposition 4.1:

4.2. PROPOSITION. Let S c [n - I] be us in Proposition 4.1, and let

A t- n. The number of permutations 71E G, such that D(K) c S and such that
Z2(x) has shape I is K,,f ‘, where K,, is the Kostku number and f’ is the
number of SYTs of shape 1. 1
We now return to the action of 6, of B”. 2n is well known to be
Cohen-Macaulay; in fact, A(&) is just the first barycentric subdivision of
the boundary of an (n - 2)-simplex. Let S = {n,, n2 ,..., nk} < c [n - I], and
let p = (n,, n2 - n, ,..., nk - nk-, , n - n&. The subgroup of 6, fixing a given
maximal chain of (~8”)~ is just G,, so a, is just the Frobenius representation
A(U). The following result is due to Solomon [29, Sect. 61, though he does
not state it quite so explicitly.

4.3. THEOREM. Let I t- n and let S be as above. Then the multiplicity

b,(A) of (A} in the representation /Is of 6, is equal to the number of SYT of
shape 1 and descent set S. Equivalently, f ‘b,(A) is equal to the number of
permututions x E G,, such that D(x) = S and such that Z2(x) has shape A.
ProoJ Let u&I) (resp., b;(A)) be the number of SYT r of shape 1 with
D(s) c S (resp. D(t) = S). Thus by Proposition 4.1, u;(A) = KAG, with
P = (n,, fl, - n,,..., n - n,J as above. Let a&) be the multiplicity of (A) in
a,. Thus from Theorem 1.1,

a&> = -T &(A), for all S c [n - 11. (14)

T-2

Since as = d(A), we have u,(A) = a#). But clearly

u;(l) = -7 b;(A), for all S c [n - 11.

TYS

Since (14) uniquely determines b,(A), it follows that b,(A) = b;(A). 1

Theorem 4.3 in effect accomplishes the following. The representation
a,,- 1, is just the regular representation of G,,, of degree n!, and the formula
%-II =Csc,n-ds d ecomposes the regular representation into a sum of
2”-’ smaller representations. The multiplicity of {A} in aIn- ,r is just f ‘, the
number of SYT of shape A. Hence we would like to assign to each SYT r of
shape 1 a set S c [n - I] such that the number of r’s which are assigned S is
142 RICHARD I’. STANLEY

the multiplicity 6,(l) of {A} in ps. Theorem 4.3 shows that we may take
S = D(t). Alternatively, we would like to assign to every rr E 6, a partition
A F n and a set S c ]n - l] such that the number of n’s which are assigned
(1, S) is b,(lZ)P. Ag ain Theorem 4.3 shows that we may take S = D(n) and
1 to be the shape of Z,(z). Moreover, these two ways to determine b,(A) are
essentially equivalent, i.e., D(n) = D(r), where r = Z,(n). These propitious
circumstances will be exactly duplicated in Section 6 for the hyperoctahedral
group (Weyl group of type B,).
Note that if the SYT t has descent set S, then the transpose t’ has descent
set S= [n - l]\S. If (A) is an irreducible representation of 6, and II’ is the
conjugate partition to 2, then it is well known that {A’} = (sgn)(A}, where
sgn denotes the sign representation of 6, (the irreducible representation
( 1”)). Hence we see that Bs = (sgn) ps. On the other hand, we have already
mentioned that Id(58n)] is an (n - 2)-sphere sne2. The degree of 7cE 6,
acting on A(.%‘,,) is just sgn z, so that Theorem 2.4 gives an independent
proof (avoiding any special knowledge of G,) that ps = (sgn) /?, .

5. THE SUBSPACE LATTICE 28,,(q)

Let 23”(q) denote the lattice of subspaces, ordered by inclusion, of an n-

dimensional vector space V,(q) over the finite field GF(q). 2’“(q) may be
regarded as a “q-analogue” of the boolean algebra .9$, since in general a
counting formula involving 3’“(q) reduces to a corresponding formula for 9’,,
when we put q = 1. The group GL,(q) of all linear automorphisms of V,,(q)
acts on 9’,,(q) in an obvious way. If S c [n - 1 ] as in Proposition 4.1, then
define the representation A@), of GL,(q) to be the permutation represen-
tation of GL,(q) on the maximal chains of 3Pn(q)s, or equivalently, the
induction of the trivial representation of GL,(q), to GL,(q), where GL,(q),
is the subgroup of GL,(q) fixing a given maximal chain of 3?Jq)s. Thus
A(u), in the q-analogue of the representation A@) of 6,, and A@), = aptq).
Steinberg [34] has shown that the representation A@), decomposes into
irreducibles parallel to the decomposition of A@) into irreducibles. More
precisely, for each A t n there exists an irreducible representation {A), of
GL,(q) (but unlike the situation for G,, there are many more irreducible
representations of GL,(q)) such that the multiplicity of {A}, in A(p), is the
Kostka number K,,, and no other irreducibles occur in A@),. Thus the
proof of Theorem 4.3 can be duplicated to yield:

5.1. THEOREM. Let ,I t n and let S be as above. Then the multiplicity

b,(il, q) of {A}, in the representation /3s of GL,(q) is equal to the number of
SYT of shape 1 and descent set S. 1
 GROUPS ACTING ON FINITE POSETS 143

When S = [n - l](so p= (1, l,..., 1)) then the iepresentation {,u}, of

GL,(q) is the Steinberg representation, of degree q’z’. Since 5Pn(q) is well
known and easily seen to be a geometric lattice, we recover from Theorem
3.1 the known result that the Steinberg representation of GL,(q) can be
realized by matrices with entries 0, f 1.

6. THE LATTICE OF FACES OF A CROSS-P• LYTOPE

Let 5?,, denote the set of all faces (including 0 and K,) of the n-
dimensional cross-polytope K,, ordered by inclusion. (K, may be defined as
the convex hull of the points fe,, 1 < i ( IZ, where e, ,..., e, are the unit coor-
dinate vectors in I?“. It is the natural n-dimensional generalization of the
octahedron K,. The dual %5$of @n is just the lattice of faces of an n-cube.)
gn is an Eulerian lattice of rank n + 1. The group of symmetries of K, is the
hyperoctahedral group B, (or Weyl group of type B,--we are denoting the
group by its “type”) of order 2”n!. Thus B, acts on gn’,, and we wish to
describe the representations a, and /Is for all S c [n]. First we will review
some facts concerning the representations of B, . Our basic reference is [ 13 1.
The irreducible representations of B, can be put in a natural one-to-one
correspondence with ordered pairs (A, ,u) of partitions such that ]A 1+ ],u] = IZ.
We call (&,u) a double partition of n, and following [ 131 denote the
corresponding representation by {A;p}. If (A 1= k (so ],u] = n - k), then
deg (2; p I= G)fAf”, where f” = deg {v}, the degree of the irreducible
representation {v} of G ,“, . The trivial representation of B, is given by {0; n},
while the sign representation (i.e., the representation of degree one whose
value on each reflection is -1) is given by { 1”; 0). The elements of B, may
be identified in an obvious way with barred permutations of [n], i.e.,
permutations ala2 +.. a, of [n] with bars over some of the ai’s. Now given
two sequences A = (A1,..., A,) and ,u = (ui ,...,pJ of positive integers with
ZAi + Cpj = n, define a subgroup BtA,,, of B, by

BcA,,, = G,, x -.a x GAr x B,, x ..a x B,$

(obvious imbedding). Let d(A;p) denote the induction of the trivial represen-
tation of BCA,,, to B,. The decomposition of A@;,u) into irreducibles has
been determined by Geissinger and Kinch [ 13, Theorem 111.51. In our
notation, their result may be stated as follows.

6.1. LEMMA. Let A and p be as above, and let (a, p) be a double

partition of n. The multiplicity of {a; /?} in A(& ,a) is given by
144 RICHARD P. STANLEY

where v ranges over all sequences(v 1,..., vr) E N’ with vi <Ai, and where
(A - v) U p denotes the sequence(,I, - v, ,..., I, - v,, p, ,..., ,u,). #
We wish to give a combinatorial description of (15) analogous to the
description of K,, in Proposition 4.1, in the case where lu has only one part.
Let (a,@ be a double partition of n. A double standard Young tableau
(DSYT) of shape (a,@ is an ordered pair (rr,, 7cJ of RCSPPs, such that 71,
has shape a, 7c2has shape /?, and every integer i E [n] appears exactly once
in 7c,U R,. The descent set O(rr, , nJ of (n,, 7cJ is defined by

O(q) n2) = {i : i and i t 1 both appear in the same zj, and

i t 1 appears in a lower row than i)
u (i : i appears in rr, and i t 1 in q, or
i = n and n appears in x1}.

For instance, the DSYT

247 16
5 3 8

has descent set (2,4,5,7}, while

145 27
69 3
8

has descent set (1,2,5,6, 7,9),

6.2. PROPOSITION. Let A= (A, ,..., ,I,.), ,u, a, /I be as in Lemma 6.1.

Assume,u = (m), the partition with onepart equal to m. Then the multiplicity
of {a;P} in 44~) =4& m> is equal to the number of DSYTs of shape
(a,/?) whose descent set is contained in the set S = (2,) A, t AZ,...,
A, t A* + *** +A,).

Proof. The term KaVKB,CI-UPUc in (15) counts the number of pairs

(z,, IL*) of RCSPPs such that 71, has shape a and type v, and nITzhas shape j3
and type (A - v) Up. Given such a pair (n,, 7~~) and assuming p = (m),
replace the li r’s which appear in rr, and rr2 by the numbers A, t A, t a.. t
~i-1+1, II,+A~+...~-&-~+~ ,..,, k,+I,+...+A,, by first tilling in
A, t **a t Ai-, + l,***, I1 t **' + ii-I + Ai - vi from left to right in x2 and
then A, t ... t I,- vi t l,..., 1, + . . . + 1, from left to right in n,. Then
replace the m occurences of r t 1 in n2 by the numbers A, t ... t 13rt l,...,
 GROUPS ACTING ON FINITE POSETS 145

1, + *-* + 2, + m from left to right. For instance, if A = (3, 1,3, 1,2),

p = (2), a = (3,3, l), B = (2,2, I), and (7~~)n,) is given by
113 13
235 56
4 6
then we obtain the arrays
237 15
4 6 10 9 12
8 11

In general we obtain a DSYT (ri , r2) of shape (a, /Q. Since v < A, it follows
that D(r,, r2) c S. This correspondence can easily be reversed, so the proof
follows. I

EXAMPLE. The multiplicity of (3 1; 13) in d( 113 1, 1) is 5, corresponding

to the DSYTs
145 3 145 2 125 3 156 2 145 2
2 6 6 3 4 6 4 3 3 6
7 7 7 7 7

Just as for the case G,, we may ask whether there is a version of
Proposition 6.2 which deals directly with the elements of B, themselves. In
other words, we seek an analogue z of Schensted’s correspondence Z for
which Proposition 4.2 exends to B,.
We proceed to define the correspondence f. Let 7c= a’,$, ... i,, E B,,
where the overhead wiggles may be either blanks or bars. Let ai, ui2 . . . ail be
the subsequence of unbarred elements. Apply Schensted’s correspondence to
the two-line array (or “generalized permutation”) ($ $* 1::2,). This yields a
pair (a,, ri) of RCSPPs of the same shape. The entries of 0, consist of
ai,,.-, a,,, while the entries of ri consist of i,,..., i,. Similarly if aj,uj, ..a uj, is
the subsequence of barred elements, apply Schensted’s correspondence to
($, $ 1::$ ) to obtain a pair (cT~, Q). Finally define @r) = (a, T), where
cr = (a,, u2) and r = (r,, r2). For instance, if x = 4 1 7 3 6 9 2 8 3, then

2 3 6 7 8 2 3 6
= (a,, fl),
78
35 15
= (a, 7 72).
I46 RICHARD P. STANLEY

Hence J?(n) = (a, t), where

Clearly this is a one-to-one correspondence, since given (0, t) where

u = (ci, u2) and r = (ri, r2), we can reconstruct uniquely the two-line arrays
.?-‘(a,, r,) and C-l(u2, rz).
Now given rcE B,, we first give an ad hoc definition of the descent set
o(n). Namely, if z = a’, a’, . . . c,, as above, then define

D(n) = (i:q > ai+, or ai>ai+, or

ai < ai,, or ai > tii+l or (16)
i=n and (I, is unbarred}.

Here a notation such as ai ( ti,, , means that ai < ai+ i (as integers), ai is
unbarred, and a,, i is barred. Thus for instance O(;i 1 7 5 6 9 2 8 5) =
(3,698).

6.3. PROPOSITION. IfZ(7r) = (u, t), then D(7r) = D(r).

ProoJ Let r = (r,, rJ. By the ordinary Schensted correspondence, we

have:
(a) i + 1 lies below i in r, if and only if a,+i > a, (both unbarred),
(b) i + 1 lies below i in r2 if and only if ai+ I > ii,
(c) i appears in r, and if 1 in r2 if and only if ai is unbarred and aifl
is barred, and
(d) n appears in r, if and only if a, is unbarred.
The proof follows from the definition of D(r,, rJ. 1
The definition (16) of O(x) can be made less ad hoc. The group B, is
generated by simple reflections s,,..., s, corresponding to the Dynkin
diagram:

Multiplication can be defined as follows: If 1 <i< n - 1 and

x= aI’ ,a22 . . . a.T
n where the j above uj is either blank or a bar, then
si7r= (I, ;* *** ;;+; 6, *.* n:,, if 1 < i < n - 1. Moreover, s,z replaces a bar
over u, by a blank or vice versu. If now 72E B,, recall that the length l(z) is
the least integer I for which one can write a-= silsiz . . . si,. It is then easy to
 GROUPS ACTING ON FINITE POSETS 147

check that i E D(X) if and only if Z(Si~) > I(n). Equivalently, sin > II in the
Bruhat order of B,, as defined, e.g., in (331.
Note. The corresponding result for n E 6, states that i E D(n) if and
only if sire < n in the Bruhat order of G,. The reason for the reversed
inequality (i.e., sin > 7c in B, and sir < z in G,,) arises from our desire to
adhere to the notation of [ 131.
It is now easy to determine the analogue of Theorem 4.3 for the cross-
polytopal lattice %$ Since d(‘Z,J is just the first barycentric subdivision of
the boundary complex of the cross-polytope K,, it follows that gn is
Cohen-Macaulay. Let S = {n,, n2 ,..., nk}< c [n]. Set A= (n,, n2 -n ,,...,
nk - nk- i) and m = n - nk. Then the subgroup of B, fixing a maximal chain
. *
in (%)s 1sJust &,,. Hence a, is just the representation A(& m) of B,.

6.4. THEOREM. Let P = gn and S c [n]. Let G = B, be the group of

automorphisms of P. Then the multiplicity b,(a,& of the representation
{a; p} of B, in the representation ps is equal to the number of DSYTs of
shape (a,& whose descent set is equal to S. Equivalently, f ((1’4’bs(a,P) is
equal to the number of elements n E B, such that D(n) = S and such that
F2(x) has shape (a, /?). Here f (am = deg{a; p}.
Proof. The proof parallels that of Theorem 4.3. Let a&(a,P) (resp.
bk(a, /3)) be the number of DSYT t of shape (a,/?) with D(t) c S (resp.
D(s) = S). By definition,

4(a,P>= x &(a,PI,
TCS
for all S c [n].

Let a,(a,p) be the multiplicity of {a; p} in as. If S = {n,, n, ,..., nk}< then
set A= (n,, n2 - n, ,..., nk - nk-J and m = n - nk. Since a, = A@, m), by
Proposition 6.3 we have a,(a, p) = ai(a, p). By Theorem 1.1,

as(a,
P>= T’;‘S
r bT(aT
P), for all S c [n].

It follows that b,(a, p) = bk(a, p) for all S c [n]. m

Note that Theorem 2.4 applies to gn since, as we’ve already remarked,
d(GQ is isomorphic to the first barycentric subdivision of the boundary of
the cross-polytope K,. Since the sign representation of B, is { 1”; a}, we
conclude ps = { 1”; 0) &, where s = [n]\S. This checks with Theorem 6.4,
since { 1”; 0}(a;/3} = {p’; a’} (where ’ denotes conjugate partition), and if
(r,, rz) is a DSYT then D(z,, z2) = [n]\D( r:, r:) (where ’ denotes transpose).
Note. While the generalization E of Schensted’s correspondence which
we have defined is certainly adequate for our purposes, it fails to possess a
148 RICHARD P.STANLEY

crucial property of interest for other applications. Namely, it is not true that
two elements 71,u of B, satisfying ,$i(z) = J?,(o) lie in the same cell of B,, as
defined in [13].

7. THE LATTICE OF PARTITIONS OF AN ~-SET

Let n, denote the poset (actually a lattice) of all partitions of [n], ordered
by refinement. Thus the elements of n,, are sets p = {B, ,..., Bk}, where the
B[s are pairwise-disjoint nonvoid subsets of [n] with union In]. Moreover,
{B; s.., B;) < {B, ,..., Bk} in n, if and only if every B: is contained in some
B,. Z?, is well known to be a geometric lattice of rank n - 1 [5, p. 951, and
therefore by the sentence preceding Theorem 3.1 is Cohen-Macaulay. The
symmetric group 6, acts on n,, in an obvious way, and we can ask for the
decomposition of the representations as and & for S c [n - 21. In
particular, it is again well known [5, p. 93, Ex. 7; 2.5, p. 3591 that in n,,
,u@, 1) = (-1)“-’ (n - l)!. Thus by (1 l), /31n-21 is a representation of 6, of
degree (n - l)!, and we can try to describe it more explicitly.
The crucial fact we need is the following recent result of Hanlon
[ 16, Theorem 41.

7.1. LEMMA. Let 71E G,,, and let l7; denote the sublattice of ll,, fixed
pointwise by II. Let ,u, denote the M6bius function of IZZ. Then

,u(,@, 1) = (-l)d-’ ,u(n/d)(d - I)! (n/d)d-‘, ifz is a product of d

cycles of length n/d
= 0. otherwise.

(Here ,u(n/d) notes the usual number-theoretic Miibius function.) I

Remark. Hanlon actually computes ,uJx,, I), where x, is the meet of

the coatoms of Z7:. It is easy to see (alternatively, this follows from [ 16,
Lemma 21) that x, = 6 if and only if all cycles of 7chave the same length.
Since in any (finite) lattice ~(0, I) = 0 unless 0 is a meet of coatoms
[25, p. 3491, Lemma 7.1 follows from Hanlon’s results.
Now let C, be a cyclic subgroup of 6, of order n generated by an n-cycle
0. Let [ = e2ni’n, and (by slight abuse of notation) identify [ with the
representation (or character) of C, whose value c(a) at u is c. Define the
induced representation

w,, = ind,%(c). (17)

 GROUPS ACTING ON FINITE POSETS 149

7.2. LEMMA. Let KE 6,. Then

(w,, n> = P(@)(d - I)! (n/dY- l, if n is a product of d

cycles of length n/d
= 0, otherwise.

ProoJ: The usual formula (e.g., [15, Theorem 16.7.21) for the character
of an induced representation yields

where C, is the conjugacy class of 6, containing 71. Hence (w,, rr) = 0

unless din and 7chas d cycles of length n/d. In this case c(o) runs through all
primitive (n/d)th roots of unity and ] C,] = n!/(n/d)d d!. Since the sum of the
primitive (n/d)th roots of unity is ,u(n/d) (e.g., [ 17, (16.64)]), the proof
follows. I

7.3. THEOREM. Let G = 6, act on P= 17, in the obvious way. Then

/3rn-21= (sgn) vn. (Note that if n f 2 (mod 4), then either (-l)“-d = 1 or
,u(n/d) = 0 for all din. Thus in this case /3r,-2, = vn and /?r,,-2, is “self-
conjugate.“)

ProoJ: Let II E 6,. By (4), we have

(-1)“-’ <Pr”-*~,~)=~,“-2](~)=~(~(~,)=)=1S(~(~~>>.

Thus by (8)

~ln-*,‘n)=(--I)“-‘~U,(~,~),
and the proof follows by comparing Lemmas 7.1 and 7.2. I

7.4. COROLLARY. Let p be an odd prime. Then the representation Pra- 2,

of 6, given in Theorem 7.3 is equal to “(l/p)th of the regular represen-
tation” of Gi. More precisely, ifA l-p then the multiplicity of {A} in &, _ 21is
equal to (f /p), where (x) denotes the nearest integer to x.

Proof. Let x” be the character of {A}, and let u be an n-cycle. Since

&-21 = w,, by Frobenius reciprocity the desired multiplicity is given by

f $ ~“(a’) e-2nu’p = +- (fA -x(u)).

150 RICHARD P. STANLEY

It is well known (e.g., [37, Lemma 4.1 I]) that

x”(a) = (- 1)j- 1, if A=@-i,l,l, . ... l),O<i<p

i

= 0, otherwise.

From this the proof is immediate. 1

7.5. COROLLARY. For any n > 1, the representation w,, of G, can be

realized by matrices whoseentries are all equal to 0, + 1, or - 1.
Proof: Follows from Theorems 3.1 and 7.3, and the fact that ZZ, is a
geometric lattice. I

7.6. COROLLARY. The group G = G,-, acts on [n] by permuting [n - l]

and Jxing n. Extend this action to II,. Then the corresponding represen-
tation /?,n-2I of 6,-, is isomorphic to the regular representation.
Proof. By Theorem 7.3, the representation P,n-21 is the restriction of
(sgn) y/, to 6,-i. By Lemma 7.2, it follows that (Pin-2,, z) = 0 unless 71= 1.
Since deg&,-,, = (n - l)!, the proof follows. I
Theorem 7.3 completely determines pin-21 for the action of G, on n,. We
have been unable to find an analogous description of j?, for arbitrary
Sc[n-21. W e h ave, however, obtained some information on the problem
of computing the multiplicity (which we will denote by b, = b,(n)) of the
trivial representation {n} in ps. Let as = a,(n) denote the multiplicity of {n}
in as. Thus a, = ](n,),/C,l, the number of orbits of maximal chains of
(n,), under the action of 6,. I am grateful to I. Gessel and J. Shearer for
supplying (independently) a proof of the following result.

7.7. THEOREM. Define integers A,, by

x A,x”/n! =tanx+secx.
H>O
Then al,-,Jn) = A,-,.
Proof. We seek the number of orbits of maximal chains of n,, under the
action of G,. For convenience denote this number by c,. We obtain exactly
two representatives 6 =x0 < x, < .e. < x,-i = 1 of each orbit as follows:
Choose 1 Q i < n - 1. Then choose the partition x,-~ of [n] to consist of
two specified blocks B, and B,, where JB, ( = i and IB21= n - i. (We get two
orbit representatives because B, and B, could be interchanged.) Next choose
 GROUPS ACTING ON FINITE POSETS 151

representatives of the orbits of maximal chains of n(B,) and n(B,) under

n(B,) and n(B,) in cic”_i ways. Finally “interweave” these two maximal
chains in (I:: ) ways to obtain a maximal chain of l7, containing x,-*. It
follows that
n-1
n-2
2c, = x cicnmi, n > 2, Cl = 1.
i=l i i- 1 1
Comparing with the well-known recurrence [8, p. 2581

2A IIt1 AiA,-i, n>O,A,= 1,

yields the proof. I

By Theorem 1.1, we have

c b,(n)=A.-,. (18)
Sc[n-21
Since fl,, is Cohen-Macaulay, each b,(n) > 0. Moreover, A, is equal to the
number of alternating permutations in G,, i.e., permutations 71E 6,
satisfying D(n) = (1, 3, 5,..., 2[n/2] - 1) (e.g., [8, p. 2581). This suggests the
following problem:
Problem. Find a combinatorial interpretation of b,(n) as the number of
alternating permutations in G,- r with a certain property (parametrized by
S c [n - 2]), so that (18) is evident.
It is easy to see that if 1 < i < n - 2, then b(,,(n) =pnpi(n) - 1, where
p,Jn) denotes the number of partitions of n into k parts. We also have the
following result.

7.8. PROPOSITION. Let 1 < i < n - 2. Then bli,(n) = 0.

Proofi If i = n - 2, then by Theorem 7.3, b,,-,](n) is equal to the
multiplicity of {n} in (sgn) t,un. Let cr E 6, be an n-cycle. By Frobenius
reciprocity,

1 n!’
b,,-,I(n) =; + ewznijln sgn(oj)
J-0
1 n?’ e-*nii/n(-l)(n-I)j = 0.
=-
n Jeo

If 1 < i < n - 3, then the result &(n) = 0 is due to P. Hanlon (private

communication), using the case i = n - 2 to deduce the general result.
Hanlon’s proof will appear elsewhere. I
152 RICHARD p. STANLEY

Even if the numbers b,(n) cannot be determined explicitly, it may still be

possible to say when b,(n) = 0. Table 1 shows that the sufficient condition of
the previous proposition is not necessary. In regard to this table, we remark
that it is not diflicult to see that if S c [M] and if n > 2m, then b,(n) depends
only on S, not on n.
There is an alternative method of computing al,,-,, (n) which leads to a
curious combinatorial identity.

7.9. PROPOSITION. Define an array g(k, n) of rational numbers as

follows:
g(0, o>= 1, gk n) = 0 if k < 0 or n < 0,
g(k, n) = f(n - 2k + 2) g(k - 1, n - 1) + f(n - 2k) g(k, n - I),
if k, n > 0, (k, n) # (0,O).

Theng(k,n)=Oifk>[f(n+l)],and
I@+ I)/21
c g(k, n) =A,.
k=

TABLE I

Some Values of b,(n)

s n=3 n=4 n=5 n=6 II=1 n)8

0 1 1 1 1 1 1

1 0 0 0 0 0 0
2 1 1 1 1 1
3 1 2 2 2
4 2 3 4

12 0 0 0 0
13 I 1 1
14 1 2
23 1 1 1
24 3 5
34 1 2

123 0 0 0
124 0 0
134 2 3
234 1 1

1234 0 0 0
 GROUPS ACTING ON FINITE POSETS 153

ProoJ By Burnside’s lemma (e.g., [8, p. 249]),

a],-,]@+ 1)= (n : l)! =.;+, ‘FiX(71)‘3

where Fix(n) is the set of maximal chains C of f7,,+, fixed by II. Since a
maximal chain C of fl” + 1 is obtained from the partition into n singletons by
merging two blocks at a time, we easily see that Fix(z) = 0 unless rr* = 1.
Let f(k, n) = 1Fix(x)/, w here x contains k 2-cycles and n + 1 - 2k fixed
points. There are (n + 1)!/2kk! (n + 1 - 2k)! such z, so

qn-l](n+ 1)= ‘@+?*I f (k n> (20)

kyo 2&k! (n + 1 - 2k)! *

Suppose K has k 2-cycles and n + 1 - 2k fixed points, and consider a

maximal chain d < x1 < a.. < x,-~ < 1 in n,,, fixed by x. The partition x,
has a unique two-element block (a, 6}, and we have the following two alter-
natives:
(i) (a, b) is a 2-cycle of n (k possibilities).
(ii) LI and b are fixed points of 7c ((“+\-2k) possibilities).

In the first case, there aref(k - 1, n - 1) ways to choose x2, x3 ,..., x,-, . In
the second case, there aref(k, n - 1) ways to choose x2, x3 ,..., x,-, . Hence
f(O, 0) = 1, f(k n) = 0 if k < 0 or n < 0,

f(k,n)=kf(k- l,n- l)+ (nii-2k)f(k,n- l),

if k,n>,O,(k,n)#(O,O). (21)

Now set

g(k, n) =f(k, n)/2kk!(n + 1 - 2k)!.

It is immediate from (21) that g(k, n) satisfies (19). Comparing Theorem 7.7
and (20) yields the proof. 1
Ira Gessel has found a proof of Proposition 7.9 which avoids Bumside’s
lemma or other group-theoretical considerations.

8. BARYCENTR~C SUBDIVISION

Let P be a finite poset with 0 and 1. The simplicial complex d(P) may
itself be regarded as a poset, with its faces being ordered by inclusion. Thus
154 RICHARD p. STANLEY

A(P) has a d consisting of the void face 0. Let a(P) denote A(P) with a ?
adjoined. If P is graded of rank II, then so is d(P). In fact, A@(P)) is just the
first barycentric subdivision of A(P). Thus d(P) is Cohen-Macaulay if and
only if P is, and any group G c &it P also acts in an obvious way on a(P).
In this section we will compute a;(‘) and /I$“’ in terms of a: and /I’,.

8.1. PROPOSITION. Let P and G be as above, and let S=

Is ,,..., sk}< c [n - 11. Then

6(P) = sk
\‘ a’,.
as
s1, s;! - s, ,..., Sk- Sk-, Tc[n-11-
ITi’Sk

Proof: Let 0 < x1 < . . . < xk < 1 be a maximal chain C in A(P)s. Thus xi
is a chain in P - {a, I} with si elements, and xi-i c xi (as subsets of P). If
g E G, then g fixes C if and only if it fixes xk. It follows that a maximal
chain of a(P)s fixed by g is obtained by: (a) choosing a subset T of [n - l]
with 1TI = sk, (b) choosing a maximal chain xk of P, in (a;, g) ways, and
(c) choosing subchains x, c ... cxk-r cxk of xk with ]xi] =si in
(S,+S,.?., Sk-Sk-, ) ways. From this the proof follows. 1

8.2. LEMMA. Let TC [n - l] and 0 < i < n. Define H,(T) to be the

number of permutations 71= a, a2 . .. a,, E 6, with descent set D(x) = T and
with a,, = n - i. Zf S = {sl ,..., sk}< c [n - 11, then

‘k
c H,(T)= ) ( n”:s;:lJ.
TCS s,,s,--s ,,...r Sk-Sk-1

Proof: If D(n)cS and a,=n-i, then l<aSk+, <as,+*< .a. <

a,- i Q n - i - 1. There are therefore (~:~,:‘r) ways to choose ask+,,..., a,-, .
The remaining aj’s must satisfy a, < a2 < ..+ < a,,,a,,+, (a,,,, ( . .. (
a S*Y*, aSk-,+ 1 < as. < ask and can therefore be chosen in (s, , s2-s, ,.Sk
.., sk-sk~,)
ways. From this the proof follows. 1

8.3. THEOREM. Let P and G be as above, and let S c [n - 11. Set

Tc[Pl-I]
ITI=i

Then
n-1
pi(‘) = C Hi(S) 6iy (22)
i=O

where H,(S) is defined in Lemma 8.2.

 GROUPS ACTING ON FINITE POSETS 155

Proof: Let & denote the right-hand side of (22). By Theorem 1.1, it
suffices to show that

where a$‘) is given by Proposition 8.1. Let S = isI ,..., sk}<. Now

= c c H,(T) c c (-l)‘v-W’ a;
TCS i IVI=i WcV

=; 4 (& wYv-w’ & %#-3)

c

the latter step by the previous lemma. A well-known combinatorial identity

is equivalent to

~~~(-l)i-w(“r’;“)(n”-t-:)=l, if sk=w

= 0, if sk f W.

(This may be proved, e.g., by multiplying the two power series (1 - x)“-”
and xW+(l -x)-(“-“‘).) It follows that

as was to be proved. m

9. UNIMODALITY

A sequence a,, a1,..., a,, of real numbers is called symmetric if a, = a,,-i,

and unimodal ifao~aa,~...~a,>,aj+l~...~a, for somej. Given n, a

582a/32/2-3
156 RICHARD P.STANLEY

polynomial a, + ai q + . . . + anqn is said to be symmetric or unimodal if the

sequence a,, a, ,..., a, of coefficients is symmetric or unimodal, respectively.
We will show how the theory of groups acting on posets can be used to show
that certain sequences are unimodal. In particular, we obtain what seems to
be the simplest known proof that the q-binomial coefficient [ z ] is unimodal;
and we obtain some new examples of unimodal sequences.
Let r= (X, Y, E) be a finite bipartite graph, i.e., X and Y are finite
disjoint sets of vertices, and E c X x Y is a set of edges. The adjacency
matrix A, of r is the matrix whose rows are indexed by X and columns by
Y, and whose (v, w)-entry is 1 if (v, w) E E, and 0 if (v, w) @E. An
automorphism g of r is a bijection g: X U Y + XV Y such that
(v, w) E E e (gv, gw) E E. (In particular, gX = X, gY = Y.) If #i and d2 are
(complex, linear) representations of a finite group G, write #i < & to mean
that the virtual representation & - 4, is a genuine representation, i.e., if
mi@) is the multiplicity of the irreducible representation p in di, then
m,(p) < mz@) for all p.
The following lemma is well-known (e.g., [9, p. 22]), but since it is usually
stated in a more general form we will include a proof.

9.1. LEMMA. Let I be a bipartite graph as above, and let G be a group

of automorphismsof I. Let #x and 0, denote the permutation representations
of G acting on X and Y, respectively. If rank A, = (Xl, then gx < $y.
Proof. Let CX and CY denote the complex vector spaces with bases X
and Y, respectively. Define a linear transformation 6: GX + CY by setting

4(v) = z: w,
WEY
(U.W)EE

where v E X. The matrix of 0 with respect to the bases X and Y is A,, so

rank $ = 1x1. Hence 4 is injective. On the other hand, since each g E G is an
automorphism of r, it follows that d is a G-homomorphism. From this the
proof is immediate. 1
Now let P be a graded poset of rank n with d and 1, and with rank
function r. Let Pi = {x E PI r(x) = i). Let Ti denote the bipartite graph
obtained by restricting P to Pi U Pi+ i, 0 Q i < n. If rank Ar, = min (1PiJ,
IP,, I I}, for all 0 < i < n, then we say that P is ample. If the sequence JP,l,
(P, (,..., ]P, ] is unimodal, then we say that P is unimodul.

9.2. THEOREM. Let P (as above) be unimodal and ample. Suppose that
G c Aut P and that p is an irreducible representation of G. Let mi@) denote
the multiplicity of p in the action (permutation representation) di of G on Pi.
Then the sequencem,,(p), m,@) ,..., m,,(p) is unimodal. In particular (taking p
 GROUPS ACTING ON FINITE POSETS 157

to be trivial), if m, = 1PJG), the number of orbits of #i, then the sequence

mo3m, ,..., m, is unimodal.
Proof: If ]P,I Q ]PI+ i 1, then by the previous lemma #i < (i+ i so
m,(p) < m,, i@). All inequalities are reversed when ] PiI > 1Pi+ 1I, so the
proof follows. I
Though Theorem 9.2 is quite elementary and simple to prove, it does lead
to some interesting consequences.

9.3. PROPOSITION. The boolean algebra 2,, is ample. 1

The are several elementary proofs of this result in the literature, e.g., [ 11,
Lemma 5.1; 14, p. 131 (where an explicit right inverse to the adjacency
matrices Ari, 0 < i < in, is constructed), and [ 18 1.

9.4. PROPOSITION. Let G a group of permutations of the n-elementset S.

Let p be an irreducible representation of G, and let m,(p) be the multiplicity
of p in the action of G on the set (s) = { Tc S: 1TI = i}. Then the sequence
m,@), ml@),..., m,,(p) is symmetric and unimodal.

Proof. Unimodality is an immediate consequence of Theorem 9.2,

Proposition 9.3, and the fact that ~8’” is unimodal. Since the actions of G on
( f) and (nS-i) are isomorphic, we have m,(p) = m,- i@). I
In particular, taking p to be trivial in Proposition 9.4 we deduce the
following result of White [35, Corollary 71.

9.5. COROLLARY. Let G and S be as above. If mi denotes the number

I( f )/GI of orbits of G on (s), then the sequencem,, m,,..., m, is symmetric
and unimodal. I
Following G. D. James, D. White and others, the wreath product 6, - 6,
acts (as its defining representation) on a k x I rectangle R of squares by
permuting the elements of each row independently, and by permuting the k
rows among themselves. Thus 6, - 6, acts on the set (‘I ), and each orbit of
this action contains a unique Ferrers diagram. (See, e.g., [ 1, p. 71 for the
notion of a Ferrers diagram.) Hence if pk,(i) denotes the number of Ferrers
diagrams fitting in a k x 1 rectangle, we deduce that the sequence

P,#), PA )wp,,(kO
is symmetric and unimodal. It is well known [ 1, Theorem 3.11 that

k+l=(1-4k+w
-4k+‘-‘)
..(1-pl+‘),
(23)
1
I
Pklti) 4’ =
[ 1
k
(1 -qk)(l --k-i) a** (1-q)
158 RICHARD P. STANLEY

the q-binomial coefficient. Hence we obtain what appears to be the simplest

proof to date of the following corollary. See [32, p. 1321 for a brief history
of this result.

9.6. COROLLARY. For any integers k, l> 0, the q-binomial coefficient

[ kk+‘] is symmetric and unimodal (of degree kl). 1

9.7. COROLLARY. For any integers k, 1 > 0, the polynomial

(1 + q)k’ - [“: ‘1 is symmetric and unimodal.
Proof: Let G = G, - G, act on R as above, and let X* be the set of all
inequivalent non-trivial irreducible representations of G. By Proposition 9.4,
the sequence t,, t, ,..., t,, where ti = CpEX* m,(p), is symmetric and unimodal.
Since Zt[q’ = (1 + q)k’ - [k:‘], the proof follows. 1
We next give a q-analogue of Proposition 9.4. As in Section 5, let AYa(q)
denote the lattice of all subspaces of an n-dimensional vector space V,,(q)
over GF(q). The following result is due to Kantor [ 181.

9.8. PROPOSITION. The lattice S”(q) is ample. 1

9.9. COROLLARY. Let G be a subgroup of GL,(q), the group of linear

automorphisms of V,,(q). Let p be an irreducible representation of G, and let
m,.(p) be the multipliticity of p in the natural action Qi of G on the Grassmann
variety G,[ of all i-dimensional subspacesof V,,(q). Then the sequence
m,(p), m,(j),..., m,@) is symmetric and unimodal.
Proof: Unimodality follows from Theorem 9.2 and Proposition 9.8.
Symmetry follows from the fact that the action of a finite group G on a
vector space V over R is isomorphic to the action of G on the dual space
Horn& R). i
It seems difficult to find additional interesting examples of ample posets
with non-trivial automorphisms. We will conclude with one very specialized
result of this nature. Following [ 331, let L(m, n) denote the poset (actually a
distributive lattice) of all Ferrers diagrams fitting in an m X n rectangle,
ordered by inclusion, Equivalently, L(m, n) consists of all integer sequences
a = (a, ,..., a,) satisfying 0 < a, < . .. < a,,, < n, ordered component-wise (i.e.,
a <b if a, < bi for all i).
9.10. PROPOSITION. L(m, n) is ample.
Proof. Let Ai denote the adjacency matrix of the bipartite graph ri
obtained by restricting L(m, n) to ranks i and i + 1, for 0 < i < mn. By
Pieri’s formula in the Schubert calculus, Ai is the matrix of the linear
transformation o: H2i(Gm+n,m; C) + H2(i+1)(G,+n,m; C), with respect to the
 GROUPS ACTING ON FINITE POSETS 159

basis of Schubert cycles, obtained by multiplication by a hyperplane section

o. By the hard Lefschetz theorem, if 0 <i < tmn then the matrix
An-i-IAn-i-2 *** Ai+, A, is invertible. Hence Ai has full rank, as desired. 1
For further information on the concepts used in the above proof, see e.g.,
[33, especially pages 175-1761. A more elementary proof appears in [24].

9.11. PROPOSITION. Let n be a positive integer. Then the two

polynomials

2n
P,(q) =
[ 1
n
2n
+ (1 + q)(l + q3)(1 + q5) *** (1 + qzn-73

P,(q) = - (1 + q)(l + q3)(1 + q5) ..* (1 + q2”-‘)

[ n I
are symmetric and unimodal.
Proof. The group C, of order 2 acts on an n x n rectangle R by
interchanging rows and columns, and thus on L(n, n). Let E denote the trivial
character of C, and x the irreducible character of order 2. An element Y of
L(n, n) (i.e., a Ferrets diagram fitting in R) is fixed by C, if and only if it is
self-conjugate. Hence the number of orbits m,(c) of C, acting on L(n, n)i is
given by f(p,,(i) + c,,(i)), where p,,(i) = ] L(n, n)i\ and c,(i) is the number of
self-conjugate Ferrers diagrams in R of cardinality i. By (23),
2n
x PA)
i
d =
[ 1
n *

On the other hand, it is well-known (e.g., [ 17, Theorem 3471) that

s c,(i)q’= (1 +q)(l +q3)(1 +q5) ..a (1 +q2”-l).

Since L(n, n) is ample (by Proposition 9.9) and unimodal (since [ ‘,” ] is a
unimodal polynomial, e.g., by Corollary 9.6), it follows from Theorem 9.2
that iP,(q), and therefore P,(q), is unimodal.
Since mi(e) + m,(x) = 1L(n, n)il =p,,(i), we have

Hence by Theorem 9.2, P,(q) is also unimodal. The symmetry of P,(q) and
P,(q) is obvious. I
The fact that P,(q) is unimodal is hardly surprising. In fact, it follows
from [23] that for some k > 0 (independent of n), if we set (1 + q)
160 RICHARD P. STANLEY

(1 +q3)-* (1 +q ‘“-‘) = C::, c,(i) qi, then the sequence c,(k), c,(k + l),...,
c,(n* - k) is unimodal. Quite probably k = 3 for n large, which should be
provable by the methods of [23]. At any rate, since the sum of symmetric
unimodal polynomials of the same degree is symmetric and unimodal, we
conclude that “most of’ P,(q) is unimodal. On the other hand, there seems
to be no simple reason for expecting P,(q) to be unimodal.

ACKNOWLEDGMENTS

I am grateful to W. Kantor, G. Lusztig and J. Munkres for several helpful discussions.

REFERENCES

1. G. E. ANDREWS, “The Theory of Partitions,” Addison-Wesley, Reading, Mass., 1976.

2. K. BACLAWSKI, Cohen-Macaulay ordered sets, J. Algebra 63 (1980), 226-258.
3. K. BACLAWSKI, Cohen-Macaulay connectivity and geometric lattices, preprint.
4. K. BACLAWSKI AND A. BJ~RNER, Fixed points in partially ordered sets, Advances in
Math. 31 (1979), 263-287.
5. G. BIRKHOFF, “Lattice Theory,” 3rd ed., American Mathematical Society, Providence,
RI., 1967.
6. A. BJ&NER, Shellable and Cohen-Macaulay partially ordered sets, Trans. Amer. Math.
Sot. 260 (1980), 159-183.
7. A. BJ~RNER, On the homology of geometric lattices, Algebra Universalis, to appear.
8. L. COMTET, “Advanced Combinatorics,” Reidel, Boston, 1974.
9. P. DEMBOWSKI, “Finite Geometries,” Springer-Verlag, 3erlin/Heidelberg/New York,
1968.
10. J. FOLKMAN, The homology groups of a lattice, J. Moth. Mech. 15 (1966). 631-636.
1 I. W. FOODY AND A. HEDAYAT, On theory and applications of BIB designs with repeated
blocks, Ann. Statist. 5 (1977), 932-945.
12. P. F. GARST, Cohen-Macaulay complexes and group actions, thesis, Univ. of Wiscon-
sin-Madison, 1979.
13. L. GEISSINGER AND D. KINCH, Representations of the hyperoctahedral groups, J. Algebra
53 (1978), l-20.
14. R. L. GRAHAM, S.-Y. R. LX, AND W.-C. W. LI, On the structure of t-designs, SIAM J.
Algebraic Discrete Meth. 1 (1980), 8-14.
15. M. HALL, JR., “The Theory of Groups,” Macmillan Co., New York, 1959.
16. P. HANLON, The fixed-point partition lattices, Pacific J. Math. 96 (1981), 3 19-341.
17. G. H. HARDY AND E. M. WRIGHT, “An Introduction to the Theory of Numbers,” 4th ed..
Oxford Univ. Press, London, 1960.
18. W. M. KANTOR, On incidence matrices of finite projective and afline spaces, Math. Z.
124 (1972), 315-318.
19. D. KNUTH, “The Art of Computer Programming,” Vol. 3, Addison-Wesley, Reading,
Mass., 1973.
20. D. KAZHDAN AND G. LUSZTIG, Representations of Coxeter groups and Hecke algebras,
Invent. Math. 53 (1979), 165-184.
21. G. LUSZTIG, “The Discrete Series of GL, over a Finite Field,” Princeton Univ. Press,
Princeton, N.J., 1974.
22. J. MUNKRES, Topological results in combinatorics, preprint.
 GROUPS ACTING ON FINITE POSETS 161

23. A. M. ODLYZKO AND B. RICHMOND, On the unimodality of some partition polynomials,

preprint.
24. R. PROCTOR, Solution of two difficult combinatorial problems with linear algebra, Amer.
Math. Monthly, to appear.
25. G.-C. ROTA, On the foundations of combinatorial theory, I. Theory of M6bius functions,
2. Wuhrsch. Verw. Gebiete 2 (1964), 340-368.
26. C. SCHENSTED, Longest increasing and decreasing subsequences. Cnnad. J. Muth. 13
(1961), 179-191.
27. M. P. SCH~TZENBERGER,Quelques remarques sur une construction de Schensted, Math.
&and. 12 (1963), 117-128.
28. L. SOLOMON, The orders of the finite Chevalley groups, J. Algebra 3 (1966), 376-393.
29. L. SOLOMON, A decomposition of the group algebra of a finite Coxeter group, J. Algebra
9 (1968),220-239.
30. R. STANLEY, Finite lattices and Jordan-H&der sets, Algebra Universalis 4 (1974).
361-371.
31. R. STANLEY, Balanced Cohen-Macaulay complexes, Trans. Amer. Math. Sot. 249
(1979),361-371.
32. R. STANLEY, Unimodal sequences arising from Lie algebras, in “Combinatorics,
Representation Theory and Statistical Methods in Groups” (T. V. Narayana.
R. M. Mathsen and J. G. Williams, Eds.), pp. 127-136, Dekker, New York. 1980.
33. R. STANLEY, Weyl groups, the hard Lefschetz theorem, and the Sperner property, SIAM
J. Algebraic and Discrete Meth. 1 (I980), 168-184.
34. R. STEINBERG, A geometric approach to the representation of the full linear group over a
Galois field, Trans. Amer. Math. Sot. 71 (1951), 274-282.
35. p. WHITE, Monotonicity and unimodality of the pattern inventory, Advances in Math. 38
(1980), 101-108.
36. G. JAMES AND A. KERBER, “The Representation Theory of the Symmetric Group.”
Addison-Wesley, Reading, Mass., 1981.
37. G. DE B. ROBINSON,“Representation Theory of the Symmetric Group,” Univ. of Toronto
Press, Toronto, 196 1.