# Reflection groups in hyperbolic spaces

and the denominator formula

for Lorentzian Kac–Moody Lie algebras

This is a continuation of our ”Lecture on Kac–Moody Lie algebras of the arithmetic type” [25].

We consider hyperbolic (i.e. signature ) integral symmetric bilinear form (i.e. hyperbolic lattice), reflection group , fundamental polyhedron of and an acceptable (corresponding to twisting coefficients) set of vectors orthogonal to faces of (simple roots). One can construct the corresponding Lorentzian Kac–Moody Lie algebra which is graded by .

We show that has good behavior of imaginary roots, its denominator formula is defined in a natural domain and has good automorphic properties if and only if has so called restricted arithmetic type. We show that every finitely generated (i.e. is finite) algebra may be embedded to of the restricted arithmetic type. Thus, Lorentzian Kac–Moody Lie algebras of the restricted arithmetic type is a natural class to study.

Lorentzian Kac–Moody Lie algebras of the restricted arithmetic type have the best automorphic properties for the denominator function if they have a lattice Weyl vector . Lorentzian Kac–Moody Lie algebras of the restricted arithmetic type with generalized lattice Weyl vector are called elliptic (if ) or parabolic (if ). We use and extend our and Vinberg’s results on reflection groups in hyperbolic spaces to show that the sets of elliptic and parabolic Kac–Moody Lie algebras with generalized lattice Weyl vector and lattice Weyl vector are essentially finite.

We also consider connection of these results with the recent results by R.Borcherds.

## §0. Introduction

In [25], it was shown that a finitely generated symmetrizable Kac–Moody algebra has ”good behavior” of imaginary roots if and only if it is finite, affine, of rank two or of hyperbolic arithmetic type. Hyperbolic arithmetic type means that Weyl group is a group generated by reflections in a hyperbolic space of dimension with a fundamental polyhedron of finite volume. This paper is a natural continuation of these studies.

We consider a hyperbolic (i.e. of signature ) integral symmetric bilinear form (i.e. a hyperbolic lattice), a reflection group , a fundamental polyhedron of and an acceptable (corresponding to twisting coefficients) set of vectors orthogonal to faces of (simple real roots). Using these data, one can construct the corresponding Lorentzian Kac–Moody algebra which is graded by (see Sects 2.1 and 2.2).

We show that has ”good behavior” of imaginary roots, its denominator formula is defined in a natural domain and has good automorphic properties if and only if this algebra has so called restricted arithmetic type (see Sect. 2.2). This means that the semi-direct product has finite index in . Here

is the ”group of symmetries” of the fundamental polyhedron.

We show that every finitely generated (i.e. is finite) Lorentzian Kac–Moody algebra has an embedding to a Lorentzian Kac–Moody algebra of restricted arithmetic type with the same lattice (see Sect. 2.2). Thus, it is natural to study Lorentzian Kac–Moody algebras of restricted arithmetic type.

The denominator formula of a Lorentzian Kac–Moody algebra of restricted arithmetic type has the best automorphic properties if this algebra has a lattice Weyl vector (see Sect. 2.3 and also 2.4). A lattice Weyl vector is an element such that

A Lorentzian Kac–Moody algebra with a generalized lattice Weyl vector (see Definition 1.4.9) is called elliptic if it has restricted arithmetic type and ; and is called parabolic if it has restricted arithmetic type and , and there does not exist a generalized lattice Weyl vector with negative square. Ellipticity is equivalent to finiteness of the index . Parabolicity is equivalent to restricted arithmetic type and existence of such that and for any where is infinite. The corresponding lattice is called elliptic reflective and parabolic reflective respectively.

We use and extend our’s and É.B. Vinberg’s results on reflection groups in hyperbolic spaces to show that sets of primitive elliptic and parabolic reflective lattices of are finite (see Sect. 1.1). For elliptic case this was known 15 years ago. Thus, we extend this finiteness result for the parabolic case. Surprisingly, exactly the same method which was used for the elliptic case is successful for the parabolic one. This shows that these two cases are very similar, and the method which had been used for the elliptic case is very natural.

We apply the main geometrical result using to obtain finiteness results above, to show that the set of elliptic Lorentzian Kac–Moody algebras with a lattice Weyl vector is finite for . For the parabolic case, we obtain the same result if we additionally suppose that the index for a fixed constant . Here is the stabilizer subgroup of . See Sect. 1.3. Example 1.3.4 demonstrates that finiteness may not be true for the parabolic case without this additional condition.

At last, in Sect. 2.4 we consider connection of these our results with the recent results by R. Borcherds.

This paper was written during my stay at Steklov Mathematical Institute, Moscow and Mathematical Institute of Göttingen University (January—March 1995). I am grateful to these Institutes for hospitality.

I am grateful to Professor É.B. Vinberg for very useful remarks. I am grateful to Professor I. R. Shafarevich for his interest to and support of these my studies.

I plan to publish this paper in Math. Russian Izvest.

## §1. Some results on reflection groups of integral hyperbolic lattices

### 1.1. Elliptic and parabolic reflective lattices and reflection groups

One can consider this section as a complement to our papers [19], [20] and to É.B. Vinberg [31]. In [19] and [20] we used signature for a hyperbolic form and used one letter to denote the form and the space. Here we use two letters and to denote a form. Also, here we use signature for a hyperbolic form. These notations are standard for Lie algebras theory.

First, we extend results of the papers above to ”reflection groups with a cusp”.

Let

be a hyperbolic (i.e. of signature ) integral symmetric bilinear form over . Here is a free -module of a finite rank. To be shorter, we call as hyperbolic lattice. We consider the corresponding cone

choose its half-cone and consider the corresponding hyperbolic space where denote the set of positive numbers. Thus, a point of is a ray where . The distance in is defined by the formula:

With this distance the curvature is equal to . Each element with defines the half-space

which is bounded by the hyperplane

The element is defined by the half-space (respectively, by the hyperplane ) up to multiplication on elements of (respectively, on elements of of non-zero real numbers). The is called orthogonal to the half-space (respectively, to the hyperplane ).

Let be the subgroup of of the index which fixes the half-cone . It is well-known (this an easy corollary of the arithmetic groups theory) that is discrete in and has a fundamental domain of finite volume. If defines a reflection in a hyperplane of , then for with . Here

and if and only if

In particular, if is primitive in , this is equivalent to

Obviously, is the reflection in the hyperplane which is orthogonal to . The reflection changes places half-spaces and . The automorphism is called reflection of the lattice . Any subgroup of (or the corresponding discrete group of motions of ) generated by a set of reflections is called a reflection group.

We denote by the subgroup of generated by all reflections of . A lattice is called reflective if index is finite; equivalently, has a fundamental polyhedron of finite volume in . É.B. Vinberg, in particular, proved in [29] that any arithmetic reflection group in a hyperbolic space with the field of definition is a subgroup of finite index for one of reflective hyperbolic lattices . For we denote by the lattice which one gets multiplying on the form of . Obviously, is reflective if does.

Almost 15 years ago there was proved

###### Theorem 1.1.1 ([19, Appendix, Theorem 1] and [20, Theorem 5.2.1])

For a fixed , the set of reflective hyperbolic lattices is finite up to similarity and isomorphism.

The proof was based on a purely geometrical result on convex polyhedra of finite volume in hyperbolic spaces which we want to formulate.

A convex polyhedron in a hyperbolic space is an intersection

of several half-spaces orthogonal to elements with . We suppose that is locally finite in . Then the minimal set above is defined canonically up to multiplication of its elements by positive reals. Then it is called the set of vectors orthogonal to faces of (and directed outward) or shortly: orthogonal to . We always suppose that has this property. The polyhedron is called non-degenerate if it contains a non-empty open subset of . A non-degenerate polyhedron is called elliptic (equivalently, it has finite volume) if it is a convex envelope of a finite set of points in or at infinity of . Then is finite. The proof of Theorem 1.1.1 was based on the following result:

###### Theorem 1.1.1 ([19, appendix, Theorem 1])

Let be an elliptic (equivalently, of finite volume) non-degenerate convex polyhedron in hyperbolic space of . Then there are elements with the following properties:

(a) ;

(b) the Gram diagram of the elements is connected (i.e. one cannot divide the set on two non-empty subsets being orthogonal to one another).

(c) (other speaking, we have inequality if we normalize elements by the condition ), .

To prove Theorem 1.1.1, one should apply Theorem 1.1.1 to the fundamental polyhedron of and elements which belong to the lattice . See [20, Theorem 5.2.1].

Further, we name reflection subgroups of finite index and the corresponding reflective hyperbolic lattices also as elliptic reflection groups and elliptic reflective hyperbolic lattices respectively.

We want to extend results on elliptic reflection groups and elliptic reflective lattices above to the following situation.

Let be a hyperbolic lattice, and a reflection subgroup. Let be a fundamental polyhedron of . Let

be the group of symmetries of . Clearly, then the semi-direct product where is a normal subgroup in .

###### Definition Definition 1.1.2

A reflection group is parabolic if the group is infinite but it has finite index in (this means that has finite index) and there exists an element with such that for any . One can easily see (replacing by if necessary) that and (equivalently, ). We call the primitive element (or the point at infinity of ) which satisfies this condition the cusp of . One can easily see that the cusp (and the point ) is unique. A hyperbolic lattice is called parabolic reflective if contains a parabolic reflection subgroup .

We want to prove that Theorem 1.1.1 is also valid for parabolic reflective hyperbolic lattices .

###### Theorem 1.1.3

For a fixed , the set of parabolic reflective hyperbolic lattices is finite up to similarity and isomorphism.

###### Demonstration Proof

We prove an analog of Theorem 1.1.1 for appropriate ”parabolic polyhedra”. Let us fix a point at infinity of . Thus, , and .

Recall that a horosphere with the center is the set of all lines in containing the . The line , is the set . We fix a constant . Then there exists a unique such that and . Let and the corresponding elements we have defined above. Let

The horosphere with this distance is an affine Euclidean space. If one changes the constant , the distance is multiplying by a constant. The set

is a sphere in touching at the . Moreover, the sphere is orthogonal to every line at the corresponding to the point . The distance of induces an Euclidean distance in which is similar to the distance 1.1.11. The set is identified with and is also called horosphere.

Let . The set

is called the cone with the vertex and the base .

A non-degenerate locally finite polyhedron in is called parabolic (relative to the point ), if the conditions 1) and 2) below are valid:

1) is finite at the point , i.e. the set is finite.

2) For any elliptic polyhedron (i.e. is a convex envelope of a finite set of points in ), the polyhedron is elliptic.

A parabolic polyhedron is called restricted parabolic if the set

is finite.

Geometrically this means that all hyperplanes , are touching of a finite set of horospheres with the center .

###### Theorem 1.1.3

Theorem 1.1.1 is also valid for any restricted parabolic polyhedron in hyperbolic space of . Thus, there are elements with the following properties:

(a) ;

(b) the Gram diagram of the elements is connected (i.e. one cannot divide the set on two non-empty subsets orthogonal to one another).

(c) (other speaking, we have inequality if we normalize elements by the condition ), . (Unlike Theorem 1.1.1, one has a non-strict inequality to the right.)

A fundamental polyhedron of a parabolic reflection group of a hyperbolic lattice is restricted parabolic with respect to the cusp of the group , and the above statement holds for .

###### Demonstration Proof

For the proof, we normalize orthogonal vectors of a polyhedron by the condition . We normalize by this condition orthogonal vectors to all hyperplanes and half-spaces below.

To prove Theorem 1.1.1 in [19, Appendix], we had fixed a point inside the polyhedron and had used the following formula for of two elements with (see [19, Appendix, formula (2.1)]):

Here and are angular openings of cones with a vertex and bases and respectively. For non-intersecting hyperplanes and , the angle is the angular opening of the cone with the vertex which intersects the previous cones and has minimal angular opening (this is the minimal touching cone of two cones above). Here the cones and angles are taken in the sense of hyperbolic geometry. Using analytic continuation, one can obviously generalize this formula for arbitrary elements with . One should consider the plane section containing and orthogonal to the hyperplanes and , and define an appropriate orientation of all plane angles of the section.

To prove Theorem 1.1.3, we use similar formula. Assume that the polyhedron is restricted parabolic with respect to the point at infinity. For an element with , we consider the ”angle” . Then one has the following analog of the formula 1.1.15 above.

Here like above, for with and non-intersecting hyperplanes , , the element is orthogonal to the hyperplane with and , which intersects and only by infinite points and has minimal .

Here behaves like an angular opening of the cone with vertex and the base . If are consecutive vertices at infinity of a convex polygon on a plane and are orthogonal to lines and respectively, have and , and , then

Formulae 1.1.15, 1.1.16 and 1.1.17 are formulae of elementary analytic 2-dimensional hyperbolic geometry if one considers the plane which contains and is orthogonal to and .

Using the formula 1.1.1) and ”angles” , , instead of for the formula 1.1.15, the proof of Theorem 1.1.3 for a restricted parabolic polyhedron is completely the same as for Theorem 1.1.1 (see the proof of Theorem 1 in [19, Appendix]).

Let us prove the last statement. Let be a fundamental polyhedron for the action of on the horosphere . Let be the cone with the vertex and the base . Then is a fundamental polyhedron for the semidirect product which has finite index in . Then is an elliptic polyhedron. It follows that is a parabolic polyhedron with respect to . The set of hyperplanes , , of faces of the polyhedron , which are also hyperplanes of faces of the polyhedron , is finite. It follows that is finite, and is restricted parabolic.

This finishes the proof of Theorem 1.1.3.

Now Theorem 1.1.3 follows from Theorem 1.1.3 (like Theorem 1.1.1 follows from Theorem 1.1.1). See the proof of Theorem 5.2.1 in [20]. In [20] this is done over an arbitrary appropriate field. Over the proof is very easy.

This finishes the proof of Theorem 1.1.3.

È.B. Vinberg [31] has shown that for an elliptic reflective hyperbolic lattice the rank . F. Esselmann [8] improved this result and has shown that . This estimate for elliptic reflective hyperbolic lattices is exact. R. Borcherds [1] has proved that the maximal even sublattice of the odd unimodular hyperbolic lattice of the rank is elliptic reflective. Using Vinberg’s method one can prove that the rank of parabolic reflective lattices is also absolutely bounded. An easy estimate one can get is for parabolic reflective lattices . Here we use existence of the Leech lattice and two different even unimodular positive lattices of the rank . But one can expect that the exact estimate here should be . J. Conway [7] proved that the even unimodular hyperbolic lattice of the rank is parabolic reflective.

We remark that general results which bound dimension of arbitrary (not necessarily arithmetic) reflection groups of so called parabolic and hyperbolic type in hyperbolic spaces were obtained in [23] (see also [24]). They generalize results of author [20], É. B. Vinberg [31], M. N. Prokhorov [27] and A. G. Khovanskii [14] which bound dimension of elliptic (i.e. with a fundamental polyhedron of finite volume) reflection groups in hyperbolic spaces.

### 1.2. Twisting coefficients

For Kac–Moody algebras and generalized Kac–Moody algebras which we consider later, the hyperbolic lattice up to similarity , , is the invariant of the algebra or of the corresponding generalized Cartan matrix. (We will consider only indecomposable generalized Cartan matrices.) Thus, it is natural to normalize to be primitive or even primitive.

Remind that a lattice is even if is even for any . Otherwise, the lattice is called odd. The lattice is primitive (respectively, even primitive) if is not a lattice (respectively even lattice) for any natural and . Difference between these two normalizations is that if is primitive and odd, then will be primitive even. In most results below, it does not matter which of these two normalizations is chosen. Thus, below, ”primitive” (lattice) means primitive or even primitive if we don’t say exactly which normalization we choose.

Let be a primitive hyperbolic lattice. We fix a reflection group and a fundamental polyhedron of . Let be the set of primitive elements of orthogonal to faces of . Let be the group of symmetries of (see 1.1.10).

###### Definition Definition 1.2.1

A function is called twisting coefficients function if

and the subgroup

has finite index in . Number is called a twisting coefficient of the .

Clearly, we can reformulate this definition as follows:

Let us consider a new set of orthogonal vectors to :

This set is called acceptable if

and the subgroup

has finite index in . Obviously, 1.2.1 and 1.2.1 are equivalent to 1.2.2 and 1.2.2. Thus, an acceptable sets is equivalent to a twisting coefficient function. If is acceptable, then

is a twisting coefficient function.

We need to estimate , for , and for where is acceptable.

###### Proposition 1.2.2

Let be the exponent of the discriminant group . Let , and be the maximal natural number such that . Then , and the satisfies 1.2.1 if and only if . In particular, , and for any acceptable and one has .

###### Demonstration Proof

This is trivial.

###### Remark Remark 1.2.3

Let be an acceptable set above. We denote

We consider elements where is a countable set. Then the Gram matrix of elements defines a generalized Kac–Moody algebra (see [2]) with the set of simple real roots , set of imaginary simple roots , and the Weyl group . One can consider (respectively, ) as an ”extended root sublattice generated by real simple roots” (respectively, ”extended root lattice” (generated by real and imaginary simple roots)) modulo kernel of the canonical symmetric bilinear form. Thus, Proposition 1.2.2 describes all possibilities for the part of this matrix connected with real simple roots.

### 1.3. Elliptic and Parabolic reflection groups with the lattice Weyl vector

We fix a primitive even elliptic or parabolic reflective lattice and elliptic or parabolic reflection group . Let be a fundamental polyhedron of and an acceptable set of orthogonal vectors to . To be shorter, we name the pair elliptic or parabolic pair respectively.

###### Definition Definition 1.3.1

An element is called lattice Weyl vector of if

Evidently, the Weyl vector if does exist. Evidently, and is invariant with respect to the automorphism group . It follows that if is elliptic, and and if is parabolic where is the cusp. Since generates (by Theorems 1.1.1 and 1.1.3), the Weyl vector is evidently unique for the fixed subset .

We want to show that the set of elliptic or parabolic pairs with the lattice Weyl vector is essentially finite up to isomorphism. Here the pair is isomorphic to a pair if their exists an isometry of lattices such that .

###### Theorem 1.3.2

For , the set of elliptic pairs with a lattice Weyl vector is finite up to isomorphism. Here is a primitive even elliptic reflective lattice, an elliptic (i.e. of finite index in ) reflection subgroup, a fundamental polyhedron of , and an acceptable set of all orthogonal vectors to .

###### Demonstration Proof

We fix one of primitive even elliptic reflective hyperbolic lattices of the (we already know that their set is finite). Let be an acceptable pair with a lattice Weyl vector.

By Theorem 1.1.1 and Proposition 1.2.2, there are elements such that they generate a sublattice of finite index and Gram matrix of these elements has bounded integral coefficients. Thus, there exists only a finite set of possibilities for these Gram matrices.

Let us fix one of possibilities above for the Gram matrix . We have: where the embedding is defined by the Gram matrix above. Thus, there exists only a finite set of possibilities for the intermediate lattice .

Let us fix one of possibilities above for . Since the lattice is non-degenerate, there exists the unique element such that satisfies Definition 1.3.1 for the subset .

By Proposition 1.2.2, is bounded for . It follows that the set

is finite. It follows that we have only finite set of possibilities for the subset .

It follows Theorem.

For the parabolic case, the finiteness result will be the following.

###### Theorem 1.3.3

For any parabolic pair with a lattice Weyl vector the group has finite index in the Euclidean crystallographic group .

We fix a constant . Then for , the set of parabolic pairs with a lattice Weyl vector is finite up to isomorphism if index . Here is a primitive even parabolic reflective lattice, a parabolic reflection subgroup with the cusp , a fundamental polyhedron of , and an acceptable set of orthogonal vectors to .

###### Demonstration Proof

Let us prove the first statement.

Let be the parabolic pair corresponding to a parabolic reflection group with a fundamental polyhedron and an acceptable set of vectors orthogonal to , and with a Weyl vector where is the cusp of .

Since , we have and has finite index in . Let be a fundamental domain of on the horosphere where , and is the cone with the vertex and the base . Then is a fundamental domain of finite volume for in . Since is Weyl vector, then for any . Thus, there does not exist a face of which contains the cusp ; on the other hand, belongs to . It follows that the fundamental domain on the horosphere has finite volume. Since is discrete in , it follows that has finite index in . This proves the first statement.

Let us prove second statement. We are arguing like for the proof of Theorem 1.3.2 (using Theorems 1.1.3 and 1.1.3 instead of 1.1.1 and 1.1.1). But at the end of the proof we should use that number of subgroups is finite if index , and replace the set 1.3.2 by the set

Here (like above) is a fundamental domain for on the horosphere , , and is the hyperplane which is orthogonal to .

This finishes the proof.

The next example demonstrates that the condition for index is essential in Theorem 1.3.3.

###### \examplename Example 1.3.4

Let us consider an even primitive hyperbolic lattice of the rank where

Here where is a fundamental triangle with zero angles (i.e, it has vertices at infinity) of the reflection group generated by reflections in all elements such that . This is one of examples of -reflective lattices of the rank which were described in [18].

Let . Here where . We have , and .

Denote

One can easily check that

for . Here , and .

Let us consider such that

The is the parallel translation on the horosphere which sends the line to the line and the triangle to the triangle which has the same vertex and the same side with the triangle .

For , we define an infinite fundamental polygon for a reflection subgroup with the acceptable set of vectors as follows:

Clearly, is an infinite polygon with zero angles. It follows that is a fundamental polygon for a subgroup generated by reflections in all elements of . By our construction, generates a subgroup of finite index in . Since 1.3.4, the set is acceptable and is the Weyl vector for . It follows that is a parabolic reflection group with the cusp .

Obviously, all pairs are different because elements have square or , and exactly consecutive sides of have orthogonal vectors from with the square .

### 1.4. Reflection groups of arithmetic type

We consider hyperbolic lattices and reflection groups . Let be a fundamental polyhedron of and an acceptable set of orthogonal vectors to . We want to define a class of these groups which is interesting from the view-point of Kac–Moody algebras. From the point of view of corresponding Kac–Moody algebras, the next definition means that imaginary roots ”behave very nice” (see Sect. 2.2 below).

###### Definition Definition 1.4.1

Consider an integral cone (semi-group)

and the corresponding integral dual cone

The group has arithmetic type if

Equivalently, this means that for any