## |

Any closed, simple, piecewise curve
defines a Hamiltonian flow, the *billiard flow* inside the
*billiard table* *T* [29, 32, 31, 16].
The *billiard map* of the table *T* is the Poincaré map
defined by the standard cross-section of the billiard flow.
The study of the billiard map for a convex table *T* goes back to
Birkhoff [2] and the theory is still far from being completed.
In this work we assume that *T* is convex and and use
the standard facts about the *phase space* of the billiard map
.

Let now be a closed convex curve with perimeter
. By varying the string length
we obtain all billiard tables with the caustic
. The corresponding billiard maps
form a one-parameter family of *twist maps* which is determined by .
Our investigatigation of this family
emphasis on the relationship between the
geometric shape of and the dynamics of the family
of billiard maps.

Let be the invariant circle of
corresponding to . Due to the time-reversal symmetry,
there are two invariant circles in associated with .
We make a choice of by requiring that the
*rotation number* of is less than or equal to
1/2. In the special case when is an interval,
the tables
are the ellipses with foci and .
The invariant circle is then the lower of the two invariant
circles in with the rotation number 1/2 (see Figure 3).

The invariant circle moves monotonically upward in the phase space as runs from to infinity (Figure 4).

We state now the main results of our work. There are two
theorems. Our first result
is that there is a Baire generic set of caustics
for which the rotation function
is a devil's staircase. More precisely,
has the property that for any rational *p*/*q*, the interval
has a nonempty interior.
Some geometrical properties of imply that
has nonempty interior. We apply this to show that
contains all polygons and curves with a flat point.

In the second theorem, we
study then the motion of Birkhoff periodic orbits of a fixed rotation
number in the phase space .
Let be the interval of the set of parameters
for which there exists a Birkhoff periodic orbit on the curve .
For , all Birkhoff periodic orbits with rotation number *p*/*q* are
above . For , they are below .
Global index arguments imply that for
the curve separates the set
of Birkhoff periodic orbits into two nonempty subsets. We prove
that for near a Birkhoff periodic set
of index -1 approaches from above. This set continues as
a set of index +1 below for near .
At , a bifurcation occurs and two Birkhoff periodic sets of
index -1 are created on . The picture for near
is obtained by an obvious symmetry from the case at :
a Birkhoff periodic set of index +1 approaches from above the curve
, collides with two Birkhoff periodic sets on
and leaves as a Birkhoff periodic set of
index -1. (see Figures 12,13,14).

The theme of relations between the shape of the billiard table and the
billiard dynamics is well reflected in the literature. Here, we mention
only a few papers which are especially close to our direction.

A problem which is closely related to our work is how to decide
whether a specific billiard table has an invariant circle or not.
We mention the theorem of Mather [19] about the global
absence of invariant circles in the case when the billiard table has a
flat point, the result of Hubacher [13] about the absence of
invariant circles near the table if the curvature of the table has a
discontinuity. An other related result is the theorem of Gruber
[6, 7] which tells that generically in the Hausdorff topology
on the set of convex tables, a billiard table has no caustics.
On the other hand, if the table is smooth enough and the curvature
is strictly positive, Lazutkin's result [17]
assures in this case the existence of infinitely many caustics near
the boundary of the table.
The inverse problem of finding billiard tables with a special
invariant circle or caustics is interesting both
from the geometrical and from the dynamical point of view.
Bialy [1] showed that if the phase space
of the billiard map is foliated by a continuous family of homotopically
non-trivial invariant circles, then the billiard table is a disc.
There are billiard tables with flat invariant circles different from
the equator [8]. The size of
the area which is free of caustics can be estimated from the shape of
the table [9].
For the billiard in an ellipse, there are caustics for any rotation number
and the table can be obtained from any of them
by the string construction. Elliptical billiards are believed
to be special. According to a conjecture attributed to Birkhoff
and stated by Poritsky [26], they are the only integrable billiards
in the sense that a subset of full Lebesgue measure of the phase space is
foliated by invariant circles. This Birkhoff-Poritsky problem is still open.

The paper is organized as follows. In section 2, we establish notation and collect preliminary results. In section 3, we investigate the rotation function and show that it is Baire generically a devil's staircase. If there exists some for which every orbit on is periodic, a caustic is called exceptional. We will show in section 4 that this property is quite restrictive. While tables with exceptional caustics with rotation number 1/3 have been constructed in [14] and his construction probably generalizes to any possible rational number, it is not known whether a table exists which has two exceptional invariant caustics. In section 5, we investigate the passage of the Birkhoff periodic orbits through the invariant circle as varies from to .

** Next:** Preliminaries
**Up:** Billiards that share a
** Previous:** Billiards that share a

Wed Jul 8 11:57:32 CDT 1998