Stability of nonlinear normal modes of symmetric Hamiltonian systems

A nonlinear normal mode of a Hamiltonian system is a periodic solution near equilibrium with period close to that of a periodic trajectory of the linearised vector field. This paper is a sequel to the preceding paper in this issue, in which the authors developed methods for proving the existence of nonlinear normal modes of a Hamiltonian system that is invariant under the action of a compact Lie group. Here they show how to compute the spectral stability of these nonlinear normal modes. The methods are a natural extension of those used in the previous paper, namely, Birkhoff normal form and singularity theory. In particular they prove that under certain nondegeneracy conditions, Floquet exponents computed from a truncated Hamiltonian in normal form determine the spectral stability for the corresponding solutions of the full Hamiltonian. Applications include systems with O(2) and Dn symmetry, systems with five degrees of freedom and O(3) symmetry (modelling some vibrational modes of a liquid drop), and resonant time-reversible systems with Z2 symmetry (such as the planar spring pendulum). They give explicit reductions to Birkhoff normal form for Hamiltonians of the form 'kinetic+potential' where the potential is invariant under O(2) or D n. The analysis of systems with O(3) symmetry uses the dynamical invariance of fixed-point spaces of subgroups to simplify the computations considerably.