Copyright © Cambridge University Press 2015. We study Landau damping in the 1+1D Vlasov-Poisson system using a Fourier-Hermite spectral representation. We describe the propagation of free energy in Fourier-Hermite phase space using forwards and backwards propagating Hermite modes recently developed for gyrokinetic theory. We derive a free energy equation that relates the change in the electric field to the net Hermite flux out of the zeroth Hermite mode. In linear Landau damping, decay in the electric field corresponds to forward propagating Hermite modes; in nonlinear damping, the initial decay is followed by a growth phase characterized by the generation of backwards propagating Hermite modes by the nonlinear term. The free energy content of the backwards propagating modes increases exponentially until balancing that of the forward propagating modes. Thereafter there is no systematic net Hermite flux, so the electric field cannot decay and the nonlinearity effectively suppresses Landau damping. These simulations are performed using the fully-spectral 5D gyrokinetics code SpectroGK, modified to solve the 1+1D Vlasov-Poisson system. This captures Landau damping via Hou-Li filtering in velocity space. Therefore the code is applicable even in regimes where phase mixing and filamentation are dominant.