Abstract
We study an unconventional quantum Hall effect for the surface states of ultrathin Floquet topological insulators in a perpendicular magnetic field. The resulting band structure is modified by photon dressing and the topological property is governed by the low-energy dynamics of a single surface. An exchange of symmetric and antisymmetric surface states occurs by reversing the light's polarization. We find a novel quantum Hall state in which the zeroth Landau level undergoes a phase transition from a trivial insulator state, with Hall conductivity σyx=0 at zero Fermi energy, to a Hall insulator state with σyx=e2/2h. These findings open new possibilities for experimentally realizing nontrivial quantum states and unusual quantum Hall plateaus at (±1/2,±3/2,±5/2,...)e2/h.