Mixed-anion compounds currently attract great attention, and there are recent indications that they may also be of interest for thermoelectric applications. In this context, the lattice thermal conductivities of mixed-anion Ba4Sb2Se and Ba4Sb2Te are investigated using density functional theory and Boltzmann transport theory. We observe a 24% increase in the lattice thermal conductivity at room temperature when the atomic mass increases from Se to Te, which is counterintuitive given that lighter atoms typically result in higher phonon group velocities and lower phonon scattering rates. This anomalous behavior is attributed to a specific weak Ba–Se bond in Ba4Sb2Se as compared to the corresponding Ba–Te bond in Ba4Sb2Te, which generates numerous low-frequency optical phonons with low group velocities and enhances the phonon scattering. These findings provide avenues to customize the lattice thermal conductivity without the usual reliance on heavy atoms.