The 3-cycle latency refers to simple NEON ALU instructions (VADD.Ix, VORR, VAND, etc). Scalar ALU instructions are still single-cycle.
Note that these numbers are from Cortex-A15 and Krait which are expected to be similar to Swift, but I didn't measure Swift itself to know for sure.