Matt Hauger:

You’d think that muting my iPhone via the side-toggle would prevent the device’s speaker from making noise. Only it doesn’t. This mute switch applies only to the phone’s ringer and app notifications. Even when muted, the phone plays back media at full volume.

I always end up trying that ninja-like move where you swipe the volume slider to zero before any noise plays.