I think there are a few other edge cases. Siri has its own volume controls while you are using siri. I think phone calls have their own volume too. And while you are in the camera volume buttons become 'take photo' buttons.
In Siri and a phone call, you're still adjusting the speaker volume. The current setting appears to be independent of random apps that make noise, but it's still speaker volume vs. ringer volume.
