I have Bose noise canceling headphones, and my experience with them is that they do a really good job of blocking conversational noise if you use them with music.
Same experience here. The downside is situations where people end up standing behind or near me to get my attention. Have been startled many times. Still, for an open office they seem a must have.
Noise cancelling headphones can cancel noise without any music playing.
It's not 100%, but they'll get rid of like 80% of the background noise without playing anything. To get that last 20% yeah you need something playing to help mask it(even at an extremely low volume).
Sometimes I wear mine with nothing playing while at work just to get rid of random office chatter.