That was my experience, but I am glad that I tried again. Player of Games is much more explicit in the world building (almost too much), and then a good balance is found in Use of Weapons. So I always recommend reading those two first before circling back to really understand the value of Consider Phlebas as an outside view on the Culture (and really, The West).