Spotify's size and consequences of not being on the Play Store are exactly why Google would do this.

Google gives no shits if random app developer isn't on Play. And so holds firm on their cut.

Play without Spotify makes a lot of users ask "Why isn't Spotify on Play?" And probably follow-up with "How do I get Spotify? Download another app store? Okay, I'll do that."

So essentially, the money that Google ceded to Spotify was approximately what they thought discouraging users from installing alternate app stores was worth.