Comparing median household income to median house prices is probably 80% of what you need to explain the housing market, and this article does a pretty good job. The essence is that you can't sustainably spend more on a house than you make, so there is no reason to expect the historical ratio to change more.
The only way you can reasonably expect that ratio to change in a dramatic way the way it did is if there was some underlying structural change in how people live. For example, a massive migration to dense cities where you're not spending hundreds a month for a car means you can, spend a larger proportion on housing, pushing that ratio up. But of course, we know that hasn't happened and will take decades if it ever does.
The only way you can reasonably expect that ratio to change in a dramatic way the way it did is if there was some underlying structural change in how people live. For example, a massive migration to dense cities where you're not spending hundreds a month for a car means you can, spend a larger proportion on housing, pushing that ratio up. But of course, we know that hasn't happened and will take decades if it ever does.