Economies of scale.

If alternatives get more expensive more people use rail, and the cost per rail rider drops.

Historically US passenger service was secondary to express mail service. Without express mail service provided by the same trains, passenger service became unprofitable.

The timeline roughly matches up with that argument, but mail and people transitioned for similar reasons.

Or it just means nobody travels.

People will travel one way or another. They'll just prioritize the factors that are important to them.

Or the whole market shrinks because demand is fairly elastic.

To some degree.

At least in the US, people will tend to drive--perhaps shorter distances--if long distance travel gets too expensive.