The "infinitely many steps" does sound fishy to me, because I wonder whether the number of axioms in True Arithmetic is countably infinite or uncountably so, and whether it makes a difference (I think it does).
But yes, the comparison is an analogy, certainly not correct at every level.
The set of all statements is easily seen to be countable, so the subset of statements being true, being infinite, is countable as well.
It does not make any difference, though, since what matters here is, as I said, recursive enumerability of axioms. Anyway, transfinite induction lets one use "pick next element" arguments even on uncountable sets.
But yes, the comparison is an analogy, certainly not correct at every level.