You're essentially correct, so I +1'd you. I don't think speed reading only works if the text is "worthless", but rather if the text is intuitively friendly and fairly straightforward information. Stories fit this criteria, news reports, lots of blogs, comments, etc. Math books require you to think about abstract concepts and information has to be digested, you can't simply get it through reading over something once like you can with stories.
Very true, but other techniques from the school of thought to which speed reading belongs do still apply to math texts, I think.
For instance, many people, if they start getting hung up on some difficult material in a math text will slow down (an ok thing to do), then if they have more trouble, they slow down more (maybe also ok). But at some point as progress approaches zero (several moves that seemed ok in isolation accumulate), and the reward/effort falls below that needed to have the reader keep reading the book, some other activity always occurs as a better use of time (the accumulation of ok moves has defeated the entire enterprise). Sometimes ending a book that way is probably ok, but surely it can't always be for the best (unless pretty much every book is organized so as to have monotonically increasing difficulty and monotonically decreasing worth-it-ness to the reader). So the attitude of not treating the book as sacrosanct is helpful here, if the material gets tough and there is any threat of motivation dropping, just skip to the beginning of the next section, summary, or chapter as the case may be. Later material may illuminate the previous difficult material, or maybe something else later on will be worth learning even without the stuff that got skipped (if one removed a random ingredient from every sandwich you ate, they will still have nutritional value).
Speed reading is just one of a number of methods all built on a presupposition, that between the realms of "worth reading start to finish carefully" and "not worth reading" there is a landscape of books that can be worth spending different amounts of time on. So then the issue becomes, for book type x, and time one is willing to spend trying t, what are the heuristics available for getting a positive return out of that book.
This has reminded me to order a copy of _How to Read a Book_. It has been years since I gave that book a two hour reading, and it still shapes how I think about allocating time for a book like making a bet and choosing strategies based on the time and book type.
I think if you want to read like that, you might as well skip the book and pull a summary from Wikipedia. Although, it does discuss two things I've often wondered about: can one read without subvocal narration, and could one read more effectively using something like that Spreeder app.
It's always been concentration/focus that's slowed my reading though. My mind wanders very easily.
