It strains credulity to believe that Intel wasn't aware that they were trading side-channel resistance for performance. The problems are just too deep and pervasive. None of AMD, ARM, Power, or SPARC came close to the number and severity of issues in Intel chips. There were problems in those chips, but their nature and limited scope shows that everybody had a rough idea about how far they could go before they made privilege separation worthless from a confidentiality perspective. Yes, some went a little too far, but it seems clear that Intel just said, "f-it", and stood on the gas pedal.
Hyperthreading/SMT is a trickier issue because it had obvious and even proven side-channel potential from the beginning. But 1) everybody had to hold their nose in order to compete with Intel on SMT performance, and 2) technically the operating system communities should have made the effort to keep unrelated processes from sharing an SMT'd core. And that still needs to happen--we need smarter schedulers.
That doesn't look to me like "everybody had a rough idea about how far they could go."
It is really easy for me to believe that a ton of designers could add optimizations without consideration of side channels. Nobody appreciated the vulnerabilities that speculation introduced.
(And keep in mind Intel has probably 90+% market share in the search for exploitable behavior.)
> The problems are just too deep and pervasive
One could also say that it strains credulity that the entire community failed to realize the existence of these vulnerabilities that are so fundamental to speculation, and yet here we are - that's exactly what happened.
Not all those named side-channel exploits are the same in terms of severity and difficulty to mitigate, nor are the chips vulnerable in the same way.
For example, Meltdown exposed severe negligence in Intel's design. For ARM Meltdown was limited to values of a single register, for which there's no reason to believe it was anything other than an unintentional bug--i.e. you don't get any substantial performance benefits from permitting speculation through that single register, though it perhaps simplified some other aspect of the chip.
Basically, if you go down the line Intel's issues were both more severe and pervasive, as-if they just didn't care about preventing speculation across privilege domains.
Notwithstanding the ARM's Meltdown mistake, both ARM and AMD very clearly had designs that attempted to prevent speculation across privilege domains. And they mostly succeed. The major issues are at syscalls where intra-privilege (not cross-privilege) speculation can indirectly be exploited by unprivileged callers. But like with SMT, it was always sort of understood that it was the operating system's responsibility here; there really are no good hardware mitigations.
Basically, the exploits for AMD and ARM (notwithstanding the lone register issue) are intrinsic to speculative execution, period. And everybody sort of understood this, especially in the cryptographic community with work on constant-time algorithms. It's just that everybody was too lazy to take it seriously more generally until Meltdown/Spectre lit a fire under everybody's pants. And once they began to pay attention, it immediately became clear that Intel's designs made patently and grossly unsafe design choices.
The details on IBM Power chips are spartan. I think their Meltdown issue was similar to ARM--a bug with a register--but I can't confirm that. My impression is that Power pushed the envelope more heavily than AMD and ARM, but not like Intel. Power went all-in on SMT, though, and though SMT is fundamentally anathema to cross-privilege confidentiality, Intel's and IBM's SMT implementations seem to leak more than AMD's.
Hyperthreading/SMT is a trickier issue because it had obvious and even proven side-channel potential from the beginning. But 1) everybody had to hold their nose in order to compete with Intel on SMT performance, and 2) technically the operating system communities should have made the effort to keep unrelated processes from sharing an SMT'd core. And that still needs to happen--we need smarter schedulers.