Given the number of people who need the compute but are only accessing it via APIs like HuggingFace's transformers library, which supports these chips, I don't really think that CUDA support is absolutely essential.
Most kernels are super quick to rewrite, and higher level abstractions like PyTorch and JAX make dealing with CUDA a pretty rare experience for most people making use of large clusters and small installs. And if you have the money to build a big cluster, you can probably also hire the engineers to port your framework to the right AMD library.
The world has changed a lot!
The bigger challenge is that if you are starting up, why in the world would you give yourself the additional challenge of going off the beaten path? Its not just CUDA but the whole infrastructure of clusters and networking that really gives NVIDIA an edge, in addition to knowing that they are going to stick around in the market, whereas AMD might leave it tomorrow.
When buying a supercomputer, you negotiate support contracts so it doesn't matter if AMD leaves the day after they sign the contract, you've still got your supercomputer and support for it.
I agree they need to work on their software but I also think that the availability as well as massive expense of the H100, AMD can undercut Nvidia and build a developer ecosystem if they wanted to. I think they need to hit the consumer market pretty hard and get all the local llama people hacking up the software and drivers to make things work. A cheaper large VRAM consumer card would go a long ways to getting a developer ecosystem behind them.
