The compute supply/demand for mining is designed in the bitcoin algorithm to oscillate, and the mining game is about being able to forecast a complex combo of BTCUSD price, power price, hardware price and depreciation.
Remarks like the title of this clickbait article are strictly meaningless, they assume the instantaneous price of bitcoin / power / hardware is what's used to compute profitability, when in practice mining is basically a futures market.
I've played with a bunch of RISC-V platforms, mostly SBCs in the raspi class
Beyond the potential platform fragmentation due to the variability of the ISA (a very unfortunate design choice IMO), mentioned elsewhere in this thread, what I find most frustrating is the boot process / equivalent of BIOS in that world.
My impression: complete lack of standardization, a ton of ad-hoc tools native to each vendor, a complete mess, especially when it comes to get the board to boot from devices the vendor didn't target (eg SSDs).
Until two things happen:
1. a CPU with a somewhat competitive compute power appears (so far, all the SBC's I've tried are way behind ARM and x86)
2. a unified BOOT environment which supports a broad standard of devices to boot from (SSD, network, SD-Card, hard-drives, etc...)
the whole RISC-V thing will remain a tiny niche thing, especially because when a vendor loses interest in the platform, all of the SW that is native to the platform goes to rot immediately (not that it was particularly good quality in the first place).
There are several RISC-V Linux distros where essentially all the software available for the x86-64 platform is also available on the RISC-V edition. Let’s use Ubuntu as an example.
> when a vendor loses interest in the platform
> the platform goes to rot immediately
Ubuntu will provide updates for 15 years. That does not seem very immediate.
For RVA23 hardware, I expect even new Ubuntu releases to support it up to around 2030 at least. 15 years from then will be 2045. I cannot say that I am picking up what you are laying down here.
> 2. a unified BOOT environment which supports a broad standard of devices to boot from (SSD, network, SD-Card, hard-drives, etc...)
I got the same experience tinkering with ARM devices. It soured me so much that I have decided that until ARM offers a unified boot mechanism like x86 PCs do, I will ignore it, no matter the supposed benefits.
I think OpenAI and Anthropic just downloaded the same torrents from Anna's Archive that anyone else can. But it's only OK when they do it. The rest of us get nastygrams from law offices. Anthropic actually had to cough up some bucks, for that matter.
At that point, a lot depends on the quality of the preprocessing applied to the raw text dumps. It is reportedly not that trivial to go from DumpOfSketchyRussianPirateSite.zip to a data set suitable for ingestion during pretraining. A few bad chunks of data can apparently do more harm than one would expect.
AFAIK Google scans almost everything in print as part of the Google Books initiative, so they may have been able to skip the torrenting step.
> I think OpenAI and Anthropic just downloaded the same torrents from Anna's Archive
I think you're underestimating how much chat conversation data they've gathered at this point, and how much of it is part of the training set.
None of that is available to anyone who wants to train a frontier model.
And when it comes to Google ... the hoard of data they're sitting on goes back to what? 1998? They've basically got a digital record of what happened since the birth of the internet.
reply