Key word being "on linux". On the other 99.9% of devices, I have no such choice and the filesystem required by the OS simply presents a layer of incompatibility. Modern OSs all run on essentially the same hardware (same CPU architecture even!) but store the same data in different, incompatible ways. I want the filesystem to join the list of things that are no longer OS-specific.

I want my storage hardware to have a higher level API than "block device". I don't want to format a drive in some OS-specific format, I want a built-in format that I can access from anything. I want this format to be durable, well specced so that I can easily access historical data. I want embedded devices to support richer storage functionality than afforded by FAT32+.

Consider that before S3, we stored files on servers in bespoke ways where we had to worry about permissions and filesystem differences (path too long!) and encryption and durability and a hundred things that we now... don't. Instead we have a simple API with dozens of conforming backend providers, including the ability to do it yourself. Imagine if computer storage experienced a similar revolution?

But S3 is so unified because has heavy limitations -- only basic file ops, no consistency guarantees. It actually has less capability than FAT32 in some ways (For example, no attributes, no way to say "create file but do not overwrite it")

Sure, someone can make super-limited drive which only does 3 basic ops (read, "upsert", list). Would it be useful for general purpose computing? I doubt it.

> It actually has less capability than FAT32 in some ways

I'm not sure I'd agree with that:

- "no attribute": S3 does actually have quite sophisticated ACLs and permissions that can be set at the object level

- "no way to say "create file but do not overwrite it": there are a few ways you can do that. Versioning, bucket policies, etc.

In fact I disagree with both yourself and the GP with your assumption that S3 is simple. If you want to do anything "enterprisey" with S3 then you absolutely do need to be aware of the maximum size of allowed objects (these days it's 5TB but it wasn't so long ago when it was only around 5GB). Then there's bandwidth costs (which means you need to also take into account VPC traffic and thus S3 endpoints) and ACLs (soooo many different places you can set permissions on objects - it's easy for someone untrained to get totally lost). There's different storage options in S3 too - depending on the frequency you want to request the data.

S3 is only superficially simple - but then the same can be said for any file system. If you just want somewhere to dump files then any fs will work. But when you start needing to performance tune and do all the other considerations that the GP highlighted with local server storage, then S3 quickly becomes as complicated as every other option out there.

ZFS, exFAT, and UFS are all supported on at least NT, Darwin, Linux, and I believe the major BSDs. I will happily grant that only exFAT is likely to work for embedded devices, but you do have real options.