last weekend I vibe-coded a project called `openfs` that plugs into just-bash

https://github.com/jeffchuber/just-bash-openfs

it puts a bash interface in front of s3, filesystem (real and in-memory), postgres, and chroma.

still very much alpha - but curious what people think about it.

see an example app here: https://github.com/jeffchuber/openfs-incident-app

I did a slightly less ambitious prototype a few weeks ago where I created added lazy loading of GCS files into the just-bash file-systems, as well as lots of other on-demand files. Was a lot of fun.

yeah (optional) caching is interesting to think about - incl write_through and write_back

What problem were you trying to solve ? ( not that you need to solve one. I’m just curious )

try out chroma or better yet as opus to!

that was me swyx

Multiple people have coined the idea repeatedly, way before you. The oldest comment on HN I could find was in December 2022 by user spawarotti: https://news.ycombinator.com/item?id=33856172

Here is an even older comment chain about it from 2020: https://news.ycombinator.com/item?id=23895706

Apparently, comparing low-background steel to pre-LLM text is a rather obvious analogy.

As well as that people often do think alike.

If you have a thought, it's likely it's not new.

Oh wow, great find! That’s really early days.

i didnt claim to invent it.

i claimed swyx heard it through me - which he did

you did!!

i support this

Good article - the most use cases i see of pg_vector are typically “chat over their technical docs” - small corpus - doesn’t change often / can rebuild the index - no multi-tenancy avoids much of the issues with post-filtering

Chroma implements SPANN and SPFresh (to avoid the limitations of HNSW), pre-filtering, hybrid search, and has a 100% usage-based tier (many bills are around $1 per month).

Chroma is also apache 2.0 - fully open source.

congrats to factory on the amazing product and release!

chroma stores both

As does Azure's AI search.

Hey there

Chroma is fully OSS - embedded, single-node and distributed (data and control plane). afaik lance distributed is not OSS.

We do have plans to release the crate (enabling embedded chroma in rust) - but haven't gotten around to it yet. Hopefully soon!

> Do you see all providers converging on similar alpha i.e cheap object storage, nvme drives,ssd cache to solve this?

It's not only a new pattern in search workloads, but it's happening in streaming, KV, OLTP, OLAP, etc. Yea - it's the future.

> Supabase/pgVector needs lots of resources when adding new rows to the index -> wish the resources scale up/down automatically. Instead of having to monitor and switch to the next plan.

Many ways potentially - but one way is Chroma makes all this pain go away.

We're also working on some ingestion tooling that will make it so you don't have to scale, manage or run those pipelines.

I'll for sure take a deeper look. Ingestion has been by far the biggest pain and least fun. Those infra parts hold us back from the cool things -> building agents/search

very fair!

cloud has been in private beta for a year now.

we chose to not release it to the public until we were extremely confident in the system and its characteristics.

databases are a serious business. developers trust us with their mission critical data.