It's also a misunderstood product from the outside.
Unfortunately banks had a tendency to pay a bunch of senior engineers to do a build out for 3 years and then turn it over to cheap nearshore consultants to operate for 10 years. The core of most big bank code you see was written before 2010.
Where I have seen bakeoffs and largely failed attempts to replace, the challenge is the number of vendor & open solutions you need to cobble together to replace all the use cases that KDB supports. Every year it was a slightly different combination that didn't really cover all the bases.
To replace KDB in many of these large use cases, you need - a fast columnar PB-scale database, an efficient on-disk store, a fast in-memory cache, SQL support plus with expressive query language additions for time series operations, an event bus, a stream processor, and a programming language to write more complex applications close to the data.
A lot of the newer cloud-centric offerings are more about massive parallelism/scale than about pure single thread/individual request-response performance.
This is great for the vast majority of CRUD apps with millions/billions of users.
It's even great for some financial "backtesting" type use cases where you can kick off large
It generally isn't sufficient for many of the real-time financial use cases of doing ad-hoc analytics on billions/day datasets with expected responses in ms.
The biggest complaints are its expensive, and the devs are expensive.
On the other hand you usually don't need as much hardware or dev staff to support it.
> a fast columnar PB-scale database, an efficient on-disk store, a fast in-memory cache, SQL support plus with expressive query language additions for time series operations
Sounds like ClickHouse.
And we see kdb slowly start being replaced...
Hi Steve. See my reply in above thread to Shin which calls out alot of the operational improvements we've made. Just an FYI, there's also been significant changes in how we price so it's now alot easier to start small, get started quickly and decrease the time to value with more mainstream skillsets. With the new product strategy we're seeing alot more buy side clients (hedge funds/asset managers) use kdb+/q for their workloads with KX Insights.
Where I have seen bakeoffs and largely failed attempts to replace, the challenge is the number of vendor & open solutions you need to cobble together to replace all the use cases that KDB supports. Every year it was a slightly different combination that didn't really cover all the bases.
To replace KDB in many of these large use cases, you need - a fast columnar PB-scale database, an efficient on-disk store, a fast in-memory cache, SQL support plus with expressive query language additions for time series operations, an event bus, a stream processor, and a programming language to write more complex applications close to the data.
A lot of the newer cloud-centric offerings are more about massive parallelism/scale than about pure single thread/individual request-response performance.
This is great for the vast majority of CRUD apps with millions/billions of users.
It's even great for some financial "backtesting" type use cases where you can kick off large
It generally isn't sufficient for many of the real-time financial use cases of doing ad-hoc analytics on billions/day datasets with expected responses in ms.
The biggest complaints are its expensive, and the devs are expensive. On the other hand you usually don't need as much hardware or dev staff to support it.