I've used it for calculating Value at Risk on a portfolio consisting of thousands of books and millions of positions.
With the books spread over a distributed, in-memory data grid of 20 or so nodes, the system sent a command object to each node to do the calculations in parallel in the same process context as the position data. The data was partitioned so that all positions for each book were in the same node. When the calculations were complete, the reduce process consolidated the results.
VaR requires storing interim results to roll the calculation up the hierarchy of books, but that was straightforward with this approach.
Any time you can parallelize the calculations, map-reduce is worth considering.
Incidentally, the core idea (like all good software ideas) was present in Lisp decades before Google popularized it. The words map and reduce are even in the language.
With the books spread over a distributed, in-memory data grid of 20 or so nodes, the system sent a command object to each node to do the calculations in parallel in the same process context as the position data. The data was partitioned so that all positions for each book were in the same node. When the calculations were complete, the reduce process consolidated the results.
VaR requires storing interim results to roll the calculation up the hierarchy of books, but that was straightforward with this approach.
Any time you can parallelize the calculations, map-reduce is worth considering.
Incidentally, the core idea (like all good software ideas) was present in Lisp decades before Google popularized it. The words map and reduce are even in the language.