The black box forecasting “platforms” baffle me. AWS has been pushing a forecasting module lately. They push the exact same thing to big corp and small start ups. Then they just push “forecasting” without putting it into a context. As this article describes there are different contexts and different outputs for forecasting. Platforms are great but at least until we have AI we still need data scientists to take the right steps.
Those platforms are selling a quick solution to this:
Lay CEO: We need to tell clients we have [Buzzword].
CTO: That's a buzzword. It doesn't exist, doesn't work, and/or doesn't make sense for our product. We could just use [well-understood technology, useful thing].
CEO: Implement [Buzzword] first. If it doesn't work well, do whatever you want under the hood. I just need to tell investors and clients we have [Buzzword].
> Platforms are great but at least until we have AI we still need data scientists to take the right steps.
Yes, but with some caveats.
You’re going to want to hire someone with technical chops to run the system, and interpret the results, but the days of second guessing the machine are over.
Rob Hyndman has long argued that automated time-series forecasting beats human judgement (before ML was even a thing). And even where humans could beat automated systems, they don’t scale.
My fear is these systems promise great results for execs who build models to answer new questions by themselves. And that's just irresponsible.
I do forecasting in my job, and use Hyndman's `forecast` R package. I totally rely on it's automated model creation.
That said, I only have the moderate confidence in its results because I took the time to research forecasting and started with an understanding of how to use data to answer questions. (To be clear, any lack of confidence in the results is from a fear I'm doing it wrong).
Non-data people are terrible at choosing the right data to answer the question. Sure, automating the model terms or even the choice of model is probably done better by machine, with human tweaking only needed for rare cases. But domain experts will often choose totally wrong numbers for models to predict.
If the target's obvious (e.g., quarterly revenue they already have a method for calculating), then letting execs do it themselves is probably fine. But I'd never trust them to take an abstract question, choose an appropriate measure to answer it, and then find good data without consulting a skilled analyst.
"Strategic forecasts drive high stakes decisions at longer horizons, so they should not be approached simply as a black box forecasting service, divorced from decision-making."
In summary, the author seems to argue that there is should be an inverse relationship between the time to prediction and how complicated the desired model should be.
I'm waiting for Google to productize Alpha Zero (or whatever its descendant is called nowadays) as a "C-suite as a service" thing for directors to play with.
