> Deep Learning makes feature engineering obsolete
I think even if you avoid constructing features, you are basically doing a similar process where a single change in a hyper-parameter can have significant effects:
- internal structure of a model (what types of blocks are you using and how do you connect them, what are they capable of together, how do gradients propagate?)
- loss function (great results come only if you use a fitting loss function)
- image/data augmentation (self-driving car won't work without significant augmentation at all)
- properly set-up optimizer
The good thing here is that you can automate optimization of these to a large extent if you have a cluster of machines and a way to orchestrate meta-optimization of slightly changed models. With feature engineering you just have to do all the work upfront, thinking what might be important, and often you just miss important parts of features :-(
Yep, and in doing so, you just traded "feature engineering" for graph design and data prep, no? And that's my response to these sorts. And their usual response to me is to grumble that I don't know what I'm doing. I've started tuning them out of my existence because they seem to have nothing to contribute to it.
It's a huge difference in terms of the time invested to create something that performs well. Hand crafted feature engineering is better for some tasks but for quite a few of them automated methods perform very well indeed (at least, better than I expected).
I think even if you avoid constructing features, you are basically doing a similar process where a single change in a hyper-parameter can have significant effects:
- internal structure of a model (what types of blocks are you using and how do you connect them, what are they capable of together, how do gradients propagate?)
- loss function (great results come only if you use a fitting loss function)
- category weights (i.e. improving under-represented classes)
- image/data augmentation (self-driving car won't work without significant augmentation at all)
- properly set-up optimizer
The good thing here is that you can automate optimization of these to a large extent if you have a cluster of machines and a way to orchestrate meta-optimization of slightly changed models. With feature engineering you just have to do all the work upfront, thinking what might be important, and often you just miss important parts of features :-(