SHAP is a recent de-facto standard way to get feature importance out of any model, whether you care about how much each column in your dataset contributes to a given decision or say which pixels in an image mattered the most.

https://github.com/slundberg/shap