My current take on this is that one should use a P99 (or P99.9, etc) median and anything above that is considered an anomaly.

You then are not assuming anything about the distribution.

I'm curious if others think this is a bad/good approach.