The problem is as follows. I give you a sequence of coin flips, for example TTTHHTTT, and your task is to determine whether this coin is fair. Obviously you can't answer this question with certainty.
The frequentist approach seems rather ridiculous to me. They pretend to be able to answer this question without knowing anything about coins. Instead of making the assumptions up front, they hide the assumptions in the method of determining whether the coin is fair. For example one would think that to decide whether a coin I give you is fair, you'd need some idea about what kind of coins I will give you. If we were talking about reality, you would be able to say "no it's not fair" without even looking at the data sequence, because in reality there are no fair coins. If we are in a different universe where fair coins do exist, you'd need some idea how many are fair. So obviously whether a coin is fair depends on which universe you live in, but the frequentist method is not parameterized by the universe. The assumptions are implicitly made inside the method.
The bayesian approach asks you first to state your prior ideas about coins. Then you give the bayesian your data, and he will compute for you the probability that this coin is fair. This cleanly separates the correct mathematical derivation from the subjective assumptions, instead of hiding these assumptions inside the method.
Moreover, the bayesian approach is automatic, in principle. Once you make your assumptions clear, the rest is just mechanical derivation. The frequentist approach requires divine inspiration, and is very easy to get wrong. For example you're not allowed to look at the data before formulating your hypothesis. Of course nobody does this right in practice. I've often seen people cherry pick a frequentist statistical test that proves their hypothesis. Given any data and any hypothesis, I can devise a correct frequentist statistical test that proves the hypothesis with arbitrarily large confidence.
Would that the world were about flipping coins. What's your prior distribution on the volatility parameter of a model of the 3-month implied volatility of the S&P 500 stock index? Now, justify it. What is your prior distribution on the rate at which bugs are found in a software system that runs a nuclear plant? Now, justify it. Personally, my priors are usually crap.
I agree. But with frequentist methods it's not even clear what kind of assumptions you're making. Explicitly stating your assumptions is better than implicitly hiding them in your method.
"We can see that the bayesian approach won't work well here, so lets use frequentist methods because we cannot see that they won't work well."
The problem with the nuclear plant is that we don't have much data on nuclear plant software bugs. However you could still get an idea by using bug rates in other software as your priors, and acknowledge that your priors are not perfect.
OR you could use frequentist methods to make up for the lack of data, and know you will be able to devise your method in such a way that it proves your hypothesis. Or if you use the method correctly (that is don't look at your data, and pick a statistical test beforehand) make no useful predictions at all.
"But with frequentist methods it's not even clear what kind of assumptions you're making."
I don't understand what you mean by this.
For a simple, concrete example: take the problem of fitting a distribution to a sample of real random variables. It seems that a Frequentist would make the following assumptions:
1. The data comes iid from some unknown but fixed distribution.
2. This true distribution is included within some set of distributions: eg, the set of normal distributions with real mean and non-negative real variance.
They would then use some estimator to estimate the parameters. The choice of estimator is perhaps justified by some theoretical properties, eg, consistency etc. This gives another assumption.
3. The best estimator to use is the one which is optimal with regard to properties x,y,z.
If it is not clear what assumptions I am making and I have explicitly stated some, presumably there are others which I have left unstated. Could you explain why you think the above collection is insufficient?
The problem is as follows. I give you a sequence of coin flips, for example TTTHHTTT, and your task is to determine whether this coin is fair. Obviously you can't answer this question with certainty.
The frequentist approach seems rather ridiculous to me. They pretend to be able to answer this question without knowing anything about coins. Instead of making the assumptions up front, they hide the assumptions in the method of determining whether the coin is fair. For example one would think that to decide whether a coin I give you is fair, you'd need some idea about what kind of coins I will give you. If we were talking about reality, you would be able to say "no it's not fair" without even looking at the data sequence, because in reality there are no fair coins. If we are in a different universe where fair coins do exist, you'd need some idea how many are fair. So obviously whether a coin is fair depends on which universe you live in, but the frequentist method is not parameterized by the universe. The assumptions are implicitly made inside the method.
The bayesian approach asks you first to state your prior ideas about coins. Then you give the bayesian your data, and he will compute for you the probability that this coin is fair. This cleanly separates the correct mathematical derivation from the subjective assumptions, instead of hiding these assumptions inside the method.
Moreover, the bayesian approach is automatic, in principle. Once you make your assumptions clear, the rest is just mechanical derivation. The frequentist approach requires divine inspiration, and is very easy to get wrong. For example you're not allowed to look at the data before formulating your hypothesis. Of course nobody does this right in practice. I've often seen people cherry pick a frequentist statistical test that proves their hypothesis. Given any data and any hypothesis, I can devise a correct frequentist statistical test that proves the hypothesis with arbitrarily large confidence.