Frequentist vs Bayesian
After a month-long gap, we're back on track.
This week we're trying to understand the difference between the bayesian and the frequentist perspective.
This is in no way an expert statistical commentary but just a naive and brief summarization of the topic.
What's the Conflict?
Well if you're not aware there are two perspectives on the concept of probability and how it is to be used in statistics namely Frequentism and Bayesianism.
It has been a longstanding debate that which of the two is a better approach for decision-making.
There have been and still are several advocates for both sides although the frequentist approach has had an edge and dominated statistics since the 20th century due to the objective notion that it provides.
Understanding Each
So what are these two perspectives exactly? The most significant difference is in between how each interprets "probability".
As per the Frequentist, the probability of an event is the frequency of the outcome when the process is repeated multiple times.
While for a Bayesian, the probability is the measure of the degree of belief of a certain outcome.
Like for a frequentist, the probability of an even number on a dice roll would be roughly equal to the 1/2 since when the experiment repeated multiple times, it would result in roughly those numbers.
And when in an election if you believe that there's a 60% chance that your favorite candidate would win is a bayesian approach as it is not an experiment that you can repeat over and over again but you have some beliefs which led you to believe that number.
- - -
The frequentist is more concerned about getting the right answer while a bayesian cares about what opinion you should have about some event and for them there is no right or wrong.
Frequentists do not assign probabilities to the events that can not be repeated just like 60% probability of our candidate winning the election is not a valid use case to apply probability as the event can not be repeated. But in the Bayesian approach, you can assign a probability to any kind of event whatsoever, even if it is neither random nor repeatable.
The Indicators
If you're using Bayes Formula, dealing with prior and posterior probablities, you're working from a bayesian perspective.
MLE or Maximum likelihood estimation is a frequentist's way of estimating probabilities. Confidence intervals and p-values capture the uncertainty in the frequentist methodology.
- - -
Random Variables
A frequentist considers the dataset as a random variable and the parameters that we are measuring as fixed while a bayesian considers the dataset to be fixed and the parameter we are measuring as random variable.
Like if you're calculating the average weight of all the wrestlers there ever have been given normal distribution and standard deviation, a frequentist would consider that there's a fix answer to that and would collect sample of data to get an estimate of the mean.
A bayesian would rather assign the mean weight a probability distribution based on all the possible values.
- - -
Almost everything that a frequentist can do can also be done by a bayesian perspective but the reverse situation is not true.
Bayesians using prior probability is one of their biggest criticism as it's assignment can be very subjective but it is also what Bayesians consider a missing aspect in the frequentist philosophy.
So What Should You Be?
Either be both or be none! The point is not to become exclusive to one approach all the time. Each approach has its pros and cons and which one is better depends on what type of problem you're trying to solve and what kind of decision-making you prefer.
This brief discussion is in no way provides complete coverage of the points regarding both the approaches. There is much more to it and can be argued indefinitely which one is better but understanding both is equally important!
---
