Fundamental Concepts Archives

The Variance: Measuring Dispersion

Posted on December 8, 2018 Written by The Cthaeh 2 Comments

A few posts ago I introduced you to the “three M’s” of statistics — the concepts of mean, mode, and median. Today I want to talk to you about a related concept called variance.

While the three M’s measure the central tendency of a collection of numbers, the variance measures their dispersion. That is, it measures how different the numbers are from each other.

Measuring dispersion is another fundamental topic in statistics and probability theory. On the one hand, it tells you how much you can trust the central tendency measures as good representatives of the collection. High variance usually means a lot of the numbers in the collection will be far away from those measures.

[Read more…]

Intuitive Explanation of Expected Value

Posted on November 24, 2018 Written by The Cthaeh 20 Comments

Expected value is another central concept in probability theory. It is a measure of the “long-term average” of a random variable (random process). I know this doesn’t sound too clear, but in this post I’m going to explain exactly what it means.

There are many areas in which expected value is applied and it’s difficult to give a comprehensive list. It is used in a variety of calculations by natural scientists, data scientists, statisticians, investors, economists, financial institutions, and professional gamblers, to name just a few.

[Read more…]

The Law of Large Numbers: Intuitive Introduction

Posted on November 13, 2018 Written by The Cthaeh 17 Comments

The law of large numbers is one of the most important theorems in probability theory. It states that, as a probabilistic process is repeated a large number of times, the relative frequencies of its possible outcomes will get closer and closer to their respective probabilities.

For example, flipping a regular coin many times results in approximately 50% heads and 50% tails frequency, since the probabilities of those outcomes are both 0.5.

The law of large numbers demonstrates and proves the fundamental relationship between the concepts of probability and frequency. In a way, it provides the bridge between probability theory and the real world.

[Read more…]

The Mean, the Mode, And the Median

Posted on October 1, 2018 Written by The Cthaeh 2 Comments

The concepts of mean, median, and mode are fundamental to statistics, probability theory, and anything related to data analysis as a whole. Being this important, they deserve their own introduction.

In statistics, these 3 concepts are examples of measures of central tendency. This is a fancy way of saying that they are single values that summarize collections of values.

Let’s unpack the last sentence. What exactly do I mean by ‘values’ and ‘collections of values’?

[Read more…]

Frequentist and Bayesian Approaches in Statistics

Posted on June 16, 2016 Written by The Cthaeh 53 Comments

What is statistics about?

Well, imagine you obtained some data from a particular collection of things. It could be the heights of individuals within a group of people, the weights of cats in a clowder, the number of petals in a bouquet of flowers, and so on.

Such collections are called samples and you can use the obtained data in two ways. The most straightforward thing you can do is give a detailed description of the sample. For example, you can calculate some of its useful properties:

The average of the sample
The spread of the sample (how much individual data points differ from each other), also known as its variance
The number or percentage of individuals who score above or below some constant (for example, the number of people whose height is above 180 cm)
Etc.

You only use these quantities to summarize the sample. And the discipline that deals with such calculations is descriptive statistics.

But what if you wanted to learn something more general than just the properties of the sample? What if you wanted to find a pattern that doesn’t just hold for this particular sample, but also for the population from which you took the sample? The branch of statistics that deals with such generalizations is inferential statistics and is the main focus of this post.

The two general “philosophies” in inferential statistics are frequentist inference and Bayesian inference. I’m going to highlight the main differences between them — in the types of questions they formulate, as well as in the way they go about answering them.

But first, let’s start with a brief introduction to inferential statistics.

[Read more…]

Calculating Compound Event Probabilities

Posted on April 29, 2016 Written by The Cthaeh 5 Comments

You can think of probabilities as measures of uncertainty in the occurrence of an event, the truth of a hypothesis, and so on.

These measures are numbers between 0 and 1. Zero means the event is impossible to occur and 1 means the event is certain to occur.

If you have many events of interest, you can measure their probabilities separately, but you can also measure probabilities of different combinations of these events.

Say you’re following the national soccer championships of England, Spain, and Italy. You want to calculate the probabilities of Arsenal, Barcelona, and Juventus becoming national champions next season. These probabilities are:

P(Arsenal)
P(Barcelona)
P(Juventus)

But what if you want to calculate the probability of both Arsenal and Barcelona becoming champions? Or the probability that at least one of the three teams does?

In this post, I’m going to show how probabilities of such combinations of events are calculated. I’m going to give the general formulas, as well as the intuition behind them. To do that, I’m first going to introduce a few relevant concepts from probability theory.

[Read more…]