Fundamentals of Probability and Statistics - Midterm 1

A collection of data categorized by each class of data. For a data matrix, each row (an observation) can be called a case, observational unit or experimental unit.

All variables can be categorized into two categories: numerical and categorical.

Numerical

This type of data can either be continuous or discrete. It can take a wide range of numerical values.

Categorical

If some data can be put into different (usually finite) groups, it will be categorical.

Ordinal

Ordinal variables are those that are categorical, but have a numerical order.

When we are trying to find a relationship between two variables, we need to identify which is the explanatory and which is the response variable. We usually assume that the explanatory affects the response variable.

\[ \text{explanatory} \to \text{ might affect } \to \text{response} \]

In statistics, there are two main ways we might collect data. We use observational studies or experiments.

Anecdotal Evidence

Evidence based on a personal sample, usually disregards the entire population.

Census

Sampling the entire population. Conducting a census is not very convenient. It is expensive, complex, takes long and it might be unrepresentative because the population can change.

Simple Random Sampling

To conduct an SRS we need to randomly select cases from the population.

Stratified Random Sampling

This sample consists of strata (a group made up of similar observations). We take a random sample from each stratum

Custer Random Sampling

The population is slit into clusters, which are then randomly selected (all the members of that selected cluster are included). This type of sampling can be very cheap.

When we take a small sample of a population, we are conducting an exploratory analysis. If based on the result of that sample we generalize to the population, we call that inference. Whether or not the sample we took is representative, depends on how we took that sample. If we want to ensure our sample is representative, we need to take it under random conditions.

Non-response

Occurs when a part of the sampled population does not answer for various reasons.

Voulentary Response

The sample consists of people who choose to respond because of a strong opinion.

Convenience Sample

Easy access to certain group of people will affect the sample

It is always better to have a large sample.

Population \(N\)

A well defined collection of objects.

Sample \(n\)

A sample is some observed subset of a population. Usually selected in a prescribed manner.

Univariate Data

A set of observations of a single variable.

Discrete Variable

A numerical variable with a finite number of possible values.

Continuous Variable

Another numerical variable consisting of all the values in a range.

This plot enables us to easily and quickly understand a dataset.

With a scatter plot, we get have clear view of all the data for two numerical variables.

A dot plot can only visualize uni-variate data.

A dot plot with a twist. In this dot plot, we do not just plot data on a single horizontal line. Instead, with each next piece of data we stack the data points.

It is the perfect alternative to a dot plot, when it comes to larger datasets. Instead of representing each observation with a dot, the histogram makes use of bins.

The box part of the box plot contains 50% of the data. Inside the box there is a thick line, which is exactly at the median of the distribution. The outer lines of the box are the first and third quartile. The whiskers can extend up to \(1.5*IQR\) away from the quartiles, but not always.

\begin{array}{l} \text{max upper whisker reach} = Q3 + 1.5 * IQR \\ \text{max lower whisker reach} = Q1 - 1.5 * IQR \\ \end{array}

Any outlier will always be past the maximum reach of each whisker.

A percentile exactly indicates the position of a value in a distribution. The location of a percentile can be computed with:

\[ \frac{P}{100} * (n+1) \]

A quartile also gives us an idea about the location of some value in the distribution. It divides our data into four equal parts. It allows us to easily understand larger datasets.

Q1

The first quartile is also the 25th percentile

Q2

The second quartile is also the 50th percentile

Q3

The third quartile is also the 75th percentile

With the mean, also called the average we can figure out the center of a data distribution. There are two kinds of means.

When calculating the median, the data is split in half. We can find the location of the median with \((\frac{n+1}{2})^{th}\). If the size of the data set is even, the median is the average of the two middle values.

The median is also the 50th percentile, and the second quartile.

Since the sample mean can at times be a bad representation of the distribution, we often resort to the median. To not lose the benefits of either, we can compute the trimmed mean. A trimmed mean is calculated by eliminating the smallest and largest \(n%\) of the samples.

• A 5% to 25% trimmed mean will give us the perfect range.

When a distribution looks to have a longer tail on one side rather than the other, it is said to have a certain skew. There are two ways a distribution can be skewed.

Right/Positive Skew

When a distribution has a long right tail.

Left/Negative Skew

When a distribution has a long left tail.

We can also identify the skew of a distribution by looking at its mean and median. The mean will be closer to the tail.

Right Skew

mean < median

Left Skew

mean > median

There are 4 main properties when it comes to distribution shapes.

Uni-modal

Single prominent peak

Bimodal

Two prominent peaks

Multimodal

Multiple prominent peaks

Uniform

No apparent peaks

The prominent peak of a distribution can also be called the mode. It is the value with the most procurances in a dataset.

The frequency in statistics is equal to the number of times certain variable occurs in a dataset.

This statistics describes the variability of a dataset. We start by comparing all the observations to the mean (this is called the deviation), we then square all these distances (to get a positive value) and take an average. The result of this calculation is the sample variance.

\[ s^2 = \frac{\sum_{i=1}^n {(x_i - \bar{x})^2}}{n - 1} \]

There is an alternate way to express this:

\[ s_xx = \sum(x_1 - \bar{x})^2 = \sum x_i^2 - \frac{(\sum x_i)^2}{n} \]

The proof of this is in 5.

Just like the sample variance, this statistics describes the variability of a distribution.

\[ s = \sqrt{s^2} \]

What makes the standard deviation special, is that *all the data is within 3 standard deviations relative to the mean.

It is roughly the average squared deviation from the population mean, denoted by \(\sigma^2\). The differentiating factor from the Sample Variance is that *it uses the entire population \(N\).

\[ \sigma^2 = \frac{\sum_{i=1}^n {(x_i - \bar{\mu})^2}}{n - 1} \]

Just like the Sample Standard Deviation, it is the square root of the population variance.

\[ \sigma = \sqrt{\sigma^2} \]

All other properties remain the same.

The range, primarily measures the spread of a distribution, but it can also be interpreted as the simplest measure of variation.

\[ \text{Range} = X_{max} - X_{min} \]

Disadvantage

It only considers the two most extreme observations.

As stated before, outliers are pieces of data which our outside the range of the \(Q1/Q3 \pm 1.5*IQR\) scope.

• Importance of outliers
  • Identifying extreme skew
  • Identifying collection/entry errors
  • Providing insight into data features

• If we have a skewed distribution, it is better to use the median and IQR to describe the center and spread.
• However, for a symmetric distribution it is more helpful to use the mean and standard deviation. Here we use the mean because in symmetric distributions \(\text{median} \approx \text{mean}\).

For any population. The percentage of data within the interval \(\mu \pm k\sigma\) is at least \(100(1 - \frac{1}{k^2})\) percent.

The empirical rule heavily relies on the Chebychev's Theorem, for measuring percentages of a distribution. For any symmetric distribution, we can apply the following rules:

• 68% of the data falls under \(\mu \pm 1 \sigma\)
• 95% of the data falls under \(\mu \pm 2 \sigma\)
• 99.7% of the data falls under \(\mu \pm 3 \sigma\)

Z-scores are one of the most popular metrics for comparing a certain data point to the rest of the distribution. There are two types of z-scores, a population z-score and a sample z-score. The only difference is that they use the population and sample mean respectively, same applies for the standard deviations.

\[ z = \frac{x - \bar{x}/\mu}{s/\sigma} \]

Correlation is a measure of how associated two variables are. The most commonly used type of correlation is the Pearson Correlation.

When thinking about correlation and association, we must remember that association does not imply causation.

With categorical data, it is useful to construct a tabular frequency distribution. The general formula for a sample proportion is as follows:

\[ p = {x \over n} \]

Given categorical data and their frequencies we can construct a contingency table.

When testing a hypothesis, we need two. One being the null hypothesis (\(H_0\)) and the other, alternative hypothesis (\(H_a\)).

Null Hypothesis

There is nothing going on

Alternate Hypothesis

There is something going on.

When testing a hypothesis, we make a discussion as to how unlikely the unlikely outcome is. The entire process of testing a hypothesis is based on the assumption that the null hypothesis is true.

The way we denote the sample space of an experiment is by \(S\). This is the set of all possible outcomes of that experiment.

An event is a set which contains certain outcomes contained the sample space \(S\). If an event only has one outcome, it is called simple, otherwise we call it compound. If the outcome of an experiment is contained in the event \(A\), we can say that the event \(A\) has occurred.

When conducting an experiment, we use probability to precisely measure the change that some event \(A\) will occur. All probability should satisfy the following axioms.

• For any event \(A\), \(P(A) \ge 0\)
• \(P(S) = 1\)
• If \(A_1, A_2 A_4, \dots\) is an infinite collection of disjoint events, then \(P(A_1 \cup A_2 \cup A_3 \cup \cdots) = \sum_{i=1}^\infty P(A_1)\)

• \(P(A) + P(A^\prime) = 1\)
• \(P(A) \le 1\)

quite a mouth-full

If we have some compound event \(A\), which consists of \(n\) events \(E_i\), we can solve for \(P(A)\) using the following rule.

\[ P(A) = \sum_{E_i \in A} P(E_i) \]

If we can find a way to define all events \(E\), we can solve for \(P(A)\) in terms of some constant.

If we have some sort of experiment in which the outcome of some event is equally as likely as that of another, we say that we have equally likely outcomes. This means that for each event, the probability is \(p = {1 \over N}\) where \(N\) is the number of events. The simplest example of this is a coin toss or a die roll.

If we have an event which consists of ordered pairs of objects and we want to know the number of pair there are, we can use the below mention proposition. What we mean by an ordered pair, is a set of two elements, where the order of the elements matters.

If the first element of an ordered pair can be selected in \(n\) ways, and for each of these \(n\) ways the second element of the pair can be selected in \(m\) ways, then the number of pair is \(n*m\).

It can be very useful to build something called a tree diagram. With this diagram, we can visualize all possibilities.

When looking at the graph from left to right, each line connecting two nodes, is called an n-th generation branch. With each new branch, \(n\) increases.

With the tree diagram, we can easily calculate givens and conjunctions.

A permutation is an ordered combination. Just like in combinations, we either can or cannot repeat elements.

A combination is a subset, that has no order. For this subset, we can consider two states where we can repeat elements or cannot repeat elements.

The probability of an event might not always be straight-forward, and we may need to find ways to figure it out. With conditional probability, we can find how some event \(A\) might affect \(B\). We use the notation \(P(A\vert B)\) to represent probability of A given that B has occured. The definition of conditional probability for two events is:

\[ P(A\vert B) = \frac{P(A \cap B)}{P(B)} \]

Multiplication rule for \(P(A \cap B)\)

This rule is very important for solving certain problems where we know \(P(A\vert B)\) and \(P(B)\). With these two and the above formula for conditional probability, we can know that \(P(A\cap B) = P(A\vert B)* P(B)\).

This theorem allows us to compute the probability of some event, based on knowing other conditional probabilities concerning that event. In this theorem, it is assumed that two events \(A\) and \(B\) are independent. The formula is as follows:

\[ P(A\vert B) = \frac{P(B\vert A) * P(A)}{P(B)} \]

You can further expand this formula using the Law of Total Probability.

\[ P(A\vert B) = \frac{P(B\vert A) * P(A)}{P(B\vert A) * P(A) + P(B \vert \neg A) * P(\neg A)} \]

In a lot of previous concepts, we have assumed independence. Independence occurs, if

\[ P(A \vert B) = P(A) \]

Another way we can check for independence is with the multiplication rule. Which states that

\[ P(A \cap B) = P(A) * P(B) \]

We can also use this the other way around, if we know that two events are independent, we can find the probability of their conjunction by multiplication. This concept can be extended to any number of events.