Correlation

Correlation and Dependence [6]

In statistics, dependence or association is any statistical relationship, whether causal or not, between two random variables or bivariate data. Correlation is any of a broad class of statistical relationships involving dependence, though in common usage it most often refers to the extent to which two variables have a linear relationship with each other. Familiar examples of dependent phenomena include the correlation between the physical statures of parents and their offspring, and the correlation between the demand for a product and its price.

Correlations are useful because they can indicate a predictive relationship that can be exploited in practice. For example, an electrical utility may produce less power on a mild day based on the correlation between electricity demand and weather. In this example there is a causal relationship, because extreme weather causes people to use more electricity for heating or cooling. However, in general, the presence of a correlation is not sufficient to infer the presence of a causal relationship (i.e., correlation does not imply causation).

Correlation Coefficient [2]

A correlation coefficient is a number that quantifies a type of correlation and dependence, meaning statistical relationships between two or more values in fundamental statistics.

Pearson Correlation Coefficient [5]

In statistics, the Pearson correlation coefficient (PCC, pronounced /ˈpɪərsən/), also referred to as the Pearson's r or Pearson product-moment correlation coefficient (PPMCC), is a measure of the linear correlation between two variables $X$ and $Y$. It has a value between $+1$ and $−1$, where $1$ is total positive linear correlation, $0$ is no linear correlation, and $−1$ is total negative linear correlation. It is widely used in the sciences. It was developed by Karl Pearson from a related idea introduced by Francis Galton in the 1880s.

Time-Series Correlation [4]

The first step is to find a way of measuring how similar two time series are. There are countless way of doing this, depending on the underlying assumptions of your data. The most used one for those applications is called correlation. The correlation between two functions (or time series) is a measure of how similarly they behave. It can be expressed as:

$corr(X,Y) = \frac{cov(X,Y)}{std(X)std(Y)}$

with $std\left(X\right)$ and $mean\left(X\right)$ being the standard deviation and the mean of $X$, respectively:

$std(X) = \sqrt[]{ \frac{1}{N}\displaystyle \sum_{i=1}^N [X_i - mean(X)^2] }$

$mean(X) = \frac{1}{N}\displaystyle \sum_{i=1}^NX_i$

The mean is simply the average of the whole time series. The standard deviation, instead, indicates how much the points of the series tends distance themselves from the mean. This quantity is often associated with variance, defined as:

$var(X) = std(X)^2$

When the variance is zero, all the points in the series are equal to the mean. A high variance indicates that the points are scattered around.

The term $cov(X,Y)$ represents the covariance between $X$ and $Y$, which generalises the concept of variance over two time series instead of one. The covariance provides a measure of how much two time series change together. It does not necessarily account for how similar they are, but for how similarly they behave. More in detail, it captures whether the two time series increase and decrease at the same time.

The covariance is calculated as follow:

$cov(X,Y) = \frac{1}{N}\displaystyle \sum_{i=1}^N[X_i - mean(X)][Y_i = mean(Y)]$

and is easy to see that indeed $cov(X,X) = var(X)$.

Looking back to the definition of correlation, it is now easy to understand what is trying to capture. It’s a measure of how similarly $X$ and $Y$ behave, normalised by their variance to obtain a value between $-1$ and $+1$. When both time series tend to increase (or decrease) over time with a similar fashion, they will be positively correlated. Conversely, if one goes up and the other goes down, they will be negatively correlated.

Autocorrelation Function [4]

The idea behind the concept of autocorrelation is to calculate the correlation coefficient of a time series with itself, shifted in time. If the data has a periodicity, the correlation coefficient will be higher when those two periods resonate with each other.

The first step is to define an operator to shift a time series in time, causing a delay of $t$. This is known as the lag operator:

$lag(X_i, t) = X_i-t$

The autocorrelation of a time series with lag $t$ is defined as:

$autocorr(X,t) = \frac{cov[X,lag(X,t)]}{std[X]std[lag(X,t)]} = \frac{cov[X,lag(X,t)]}{var(X)} = \frac{\displaystyle \sum_{i=1}^N[X_i - mean(X)][X_{i-t} - mean(X)]}{\displaystyle \sum_{i=1}^N[X_i - mean(X)]^2}$

The Correlogram [4]

Autocorrelation is a relatively robust technique, which doesn’t come with strong assumptions on how the data has been created. If in the previous post we have used a synthetic sales data, this time we can confidently use real analytics:

This is the plot for the autocorrelation function, also known as correlogram:

Example

$mean(X) = \frac{1}{N}\displaystyle \sum_{i=1}^NX_i$ = $\frac{(1 + 2 + 3 + 4 + 5)}{5}$ = $3$

Calculate the Variance - Steps

1. Calculate the mean of the sample.
2. Subtract the mean from each data point.
3. Square each result.
4. Find the sum of the squared values.
5. Divide by $n - 1$, where $n$ is the number of data points.

Calculate the Autocorrelation

$autocorr(X,t) = \frac{cov[X,lag(X,t)]}{std[X]std[lag(X,t)]} = \frac{cov[X,lag(X,t)]}{var(X)} = \frac{\displaystyle \sum_{i=1}^N[X_i - mean(X)][X_{i-t} - mean(X)]}{\displaystyle \sum_{i=1}^N[X_i - mean(X)]^2}$

$t = 1$

Numerator

Denominator

$\frac{4}{10} = 0.4$

Verify in R

Code

data <- c(1,2,3,4,5)
summary(data)
acfTs <- acf(data, na.action = na.pass)
acfTs

Output

> data <- c(1,2,3,4,5)
> summary(data)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
      1       2       3       3       4       5 
> acfTs <- acf(data, na.action = na.pass)
> acfTs

Autocorrelations of series ‘data’, by lag

   0    1    2    3    4 
 1.0  0.4 -0.1 -0.4 -0.4 
>