Independence

• Cross-sectional data: independent across individuals

• A random draw from a distribution

• Slightly weaker version: independently but non-identically distributed (i.n.i.d.)

• Multiplicative rule: two events \(A\) and \(B\) are statistically independent iff \(P(AB) = P(A)P(B)\).

Dependence

• Natural chronic ordering
• \((z_1,\ldots,z_T)\) is a random draw from population of an infinite time series \((\ldots, Z_{-1}, Z_0, Z_1, \ldots)\)
• Dependence of nearby units, for example \((Z_1, Z_2)\)

Stationarity

• Time series is a single realization,
• … but statistical analysis requires repeated patterns
• Stationarity says patterns are independent of the absolute index
• Two forms: weakly stationary and strongly stationary
• Without a qualifier, “stationary” refers to strongly stationary by default

Weak stationarity

For all \(t\in \mathbb{N}\):

• Mean: \(\mu = E[Z_t]\)

• Variance: \(\gamma(0) = var[Z_t]\)

• Autocovariance: \(\gamma(l) = cov(Z_t, Z_{t+l})\), \(l\neq 0\)

• Autocorrelation: \(\rho(l)= cov(Z_t, Z_{t+l})/\gamma(0) = \gamma(l)/\gamma(0)\)

In words, the first two moments are independent of the absolute time index \(t\).

• White noise: \(E[Z_t]=0\), \(E[Z_t^2] = \sigma^2\), and \(E[Z_t Z_{s}] = 0\) for all \(t\neq s\).

Strong stationarity

• For every finite collection of indexes \((t_1, t_2,\ldots, t_m)\) and \(h\in \mathbb{N}\), the joint distribution of \((Z_{t_1}, Z_{t_2},\ldots,Z_{t_m})\) is the same as \((Z_{t_1+h}, Z_{t_2+h},\ldots,Z_{t_m+h})\).
• “Strong” implies “weak” if the first two moments exist
• “Weak” is more intuitive
• “Strong” simplifies asymptotic theory

Weakly dependent

• For all \(t\), as \(l \to \infty\) the variables \(Z_t\) and \(Z_{t+l}\) are statistically independent

• Various formal definitions (beyond the scope of this course, for example, ergodicity.)

• Asymptotically uncorrelated: \(corr(Z_t, Z_{t+h}) \to 0\) as \(h\to \infty\).

• All the above definitions are introduced to give LLN and CLT.

• A typical large sample result requires strong stationarity and weak dependence.

Example

A time series \(\{Z_t\}\) is generated as the following.

• \(Z_t = V + U_t\)
• \((V, U_1, \ldots, U_T)\) are mutually independent.
• \(V \sim N(0,1)\) and \(U_t \sim N(0,1)\) for all \(t\).

Questions:

• Is \(\{Z_t\}\) weakly stationary?
• Is \(\{Z_t\}\) strongly stationary?
• Is \(\{Z_t\}\) weakly dependent?

Consistency of OLS

A1. Linear model. \(\{(\mathbf{x_t}, y_t)\}_{t=1}^T\) is stationary and weakly dependent so that LLN and CLT hold.

A2. No perfect collinearity

A3. \(E[\epsilon_t | \mathbf{x_t} ] = 0\) (contemporaneous exogeneity)

Theorem: Under A1-A3, the OLS estimator \(\hat{\beta} \stackrel{p}{\to} \beta_0\).

Examples

• Static model: \(y_t = \beta_1 x_{1t} + \beta_2 x_{2t} + \beta_3 + \epsilon_t\).
  • \(E[\epsilon_t | x_{1t}, x_{2t}]=0\) requires no omitted variable and no functional misspecification.
  • Correlation between \(\epsilon_{t-1}\) (eg. inflation shock) and \(x_{1t}\) (eg. monetary policy) is allowed.
• Finite distributed lag model: \(y_t = \beta_1 x_t + \beta_2 x_{t-1} + \beta_3 x_{t-2} + \beta_4 + \epsilon_t\).
  • Start with \(E[ \epsilon_t | x_t, x_{t-1}, x_{t-2},\ldots] = 0.\) It implies \(E[ \epsilon_t | x_t, x_{t-1}, x_{t-2}] = 0.\)
• Autoregression of order 1 (AR(1)): \(y_t = \beta_1 y_{t-1} + \beta_2 + \epsilon_t\), with \(|\beta_1 | < 1\).
  • Start with \(E[\epsilon_t | y_{t-1}, y_{t-2},\ldots] = 0\). It implies \(E[y_t | y_{t-1}, y_{t-2},\ldots] = \beta_1 y_{t-1} + \beta_2\).
  • Conditioning on \(y_{t-1}\), the information of \(y_{t-2}, y_{t-3},\ldots\) is redundant (Markov property).
  • Strict exogeneity must be violated in AR(1). OLS is not unbiased.
  • Severe bias when \(\beta_1\) is close to 1.

Asymptotic normality of OLS

A4. \(var[\epsilon_t |\mathbf{x}_t ] = \sigma^2\) for all \(t\) (contemporaneously homoskedastic)

A5. \(cov(\epsilon_t, \epsilon_s | \mathbf{x}_t, \mathbf{x}_s) = 0\) for all \(t\neq s\) (No serial correlation)

Theorem: Under A1-A5, the OLS estimator is asymptotically normal.

• Gauss-Markov theorem is associated with the classical assumptions. BLUE is no longer true under A1-A5 here.