Almost Sure Uniqueness of Limits

I saw an interesting probability question:

Show that $\overline{X}_n \to \mu$ a.s. if $\sqrt{n}(\overline{X}_n - \mu)\to Z$ in distribution.

This one-sentence problem needs to be broken down into subproblems:

1. A sequence of random variables $X_n$, $n=1,2,\ldots$ converges in probability to $X$ and $Y$. That is, $X_n \to X$ in probability and $X_n \to Y$ in probability. Show that $X = Y$ almost surely.
2. Show that if $\sqrt{n}(X_n - \mu) \to Z$ in distribution for some random variable $Z$ and $\mu \in \mathbb{R}$, $X_n \to \mu$ in probability.
3. Conclude that $\overline{X}_n \to \mu$ almost surely provided that $\sqrt{n}(\overline{X}_n - \mu)\to Z$ for some random variable $Z$ and $\mu \in \mathbb{R}$.

This is how I would solve this problem:

• The first question can be taken care of using basic probability calculus. For any $\epsilon > 0$, we know that $\mathbb{P}(|X_n - X| > \epsilon/2) + \mathbb{P}(|X_n - Y| > \epsilon/2) \to 0$ due to convergence in probability for both terms. Then, by the triangle inequality, $$[|X-Y| > \epsilon] \subseteq [|X_n - X| > \epsilon/2] \cup [|X_n - Y| > \epsilon/2],$$ which, by the subadditivity of the probability measure, gives $$\mathbb{P}(|X-Y| > \epsilon) \leq \mathbb{P}(|X_n - X| > \epsilon/2) + \mathbb{P}(|X_n - Y| > \epsilon/2).$$ Taking the limit shows that $\mathbb{P}(|X-Y| > \epsilon)=0$ for every $\epsilon$. This is a very common first-order logic: to use an arbitrary positive real number to show the limit. That is, $$\mathbb{P}(X\neq Y)=\mathbb{P}(|X-Y|=0) = \mathbb{P}\left(\bigcup_{n=1}^\infty [|X-Y|] > \dfrac{1}{n} \right)=0.$$ $\mathbb{P}(X\neq Y)=0$ is the definition of the almost-sure uniqueness of the limits.

• The second subproblem is applying the Slutsky’s theorem and the fact that convergence in distribution to a constant (not a random variable) implies convergence in probability. $1/\sqrt{n} \to 0$ along with $\sqrt{n}(X_n - \mu)\to Z$ yields $$ \dfrac{1}{\sqrt{n}}\times \sqrt{n}(X_n - \mu) \to 0 \Rightarrow X_n - \mu \xrightarrow{\mathcal{L}} 0.$$

  Normally, $X_n \xrightarrow{\mathcal{L}} X$ doesn’t imply $X_n \xrightarrow{p} X$ because convergence in probability requires that $X_n$ and $X$ are defined on the same sample space for $\forall n$, which is not always the case. Take the normal approximation to binomial. Binomial distribution is defined on a restricted range of integers, but it certainly converges to a normal distribution. But it’s a different story when $X$ is just a constant $\mu$. To prove that $X_n \xrightarrow{\mathcal{L}} \mu$ implies $X_n \xrightarrow{p} \mu$, the constant $\mu$ permits a distribution function $F(x) = I(x \geq \mu)$. Therefore, for $\epsilon > 0$, $\mu+\epsilon$ and $\mu-\epsilon$ are points of continuity. Then, $$X_n \xrightarrow{\mathcal{L}} \mu \Rightarrow \begin{cases}F_n(\mu-\epsilon)\to F(\mu -\epsilon)=0\\ F_n(\mu+\epsilon)\to F(\mu+\epsilon) = 1\end{cases}.$$ Thus, $$\mathbb{P}(-\epsilon < X_n - \mu \leq \epsilon) = F_n(\mu+\epsilon) - F_n(\mu-\epsilon) \to 1,$$ implying $X_n \xrightarrow{p} \mu$.

• Combining all of the above, from the weak law of large numbers, $\overline{X}_n \xrightarrow{p} \mathbb{E}X_1$, and from $\sqrt{n}(\overline{X}_n - \mu) \xrightarrow{\mathcal{L}} Z$, we know that $\overline{X}_n \xrightarrow{p} \mu$. Thus, by the first subproblem, we conclude that $\mu = \mathbb{E}X_1$ almost surely. We already know from the strong law of large numbers that $\overline{X}_n \xrightarrow{a.s.}\mathbb{E}X_1$, which is $\overline{X}_n \xrightarrow{a.s.} \mu$.