#Sequence – deep mind

Convergence ca be defined in many different ways. In this post, we study the most popular way to define convergence by a metric. Note that knowledge about metric spaces is a prerequisite.

In order to define other types of convergence (e.g. point-wise convergence of functions) one needs to extend the following approach based on open sets. This, however, is not in scope of this post.

Limits of a Sequence

In this section, we apply our knowledge about metrics, open and closed sets to limits. We thereby restrict ourselves to the basics of limits. Refer to [1] for further details.

Let $X:=(X, d)$ denote a metric space. If the metric is not specified, we assume that the standard Euclidean metric is assigned.

Definition 3.1 (Sequence):
A sequence in $X$ is a function from $\mathbb{N}$ to $X$ by assigning a value $f(n)\in X$ to each natural number $n\in \mathbb{N}$ . The set $(x_n)_{n\in \mathbb{N}}$ $= (x_n) =:$ $\{x_n | \ n\in\mathbb{N} \}=$ $\{ x_1, x_2, \ldots, x_n, \ldots\}$ is called sequence of $X$ . The elements of $(x_n)_{n\in \mathbb{N}}$ are called terms of the sequence.

$\square$

Sequences in $X \subseteq \mathbb{R}^1$ are called real number sequences. Sequences in $X \subseteq \mathbb{R}^d$ , $d \in \mathbb{N}$ are called real tuple sequences. Note that latter definition is simply a generalization since number sequences are, of course, $d$ -tuple sequences with $d=1$ .

Let $(x_n)$ be a $d$ -tuple sequence in $X$ equipped with property $E$ . Property $E$ holds for almost all terms of $(x_n)$ if there is some $m\in \mathbb{N}$ such that $E$ is true for infinitely many of the terms $x_k$ with $k\geq m$ .

Note that a sequence can be considered as a function with domain $\mathbb{N}$ . We need to distinguish this from functions that map sequences to corresponding function values. Latter concept is very closely related to continuity at a point.

Sequences are, basically, countably many ( $1$ – or higher-dimensional) vectors arranged in an ordered set that may or may not exhibit certain patterns.

Example 3.1:
a) The sequence $\{ \frac{1}{1}, \frac{1}{2}, \frac{1}{3}, \frac{1}{4}, \ldots \}$ can be written as $( \frac{1}{n} )_{n\in \mathbb{N}}$ and is nothing but a function $h:\mathbb{N} \rightarrow \mathbb{R}$ defined by $h(n)=\frac{1}{n}$ .

Plot of harmonic series — Plot of $h(n)$ for $n\in \{1, 2, \ldots, 49\}$

b) Let us now consider the sequence $\{ +1, -1, +1, -1, \ldots \}$ that can be denoted by $\{(-1)^n\}_{ n\in \mathbb{N} \cup \{ 0 \} }$ . The range of the function only comprises two real figures $\{+1,-1\}$ .

c) Now, let us consider the sequence $\{1, 2, 3, \ldots \} = \{n\}_{n\in \mathbb{N}}$ . Here, each natural $n$ is mapped on itself.

d) Consider $(\frac{n-1}{n})_{n\in \mathbb{N}} := \{0, \frac{1}{2}, \frac{2}{3}, \frac{3}{4}, \ldots \}$ , that can also be written as $\{\frac{n-1}{n}\}_{n\in \mathbb{N}} = \{1-\frac{1}{n}\}_{n\in \mathbb{N}}$ .

Plot of the sequence $(\frac{n-1}{n})_{n\in \mathbb{N}}$

e) Consider the 2-tuple sequence $(x_n)_{n\in \mathbb{N}} := ((1+\frac{1}{n})^n, 1-\frac{1}{n})$ in $\mathbb{R}^2$ .

Plot of 2-tuple sequence for the first 1000 points that seems to head towards a specific point in $\mathbb{R}^2$ .

$\square$

We might think that some of the above examples contain patterns of vectors (e.g. number) that are “getting close” to some other vector (e.g. number). Other sequences may not give us that impression. We are interested in what the long-term behavior of the sequence is:

What happens for large values of $n$ ?
Does the sequence approach a (real) vector/number?

Note that it is not necessary for a convergent sequence to actually reach its limit. It is only important that the sequence can get arbitrarily close to its limit.

Convergence of a Sequence

Now, let us try to formalize our heuristic thoughts about a sequence approaching a number $x\in \mathbb{R}^n$ arbitrarily close by employing mathematical terms.

By writing

(1) $\begin{align*}\lim_{n \rightarrow \infty}{x_n} = x,\end{align*}$

we mean that for every real number $\epsilon>0$ there is an integer $N\in \mathbb{N}$ , such that

(2) $\begin{align*}d(x_n, x) < \epsilon\end{align*}$

whenever $n\geq N$ . A sequence $(x_n)$ that fulfills this requirement is called convergent. We can illustrate that on the real line using balls (i.e. open intervals) as follows.

Note that $d(x_n, x) < \epsilon$ represents an open ball $B(x, \epsilon)$ centered at the convergence point or limit x. For instance, for $\epsilon=0.5$ we have the following situation, that all points (i.e. an infinite number) smaller than $\frac{1}{2}$ lie within the open ball $B(0, \epsilon=0.5)$ . Those points are sketched smaller than the ones outside of the open ball $B(0, \epsilon=0.5)$ .

Accordingly, a real number sequence is convergent if the absolute amount is getting arbitrarily close to some (potentially unknown) number $x$ , i.e. if there is an integer $N\in \mathbb{N}$ such that $|x_n - x|<\epsilon$ whenever $n\geq N$ .

Convergence actually means that the corresponding sequence gets as close as it is desired without actually reaching its limit. Hence, it might be that the limit $\lim_{n \rightarrow \infty}{x_n}=x_0$ of the sequence $x_n$ is not defined at $x_0$ but it has to be defined in a neighborhood of $x_0$ .

This limit process conveys the intuitive idea that $x_n$ can be made arbitrarily close to $x\in \mathbb{R}$ provided that $n$ is sufficiently large. “Arbitrarily close to the limit $x$ ” can also be reflected by corresponding open balls $B(x, \epsilon)$ , where the radius $\epsilon>0$ needs to be adjusted accordingly.

The sequence can also be considered as a function $f:\mathbb{N} \rightarrow \mathbb{R}$ defined by $f(n)=x_n$ with

(3) $\begin{align*}\lim_{n \rightarrow \infty}{x_n} = x\end{align*}$

whenever $n\geq N$ .

If there is no such $x\in \mathbb{R}$ , the sequence $\{x_n\}$ is said to diverge. Please note that it also important in what space the process is considered. It might be that a sequence is heading to a number that is not in the range of the sequence (i.e. not part of the considered space). For instance, the sequence Example 3.1 a) converges in $\mathbb{R}$ to 0, however, fails to converge in the set of all positive real numbers (excluding zero).

The definition of convergence implies that $x_n \rightarrow x$ if and only if $d(x_n, x) \rightarrow 0$ . The convergence of the sequence $\{d(x_n, x) \}$ to 0 takes place in the standard Euclidean metric space $(\mathbb{R}^1, |\cdot|)$ .

While a sequence $\{x_n\}$ in a metric space $(V, d)$ does not need to converge, if $\{x_n\} \rightarrow x \in \mathbb{R}$ its limit is unique. Notice, that a ‘detour’ via another convergence point $y\in V$ (triangle property) would turn out to be the direct path with respect to the metric as $n \rightarrow \infty$ .

A convergent sequence $x_n \rightarrow x$ is also bounded. We can prove this intuitive statement by setting $\epsilon:=1$ . Hence, it exists a $N\in \mathbb{N}$ , such that $|x_n - x| < 1$ for all $n>N$ . This implies

(4) $\begin{align*}|x_n - x| &<1 \ \text{for all } n>N \\\Rightarrow |x_n| &< |x_n - x| +|x| < 1 +|x|\end{align*}$

for all $n > N$ . Let $m := \max(|x_k| , 1 +|x|)$ for $k\in\{1, \ldots, N\}$ then the assertion $|x_n|<m$ follows immediately.
$\square$

Let us re-consider Example 3.1, where the sequence a) apparently converges towards $zero$ . Sequence b) instead is alternating between $+1$ and $-1$ and, hence, does not converge. Note that example b) is a bounded sequence that is not convergent. Sequence c) does not have a limit in $\mathbb{R}$ as it is growing towards $\infty$ and is therefore not bounded. However, sequence d) converges towards 1. Finally, 2-tuple sequence e) converges to the vector $(e, 1)\in \mathbb{R}^2$ .

If we consider one of the converging examples carefully, we will notice that we can chose an arbitrarily small $\epsilon >0$ and we will find a correspondingly large $N\in \mathbb{N}$ , such that $|x_n - x| < \epsilon$ with $n>N$ .

For instance, let us define $\epsilon$ to be $0,0000001$ in Example 3.1 a). Then, we can set $N:=1,000,000$ and $\frac{1}{n} - 0| < 0,0000001$ provided that $n>1,000,000$ .

$\square$

A sequence $\{x_n\}$ is called increasing if $x_n \leq x_{n+1}$ for all $n\in \mathbb{N}$ . If an increasing sequence is bounded above, then $\{x_n\}$ converges to the supremum $\sup\{x_1, x_2, \ldots, x_n\}$ of its range.

If you want to get a deeper understanding of converging sequences, the second part (i.e. Level II) of the following video by Mathologer is recommended.

Cauchy Sequences

If a sequence $\{x_n\}_{n\in \mathbb{N}}$ converges to a limit $x$ , its terms must ultimately become close to its limit $x$ and hence close to each other. That is, two arbitrary terms $x_l$ and $x_k$ of a convergent sequence $\{x_n\}$ become closer and closer to each other provided that the index of both are sufficiently large.

Theorem 3.1 (Convergent and Cauchy Sequences):

Assume that $\{x_n\}_{n\in \mathbb{N}}$ converges in a metric space $(X, d)$ to a limit $x$ . Then for every real $\epsilon^*>0$ there is an integer $N^*$ such that

$\begin{align*}d(x_l, x_k) < \epsilon^* \quad \text{whenever } l,k\geq N^*.\end{align*}$

Proof: $x$ is the limit of $\{x_n\}_{n\in \mathbb{N}}$ , i.e. $\lim_{n \rightarrow \infty} x_n = x$ . Assume $\epsilon^* > 0$ is given. Due to the fact that $\lim_{n \rightarrow \infty} x_n = x$ , we can choose an integer $N$ , such that $d(x_n, x)< \epsilon=\frac{\epsilon^*}{2}$ for all $n>N$ . If $l, k>N$ is valid, we can conclude

$\begin{align*}d(x_l, x_k) \leq d(x_l, x) + d(x_k, x) < \frac{\epsilon^*}{2} + \frac{\epsilon^*}{2} = \epsilon^*\end{align*}$

by employing the triangle inequality of the metric.

$\square$

The last proposition proved that two terms of a convergent sequence becomes arbitrarily close to each other. This property was used by Cauchy to construct the real number system by adding new points to a metric space until it is ‘completed‘. Sequences that fulfill this property are called Cauchy sequence.

Definition 3.2 (Cauchy Sequence):
A sequence $\{x_n\}_{n\in \mathbb{R}}$ is called Cauchy sequence, if the following condition holds true:

(5) $\begin{align*} \forall \epsilon \ \exists N^* \in \mathbb{N} \text{ such that } \forall l,k > N^*:\\ d(x_l, x_k) < \epsilon \end{align*}$

$\square$

Note that all pairs of terms with index greater than $N^*$ need to get close together. It is not sufficient to require that two consecutive terms get close together.

In the following example, we consider the function and sequences that are interpreted as attributes of this function. If we consider the points of the domain and the function values of the range, we get two sequences that correspond to each other via the function. This concept is closely related to continuity.

Example 3.2 (Non-Complete Space):
If we consider $\mathbb{Q}$ embedded in $\mathbb{R}$ such that symbols such as $\sqrt{2}$ and $e$ can be interpreted and used. The letter $x$ is assumed to represent a rational number in this example.

$\begin{align*} f(x):= \frac{1}{x^2-2} \quad \text{with } 0\leq x\leq 2.\end{align*}$

Example for incompletness of rational numbers — Function graph of $f$ with singularities at $\pm$ 2

Considering the sequence $\lim_{x\rightarrow 2}{f(x)}$ in $\mathbb{Q}$ shows that the actual limit $\sqrt{2}\in \mathbb{R} \setminus \mathbb{Q}$ is not contained in $\mathbb{Q}$ .

$\square$

Hence, a key question is:

What condition on a sequence of numbers is necessary and sufficient for the sequence to converge to a limit but does not explicitly involve the limit?

If we already knew the limit in advance, the answer would be trivial. In general, however, the limit is not known and thus the question not easy to answer. It turns out that the Cauchy-property of a sequence is not only necessary but also sufficient. That is, every convergent Cauchy sequence is convergent (sufficient) and every convergent sequence is a Cauchy sequence (necessary).

Note that every Cauchy sequence is bounded. To see this set $\epsilon:=1$ , then there is a $N^* \in \mathbb{N}$ : $(l, k>N^* \Rightarrow d(x_l, x_k)<1$ and thus $d(x_k, x_{N^*})<1$ for all $k>N^*$ . This means that all points $x_k$ with $k>N^*$ lies within a ball of radius 1 with $x_{N^*}$ as its center.

Theorem 3.2 (Cauchy Sequences & Convergence):

In an Euclidean space $\mathbb{R}^d$ every Cauchy sequence is convergent.

Proof: Let $\{x_n\}$ be a Cauchy sequence in $\mathbb{R}^d$ and let $T:=\{x_1, x_2, \ldots\}$ be the range of the sequence. If $T$ is finite, then all except a finite number of the terms $\{x_n\}$ are equal and hence $\{x_n\}$ converges to this common value.

Now suppose $T$ is infinite. We use the Balzano-Weierstrass Theorem to show that $T$ has an accumulation point $x_0$ , and then we show that $\{x_n\}$ converges to $x_0$ . First, recall that each Cauchy sequence is bounded. Hence, since $T$ is infinite there must be an accumulation point $x_0$ according to the Bolzano-Weierstrass Theorem.
Now let $\epsilon >0$ , then there exist a $N^*$ such that $d(x_k, x_l)< \frac{\epsilon}{2}$ whenever $k,l\geq N^*$ . Hence, if $k\geq N^*$ we have

$d(x_k, x_0) \leq d(x_k, x_l) + d(x_l, x_0) < \frac{\epsilon}{2}+\frac{\epsilon}{2}=\epsilon$ ,

so $\lim_{k \rightarrow \infty}{x_k}=x_0$ .
$\square$

Let us furthermore connect the concepts of metric spaces and Cauchy sequences.

Definition 3.3: (Complete Metric Space & Banach Space)
A metric space $(X, d)$ is called complete (or a Cauchy space) if every Cauchy sequence of points in $X$ has a limit that is also in $X$ .
A Banach space is a complete normed vector space, i.e. a real or complex vector space on which a norm is defined.

$\square$

Complete and Banach space will become important in Functional Analysis, for instance.

One-Sided Limit of a Function

In this section it is about the limit of a sequence that is mapped via a function to a corresponding sequence of the range. As mentioned before, this concept is closely related to continuity. Let $X:=(X, d)$ denote the standard metric space on the real line with $X=(\mathbb{R}, |\cdot|)$ and $f:X \rightarrow \mathbb{R}$ .

Consider the Heaviside function as shown below. In the one-dimensional metric space $X=(\mathbb{R}, |\cdot|)$ there are only two ways to approach a certain point $x_0\in \mathbb{R}$ on the real line. For instance, the point $x_0=0$ can be either be approached from the negative (denoted by $\nearrow$ ) or from the positive (denoted by $\swarrow$ ) part of the real line. Sometimes this is stated as the limit $x_0$ is approached “from the left/righ” or “from below/above”.

The Heaviside function does not have a limit at $x_0=0$ , because if you approach 0 from positive numbers the value is 1 while if you approach from negative numbers the value is 0. As we know, the limit needs to be unique if it exists.

Hence, by writing the left-sided limit

(6) $\begin{align*}\lim_{x \nearrow a}{f(x)} = y_0,\end{align*}$

we mean that for every real number $\epsilon>0$ , there is a $\delta>0$ , such that

(7) $\begin{align*}0 < a-x < \delta \ \Rightarrow \ | f(x), y_0| < \epsilon.\end{align*}$

Left-sided means that the $x$ -value increases on the real axis and approaches from the left to the limit point $x_0$ . Hence, $x_0-x$ is positive.

Accordingly, by writing the right-sided limit

(8) $\begin{align*}\lim_{x \swarrow a}{f(x)} = y_0,\end{align*}$

we mean that for every real number $\epsilon>0$ , there is a $\delta>0$ , such that

(9) $\begin{align*}0 < x-a < \delta \ \Rightarrow \ | f(x), y_0| < \epsilon.\end{align*}$

Right-sided means that the $x$ -value decreases on the real axis and approaches from the right to the limit point $x_0$ . Hence, $x-x_0$ is positive.

$\square$

Consider that the left-sided and right-sided limits are just the restricted functions, where the domain is constrained to the “right-hand side” or “left-hand side” of the domain relative to its limit point $x_0\in X$ .

That is, for $X$ being the metric space the left-sided and the right-sided domains are $X_l :=\{x\in X| x<a\}$ and $X_r :={x\in X| x<a}$ , respectively. If we then consider the limit of the restricted functions $f_{|X_l}$ and $f_{|X_r}$ , we get an equivalent to the definitions above.

Having said that, it is clear that all the rules and principles also apply to this type of convergence. In particular, this type will be of interest in the context of continuity.

Literature:
[1]

[2]

[3]

Tag: #Sequence

Sequences in Metric Spaces

Limits of a Sequence

Convergence of a Sequence

Cauchy Sequences

One-Sided Limit of a Function