The Chinese Remainder Theorem and Lagrange Interpolation

by t0rajir0u, Nov 6, 2008, 3:24 AM

Today I'd like to discuss two results that are used much more often than they are proved. A strong understanding of both these theorems is based on notions of linearity. Moreover, the following statements of the two theorems will make it clear that they are essentially the same.

The Chinese Remainder Theorem: Let $ m_1, m_2, ... m_k$ be pairwise relatively prime positive integers greater than $ 1$ and let $ r_1, r_2, ... r_k$ be integers. The system of congruences

$ x \equiv r_1 \bmod m_1$
$ x \equiv r_2 \bmod m_2$
...
$ x \equiv r_k \bmod m_k$

has a unique solution $ \bmod m_1 m_2 ... m_k$. In particular, it has a unique solution $ 0 \le x < m_1 m_2 ... m_k$.

The Lagrange Interpolation Theorem: Let $ x_1, x_2, ... x_k$ be distinct elements of a field $ F$ and let $ y_1, y_2, ... y_k \in F$. The system of polynomial congruences

$ P(x) \equiv y_1 \bmod (x - x_1)$
$ P(x) \equiv y_2 \bmod (x - x_2)$
...
$ P(x) \equiv y_k \bmod (x - x_k)$

has a unique solution $ \bmod (x - x_1)(x - x_2)...(x - x_k)$. In particular, it has a unique solution of degree $ k-1$.

(Note that the condition $ P(x) \equiv y_i \bmod (x - x_i)$ is exactly equivalent to the condition that $ P(x_i) = y_i$ by the remainder theorem.)


Lagrange interpolation is usually stated in a way that is very different from the above; in particular, we usually derive an explicit formula for $ P(x)$. Rather than doing so immediately, we will first derive an explicit form for the solution to the problem posed by the Chinese Remainder Theorem; both solutions use the same fundamental idea, which is this:

The operation that takes a polynomial (respectively, an integer) to its values at some distinct points (respectively, its remainders modulo some coprime integers) is linear and invertible.

This suggests the following elegant idea: suppose we could find integers $ u_1, u_2, ... u_k$ such that

$ u_i \equiv 1 \bmod m_i$
$ u_i \equiv 0 \bmod \frac{m_1 m_2 ... m_k}{m_i}$

for all $ i$. These integers form a basis for the solutions we want in the following sense: the $ x$ that solves our original system of congruences is

$ x = a_1 u_1 + a_2 u_2 + ... + a_k u_k$.

This is easily verified by adding together all of the congruences just described. It remains to actually find $ u_1, u_2, ... u_k$. But all we have to do is set $ u_i = \frac{m_1 m_2 ... m_k}{m_i} v_i$ for some integer $ v_i$ and then compute the multiplicative inverse of $ \frac{m_1 m_2 ... m_k}{m_i} \bmod m_i$, which will be equal to $ v_i$. (This is why we need the moduli to be relatively prime.) This can be done using the extended Euclidean algorithm and the resulting answer is unique, hence the resulting $ x$ is unique. (This does not quite constitute a proof, but I'll let you fill in the details: in terms of the $ u_i$s, what happens if $ x$ is not unique?)

The exact same ideas work on polynomials: suppose we could find polynomials $ P_i(x)$ such that

$ P_i(x_i) = 1$
$ P_i(x_j) = 0, j \neq i$

for all $ i$. Then the polynomial we want is

$ P(x) = y_1 P_1(x) + y_2 P_2(x) + ... + y_k P_k(x)$.

Again, this is easily verified by adding together all of the identities (which can be viewed as congruences in the sense described above). Here, though, it is a little more straightforward to compute the polynomials $ P_i$ (and this is how Lagrange interpolation is usually stated): if $ P_i(x_j) = 0$ then

$ P_i(x) = c_i(x - x_1)...(x - x_{i-1})(x - x_{i+1})...(x - x_k)$

where $ P_i(x_i) = 1$, hence

$ c_i = \frac{1}{(x_i - x_1)...(x_i - x_k)}$.

This is exactly analogous to our discussion of multiplicative inverses $ \bmod m_i$; finding a remainder $ \bmod x - x_i$ is the same thing as computing $ P_i(x_i)$. If the form of the Lagrange interpolating polynomials ever seemed mysterious to you, I hope this discussion puts them in the proper context.

Why is computing a remainder $ \bmod m_i$ so exactly analogous to substituting a value $ x_i$ into a polynomial? Well, in one sense it's because polynomial congruences work pretty much the same way as regular congruences, but there has to be something more going on here. When we talk about congruences we are really talking about ideals and quotients with respect to ideals. When we talk about values of a polynomial at points, we are talking about polynomial ideas generated by polynomials of the form $ x - x_i$; quotients with respect to those ideals give us values at points. In other words, the question of whether a polynomial has a root at a given point is equivalent to the question of whether it belongs in a given ideal; we are talking algebraic geometry again.

We are not done. I have avoided talking about Lagrange interpolation in the past because it is not, computationally, an efficient way to solve interpolation problems, especially interpolation problems with some additional structure. I have talked about Newton interpolation, which in more general form looks like this: write

$ P(x) = d_1 + d_2(x - x_1) + d_3(x - x_1)(x - x_2) + ...$

where each term has an additional factor. Then

$ P(x_1) = d_1 = y_1$
$ P(x_2) = d_1 + d_2(x_2 - x_1) = y_2 \Leftrightarrow d_2 = \frac{y_2 - d_1}{x_2 - x_1}$
$ P(x_3) = d_1 + d_2(x_3 - x_1) + d_3(x_3 - x_1)(x_3 - x_2) = y_3 \Leftrightarrow \text{etc.}$

This gives a nice proof both of existence and of uniqueness. (Note the correspondence to the Newton polynomials if $ x_i = i$.) In linear algebra terms, Lagrange interpolation is equivalent to inverting a Vandermonde matrix (albeit not directly), whereas Newton interpolation is equivalent to inverting a triangular matrix, which is much easier.

The obvious question from here is whether this idea works when applied to CRT. In fact, it does. Write

$ x = p_1 + p_2 m_1 + p_3 m_1 m_2 + ...$

where again each term has an additional factor. Then

$ x \equiv p_1 \equiv r_1 \bmod m_1$
$ x \equiv r_1 + p_2 m_1 \equiv r_2 \bmod m_2 \Leftrightarrow p_2 \equiv \frac{r_2 - r_1}{m_1} \bmod m_2$
$ x \equiv r_1 + p_2 m_1 + p_3 m_1 m_2 \equiv r_3 \bmod m_3 \Leftrightarrow \text{etc.}$

which again provides a nice proof both of existence and of uniqueness. Moreover, I have not seen this proof of CRT before, although it is almost certainly known. Slightly revised, and put into reverse, it provides a proof of the uniqueness of mixed radix representation.


So far our discussion of the two theorems has been identical and has fruitfully revealed some interesting ideas. But there is a big advantage in the CRT regime that we don't have in the Lagrange interpolation regime: finiteness. The following can be considered a counting proof, and it begins in the opposite direction as the other.

Let $ M = m_1 m_2 ... m_k$. Define the function $ f(x) : \mathbb{Z}/M\mathbb{Z} \to \mathbb{Z}/m_1\mathbb{Z} \times \mathbb{Z}/m_2\mathbb{Z} \times ...$ that takes $ x$ to its $ k$-tuples of remainders

$ (x \bmod m_1, x \bmod m_2, .... x \bmod m_k).$

We claim that this function is injective. Suppose $ f(x) = f(y)$. Then $ x \equiv y \bmod m_1, m_2, ... m_k$, hence $ x \equiv y \bmod M$ (because of pairwise relative primality), hence $ x = y$ in $ \mathbb{Z}/M\mathbb{Z}$. On the other hand, $ f$ takes a finite set of size $ M$ into a Cartesian product of finite sets of size $ m_1, m_2, ... m_k$, which has total size $ M$. It follows that $ f$ must be surjective (why?), hence a bijection. The statement that $ f$ has an inverse is precisely the statement of the Chinese Remainder Theorem.

Of course, the above argument gives no method to explicitly construct the inverse function $ f^{-1}(x)$, but there are important lessons to draw from this proof. In the context of our previous discussion, we took advantage of the fact that $ f^{-1}$ was linear (in the appropriate sense) to describe ways to invert it, but if you think about what scalar multiplication really means here, what we're really saying is that $ f$ is a ring isomorphism. This doesn't directly help us translate this proof to the Lagrange interpolation regime (if we try, the proof looks suspiciously like our earlier ideas), but it does suggest the correct generalization of both results.


Practice Problem 1: State and prove the appropriate generalization of both of the main theorems above in a commutative ring $ R$ in terms of a collection of pairwise relatively prime ideals $ I_1, I_2, ... I_k$.

Practice Problem 2 (CRT): a) Show that there exist $ n$ consecutive composite positive integers for any $ n$.

b) (Putnam 1955) Show that there exist $ n$ consecutive positive integers that are not squarefree for any $ n$.

c) Show that there exist $ n$ consecutive positive integers that cannot be written as the sum of two squares for any $ n$.

d) Do there exist positive integers $ a < b$ such that for any $ k$ between $ a$ and $ b$ either $ \gcd(a, k) > 1$ or $ \gcd(b, k) > 1$?

Practice Problem 3 (Lagrange interpolation): a) The polynomial $ P(x)$ of degree $ n$ satisfies $ P(k) = 2^k, k = 0, 1, 2, ... n$. Compute $ P(n+1)$. (Compare with the solution by Newton interpolation.)

b) Prove the validity of Lagrange interpolation by computing the Vandermonde determinant and showing that it is nonzero. (Hint: the determinant is a homogeneous polynomial in the $ x$ values. When must this determinant be zero? What degree does the determinant have?)

Practice Problem 4: Give the proof of the uniqueness (and existence) of mixed radix representation I hinted at. (The proof begins like this: given natural numbers $ b_1, b_2, ...$ and a natural number $ x$ compute $ x \bmod b_1$. Now set $ x' = \frac{x - x \bmod b_1}{b_1}$ and compute $ x \bmod b_2$. Now...)

Comment

2 Comments

The post below has been deleted. Click to close.
This post has been deleted. Click here to see post.
My professor of abstract algebra said that the Chinese Remainder Theorem is one of the greatest discovery by Chinese. He gave us an example of its application on CS and IT to show us the importence of this theorem. (But I do not remember the example very clearly.)

by ifai, Nov 6, 2008, 3:25 PM

The post below has been deleted. Click to close.
This post has been deleted. Click here to see post.
:O :O I was shocked when I read the read paragraph.

Also I don't really know Lagrange Interpolation... but still, that is cool.

by Ubemaya, Dec 3, 2008, 3:47 AM

Various mathematical thoughts, ranging from problem-solving techniques to attempts to tie together related concepts.

avatar

t0rajir0u
Archives
+ March 2009
+ October 2007
+ May 2007
Shouts
Submit
  • orz $~~~~$

    by clarkculus, Jan 10, 2025, 4:13 PM

  • Insanely auraful

    by centslordm, Jan 1, 2025, 11:17 PM

  • Fly High :(

    by Siddharthmaybe, Oct 22, 2024, 8:34 PM

  • Dang it he is gone :(( /revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive/revive.

    by 799786, Aug 4, 2022, 1:56 PM

  • annoying precision

    by centslordm, May 16, 2021, 7:34 PM

  • rip t0rajir0u

    by OlympusHero, Dec 5, 2020, 9:29 PM

  • Shoutbox bump xD

    by DuoDuoling0, Oct 4, 2020, 2:25 AM

  • dang hes gone :(

    by OlympusHero, Jul 28, 2020, 3:52 AM

  • First shout in July

    by smartguy888, Jul 20, 2020, 3:08 PM

  • https://artofproblemsolving.com/community/c2448

    has more.

    -πφ

    by piphi, Jun 12, 2020, 8:20 PM

  • wait hold up 310,000
    people visited this man?!?!??

    by srisainandan6, May 29, 2020, 5:16 PM

  • first shout in 2020

    by OlympusHero, Apr 4, 2020, 1:15 AM

  • in his latest post he says he moved to wordpress

    by MelonGirl, Nov 16, 2019, 2:43 AM

  • Please revive!

    by AopsUser101, Oct 30, 2019, 7:10 PM

  • first shout in october fj9odiais

    by bulbasaur., Oct 14, 2019, 1:14 AM

128 shouts
Tags
About Owner
  • Posts: 12167
  • Joined: Nov 20, 2005
Blog Stats
  • Blog created: Dec 5, 2006
  • Total entries: 48
  • Total visits: 321832
  • Total comments: 202
Search Blog
a