WEBVTT 00:00:08.750 --> 00:00:09.310 Okay. 00:00:09.310 --> 00:00:13.070 This is a lecture where complex numbers come in. 00:00:13.070 --> 00:00:17.160 It's a -- complex numbers have slipped into this course 00:00:17.160 --> 00:00:23.480 because even a real matrix can have complex eigenvalues. 00:00:23.480 --> 00:00:27.420 So we met complex numbers there as the eigenvalues 00:00:27.420 --> 00:00:30.030 and complex eigenvectors. 00:00:30.030 --> 00:00:33.560 And we -- or -- 00:00:33.560 --> 00:00:36.410 this is probably the last -- 00:00:36.410 --> 00:00:38.870 we have a lot of other things to do about eigenvalues 00:00:38.870 --> 00:00:40.580 and eigenvectors. 00:00:40.580 --> 00:00:43.610 And that will be mostly real. 00:00:43.610 --> 00:00:46.090 But at one point somewhere, we have 00:00:46.090 --> 00:00:51.140 to see what you do when the numbers become complex numbers. 00:00:51.140 --> 00:00:54.500 What happens when the vectors are complex, 00:00:54.500 --> 00:00:57.360 when the matrixes are complex, when the -- 00:00:57.360 --> 00:01:01.640 what's the inner product of two, the dot product of two complex 00:01:01.640 --> 00:01:03.730 vectors -- 00:01:03.730 --> 00:01:06.050 we just have to make the change, just see -- 00:01:06.050 --> 00:01:09.340 what is the change when numbers become complex? 00:01:09.340 --> 00:01:14.360 Then, can I tell you about the most important example 00:01:14.360 --> 00:01:17.270 of complex matrixes? 00:01:17.270 --> 00:01:20.460 It comes in the Fourier matrix. 00:01:20.460 --> 00:01:23.480 So the Fourier matrix, which I'll describe, 00:01:23.480 --> 00:01:25.400 is a complex matrix. 00:01:25.400 --> 00:01:28.850 It's certainly the most important complex matrix. 00:01:28.850 --> 00:01:34.770 It's the matrix that we need in Fourier transform. 00:01:34.770 --> 00:01:37.930 And the -- really, the special thing that I want to tell you 00:01:37.930 --> 00:01:42.340 about is what's called the fast Fourier transform, 00:01:42.340 --> 00:01:48.000 and everybody refers to it as the FFT and it's in all 00:01:48.000 --> 00:01:50.500 computer and it's used -- 00:01:50.500 --> 00:01:54.010 it's being used as we speak in a thousand places, 00:01:54.010 --> 00:01:59.510 because it has, like, transformed whole industries 00:01:59.510 --> 00:02:03.090 to be able to do the Fourier transform fast, 00:02:03.090 --> 00:02:09.270 which means multiplying -- how do I multiply fast by that 00:02:09.270 --> 00:02:11.730 matrix -- by that n by n matrix? 00:02:11.730 --> 00:02:16.380 Normally, multiplications by an n by n matrix -- 00:02:16.380 --> 00:02:21.440 would normally be n squared multiplications, 00:02:21.440 --> 00:02:24.460 because I've got n squared entries and none of them is 00:02:24.460 --> 00:02:24.960 zero. 00:02:24.960 --> 00:02:27.780 This is a full matrix. 00:02:27.780 --> 00:02:30.700 And it's a matrix with orthogonal columns. 00:02:30.700 --> 00:02:34.140 I mean, it's just, like, the best matrix. 00:02:34.140 --> 00:02:37.700 And this fast Fourier transform idea 00:02:37.700 --> 00:02:43.680 reduces this n squared, which was slowing up 00:02:43.680 --> 00:02:47.815 the calculation of Fourier transforms down to n log(n). 00:02:51.000 --> 00:02:54.480 n log(n), log to the base two, actually. 00:02:54.480 --> 00:02:57.940 And it's this -- when that hit -- 00:02:57.940 --> 00:03:01.930 when that possibility hit, it made a big difference. 00:03:01.930 --> 00:03:08.430 Everybody realized gradually what, -- 00:03:08.430 --> 00:03:11.800 that this simple idea -- you'll see it's just a simple matrix 00:03:11.800 --> 00:03:15.060 factorization -- but it changed everything. 00:03:15.060 --> 00:03:15.630 Okay. 00:03:15.630 --> 00:03:19.250 So I want to talk about complex vectors and matrixes 00:03:19.250 --> 00:03:22.610 in general, recap a little bit from last time, 00:03:22.610 --> 00:03:27.450 and the Fourier matrix in particular. 00:03:27.450 --> 00:03:28.940 Okay. 00:03:28.940 --> 00:03:31.360 So what's the deal? 00:03:31.360 --> 00:03:32.210 All right. 00:03:32.210 --> 00:03:36.500 The main point is, what about length? 00:03:36.500 --> 00:03:40.560 I'm given a vector, I have a vector x. 00:03:40.560 --> 00:03:44.310 Or let me call it z as a reminder that it's complex, 00:03:44.310 --> 00:03:45.260 for the moment. 00:03:45.260 --> 00:03:48.950 But I can -- later I'll call the components x. 00:03:48.950 --> 00:03:50.740 They'll be complex numbers. 00:03:50.740 --> 00:03:53.090 But it's a vector -- 00:03:53.090 --> 00:03:56.500 z1, z2 down to zn. 00:03:56.500 --> 00:04:01.590 So the only novelty is it's not in R^n anymore. 00:04:01.590 --> 00:04:04.370 It's in complex n dimensional space. 00:04:04.370 --> 00:04:07.510 Each of those numbers is a complex number. 00:04:07.510 --> 00:04:14.750 So this z,z1 is in C^n, n dimensional complex space 00:04:14.750 --> 00:04:17.700 instead of R^n. 00:04:17.700 --> 00:04:20.750 So just a different letter there, but now the point 00:04:20.750 --> 00:04:23.200 about its length is what? 00:04:23.200 --> 00:04:29.190 The point about its length is that z transpose z is no good. 00:04:34.800 --> 00:04:38.530 z transpose z -- if I just put down z transpose here, 00:04:38.530 --> 00:04:42.090 it would be z1, z2, to zn. 00:04:45.580 --> 00:04:47.990 Doing that multiplication doesn't give me 00:04:47.990 --> 00:04:50.090 the right thing. 00:04:50.090 --> 00:04:51.300 W-Why not? 00:04:51.300 --> 00:04:57.390 Because the length squared should be positive. 00:04:57.390 --> 00:05:01.930 And if I multiply -- suppose this is, like, 1 and i. 00:05:01.930 --> 00:05:06.560 What's the length of the vector with components 1 and i? 00:05:06.560 --> 00:05:08.550 What if I do this, so n is just two. 00:05:08.550 --> 00:05:11.380 I'm in C^2, two dimensional space, 00:05:11.380 --> 00:05:16.510 complex space with the vector whose components are 1 and i. 00:05:16.510 --> 00:05:17.310 All right. 00:05:17.310 --> 00:05:23.320 So if I took one times one and i times i and added, 00:05:23.320 --> 00:05:27.150 z transpose z would be zero. 00:05:27.150 --> 00:05:29.310 But I don't -- that vector is not -- 00:05:29.310 --> 00:05:32.030 doesn't have length zero -- the vector with the components 1 00:05:32.030 --> 00:05:33.590 and i -- 00:05:33.590 --> 00:05:41.710 this multiplication -- what I really want is z1 conjugate z1. 00:05:41.710 --> 00:05:46.510 You remember that z1 conjugate z1 is -- 00:05:46.510 --> 00:05:50.370 so you see that first step will be z1 conjugate z1, 00:05:50.370 --> 00:05:56.820 which is the magnitude of z1 squared, which is what I want. 00:05:56.820 --> 00:06:00.550 That's, like, three squared or five squared. 00:06:00.550 --> 00:06:11.170 Now, if it's -- if z1 is i, then I multiplied by minus i gives 00:06:11.170 --> 00:06:19.290 one plus one, so the component of length -- the component i, 00:06:19.290 --> 00:06:21.731 its modulus squared is plus 00:06:21.731 --> 00:06:22.230 one. 00:06:22.230 --> 00:06:22.990 That's great. 00:06:22.990 --> 00:06:25.560 So what I want to do then is do that -- 00:06:25.560 --> 00:06:30.680 I want z1 bar z1, z2 bar z2, zn bar zn. 00:06:30.680 --> 00:06:33.210 And remember that -- you remember this complex 00:06:33.210 --> 00:06:33.900 conjugate. 00:06:33.900 --> 00:06:38.270 So -- so there's the point. 00:06:38.270 --> 00:06:41.030 Now I can erase the no good and put 00:06:41.030 --> 00:06:48.800 is good, because that now gives the answer zero for the zero 00:06:48.800 --> 00:06:52.590 vector, of course, but it gives a positive length 00:06:52.590 --> 00:06:55.780 squared for any other vector. 00:06:55.780 --> 00:06:58.890 So it's a -- it's the right definition of length, 00:06:58.890 --> 00:07:04.370 and essentially the message is that we're always going to be 00:07:04.370 --> 00:07:06.750 taking -- 00:07:06.750 --> 00:07:09.400 when we transpose, we also take complex conjugate. 00:07:09.400 --> 00:07:13.670 So let's -- let's find the length of one -- 00:07:13.670 --> 00:07:19.890 so the vector one i, that's z, that's that vector z. 00:07:19.890 --> 00:07:23.490 Now I take the conjugate of one is one, the conjugate of i is 00:07:23.490 --> 00:07:24.650 minus i. 00:07:24.650 --> 00:07:29.340 I take this vector, I get one plus one -- 00:07:29.340 --> 00:07:29.840 I get 00:07:29.840 --> 00:07:30.339 two. 00:07:33.050 --> 00:07:36.810 So that's a vector and that's a vector of length -- square root 00:07:36.810 --> 00:07:37.740 of two. 00:07:37.740 --> 00:07:40.560 Square root of two is the length and not the zero 00:07:40.560 --> 00:07:43.830 that we would have got from one minus i squared. 00:07:43.830 --> 00:07:46.210 Okay. 00:07:46.210 --> 00:07:49.250 So the message really is whenever we transpose, 00:07:49.250 --> 00:07:51.660 we also take conjugates. 00:07:51.660 --> 00:07:57.300 So here's a symbol -- one symbol to do both. 00:07:57.300 --> 00:08:03.890 So that symbol H, it stands for a guy named Hermite, 00:08:03.890 --> 00:08:06.710 who didn't actually pronounce the H, 00:08:06.710 --> 00:08:09.630 but let's pronounce it -- 00:08:09.630 --> 00:08:14.040 so I would call that z Hermitian z. 00:08:14.040 --> 00:08:20.940 I'll -- let me write that word, Herm- so his name was Hermite, 00:08:20.940 --> 00:08:27.240 and then we make it into an adjective, Hermitian. 00:08:27.240 --> 00:08:32.000 So z Hermitian z. z H z. 00:08:32.000 --> 00:08:34.360 Okay. 00:08:34.360 --> 00:08:37.510 So, that's the -- 00:08:37.510 --> 00:08:40.730 that's the, length squared. 00:08:40.730 --> 00:08:43.409 Now what's the inner product? 00:08:43.409 --> 00:08:45.000 Well, it should match. 00:08:45.000 --> 00:08:49.850 The inner product of two vectors -- 00:08:49.850 --> 00:08:56.060 so inner product is no longer -- 00:08:56.060 --> 00:08:59.230 used to be y transpose x. 00:08:59.230 --> 00:09:01.680 That's for real vectors. 00:09:01.680 --> 00:09:04.970 For complex vectors, whenever we transpose, 00:09:04.970 --> 00:09:07.220 we also take the conjugate. 00:09:07.220 --> 00:09:12.110 So it's y Hermitian x. 00:09:12.110 --> 00:09:14.160 Of course it's not real anymore, usually. 00:09:14.160 --> 00:09:19.610 That -- the inner product will usually be complex number. 00:09:19.610 --> 00:09:24.100 But if y and x are the same, if they're the same z, 00:09:24.100 --> 00:09:29.240 then we have z -- z H z, we have the length squared, 00:09:29.240 --> 00:09:31.950 and that's what we want, the inner product of a vector with 00:09:31.950 --> 00:09:35.490 itself should be its length squared. 00:09:35.490 --> 00:09:40.230 So this is, like, forced on us because this is forced on us. 00:09:40.230 --> 00:09:43.750 So -- so this z -- this -- 00:09:43.750 --> 00:09:45.660 everybody's picking up what this equals. 00:09:45.660 --> 00:09:52.170 This is z1 squared plus zn squared. 00:09:52.170 --> 00:09:53.820 That's the length squared. 00:09:53.820 --> 00:09:57.080 And that's the inner product that we have to go with. 00:09:57.080 --> 00:10:00.560 So it could be a complex number now. 00:10:00.560 --> 00:10:02.790 One more change. 00:10:02.790 --> 00:10:04.750 Well, two more changes. 00:10:04.750 --> 00:10:08.780 We've got to change the idea of a symmetric matrix. 00:10:08.780 --> 00:10:11.870 So I'll just recap on symmetric matrixes. 00:10:11.870 --> 00:10:21.610 Symmetric means A transpose equals A, but not -- 00:10:21.610 --> 00:10:30.430 no good if A is complex. 00:10:30.430 --> 00:10:33.890 So what do we instead -- 00:10:37.490 --> 00:10:40.360 that applies perfectly to real matrixes. 00:10:40.360 --> 00:10:42.760 But now if my matrixes were complex, 00:10:42.760 --> 00:10:51.910 I want to take the transpose and the conjugate to equal A. 00:10:51.910 --> 00:10:55.430 So there's -- that's the -- the right complex version 00:10:55.430 --> 00:10:56.780 of symmetry. 00:10:56.780 --> 00:10:59.470 The com- the symmetry now means when 00:10:59.470 --> 00:11:03.180 I transpose it, flip across the diagonal and take conjugates. 00:11:03.180 --> 00:11:07.250 So, for example -- here would be an example. 00:11:07.250 --> 00:11:09.480 On the diagonal, it had better be real, 00:11:09.480 --> 00:11:15.760 because when I flip it, the diagonal is still there and it 00:11:15.760 --> 00:11:18.410 has to -- and then when I take the complex conjugate it has 00:11:18.410 --> 00:11:21.240 to be still there, so it better be a real number, 00:11:21.240 --> 00:11:24.580 let me say two and five. 00:11:24.580 --> 00:11:27.790 What about entries off the diagonal? 00:11:27.790 --> 00:11:32.200 If this entry is, say, three plus i, 00:11:32.200 --> 00:11:36.630 then this entry had better be -- 00:11:36.630 --> 00:11:40.070 because I want whatever this -- when I transpose, 00:11:40.070 --> 00:11:42.790 it'll show up here and i conjugate. 00:11:42.790 --> 00:11:47.000 So I need three minus I there. 00:11:47.000 --> 00:11:51.400 So there's a matrix with -- 00:11:51.400 --> 00:11:55.640 that corresponds to symmetry, but it's complex. 00:11:55.640 --> 00:12:01.550 And those matrixes are called Hermitian matrixes. 00:12:01.550 --> 00:12:03.170 Hermitian matrixes. 00:12:03.170 --> 00:12:06.210 A H equals A. 00:12:06.210 --> 00:12:08.790 Fine. 00:12:08.790 --> 00:12:13.140 Okay, that's -- and those matrixes have real eigenvalues 00:12:13.140 --> 00:12:15.950 and they have perpendicular eigenvectors. 00:12:15.950 --> 00:12:18.760 What does perpendicular mean? 00:12:18.760 --> 00:12:21.760 Perpendicular means the inner product -- 00:12:21.760 --> 00:12:23.585 so let's go on to perpendicular. 00:12:28.180 --> 00:12:31.410 Well, when I had perpendicular vectors, for example, 00:12:31.410 --> 00:12:37.820 they were like q1, q2 up to qn. 00:12:37.820 --> 00:12:41.630 That's my -- q is my letter that I use for perpendicular. 00:12:41.630 --> 00:12:43.190 Actually, I usually -- 00:12:43.190 --> 00:12:45.840 I also mean unit length. 00:12:45.840 --> 00:12:49.010 So those are perpendicular unit vectors. 00:12:49.010 --> 00:12:51.030 But now what does -- so it's a -- 00:12:51.030 --> 00:12:54.730 orthonormal basis, I'll still use those words, 00:12:54.730 --> 00:12:58.560 but how do I compute perpendicular? 00:12:58.560 --> 00:13:00.770 How do I check perpendicular? 00:13:00.770 --> 00:13:07.370 This means that the inner product of qi with qj -- 00:13:07.370 --> 00:13:11.510 but now I not only transpose, I must conjugate, right, 00:13:11.510 --> 00:13:20.210 to get zero if i is not j and one if i is j. 00:13:20.210 --> 00:13:25.950 So it's a unit vector, meaning unit length, orthogonal -- 00:13:25.950 --> 00:13:28.930 all the angles are right angles, but these are angles 00:13:28.930 --> 00:13:33.020 in complex n dimensional space. 00:13:33.020 --> 00:13:37.155 So it's q1, q on- qi bar transpose. 00:13:39.790 --> 00:13:44.310 Or, for short, qi H qj. 00:13:44.310 --> 00:13:46.650 So it will still be true -- so let me -- 00:13:46.650 --> 00:13:51.230 again I'll create a matrix out of those guys. 00:13:51.230 --> 00:13:56.090 The matrix will have these q-s in its columns, q2 to qn. 00:14:00.320 --> 00:14:03.400 And I want to turn that into matrix language, 00:14:03.400 --> 00:14:05.130 just like before. 00:14:05.130 --> 00:14:06.690 What does that mean? 00:14:06.690 --> 00:14:08.800 That means I want all these inner products, 00:14:08.800 --> 00:14:13.920 so I take these columns of Q, multiply by their rows -- 00:14:13.920 --> 00:14:17.250 so it was -- it used to be Q -- it used to be Q transpose Q 00:14:17.250 --> 00:14:18.160 equals I, right? 00:14:20.840 --> 00:14:22.535 This was an orthogonal matrix. 00:14:27.100 --> 00:14:30.490 But what's changed? 00:14:30.490 --> 00:14:33.290 These are now complex vectors. 00:14:33.290 --> 00:14:37.730 Their inner products are -- involve conjugating the first 00:14:37.730 --> 00:14:39.920 factor. 00:14:39.920 --> 00:14:44.240 So it's not -- it's the conjugate of Q transpose. 00:14:44.240 --> 00:14:47.050 It's Q bar transpose Q. 00:14:47.050 --> 00:14:48.590 Q H. 00:14:48.590 --> 00:14:55.240 So can I call this -- let me call it Q H Q, which is I. 00:14:55.240 --> 00:14:57.260 So that's our new -- 00:14:57.260 --> 00:14:59.550 you -- you see I'm just translating, and the -- 00:14:59.550 --> 00:15:07.300 the book h- on one page gives a little dictionary of the right 00:15:07.300 --> 00:15:12.900 words in the real case, R^n, and the corresponding words 00:15:12.900 --> 00:15:17.120 in the complex case for the vector space C^n. 00:15:17.120 --> 00:15:19.600 Of course, C^n is a vector space, 00:15:19.600 --> 00:15:22.650 the numbers we multiply are now complex numbers -- 00:15:22.650 --> 00:15:27.770 we're just moving into complex n dimensional space. 00:15:27.770 --> 00:15:29.700 Okay. 00:15:29.700 --> 00:15:34.450 Now -- actually, I have to say we changed the word symmet- 00:15:34.450 --> 00:15:38.500 symmetric to Hermitian for those matrixes. 00:15:38.500 --> 00:15:43.300 People also change this word orthogonal into another word 00:15:43.300 --> 00:15:52.730 that happens to be unitary, as a word that applies -- 00:15:52.730 --> 00:15:56.540 that signals that we might be dealing with a complex matrix 00:15:56.540 --> 00:15:57.320 here. 00:15:57.320 --> 00:15:59.940 So what's a unitary matrix? 00:15:59.940 --> 00:16:02.960 It's a -- it's just like an orthogonal matrix. 00:16:02.960 --> 00:16:10.800 It's a square, n by n matrix with orthonormal columns, 00:16:10.800 --> 00:16:13.970 perpendicular columns, unit vectors -- 00:16:13.970 --> 00:16:20.010 unit vectors computed by -- and perpendicularity computed 00:16:20.010 --> 00:16:23.820 by remembering that there's a conjugate as well 00:16:23.820 --> 00:16:25.250 as a transpose. 00:16:25.250 --> 00:16:26.270 Okay. 00:16:26.270 --> 00:16:28.520 So those are the words. 00:16:28.520 --> 00:16:32.880 Now I'm ready to get into the substance of the lecture which 00:16:32.880 --> 00:16:38.290 is the most famous complex matrix, which happens 00:16:38.290 --> 00:16:41.500 to be one of these guys. 00:16:41.500 --> 00:16:50.240 It has orthogonal columns, and it's named after Fourier 00:16:50.240 --> 00:16:52.800 because it comes into the Fourier transform, 00:16:52.800 --> 00:16:55.380 so it's the matrix that's all around us. 00:16:55.380 --> 00:16:55.880 Okay. 00:16:58.450 --> 00:17:05.660 Let me tell you what it is first of all in the n by n case. 00:17:05.660 --> 00:17:09.750 Then often I'll let n be four because four is a good size 00:17:09.750 --> 00:17:11.150 to work with. 00:17:11.150 --> 00:17:13.369 But here's the n by n Fourier matrix. 00:17:17.790 --> 00:17:20.319 Its first column is the vector of ones. 00:17:24.310 --> 00:17:26.319 It's n by n, of course. 00:17:26.319 --> 00:17:30.420 Its second column is the powers, the -- 00:17:30.420 --> 00:17:35.370 actually, better if I move from the math department 00:17:35.370 --> 00:17:39.910 to EE for this one half hour and then, please, 00:17:39.910 --> 00:17:42.100 let me move back again. 00:17:42.100 --> 00:17:43.870 Okay. 00:17:43.870 --> 00:17:46.170 What's the difference between those two departments? 00:17:46.170 --> 00:17:50.640 It's just math starts counting with one 00:17:50.640 --> 00:17:56.180 and electrical engineers start counting at zero. 00:17:56.180 --> 00:17:57.840 Actually, they're probably right. 00:17:57.840 --> 00:18:00.720 So anyway, we'll give them -- humor them. 00:18:00.720 --> 00:18:05.090 So this is really the zeroes column. 00:18:05.090 --> 00:18:08.030 And the first column up to the n-1, that's 00:18:08.030 --> 00:18:11.780 the one inconvenient spot in electrical engineering. 00:18:11.780 --> 00:18:14.760 All these expressions start at zero, no problem, 00:18:14.760 --> 00:18:16.720 but they end at n-1. 00:18:16.720 --> 00:18:24.000 Well, that's -- that's the difficulty of Course 6. 00:18:24.000 --> 00:18:27.750 So what's -- they're the powers of a number that I'm going 00:18:27.750 --> 00:18:28.660 to call W -- 00:18:28.660 --> 00:18:34.840 W squared, W cubed, W to the -- now what is the W here? 00:18:34.840 --> 00:18:36.060 What's the power? 00:18:36.060 --> 00:18:39.000 This was the zeroes power, first power, second power, 00:18:39.000 --> 00:18:43.290 this will be n minus first power. 00:18:43.290 --> 00:18:45.130 That's the column. 00:18:45.130 --> 00:18:46.500 What's the next column? 00:18:46.500 --> 00:18:51.920 It's the powers of W squared, W to the fourth, W to the sixth, 00:18:51.920 --> 00:18:55.870 W to the two n-1. 00:18:55.870 --> 00:18:58.010 And then more columns and more columns 00:18:58.010 --> 00:19:00.060 and more columns and what's the last column? 00:19:02.790 --> 00:19:07.361 It's the powers of -- 00:19:07.361 --> 00:19:07.860 let's see. 00:19:07.860 --> 00:19:10.860 We -- actually, if we look around rows, w- 00:19:10.860 --> 00:19:12.520 this matrix is symmetric. 00:19:12.520 --> 00:19:17.220 It's symmetric in the old not quite perfect way, 00:19:17.220 --> 00:19:21.100 not perfect because these numbers are complex. 00:19:21.100 --> 00:19:25.370 And so it's -- that first row is all ones. 00:19:25.370 --> 00:19:28.740 One W W squared up to W to the n-1. 00:19:28.740 --> 00:19:34.500 That's the last column is the powers of W to the n-1, 00:19:34.500 --> 00:19:38.841 so this guy matches that, and finally we get W to something 00:19:38.841 --> 00:19:39.340 here. 00:19:43.060 --> 00:19:46.670 I guess we could actually figure out what that something is. 00:19:46.670 --> 00:19:48.610 What are the entries of this matrix? 00:19:48.610 --> 00:19:57.490 The i j entry of this matrix are -- 00:19:57.490 --> 00:20:00.420 I going to -- are you going to allow me to let i go from zero 00:20:00.420 --> 00:20:01.890 to n minus one? 00:20:01.890 --> 00:20:07.180 So i and g go from zero to n-1. 00:20:07.180 --> 00:20:10.870 So the one -- the zero zero entry is a one -- 00:20:10.870 --> 00:20:15.670 it's just this same W guy to the power i times j. 00:20:21.960 --> 00:20:22.460 Let's see. 00:20:22.460 --> 00:20:24.690 I'm jumping into formulas here and I have to tell you 00:20:24.690 --> 00:20:27.903 what W is and then you know everything about this matrix. 00:20:30.960 --> 00:20:34.520 So W is the -- well, shall we finish here? 00:20:34.520 --> 00:20:37.890 What was this -- this is the (n-1) (n-1) entry. 00:20:37.890 --> 00:20:41.380 This is W to the n-1 squared. 00:20:41.380 --> 00:20:46.140 Everything's looking like a mess here, because we have -- 00:20:46.140 --> 00:20:51.890 not too bad, because all the entries are powers of W. 00:20:51.890 --> 00:20:53.830 There -- none of them are zero. 00:20:53.830 --> 00:20:56.600 This is a full matrix. 00:20:56.600 --> 00:20:59.840 But W is a very special number. 00:20:59.840 --> 00:21:08.720 W is the special number whose n-th power is one. 00:21:08.720 --> 00:21:12.780 In fact -- well, actually, there are n numbers like that. 00:21:12.780 --> 00:21:14.620 One of them is one, of course. 00:21:14.620 --> 00:21:19.850 But the one we -- the W we want is -- 00:21:19.850 --> 00:21:23.470 the angle is two pi over n. 00:21:28.060 --> 00:21:29.130 Is that what I mean? 00:21:29.130 --> 00:21:32.740 n over two pi. 00:21:32.740 --> 00:21:34.310 No, two pi over n. 00:21:34.310 --> 00:21:42.540 W is E to the I and the angle is two pi over n. 00:21:42.540 --> 00:21:43.550 Right. 00:21:43.550 --> 00:21:47.590 Where is this W in the complex plane? 00:21:47.590 --> 00:21:49.870 It's -- it's on the unit circle, right? 00:21:49.870 --> 00:21:56.160 This is -- it's the cosine of two pi over n plus I times 00:21:56.160 --> 00:21:59.620 the sine of two pi over n. 00:21:59.620 --> 00:22:02.840 But actually, forget this. 00:22:02.840 --> 00:22:09.050 It's never good to work with the real and imaginary parts, 00:22:09.050 --> 00:22:14.130 the rectangular coordinates, when we're taking powers. 00:22:14.130 --> 00:22:17.770 To take that to the tenth power, we can't see what we're doing. 00:22:17.770 --> 00:22:19.890 To take this form to the tenth power, 00:22:19.890 --> 00:22:22.350 we see immediately what we're doing. 00:22:22.350 --> 00:22:26.480 It would be e to the i 20 pi over n. 00:22:26.480 --> 00:22:30.570 So when our matrix is full of powers -- so it's this formula, 00:22:30.570 --> 00:22:32.690 and where is this on the complex plain? 00:22:32.690 --> 00:22:36.830 Here are the real numbers, here's the imaginary axis, 00:22:36.830 --> 00:22:40.410 here's the unit circle of radius one, 00:22:40.410 --> 00:22:44.240 and this number is on the unit circle 00:22:44.240 --> 00:22:48.860 at this angle, which is one n-th of the full way round. 00:22:48.860 --> 00:22:52.700 So if I drew, for example, n equals six, 00:22:52.700 --> 00:22:56.490 this would be e to the two pi, two pi over six, 00:22:56.490 --> 00:22:59.160 it would be one sixth of the way around, it would be 60 degrees. 00:23:03.250 --> 00:23:06.090 And where is W squared? 00:23:06.090 --> 00:23:13.120 So I -- my W is e to the two pi I over six in this case, 00:23:13.120 --> 00:23:16.760 in this six by -- for the six by six Fourier transform, 00:23:16.760 --> 00:23:22.800 it's totally constructed out of this number and its powers. 00:23:22.800 --> 00:23:24.480 So what are its powers? 00:23:24.480 --> 00:23:27.470 Well, its powers are on the unit circle, right? 00:23:27.470 --> 00:23:32.680 Because when I square a number, a complex number, 00:23:32.680 --> 00:23:37.740 I square its absolute value, which gives me one again. 00:23:37.740 --> 00:23:40.500 All the powers have -- are on the unit circle. 00:23:40.500 --> 00:23:43.250 And the -- the angle gets doubled to a hundred 00:23:43.250 --> 00:23:46.180 and twenty, so there's W squared, there's W cubed, 00:23:46.180 --> 00:23:50.660 there's W to the fourth, there's W to the fifth and there is W 00:23:50.660 --> 00:23:57.070 to the sixth, as we hoped, W to the sixth coming back to one. 00:23:57.070 --> 00:24:01.080 So those are the six -- 00:24:01.080 --> 00:24:02.620 can I say this on TV? 00:24:02.620 --> 00:24:07.130 The six sixth roots of one, and it's 00:24:07.130 --> 00:24:14.560 this one, the primitive one we say, the first one, which is W. 00:24:14.560 --> 00:24:17.840 Okay, so what -- let me change -- let me -- 00:24:17.840 --> 00:24:21.440 I said I would probably switch to n equal four. 00:24:21.440 --> 00:24:24.130 What's W for that? 00:24:24.130 --> 00:24:25.970 It's the fourth root of one. 00:24:25.970 --> 00:24:28.590 W to the fourth will be one. 00:24:28.590 --> 00:24:33.410 W will be e to the two pi i over four now. 00:24:38.820 --> 00:24:41.040 What's that? 00:24:41.040 --> 00:24:43.270 This is e to the i pi over two. 00:24:43.270 --> 00:24:46.180 This is a quarter of the way around the unit circle, 00:24:46.180 --> 00:24:53.930 and that's exactly i, a quarter of the way around. 00:24:53.930 --> 00:24:58.660 And sure enough, the powers are i, i squared, 00:24:58.660 --> 00:25:01.670 which is minus one, i cubed, which 00:25:01.670 --> 00:25:08.780 is minus i and finally i to the fourth which is one, right. 00:25:08.780 --> 00:25:12.760 So there's W, W squared, W cubed, W to the fourth -- 00:25:12.760 --> 00:25:15.050 I'm really ready to write down this Fourier 00:25:15.050 --> 00:25:18.780 matrix for the four by four case, 00:25:18.780 --> 00:25:21.790 just so we see that clearly. 00:25:21.790 --> 00:25:22.890 Let me do it here. 00:25:22.890 --> 00:25:34.760 F4 is -- all right, one one one one one one one W -- it's I. 00:25:34.760 --> 00:25:35.590 I squared. 00:25:35.590 --> 00:25:36.770 That's minus one. 00:25:36.770 --> 00:25:38.520 i cubed is minus i. 00:25:41.350 --> 00:25:43.490 I'll -- I could write i squared and i cubed. 00:25:43.490 --> 00:25:46.140 Why don't I, just so we see the pattern for sure. 00:25:46.140 --> 00:25:54.040 i squared, i cubed, i squared, i cubed, i fourth, i sixth -- 00:25:54.040 --> 00:25:58.690 i fourth, i sixth and i ninth. 00:25:58.690 --> 00:26:01.970 You see the exponents fall in this nice -- 00:26:01.970 --> 00:26:05.760 the exponent is the row number times the column number, 00:26:05.760 --> 00:26:08.770 always starting at zero. 00:26:08.770 --> 00:26:09.560 Okay. 00:26:09.560 --> 00:26:12.910 And now I can put in those numbers if you like -- 00:26:12.910 --> 00:26:20.520 one one one one, one i minus one minus i, one minus one, 00:26:20.520 --> 00:26:28.710 one minus one and one minus i minus one i. 00:26:28.710 --> 00:26:29.410 No. 00:26:29.410 --> 00:26:31.111 Yes. 00:26:31.111 --> 00:26:31.610 Right. 00:26:38.260 --> 00:26:40.965 What's -- why do I think that matrix is so remarkable? 00:26:44.520 --> 00:26:49.030 It's the four by four matrix that comes into the four point 00:26:49.030 --> 00:26:52.020 Fourier transform. 00:26:52.020 --> 00:26:55.700 When we want to find the Fourier transform, the four point 00:26:55.700 --> 00:27:01.380 Fourier transform of a vector with four components, 00:27:01.380 --> 00:27:05.540 we want to multiply by this F4 or we 00:27:05.540 --> 00:27:08.920 want to multiply by F4 inverse. 00:27:08.920 --> 00:27:10.840 One way we're taking the transform, 00:27:10.840 --> 00:27:13.820 one way we're taking the inverse transform. 00:27:13.820 --> 00:27:17.950 Actually, they're so close that it's easy to confuse the two. 00:27:17.950 --> 00:27:21.990 The inverse of this matrix will be a nice matrix also. 00:27:25.250 --> 00:27:29.190 So -- and that's, of course, what makes it -- 00:27:29.190 --> 00:27:32.370 that -- I guess Fourier knew that. 00:27:32.370 --> 00:27:34.200 He knew the inverse of this matrix. 00:27:36.970 --> 00:27:39.890 A- as you'll see, it just comes from the fact that the columns 00:27:39.890 --> 00:27:41.640 are orthogonal -- 00:27:41.640 --> 00:27:44.050 from the fact that the columns are orthogonal, 00:27:44.050 --> 00:27:51.220 we will quickly figure out what is the inverse. 00:27:51.220 --> 00:27:53.910 What Fourier didn't know -- didn't notice -- 00:27:53.910 --> 00:27:58.180 I think Gauss noticed it but didn't make a point of it 00:27:58.180 --> 00:28:00.020 and then it turned out to be really 00:28:00.020 --> 00:28:04.770 important was the fact that this matrix is so special that you 00:28:04.770 --> 00:28:10.340 can break it up into nice pieces with lots of zeroes, factors 00:28:10.340 --> 00:28:13.360 that have lots of zeroes and multiply 00:28:13.360 --> 00:28:16.650 by it or by its inverse very, very fast. 00:28:16.650 --> 00:28:17.150 Okay. 00:28:19.740 --> 00:28:22.850 But how did it get into this lecture first? 00:28:22.850 --> 00:28:26.340 Because the columns are orthogonal. 00:28:26.340 --> 00:28:29.130 Can I just check that the columns of this matrix 00:28:29.130 --> 00:28:30.265 are orthogonal? 00:28:33.480 --> 00:28:38.360 So the inner product of that column with that column 00:28:38.360 --> 00:28:41.260 is zero. 00:28:41.260 --> 00:28:48.250 The inner product of column one with column three is zero. 00:28:48.250 --> 00:28:55.850 How about the inner product of two and four? 00:28:55.850 --> 00:29:01.280 Can I take the inner product of column two with column four? 00:29:01.280 --> 00:29:04.860 Or even the inner product of two with three, let's -- let's see, 00:29:04.860 --> 00:29:06.820 does that -- 00:29:06.820 --> 00:29:07.751 let me do two and 00:29:07.751 --> 00:29:08.250 four. 00:29:15.330 --> 00:29:17.260 Okay. 00:29:17.260 --> 00:29:19.110 What -- oh, I see, yes, hmm. 00:29:19.110 --> 00:29:19.610 Hmm. 00:29:19.610 --> 00:29:30.060 Let's see, I believe that those two columns are orthogonal. 00:29:30.060 --> 00:29:33.520 So let me take their inner product and hope to get zero. 00:29:33.520 --> 00:29:36.310 Okay, now if you hadn't listened to the first half 00:29:36.310 --> 00:29:39.890 of this lecture, when you took the inner product of that 00:29:39.890 --> 00:29:42.760 with that, you would have multiplied one by one, 00:29:42.760 --> 00:29:48.090 i by minus i, and that would have given you one, 00:29:48.090 --> 00:29:51.370 minus one by minus one would have given you 00:29:51.370 --> 00:29:55.940 another one minus I by I would have been minus I squared, 00:29:55.940 --> 00:29:56.810 that's another one. 00:30:00.870 --> 00:30:04.240 So do I conclude that the inner product of columns -- 00:30:04.240 --> 00:30:05.920 I said columns two and four, that's 00:30:05.920 --> 00:30:08.550 because I forgot those are columns one and three. 00:30:12.110 --> 00:30:13.670 I'm interested in their inner product 00:30:13.670 --> 00:30:17.570 and I'm hoping it's zero, but it doesn't look like zero. 00:30:17.570 --> 00:30:19.090 Nevertheless, it is zero. 00:30:19.090 --> 00:30:20.580 Those columns are perpendicular. 00:30:20.580 --> 00:30:21.500 Why? 00:30:21.500 --> 00:30:25.390 Because the inner product -- 00:30:25.390 --> 00:30:26.520 we conjugate. 00:30:26.520 --> 00:30:29.150 Do you remember that the -- one of the vectors in the inner 00:30:29.150 --> 00:30:31.510 product has to get conjugated. 00:30:31.510 --> 00:30:34.750 So when I conjugated, it changes that i to a minus i, 00:30:34.750 --> 00:30:37.900 changes this to a plus i, changes those -- 00:30:37.900 --> 00:30:44.060 that second sine and that fourth sine and I do get zero. 00:30:44.060 --> 00:30:46.490 So those columns are orthogonal. 00:30:46.490 --> 00:30:49.130 So columns are orthogonal. 00:30:51.780 --> 00:30:55.630 They're not quite orthonormal. 00:30:55.630 --> 00:30:58.020 But I could fix that easily. 00:30:58.020 --> 00:31:02.710 They -- all those columns have length two. 00:31:02.710 --> 00:31:07.450 Length squared is four, like this -- the four I had there -- 00:31:07.450 --> 00:31:10.260 this length squared, one plus -- one squared one squared one 00:31:10.260 --> 00:31:13.910 squared one squared is four, square root is two -- 00:31:13.910 --> 00:31:17.420 so if I really wanted them -- suppose I really wanted to fix 00:31:17.420 --> 00:31:21.300 life perfectly, I could divide by two, 00:31:21.300 --> 00:31:26.653 and now I have columns that are actually orthonormal. 00:31:33.420 --> 00:31:35.560 So what? 00:31:35.560 --> 00:31:38.040 So I can invert right away, right? 00:31:38.040 --> 00:31:41.970 O- orthonormal columns means -- now I'm keeping this one half 00:31:41.970 --> 00:31:43.820 in here for the moment -- 00:31:43.820 --> 00:31:48.060 c- means F4 Hermitian, can I use that, 00:31:48.060 --> 00:31:51.620 conjugate transpose times F4 is 00:31:51.620 --> 00:31:52.120 i. 00:31:56.230 --> 00:31:59.150 So I see what the inverse is. 00:31:59.150 --> 00:32:01.840 The inverse of F4 is -- it's just like an -- 00:32:01.840 --> 00:32:03.180 an orthogonal matrix. 00:32:03.180 --> 00:32:06.230 The inverse is the transpose -- here the inverse is 00:32:06.230 --> 00:32:08.990 the conjugate transpose. 00:32:08.990 --> 00:32:10.250 So, fine. 00:32:10.250 --> 00:32:15.000 That -- that tells me that anything good that I learn 00:32:15.000 --> 00:32:19.610 about F4 I'll know the same -- 00:32:19.610 --> 00:32:21.710 I'll know a similar fact about its inverse, 00:32:21.710 --> 00:32:25.620 because its inverse is just its conjugate transpose. 00:32:25.620 --> 00:32:28.080 Okay, now -- so what's good? 00:32:28.080 --> 00:32:30.430 Well, first, the columns are orthogonal. 00:32:30.430 --> 00:32:33.870 That's a key fact, then. 00:32:33.870 --> 00:32:36.670 That's the thing that makes the inverse easy. 00:32:36.670 --> 00:32:40.740 But what property is it that leads to the fast Fourier 00:32:40.740 --> 00:32:41.460 transform? 00:32:41.460 --> 00:32:44.000 So now I'm going to talk, in these last minutes, 00:32:44.000 --> 00:32:48.010 about the fast Fourier transform. 00:32:48.010 --> 00:32:49.290 What -- here's the idea. 00:32:51.980 --> 00:32:56.030 F6, our six by six matrix, will c- 00:32:56.030 --> 00:33:02.250 there's a neat connection to F3, half as big. 00:33:02.250 --> 00:33:05.200 There's a connection of F8 to F4. 00:33:05.200 --> 00:33:09.690 There's a connection of F(64) to F(32). 00:33:09.690 --> 00:33:12.070 Shall I write down what that connection is? 00:33:12.070 --> 00:33:15.190 What's the connection of F(64) to F(32)? 00:33:15.190 --> 00:33:22.950 So F(64) is a 64 by 64 matrix whose W 00:33:22.950 --> 00:33:26.500 is the 64th root of one. 00:33:26.500 --> 00:33:31.870 So it's one 64th of the way round in F(64). 00:33:31.870 --> 00:33:35.959 And it -- do- and F(32) is a 32 by 32 matrix. 00:33:35.959 --> 00:33:37.375 Remember, they're different sizes. 00:33:40.450 --> 00:33:45.680 And the W in that 32 by 32 matrix is the 32nd root of one, 00:33:45.680 --> 00:33:50.220 which is twice as far -- that -- you sh- see that key point -- 00:33:50.220 --> 00:33:55.850 that's the -- that's how 32 and 64 are connected in the Ws. 00:33:55.850 --> 00:33:59.380 The W for 64 is one 64th of the way -- 00:33:59.380 --> 00:34:05.610 so all I'm saying is that if I square the W -- 00:34:05.610 --> 00:34:09.219 W(64), that's what I'm using for the one over -- the -- 00:34:09.219 --> 00:34:15.070 W sixty f- this Wn is either the i two pi over n -- 00:34:15.070 --> 00:34:18.690 so W(64) is one 64th of the way around it. 00:34:18.690 --> 00:34:24.139 When I square that, what do I get but W(32)? 00:34:24.139 --> 00:34:25.969 Right? 00:34:25.969 --> 00:34:31.060 If I square this matrix, I double the angle -- 00:34:31.060 --> 00:34:36.300 if I square this number, I double the angle, I get, the -- 00:34:36.300 --> 00:34:40.120 the W(32). 00:34:40.120 --> 00:34:43.239 So somehow there's a little hope here 00:34:43.239 --> 00:34:46.440 to connect F(64) with F(32). 00:34:46.440 --> 00:34:49.400 And here's the connection. 00:34:49.400 --> 00:34:49.980 Okay. 00:34:49.980 --> 00:34:53.530 Let me -- let me go back, -- 00:34:53.530 --> 00:34:54.170 yes, let me -- 00:34:54.170 --> 00:34:58.780 I'll do it here. 00:34:58.780 --> 00:35:00.520 Here's the connection. 00:35:00.520 --> 00:35:01.780 F(64). 00:35:01.780 --> 00:35:06.530 The 64 by 64 Fourier matrix is connected 00:35:06.530 --> 00:35:10.070 to two copies of F(32). 00:35:10.070 --> 00:35:13.120 Let me leave a little space for the connection. 00:35:13.120 --> 00:35:15.990 So this is 64 by 64. 00:35:15.990 --> 00:35:18.980 Here's a matrix of that same size, 00:35:18.980 --> 00:35:25.640 because it's got two copies of F(32) and two zero matrixes. 00:35:25.640 --> 00:35:31.500 Those zero matrixes are the key, because when I multiply by this 00:35:31.500 --> 00:35:35.570 matrix, just as it is, regular multiplication, I would take -- 00:35:35.570 --> 00:35:37.160 need 64 -- 00:35:37.160 --> 00:35:40.320 I would -- I would have 64 squared little multiplications 00:35:40.320 --> 00:35:41.470 to do. 00:35:41.470 --> 00:35:45.000 But this matrix is half zero. 00:35:45.000 --> 00:35:46.600 Well, of course, the two aren't equal. 00:35:46.600 --> 00:35:52.010 I'm going to put an equals sign, but there has to be some fix up 00:35:52.010 --> 00:35:53.570 factors -- 00:35:53.570 --> 00:35:57.640 one there and one there -- 00:35:57.640 --> 00:36:00.620 to make it true. 00:36:00.620 --> 00:36:05.260 The beauty is that these fix up factors will be really -- 00:36:05.260 --> 00:36:07.550 almost all zeroes. 00:36:07.550 --> 00:36:11.470 So that as soon as we get this formula right, 00:36:11.470 --> 00:36:15.670 we've got a great idea for how to get from the sixty- 00:36:15.670 --> 00:36:21.180 from the 64 squared calculations -- so this original -- 00:36:21.180 --> 00:36:25.110 originally we have 64 squared calculations from there, 00:36:25.110 --> 00:36:29.570 but this one will give us -- this is -- this will -- 00:36:29.570 --> 00:36:34.660 we don't need that many -- we only need two times 32 squared, 00:36:34.660 --> 00:36:38.120 because we've got that twice. 00:36:38.120 --> 00:36:42.440 And -- plus the fix-up. 00:36:42.440 --> 00:36:47.570 So I have to tell you what's in this fix-up matrix. 00:36:47.570 --> 00:36:50.370 The one on the right is actually a permutation matrix, 00:36:50.370 --> 00:36:55.820 a very simple odds and evens permutation matrix, the -- 00:36:55.820 --> 00:36:58.260 ones show up -- 00:36:58.260 --> 00:37:00.820 I haven't put enough ones, I really need a -- 00:37:00.820 --> 00:37:05.106 32 of these guys at -- double space and then -- 00:37:09.210 --> 00:37:12.400 you see it's -- it's a permutation matrix. 00:37:12.400 --> 00:37:14.070 What it does -- 00:37:14.070 --> 00:37:18.430 shall I call it P for permutation matrix? 00:37:18.430 --> 00:37:23.920 So what that P does when it multiplies a vector, 00:37:23.920 --> 00:37:28.770 it takes the odd -- the even numbered components first 00:37:28.770 --> 00:37:30.580 and then the odds. 00:37:30.580 --> 00:37:33.640 You see this -- this one skipping every time is going 00:37:33.640 --> 00:37:40.260 to pick out x0, x2, x4, x6 and then below that will come -- 00:37:40.260 --> 00:37:42.335 will pick out x1, x3, x5. 00:37:45.850 --> 00:37:49.650 And of course, that can be hard wired in the computer 00:37:49.650 --> 00:37:52.350 to be instantaneous. 00:37:52.350 --> 00:37:56.570 So that says -- so far, what have we said? 00:37:56.570 --> 00:38:00.960 We're saying that the 64 by 64 Fourier matrix is really 00:38:00.960 --> 00:38:04.540 separated into -- separate your vector into the odd -- 00:38:04.540 --> 00:38:07.430 into the even components and the odd components, 00:38:07.430 --> 00:38:12.820 then do a 32 size Fourier transform onto those 00:38:12.820 --> 00:38:17.420 separately, and then put the pieces together again. 00:38:17.420 --> 00:38:21.890 So the pieces -- putting them together turns them out to be I 00:38:21.890 --> 00:38:25.350 and a diagonal matrix and I and a minus, 00:38:25.350 --> 00:38:26.615 that same diagonal matrix. 00:38:29.330 --> 00:38:33.540 So the fix-up cost is really the cost of multiplying by D, 00:38:33.540 --> 00:38:38.120 this diagonal matrix, because there's essentially no cost 00:38:38.120 --> 00:38:39.020 in -- 00:38:39.020 --> 00:38:42.630 in the I part or in the permutation part, 00:38:42.630 --> 00:38:50.720 so really it's -- the fix-up cost is essentially because D 00:38:50.720 --> 00:38:52.950 is diagonal -- 00:38:52.950 --> 00:38:54.160 is 32 multiplications. 00:38:58.930 --> 00:39:00.430 That's the -- there you're seeing -- 00:39:00.430 --> 00:39:02.888 of course we didn't check the formula or we didn't even say 00:39:02.888 --> 00:39:05.030 what D is yet, but I will -- 00:39:05.030 --> 00:39:08.900 this diagonal matrix D is powers of W -- 00:39:08.900 --> 00:39:13.345 one W W squared down to W to the 31st. 00:39:19.720 --> 00:39:22.630 So you see that when I -- to do a multiplication by D, 00:39:22.630 --> 00:39:25.920 I need to do 32 multiplications. 00:39:25.920 --> 00:39:27.780 There they are. 00:39:27.780 --> 00:39:33.030 Then -- but the other, the more serious work is to do the F(32) 00:39:33.030 --> 00:39:34.910 twice on the -- 00:39:34.910 --> 00:39:37.300 separately on the even numbered and odd 00:39:37.300 --> 00:39:41.600 numbered components, so twice 32 squared. 00:39:41.600 --> 00:39:45.970 So 64 squared is gone now. 00:39:45.970 --> 00:39:47.990 And that's the new count. 00:39:50.580 --> 00:39:54.500 Okay, great, but what next? 00:39:54.500 --> 00:39:58.730 So that's -- I -- we now have the key idea -- 00:39:58.730 --> 00:40:01.110 we would have to check the algebra, 00:40:01.110 --> 00:40:06.770 but it's just checking a lot of sums that come out correctly. 00:40:06.770 --> 00:40:10.290 This is right -- the right way to see the fast Fourier 00:40:10.290 --> 00:40:15.010 transform, or one right way to see it. 00:40:15.010 --> 00:40:17.240 Then you've got to see what's the next idea. 00:40:17.240 --> 00:40:21.390 The next idea is to break the 32s down. 00:40:21.390 --> 00:40:23.450 Break those 32s down. 00:40:23.450 --> 00:40:27.430 So we have this factor, and now we have the F(32), 00:40:27.430 --> 00:40:32.630 but that breaks into some guy here -- 00:40:32.630 --> 00:40:36.510 F thirty- F six- F(16) -- 00:40:36.510 --> 00:40:37.610 F(16). 00:40:37.610 --> 00:40:43.320 Each -- each F(32) is breaking into two copies of F(16), 00:40:43.320 --> 00:40:47.950 and then we have a permutation and then the -- 00:40:47.950 --> 00:40:52.210 so this is a -- like, this was a 64 size permutation, 00:40:52.210 --> 00:40:55.410 this is a 32 size permutation -- 00:40:55.410 --> 00:40:57.252 I guess I've got it twice. 00:40:57.252 --> 00:40:57.960 So it's -- I'm -- 00:40:57.960 --> 00:41:01.120 I'm just using the same idea recursively -- 00:41:01.120 --> 00:41:05.270 recursion is the key word -- that on each of those F(32)s -- 00:41:05.270 --> 00:41:09.465 so here's zero zero -- it's just -- to get F(32) -- 00:41:13.810 --> 00:41:16.980 this is the odd even permutations -- 00:41:16.980 --> 00:41:20.580 so you see, we're -- the combination of those 00:41:20.580 --> 00:41:24.210 permutations, what's it doing? 00:41:24.210 --> 00:41:28.070 This guy separates into odds -- in -- into evens and odds, 00:41:28.070 --> 00:41:34.160 and then this guy separates the evens into the ones -- 00:41:34.160 --> 00:41:37.430 the numbers that are mult- the even evens, 00:41:37.430 --> 00:41:41.080 which means zero four eight sixteen -- 00:41:41.080 --> 00:41:48.580 and even odds, which means two, six, ten, fourteen -- 00:41:48.580 --> 00:41:52.240 and then odd evens and odd odds. 00:41:52.240 --> 00:41:55.930 You see, together these permutations then break it -- 00:41:55.930 --> 00:42:00.630 break our vector down into x, even even and three other 00:42:00.630 --> 00:42:02.090 pieces. 00:42:02.090 --> 00:42:05.070 Those are the four pieces that separately get multiplied 00:42:05.070 --> 00:42:06.650 by F(16) -- 00:42:06.650 --> 00:42:12.808 separately fixed up by these Is and Ds and Is and minus Ds -- 00:42:15.680 --> 00:42:19.340 so this count is now reduced. 00:42:19.340 --> 00:42:23.220 This count is now -- what's it reduced to? 00:42:23.220 --> 00:42:28.490 So that's going to be gone, because 32 squared -- that's -- 00:42:28.490 --> 00:42:30.680 that's the change I'm making, right? 00:42:30.680 --> 00:42:35.130 The 32 squared -- w- so -- so it's this that's now reduced. 00:42:35.130 --> 00:42:40.150 So I still have two times it, but now what's 32 squared? 00:42:40.150 --> 00:42:47.710 It's gone in favor of two sixteen squareds plus sixteen. 00:42:47.710 --> 00:42:54.055 That's -- and then the original 32 to fix. 00:43:00.170 --> 00:43:01.910 Maybe you see what's happening. 00:43:01.910 --> 00:43:04.580 Even easier than this formula is w- what's -- 00:43:04.580 --> 00:43:09.410 when I do the recursion more and more times, 00:43:09.410 --> 00:43:12.670 I get simpler and simpler factors in the middle. 00:43:12.670 --> 00:43:15.330 Eventually I'll be down to two point or one point 00:43:15.330 --> 00:43:18.320 Fourier transforms. 00:43:18.320 --> 00:43:21.170 But I get more and more factors piling up on the right 00:43:21.170 --> 00:43:21.720 and left. 00:43:21.720 --> 00:43:23.650 On the right, I'm just getting permutation 00:43:23.650 --> 00:43:24.450 matrixes. 00:43:24.450 --> 00:43:29.300 On the left, I'm getting these guys, these Is and Ds, 00:43:29.300 --> 00:43:32.490 so that there was a 32 there and -- 00:43:32.490 --> 00:43:35.320 each one of these is costing 32. 00:43:35.320 --> 00:43:39.330 Each one of those is costing And how many will there be? 00:43:42.450 --> 00:43:45.080 So you see the 32 for this original fix 00:43:45.080 --> 00:43:49.850 up, because D had 32 numbers, 32 for this next fix 00:43:49.850 --> 00:43:54.750 up, because D has 16 and 16 more. 00:43:54.750 --> 00:43:56.920 I keep going. 00:43:56.920 --> 00:44:00.320 So the count in the middle goes down to zip, 00:44:00.320 --> 00:44:04.080 but these fix up counts are all that I'm left with, 00:44:04.080 --> 00:44:10.820 and how many factors -- how many fix-ups have I got -- log in -- 00:44:10.820 --> 00:44:15.740 from 64, one step to 32, one step to 16, one step to eight, 00:44:15.740 --> 00:44:17.660 four, two and one. 00:44:17.660 --> 00:44:19.030 Six steps. 00:44:19.030 --> 00:44:20.550 So I have six fix-up -- 00:44:20.550 --> 00:44:23.540 six fix up factors. 00:44:23.540 --> 00:44:33.380 Finally I get to six times the That's my final count. 00:44:33.380 --> 00:44:41.430 Instead of 64 squared, this is log to the base two of 64 times 00:44:41.430 --> 00:44:42.960 64 -- 00:44:42.960 --> 00:44:45.190 actually, half of 64. 00:44:45.190 --> 00:44:51.600 So actually, the final count is n log to the base two of n -- 00:44:51.600 --> 00:44:52.815 that's the 32 -- 00:44:55.380 --> 00:44:57.430 a half. 00:44:57.430 --> 00:45:02.160 So can I put a box around that wonderful, 00:45:02.160 --> 00:45:07.210 extremely important and satisfying conclusion -- 00:45:07.210 --> 00:45:12.400 that the fast Fourier transform multiplies by an n by n matrix, 00:45:12.400 --> 00:45:16.110 but it does it not in n squared steps, but in one 00:45:16.110 --> 00:45:19.100 half n log n steps. 00:45:19.100 --> 00:45:21.130 And if we just -- 00:45:21.130 --> 00:45:25.070 complete by doing a count, let's suppose -- 00:45:25.070 --> 00:45:31.680 suppose -- a typical case would be two to the 00:45:31.680 --> 00:45:32.180 tenth. 00:45:35.540 --> 00:45:39.760 Now n squared is bigger than a million. 00:45:44.560 --> 00:45:47.910 So it's a thousand twenty four times a thousand twenty four. 00:45:47.910 --> 00:45:51.390 But what is n -- what is one half -- 00:45:51.390 --> 00:45:55.690 what is the new count, done the right way? 00:45:55.690 --> 00:46:02.550 It's n -- the thousand twenty four times one half, 00:46:02.550 --> 00:46:04.640 and what's the logarithm? 00:46:04.640 --> 00:46:06.180 It's ten. 00:46:06.180 --> 00:46:08.540 So times ten over two. 00:46:08.540 --> 00:46:12.880 So it's five times -- it's five times a thousand twenty four, 00:46:12.880 --> 00:46:16.690 where this one was a thousand twenty four times a thousand 00:46:16.690 --> 00:46:17.460 twenty four. 00:46:21.450 --> 00:46:25.470 We've reduced the calculation by a factor of 200 just 00:46:25.470 --> 00:46:28.830 by factoring the matrix properly. 00:46:28.830 --> 00:46:34.580 This was a thousand times n, we're now down to five times n. 00:46:34.580 --> 00:46:38.570 So we can do 200 Fourier transforms, 00:46:38.570 --> 00:46:42.160 where before we could do one, and 00:46:42.160 --> 00:46:45.720 in real scientific calculations where Fourier transforms are 00:46:45.720 --> 00:46:48.200 happening all the time, we're saving 00:46:48.200 --> 00:46:52.120 a factor of in one of the major steps 00:46:52.120 --> 00:46:55.960 of modern scientific computing. 00:46:55.960 --> 00:46:58.280 So that's the idea of the fast Fourier transform, 00:46:58.280 --> 00:47:00.020 and you see the whole thing hinged 00:47:00.020 --> 00:47:07.360 on being a special matrix with orthonormal columns. 00:47:07.360 --> 00:47:13.020 Okay, that's actually it for complex numbers. 00:47:13.020 --> 00:47:16.960 I'm back next time really to -- 00:47:16.960 --> 00:47:22.560 to real numbers, eigenvalues and eigenvectors 00:47:22.560 --> 00:47:27.190 and the key idea of positive definite matrixes 00:47:27.190 --> 00:47:28.790 is going to show up. 00:47:28.790 --> 00:47:30.660 What's a positive definite matrix? 00:47:30.660 --> 00:47:32.550 And it's terrific that this course 00:47:32.550 --> 00:47:35.520 is going to reach positive definiteness, 00:47:35.520 --> 00:47:37.840 because those are the matrixes that you 00:47:37.840 --> 00:47:40.320 see the most in applications. 00:47:40.320 --> 00:47:41.965 Okay, see you next time. 00:47:46.240 --> 00:47:47.790 Thanks.