WEBVTT 00:00:00.000 --> 00:00:11.240 -- two, one and -- okay. 00:00:11.240 --> 00:00:18.910 Here is a lecture on the applications of eigenvalues 00:00:18.910 --> 00:00:24.070 and, if I can -- so that will be Markov matrices. 00:00:24.070 --> 00:00:27.260 I'll tell you what a Markov matrix is, 00:00:27.260 --> 00:00:32.070 so this matrix A will be a Markov matrix 00:00:32.070 --> 00:00:38.890 and I'll explain how they come in applications. 00:00:38.890 --> 00:00:42.060 And -- and then if I have time, I would like to say a little 00:00:42.060 --> 00:00:46.060 bit about Fourier series, which is a fantastic application 00:00:46.060 --> 00:00:48.540 of the projection chapter. 00:00:48.540 --> 00:00:49.100 Okay. 00:00:49.100 --> 00:00:50.930 What's a Markov matrix? 00:00:50.930 --> 00:01:01.940 Can I just write down a typical Markov matrix, say .1, .2, .7, 00:01:01.940 --> 00:01:08.820 .01, .99 0, let's say, .3, .3, .4. 00:01:08.820 --> 00:01:09.970 Okay. 00:01:09.970 --> 00:01:13.922 There's a -- a totally just invented Markov matrix. 00:01:16.640 --> 00:01:19.540 What makes it a Markov matrix? 00:01:19.540 --> 00:01:22.510 Two properties that this -- this matrix has. 00:01:22.510 --> 00:01:24.970 So two properties are -- 00:01:24.970 --> 00:01:30.430 one, every entry is greater equal zero. 00:01:30.430 --> 00:01:38.720 All entries greater than or equal to zero. 00:01:38.720 --> 00:01:43.100 And, of course, when I square the matrix, 00:01:43.100 --> 00:01:45.660 the entries will still be greater/equal zero. 00:01:45.660 --> 00:01:50.150 I'm going to be interested in the powers of this matrix. 00:01:50.150 --> 00:01:54.250 And this property, of course, is going to -- stay there. 00:01:54.250 --> 00:01:58.870 It -- really Markov matrices you'll see are connected 00:01:58.870 --> 00:02:04.230 to probability ideas and probabilities are never 00:02:04.230 --> 00:02:05.120 negative. 00:02:05.120 --> 00:02:09.669 The other property -- do you see the other property in there? 00:02:09.669 --> 00:02:13.590 If I add down the columns, what answer do I get? 00:02:13.590 --> 00:02:14.490 One. 00:02:14.490 --> 00:02:17.640 So all columns add to one. 00:02:17.640 --> 00:02:25.380 All columns add to one. 00:02:28.900 --> 00:02:31.580 And actually when I square the matrix, 00:02:31.580 --> 00:02:33.820 that will be true again. 00:02:33.820 --> 00:02:38.920 So that the powers of my matrix are all 00:02:38.920 --> 00:02:42.330 Markov matrices, and I'm interested 00:02:42.330 --> 00:02:46.520 in, always, the eigenvalues and the eigenvectors. 00:02:46.520 --> 00:02:50.780 And this question of steady state will come up. 00:02:50.780 --> 00:02:53.860 You remember we had steady state for differential equations 00:02:53.860 --> 00:02:54.940 last time? 00:02:54.940 --> 00:02:57.080 When -- what was the steady state -- 00:02:57.080 --> 00:02:59.100 what was the eigenvalue? 00:02:59.100 --> 00:03:03.240 What was the eigenvalue in the differential equation case 00:03:03.240 --> 00:03:05.620 that led to a steady state? 00:03:05.620 --> 00:03:08.360 It was lambda equals zero. 00:03:08.360 --> 00:03:11.210 When -- you remember that we did an example and one 00:03:11.210 --> 00:03:15.180 of the eigenvalues was lambda equals zero, and that -- 00:03:15.180 --> 00:03:20.140 so then we had an E to the zero T, a constant one -- 00:03:20.140 --> 00:03:24.430 as time went on, there that thing stayed steady. 00:03:24.430 --> 00:03:30.760 Now what -- in the powers case, it's not a zero eigenvalue. 00:03:30.760 --> 00:03:33.740 Actually with powers of a matrix, a zero eigenvalue, 00:03:33.740 --> 00:03:36.570 that part is going to die right away. 00:03:36.570 --> 00:03:41.710 It's an eigenvalue of one that's all important. 00:03:41.710 --> 00:03:44.540 So this steady state will correspond -- 00:03:44.540 --> 00:03:49.000 will be totally connected with an eigenvalue of one and its 00:03:49.000 --> 00:03:49.910 eigenvector. 00:03:49.910 --> 00:03:53.510 In fact, the steady state will be the eigenvector 00:03:53.510 --> 00:03:55.260 for that eigenvalue. 00:03:55.260 --> 00:03:56.670 Okay. 00:03:56.670 --> 00:03:58.580 So that's what's coming. 00:03:58.580 --> 00:04:04.390 Now, for some reason then that we have to see, 00:04:04.390 --> 00:04:08.210 this matrix has an eigenvalue of one. 00:04:08.210 --> 00:04:12.040 This property, that the columns all add to one -- 00:04:12.040 --> 00:04:16.680 turns out -- guarantees that one is an eigenvalue, 00:04:16.680 --> 00:04:19.600 so that you can actually find the eigenvalue -- 00:04:19.600 --> 00:04:24.330 find that eigenvalue of a Markov matrix without computing any 00:04:24.330 --> 00:04:27.640 determinants of A minus lambda I -- 00:04:27.640 --> 00:04:30.310 that matrix will have an eigenvalue of one, 00:04:30.310 --> 00:04:32.270 and we want to see why. 00:04:32.270 --> 00:04:35.850 And then the other thing is -- 00:04:35.850 --> 00:04:39.190 so the key points -- let me -- let me write these underneath. 00:04:39.190 --> 00:04:45.837 The key points are -- the key points are lambda equal one is 00:04:45.837 --> 00:04:46.420 an eigenvalue. 00:04:53.760 --> 00:04:56.410 I'll add in a little -- an additional -- well, 00:04:56.410 --> 00:04:59.030 a thing about eigenvalues -- 00:04:59.030 --> 00:05:02.400 key point two, the other eigenval- values -- 00:05:02.400 --> 00:05:12.340 all other eigenvalues are, in magnitude, smaller than one -- 00:05:12.340 --> 00:05:14.630 in absolute value, smaller than one. 00:05:14.630 --> 00:05:18.390 Well, there could be some exceptional case when -- 00:05:18.390 --> 00:05:22.380 when an eigen -- another eigenvalue might have magnitude 00:05:22.380 --> 00:05:23.300 equal one. 00:05:23.300 --> 00:05:26.690 It never has an eigenvalue larger than one. 00:05:26.690 --> 00:05:29.230 So these two facts -- somehow we ought to -- 00:05:29.230 --> 00:05:32.600 linear algebra ought to tell us. 00:05:32.600 --> 00:05:35.440 And then, of course, linear algebra is going to tell us 00:05:35.440 --> 00:05:40.040 what the -- what's -- what happens if I take -- if -- 00:05:40.040 --> 00:05:42.720 you remember when I solve -- 00:05:42.720 --> 00:05:47.980 when I multiply by A time after time the K-th thing is A 00:05:47.980 --> 00:05:54.940 to the K u0 and I'm asking what's special about this -- 00:05:54.940 --> 00:06:01.240 these powers of A, and very likely the quiz will have 00:06:01.240 --> 00:06:06.440 a problem to computer s- to computer some powers of A or -- 00:06:06.440 --> 00:06:08.570 or applied to an initial vector. 00:06:08.570 --> 00:06:11.040 So, you remember the general form? 00:06:11.040 --> 00:06:14.180 The general form is that there's some amount 00:06:14.180 --> 00:06:17.660 of the first eigenvalue to the K-th power 00:06:17.660 --> 00:06:22.790 times the first eigenvector, and another amount 00:06:22.790 --> 00:06:25.340 of the second eigenvalue to the K-th power 00:06:25.340 --> 00:06:27.430 times the second eigenvector and so on. 00:06:27.430 --> 00:06:30.700 A -- just -- 00:06:30.700 --> 00:06:34.380 my conscience always makes me say at least once per lecture 00:06:34.380 --> 00:06:40.060 that this requires a complete set of eigenvectors, 00:06:40.060 --> 00:06:43.080 otherwise we might not be able to expand u0 00:06:43.080 --> 00:06:45.840 in the eigenvectors and we couldn't get started. 00:06:45.840 --> 00:06:49.410 But once we're started with u0 when K is zero, 00:06:49.410 --> 00:06:53.880 then every A brings in these lambdas. 00:06:53.880 --> 00:06:56.830 And now you can see what the steady state is going to be. 00:06:56.830 --> 00:07:01.630 If lambda one is one -- so lambda one equals one 00:07:01.630 --> 00:07:07.900 to the K-th power and these other eigenvalues are smaller 00:07:07.900 --> 00:07:10.280 than one -- 00:07:10.280 --> 00:07:14.560 so I've sort of scratched over the equation there to -- 00:07:14.560 --> 00:07:18.260 we had this term, but what happens to this term -- 00:07:18.260 --> 00:07:21.180 if the lambda's smaller than one, then the -- when -- 00:07:21.180 --> 00:07:25.160 as we take powers, as we iterate as we -- 00:07:25.160 --> 00:07:30.662 as we go forward in time, this goes to zero, Can I just -- 00:07:30.662 --> 00:07:32.370 having scratched over it, I might as well 00:07:32.370 --> 00:07:34.050 scratch right? further. 00:07:34.050 --> 00:07:36.880 That term and all the other terms are going to zero 00:07:36.880 --> 00:07:39.750 because all the other eigenvalues are smaller than 00:07:39.750 --> 00:07:45.440 one and the steady state that we're approaching is just -- 00:07:45.440 --> 00:07:48.780 whatever there was -- this was -- this was the -- 00:07:48.780 --> 00:07:56.990 this is the x1 part of un- of the initial condition u0 -- 00:07:56.990 --> 00:07:59.150 is the steady state. 00:08:05.120 --> 00:08:08.350 This much we know from general -- from -- you know, 00:08:08.350 --> 00:08:11.390 what we've already done. 00:08:11.390 --> 00:08:15.740 So I want to see why -- let's at least see number one, 00:08:15.740 --> 00:08:18.160 why one is an eigenvalue. 00:08:18.160 --> 00:08:20.580 And then there's actually -- 00:08:20.580 --> 00:08:23.370 in this chapter we're interested not only in eigenvalues, 00:08:23.370 --> 00:08:25.390 but also eigenvectors. 00:08:25.390 --> 00:08:28.760 And there's something special about the eigenvector. 00:08:28.760 --> 00:08:30.800 Let me write down what that is. 00:08:30.800 --> 00:08:36.700 The eigenvector x1 -- 00:08:36.700 --> 00:08:41.909 x1 is the eigenvector and all its components are positive, 00:08:41.909 --> 00:08:49.100 so the steady state is positive, if the start was. 00:08:49.100 --> 00:08:51.480 If the start was -- so -- 00:08:51.480 --> 00:08:54.780 well, actually, in general, I -- 00:08:54.780 --> 00:09:00.710 this might have a -- might have some component zero always, 00:09:00.710 --> 00:09:03.880 but no negative components in that eigenvector. 00:09:03.880 --> 00:09:04.410 Okay. 00:09:04.410 --> 00:09:08.470 Can I come to that point? 00:09:08.470 --> 00:09:11.370 How can I look at that matrix -- 00:09:11.370 --> 00:09:15.280 so that was just an example. 00:09:15.280 --> 00:09:21.000 How could I be sure -- how can I see that a matrix -- 00:09:21.000 --> 00:09:24.750 if the columns add to zero -- add to one, sorry -- 00:09:24.750 --> 00:09:30.930 if the columns add to one, this property means that lambda 00:09:30.930 --> 00:09:32.780 equal one is an eigenvalue. 00:09:32.780 --> 00:09:33.540 Okay. 00:09:33.540 --> 00:09:37.080 So let's just think that through. 00:09:37.080 --> 00:09:41.760 What I saying about -- let me ca- let me look at A, 00:09:41.760 --> 00:09:44.940 and if I believe that one is an eigenvalue, 00:09:44.940 --> 00:09:47.810 then I should be able to subtract off one times 00:09:47.810 --> 00:09:56.150 the identity and then I would get a matrix that's, what, -.9, 00:09:56.150 --> 00:09:59.160 -.01 and -.6 -- 00:09:59.160 --> 00:10:02.700 wh- I took the ones away and the other parts, of course, 00:10:02.700 --> 00:10:11.430 are still what they were, and this is still .2 and .7 and -- 00:10:11.430 --> 00:10:14.200 okay, what's -- 00:10:14.200 --> 00:10:15.700 what's up with this matrix now? 00:10:15.700 --> 00:10:19.380 I've shifted the matrix, this Markov matrix by one, 00:10:19.380 --> 00:10:24.350 by the identity, and what do I want to prove? 00:10:24.350 --> 00:10:28.940 I -- what is it that I believe this matrix -- 00:10:28.940 --> 00:10:30.760 about this matrix? 00:10:30.760 --> 00:10:33.190 I believe it's singular. 00:10:33.190 --> 00:10:36.580 Singular will -- if A minus I is singular, 00:10:36.580 --> 00:10:40.020 that tells me that one is an eigenvalue, right? 00:10:40.020 --> 00:10:43.420 The eigenvalues are the numbers that I subtract off -- 00:10:43.420 --> 00:10:45.010 the shifts -- 00:10:45.010 --> 00:10:47.880 the numbers that I subtract from the diagonal -- 00:10:47.880 --> 00:10:48.810 to make it singular. 00:10:48.810 --> 00:10:50.610 Now why is that matrix singular? 00:10:50.610 --> 00:10:55.570 I -- we could compute its determinant, 00:10:55.570 --> 00:11:00.000 but we want to see a reason that would work for every Markov 00:11:00.000 --> 00:11:05.550 matrix not just this particular random example. 00:11:05.550 --> 00:11:07.780 So what is it about that matrix? 00:11:07.780 --> 00:11:11.040 Well, I guess you could look at its columns now -- 00:11:11.040 --> 00:11:12.890 what do they add up to? 00:11:12.890 --> 00:11:13.390 Zero. 00:11:13.390 --> 00:11:21.660 The columns add to zero, so all columns -- 00:11:21.660 --> 00:11:25.240 let me put all columns now of -- of -- 00:11:25.240 --> 00:11:32.450 of A minus I add to zero, and then I 00:11:32.450 --> 00:11:40.305 want to realize that this means A minus I is singular. 00:11:45.400 --> 00:11:47.790 Okay. 00:11:47.790 --> 00:11:49.560 Why? 00:11:49.560 --> 00:11:52.210 So I could I -- you know, that could be a quiz question, 00:11:52.210 --> 00:11:54.280 a sort of theoretical quiz question. 00:11:54.280 --> 00:11:56.070 If I give you a matrix and I tell you 00:11:56.070 --> 00:12:00.740 all its columns add to zero, give me a reason, 00:12:00.740 --> 00:12:06.270 because it is true, that the matrix is singular. 00:12:06.270 --> 00:12:06.940 Okay. 00:12:06.940 --> 00:12:09.290 I guess actually -- now what -- 00:12:09.290 --> 00:12:11.680 I think of -- you know, I'm thinking of two or three ways 00:12:11.680 --> 00:12:12.480 to see that. 00:12:17.370 --> 00:12:18.410 How would you do it? 00:12:18.410 --> 00:12:21.880 We don't want to take its determinant somehow. 00:12:21.880 --> 00:12:24.520 For the matrix to be singular, well, 00:12:24.520 --> 00:12:31.180 it means that these three columns are dependent, right? 00:12:31.180 --> 00:12:33.530 The determinant will be zero when those three columns 00:12:33.530 --> 00:12:34.260 are dependent. 00:12:34.260 --> 00:12:36.650 You see, we're -- we're at a point in this course, now, 00:12:36.650 --> 00:12:40.100 where we have several ways to look at an idea. 00:12:40.100 --> 00:12:42.390 We can take the determinant -- here we don't want to. 00:12:42.390 --> 00:12:45.810 B- but we met singular before that -- 00:12:45.810 --> 00:12:47.830 those columns are dependent. 00:12:47.830 --> 00:12:50.670 So how do I see that those columns are dependent? 00:12:50.670 --> 00:12:51.925 They all add to zero. 00:12:57.020 --> 00:13:00.440 Let's see, whew -- 00:13:00.440 --> 00:13:05.070 well, oh, actually, what -- 00:13:05.070 --> 00:13:07.010 another thing I know is that the -- 00:13:07.010 --> 00:13:11.410 I would like to be able to show is that the rows are dependent. 00:13:11.410 --> 00:13:14.250 Maybe that's easier. 00:13:14.250 --> 00:13:15.940 If I know that all the columns add 00:13:15.940 --> 00:13:18.480 to zero, that's my information, how 00:13:18.480 --> 00:13:25.120 do I see that those three rows are linearly dependent? 00:13:25.120 --> 00:13:30.210 What -- what combination of those rows gives the zero row? 00:13:30.210 --> 00:13:32.920 How -- how could I combine those three rows -- 00:13:32.920 --> 00:13:38.930 those three row vectors to produce the zero row vector? 00:13:38.930 --> 00:13:42.360 And that would tell me those rows are dependent, 00:13:42.360 --> 00:13:43.910 therefore the columns are dependent, 00:13:43.910 --> 00:13:46.940 the matrix is singular, the determinant is zero -- well, 00:13:46.940 --> 00:13:47.890 you see it. 00:13:47.890 --> 00:13:49.010 I just add the rows. 00:13:49.010 --> 00:13:52.610 One times that row plus one times that row plus one times 00:13:52.610 --> 00:13:53.770 that row -- 00:13:53.770 --> 00:13:56.990 it's the zero row. 00:13:56.990 --> 00:13:58.960 The rows are dependent. 00:13:58.960 --> 00:14:03.470 In a way, that one one one, because it's multiplying 00:14:03.470 --> 00:14:07.810 the rows, is like an eigenvector in the -- 00:14:07.810 --> 00:14:10.700 it's in the left null space, right? 00:14:10.700 --> 00:14:12.870 One one one is in the left null space. 00:14:12.870 --> 00:14:23.190 It's singular because the rows are dependent -- 00:14:23.190 --> 00:14:25.380 and can I just keep the reasoning going? 00:14:25.380 --> 00:14:32.790 Because this vector one one one is -- 00:14:32.790 --> 00:14:34.940 it's not in the null space of the matrix, 00:14:34.940 --> 00:14:37.410 but it's in the null space of the transpose -- 00:14:37.410 --> 00:14:43.160 is in the null space of the transpose. 00:14:43.160 --> 00:14:45.600 And that's good enough. 00:14:45.600 --> 00:14:48.890 If we have a square matrix -- if we have a square matrix 00:14:48.890 --> 00:14:52.930 and the rows are dependent, that matrix is singular. 00:14:52.930 --> 00:14:58.630 So it turned out that the immediate guy we could identify 00:14:58.630 --> 00:15:00.470 was one one one. 00:15:00.470 --> 00:15:04.650 Of course, the -- 00:15:04.650 --> 00:15:06.480 there will be somebody in the null space, 00:15:06.480 --> 00:15:07.300 too. 00:15:07.300 --> 00:15:10.280 And actually, who will it be? 00:15:10.280 --> 00:15:13.110 So what's -- so -- 00:15:13.110 --> 00:15:15.830 so now I want to ask about the null space of -- 00:15:15.830 --> 00:15:17.400 of the matrix itself. 00:15:17.400 --> 00:15:20.890 What combination of the columns gives zero? 00:15:20.890 --> 00:15:23.450 I -- I don't want to compute it because I just made up this 00:15:23.450 --> 00:15:24.830 matrix and -- 00:15:24.830 --> 00:15:28.020 it will -- it would take me a while -- 00:15:28.020 --> 00:15:31.390 it looks sort of doable because it's three by three but wh- 00:15:31.390 --> 00:15:34.720 my point is, what -- 00:15:34.720 --> 00:15:37.350 what vector is it if we -- once we've found it, 00:15:37.350 --> 00:15:41.360 what have we got that's in the -- in the null space of A? 00:15:41.360 --> 00:15:44.060 It's the eigenvector, right? 00:15:44.060 --> 00:15:46.060 That's where we find X one. 00:15:46.060 --> 00:15:54.350 Then X one, the eigenvector, is in the null space of A. 00:15:54.350 --> 00:15:57.690 That's the eigenvector corresponding to the eigenvalue 00:15:57.690 --> 00:15:58.470 one. 00:15:58.470 --> 00:15:58.980 Right? 00:15:58.980 --> 00:16:01.330 That's how we find eigenvectors. 00:16:01.330 --> 00:16:05.280 So those three columns must be dependent -- 00:16:05.280 --> 00:16:08.090 some combination of columns -- of those three columns is 00:16:08.090 --> 00:16:11.930 the zero column and that -- the three components in that 00:16:11.930 --> 00:16:14.020 combination are the eigenvector. 00:16:14.020 --> 00:16:17.250 And that guy is the steady state. 00:16:17.250 --> 00:16:18.280 Okay. 00:16:18.280 --> 00:16:21.110 So I'm happy about the -- 00:16:21.110 --> 00:16:26.010 the thinking here, but I haven't given -- 00:16:26.010 --> 00:16:29.620 I haven't completed it because I haven't found x1. 00:16:29.620 --> 00:16:32.560 But it's there. 00:16:32.560 --> 00:16:36.330 Can I -- another thought came to me as I was doing this, 00:16:36.330 --> 00:16:39.280 another little comment that -- you -- 00:16:39.280 --> 00:16:41.500 about eigenvalues and eigenvectors, 00:16:41.500 --> 00:16:45.330 because of A and A transpose. 00:16:45.330 --> 00:16:50.800 What can you tell me about eigenvalues of A -- 00:16:50.800 --> 00:16:56.275 of A and eigenvalues of A transpose? 00:17:01.160 --> 00:17:01.660 Whoops. 00:17:04.952 --> 00:17:05.660 They're the same. 00:17:05.660 --> 00:17:11.079 They're -- so this is a little comment -- we -- it's useful, 00:17:11.079 --> 00:17:14.550 since eigenvalues are generally not easy to find -- 00:17:14.550 --> 00:17:19.180 it's always useful to know some cases where you've got them, 00:17:19.180 --> 00:17:20.210 where -- 00:17:20.210 --> 00:17:23.089 and this is -- if you know the eigenvalues of A, 00:17:23.089 --> 00:17:25.260 then you know the eigenvalues of A transpose. 00:17:25.260 --> 00:17:27.500 eigenvalues of A transpose are the same. 00:17:32.560 --> 00:17:37.890 And can I just, like, review why that is? 00:17:37.890 --> 00:17:44.280 So to find the eigenvalues of A, this would be determinate of A 00:17:44.280 --> 00:17:53.490 minus lambda I equals zero, that gives me an eigenvalue of A -- 00:17:53.490 --> 00:17:58.520 now how can I get A transpose into the picture here? 00:17:58.520 --> 00:18:00.690 I'll use the fact that the determinant 00:18:00.690 --> 00:18:06.690 of a matrix and the determinant of its transpose are the same. 00:18:06.690 --> 00:18:09.550 The determinant of a matrix equals the determinant of a -- 00:18:09.550 --> 00:18:10.520 of the transpose. 00:18:10.520 --> 00:18:13.950 That was property ten, the very last guy 00:18:13.950 --> 00:18:15.860 in our determinant list. 00:18:15.860 --> 00:18:18.280 So I'll transpose that matrix. 00:18:18.280 --> 00:18:21.680 This leads to -- 00:18:21.680 --> 00:18:24.690 I just take the matrix and transpose it, 00:18:24.690 --> 00:18:29.520 but now what do I get when I transpose lambda I? 00:18:29.520 --> 00:18:33.920 I just get lambda I. 00:18:33.920 --> 00:18:37.800 So that's -- that's all there was to the reasoning. 00:18:37.800 --> 00:18:40.000 The reasoning is that the eigenvalues of A 00:18:40.000 --> 00:18:42.370 solved that equation. 00:18:42.370 --> 00:18:44.530 The determinant of a matrix is the determinant 00:18:44.530 --> 00:18:47.480 of its transpose, so that gives me this equation 00:18:47.480 --> 00:18:50.160 and that tells me that the same lambdas 00:18:50.160 --> 00:18:53.240 are eigenvalues of A transpose. 00:18:53.240 --> 00:18:56.660 So that, backing up to the Markov case, 00:18:56.660 --> 00:19:00.880 one is an eigenvalue of A transpose and we actually found 00:19:00.880 --> 00:19:05.480 its eigenvector, one one one, and that tell us that one is 00:19:05.480 --> 00:19:08.090 also an eigenvalue of A -- but, of course, 00:19:08.090 --> 00:19:10.360 it has a different eigenvector, the -- 00:19:10.360 --> 00:19:13.530 the left null space isn't the same as the null space and we 00:19:13.530 --> 00:19:14.750 would have to find it. 00:19:14.750 --> 00:19:20.960 So there's some vector here which is x1 that produces zero 00:19:20.960 --> 00:19:22.460 zero zero. 00:19:22.460 --> 00:19:24.900 Actually, it wouldn't be that hard to find, you know, I -- 00:19:24.900 --> 00:19:26.520 as I'm talking I'm thinking, okay, 00:19:26.520 --> 00:19:29.440 I going to follow through and actually find it? 00:19:29.440 --> 00:19:32.750 Well, I can tell from this one -- look, 00:19:32.750 --> 00:19:37.650 if I put a point six there and a point seven there, 00:19:37.650 --> 00:19:43.510 that's what -- then I'll be okay in the last row, right? 00:19:43.510 --> 00:19:47.840 Now I only -- remains to find one guy. 00:19:47.840 --> 00:19:49.860 And let me take the first row, then. 00:19:49.860 --> 00:19:53.500 Minus point 54 plus point 21 -- 00:19:53.500 --> 00:19:56.790 there's some big number going in there, right? 00:19:56.790 --> 00:20:00.320 So I have -- just to make the first row come out zero, 00:20:00.320 --> 00:20:03.980 I'm getting minus point 54 plus point 21, 00:20:03.980 --> 00:20:09.420 so that was minus point 33 and what -- what do I want? 00:20:09.420 --> 00:20:11.350 Like thirty three hundred? 00:20:11.350 --> 00:20:13.980 This is the first time in the history of linear algebra 00:20:13.980 --> 00:20:16.980 that an eigenvector has every had a component 00:20:16.980 --> 00:20:18.980 thirty three hundred. 00:20:18.980 --> 00:20:21.030 But I guess it's true. 00:20:21.030 --> 00:20:24.760 Because then I multiply by minus one over a hundred -- oh no, 00:20:24.760 --> 00:20:26.830 it was point 33. 00:20:26.830 --> 00:20:29.180 So is this just -- oh, shoot. 00:20:29.180 --> 00:20:30.800 Only 33. 00:20:30.800 --> 00:20:31.880 Okay. 00:20:31.880 --> 00:20:33.030 Only 33. 00:20:33.030 --> 00:20:35.610 Okay, so there's the eigenvector. 00:20:35.610 --> 00:20:39.150 Oh, and notice that it -- that it turned -- did turn out, 00:20:39.150 --> 00:20:42.940 at least, to be all positive. 00:20:42.940 --> 00:20:45.990 So that was, like, the theory -- predicts that part, too. 00:20:45.990 --> 00:20:48.500 I won't give the proof of that part. 00:20:48.500 --> 00:20:50.600 So 30 -- 33 -- 00:20:50.600 --> 00:20:52.770 point six 33 point seven. 00:20:52.770 --> 00:20:53.420 Okay. 00:20:53.420 --> 00:20:58.750 Now those are the ma- that's the linear algebra part. 00:20:58.750 --> 00:21:00.980 Can I get to the applications? 00:21:00.980 --> 00:21:03.120 Where do these Markov matrices come from? 00:21:03.120 --> 00:21:06.360 Because that's -- that's part of this course and absolutely part 00:21:06.360 --> 00:21:07.570 of this lecture. 00:21:07.570 --> 00:21:08.080 Okay. 00:21:08.080 --> 00:21:12.270 So where's -- what's an application of Markov matrices? 00:21:12.270 --> 00:21:12.770 Okay. 00:21:17.600 --> 00:21:21.180 Markov matrices -- so, my equation, then, 00:21:21.180 --> 00:21:24.842 that I'm solving and studying is this equation u(k+1)=Auk. 00:21:28.290 --> 00:21:31.280 And now A is a Markov matrix. 00:21:31.280 --> 00:21:31.930 A is Markov. 00:21:36.060 --> 00:21:39.110 And I want to give an example. 00:21:39.110 --> 00:21:41.190 Can I just create an example? 00:21:41.190 --> 00:21:44.120 It'll be two by two. 00:21:44.120 --> 00:21:48.870 And it's one I've used before because it seems 00:21:48.870 --> 00:21:50.990 to me to bring out the idea. 00:21:50.990 --> 00:21:55.740 It's -- because we have two by two, we have two states, 00:21:55.740 --> 00:22:01.510 let's say California and Massachusetts. 00:22:01.510 --> 00:22:05.310 And I'm looking at the populations in those two 00:22:05.310 --> 00:22:08.720 states, the people in those two states, California 00:22:08.720 --> 00:22:10.900 and Massachusetts. 00:22:10.900 --> 00:22:17.040 And my matrix A is going to tell me in a -- in a year, 00:22:17.040 --> 00:22:19.230 some movement has happened. 00:22:19.230 --> 00:22:21.180 Some people stayed in Massachusetts, 00:22:21.180 --> 00:22:24.050 some people moved to California, some smart people 00:22:24.050 --> 00:22:26.300 moved from California to Massachusetts, 00:22:26.300 --> 00:22:29.240 some people stayed in California and made a billion. 00:22:29.240 --> 00:22:29.740 Okay. 00:22:29.740 --> 00:22:36.150 So that -- there's a matrix there with four entries 00:22:36.150 --> 00:22:41.044 and those tell me the fractions of my population -- 00:22:41.044 --> 00:22:41.710 so I'm making -- 00:22:41.710 --> 00:22:45.140 I'm going to use fractions, so they won't be negative, 00:22:45.140 --> 00:22:48.500 of course, because -- because only positive people are 00:22:48.500 --> 00:22:51.550 in- involved here -- and they'll add up to one, 00:22:51.550 --> 00:22:54.620 because I'm accounting for all people. 00:22:54.620 --> 00:22:57.830 So that's why I have these two key properties. 00:22:57.830 --> 00:22:59.650 The entries are greater equal zero 00:22:59.650 --> 00:23:02.890 because I'm looking at probabilities. 00:23:02.890 --> 00:23:06.050 Do they move, do they stay? 00:23:06.050 --> 00:23:09.130 Those probabilities are all between zero and one. 00:23:09.130 --> 00:23:12.290 And the probabilities add to one because everybody's 00:23:12.290 --> 00:23:12.990 accounted for. 00:23:12.990 --> 00:23:17.970 I'm not losing anybody, gaining anybody in this Markov chain. 00:23:17.970 --> 00:23:22.490 It's -- it conserves the total population. 00:23:22.490 --> 00:23:22.990 Okay. 00:23:22.990 --> 00:23:25.450 So what would be a typical matrix, then? 00:23:25.450 --> 00:23:34.820 So this would be u, California and u Massachusetts at time t 00:23:34.820 --> 00:23:36.740 equal k+1. 00:23:36.740 --> 00:23:40.680 And it's some matrix, which we'll 00:23:40.680 --> 00:23:48.730 think of, times u California and u Massachusetts at time k. 00:23:51.280 --> 00:23:54.460 And notice this matrix is going to stay the same, 00:23:54.460 --> 00:23:57.480 you know, forever. 00:23:57.480 --> 00:24:01.940 So that's a severe limitation on the example. 00:24:01.940 --> 00:24:05.090 The example has a -- the same Markov matrix, 00:24:05.090 --> 00:24:08.510 the same probabilities act at every time. 00:24:08.510 --> 00:24:09.010 Okay. 00:24:09.010 --> 00:24:11.030 So what's a reasonable, say -- 00:24:11.030 --> 00:24:16.750 say point nine of the people in California at time k 00:24:16.750 --> 00:24:19.410 stay there. 00:24:19.410 --> 00:24:24.440 And point one of the people in California 00:24:24.440 --> 00:24:26.930 move to Massachusetts. 00:24:26.930 --> 00:24:28.900 Notice why that column added to one, 00:24:28.900 --> 00:24:32.170 because we've now accounted for all the people in California 00:24:32.170 --> 00:24:33.890 at time k. 00:24:33.890 --> 00:24:36.250 Nine tenths of them are still in California, 00:24:36.250 --> 00:24:41.220 one tenth are here at time k+1. 00:24:41.220 --> 00:24:42.130 Okay. 00:24:42.130 --> 00:24:45.120 What about the people who are in Massachusetts? 00:24:45.120 --> 00:24:47.530 This is going to multiply column two, right, 00:24:47.530 --> 00:24:52.370 by our fundamental rule of multiplying matrix by vector, 00:24:52.370 --> 00:24:57.050 it's the -- it's the population in Massachusetts. 00:24:57.050 --> 00:25:06.350 Shall we say that -- that after the Red Sox, fail again, 00:25:06.350 --> 00:25:11.630 eight -- only 80 percent of the people in Massachusetts stay 00:25:11.630 --> 00:25:15.120 and 20 percent move to California. 00:25:15.120 --> 00:25:15.870 Okay. 00:25:15.870 --> 00:25:18.960 So again, this adds to one, which 00:25:18.960 --> 00:25:23.070 accounts for all people in Massachusetts where they are. 00:25:23.070 --> 00:25:26.770 So there is a Markov matrix. 00:25:26.770 --> 00:25:28.400 Non-negative entries adding to one. 00:25:28.400 --> 00:25:29.980 What's the steady state? 00:25:29.980 --> 00:25:33.314 If everybody started in Massachusetts, say, at -- 00:25:33.314 --> 00:25:34.980 you know, when the Pilgrims showed up or 00:25:34.980 --> 00:25:35.600 something. 00:25:35.600 --> 00:25:40.280 Then where are they now? 00:25:40.280 --> 00:25:45.211 Where are they at time 100, let's say, or maybe -- 00:25:45.211 --> 00:25:47.210 I don't know, how many years since the Pilgrims? 00:25:47.210 --> 00:25:49.090 300 and something. 00:25:49.090 --> 00:25:51.520 Or -- and actually where will they be, like, 00:25:51.520 --> 00:25:54.820 way out a million years from now? 00:25:54.820 --> 00:26:00.980 I -- I could multiply -- 00:26:00.980 --> 00:26:03.360 take the powers of this matrix. 00:26:03.360 --> 00:26:07.040 In fact, you'll -- you would -- ought to be able to figure out 00:26:07.040 --> 00:26:10.490 what is the hundredth power of that matrix? 00:26:10.490 --> 00:26:12.520 Why don't we do that? 00:26:12.520 --> 00:26:15.050 But let me follow the steady state. 00:26:15.050 --> 00:26:17.760 So what -- what's my starting -- 00:26:17.760 --> 00:26:28.560 my starting u Cal, u Mass at time zero is, shall we say -- 00:26:28.560 --> 00:26:30.400 shall we put anybody in California? 00:26:30.400 --> 00:26:32.920 Let's make -- let's make zero there, 00:26:32.920 --> 00:26:36.400 and say the population of Massachusetts is -- 00:26:36.400 --> 00:26:38.780 let's say a thousand just to -- okay. 00:26:43.330 --> 00:26:46.030 So the population is -- 00:26:46.030 --> 00:26:49.400 so the populations are zero and a thousand at the start. 00:26:49.400 --> 00:26:52.790 What can you tell me about this population after -- 00:26:52.790 --> 00:26:55.990 after k steps? 00:26:55.990 --> 00:27:01.420 What will u Cal plus u Mass add to? 00:27:01.420 --> 00:27:02.790 A thousand. 00:27:02.790 --> 00:27:06.260 Those thousand people are always accounted for. 00:27:06.260 --> 00:27:10.590 But -- so u Mass will start dropping from a thousand and u 00:27:10.590 --> 00:27:11.930 Cal will start growing. 00:27:11.930 --> 00:27:15.060 Actually, we could see -- why don't we figure out what it is 00:27:15.060 --> 00:27:16.420 after one? 00:27:16.420 --> 00:27:22.100 After one time step, what are the populations at time one? 00:27:24.890 --> 00:27:26.530 So what happens in one step? 00:27:26.530 --> 00:27:30.230 You multiply once by that matrix and, let's see, 00:27:30.230 --> 00:27:33.460 zero times this column -- so it's just a thousand times this 00:27:33.460 --> 00:27:39.940 column, so I think we're getting 200 and 800. 00:27:39.940 --> 00:27:42.860 So after the first step, 200 people have -- 00:27:42.860 --> 00:27:44.760 are in California. 00:27:44.760 --> 00:27:49.620 Now at the following step, I'll multiply again by this matrix 00:27:49.620 --> 00:27:52.740 -- more people will move to California. 00:27:52.740 --> 00:27:54.340 Some people will move back. 00:27:54.340 --> 00:28:00.390 Twenty people will come back and, the -- 00:28:00.390 --> 00:28:03.010 the net result will be that the California population will be 00:28:03.010 --> 00:28:08.310 above 200 and the Massachusetts below 800 and they'll still add 00:28:08.310 --> 00:28:10.191 up to a thousand. 00:28:10.191 --> 00:28:10.690 Okay. 00:28:10.690 --> 00:28:14.100 I do that a few times. 00:28:14.100 --> 00:28:15.500 I do that 100 times. 00:28:15.500 --> 00:28:18.220 What's the population? 00:28:18.220 --> 00:28:20.770 Well, okay, to answer any question like that, 00:28:20.770 --> 00:28:22.700 I need the eigenvalues and eigenvectors, 00:28:22.700 --> 00:28:23.200 right? 00:28:23.200 --> 00:28:24.090 As soon as I've -- 00:28:24.090 --> 00:28:26.340 I've created an example, but as soon 00:28:26.340 --> 00:28:28.570 as I want to solve anything, I have 00:28:28.570 --> 00:28:31.610 to find eigenvalues and eigenvectors of that matrix. 00:28:31.610 --> 00:28:32.230 Okay. 00:28:32.230 --> 00:28:33.890 So let's do it. 00:28:33.890 --> 00:28:40.010 So there's the matrix .9, .2, .1, .8 and tell 00:28:40.010 --> 00:28:43.150 me its eigenvalues. 00:28:43.150 --> 00:28:46.390 Lambda equals -- so tell me one eigenvalue? 00:28:46.390 --> 00:28:48.680 One, thanks. 00:28:48.680 --> 00:28:49.960 And tell me the other one. 00:28:53.120 --> 00:28:55.300 What's the other eigenvalue -- 00:28:55.300 --> 00:28:59.370 from the trace or the determinant -- from the -- 00:28:59.370 --> 00:29:02.510 I -- the trace is what -- is, like, easier. 00:29:02.510 --> 00:29:06.110 So the trace of that matrix is one point seven. 00:29:06.110 --> 00:29:10.040 So the other eigenvalue is point seven. 00:29:10.040 --> 00:29:13.540 And it -- notice that it's less than one. 00:29:13.540 --> 00:29:17.780 And notice that that determinant is point 72-.02, 00:29:17.780 --> 00:29:18.960 which is point seven. 00:29:18.960 --> 00:29:19.490 Right. 00:29:19.490 --> 00:29:20.190 Okay. 00:29:20.190 --> 00:29:21.900 Now to find the eigenvectors. 00:29:21.900 --> 00:29:26.120 This is -- so that's lambda one and the eigenvector -- 00:29:26.120 --> 00:29:29.950 I'll subtract one from the diagonal, right? 00:29:29.950 --> 00:29:33.440 So can I do that in light let -- in light here? 00:29:33.440 --> 00:29:35.300 Subtract one from the diagonal, I 00:29:35.300 --> 00:29:38.410 have minus point one and minus point two, 00:29:38.410 --> 00:29:39.730 and of course these are still 00:29:39.730 --> 00:29:40.940 there. 00:29:40.940 --> 00:29:44.220 And I'm looking for its -- 00:29:44.220 --> 00:29:47.360 here's -- here's -- this is going to be x1. 00:29:47.360 --> 00:29:52.170 It's the null space of A minus I. 00:29:52.170 --> 00:29:56.820 Okay, everybody sees that it's two and one. 00:29:56.820 --> 00:29:57.940 Okay? 00:29:57.940 --> 00:29:59.820 And now how about -- so that -- and it -- 00:29:59.820 --> 00:30:03.160 notice that that eigenvector is positive. 00:30:03.160 --> 00:30:08.770 And actually, we can jump to infinity right now. 00:30:08.770 --> 00:30:14.100 What's the population at infinity? 00:30:14.100 --> 00:30:17.170 It's a multiple -- this is -- this eigenvector is giving 00:30:17.170 --> 00:30:19.910 the steady state. 00:30:19.910 --> 00:30:23.280 It's some multiple of this, and how is that multiple decided? 00:30:23.280 --> 00:30:26.940 By adding up to a thousand people. 00:30:26.940 --> 00:30:31.200 So the steady state, the c1x1 -- this is the x1, 00:30:31.200 --> 00:30:36.740 but that adds up to three, so I really want two -- 00:30:36.740 --> 00:30:39.820 it's going to be two thirds of a thousand and one third 00:30:39.820 --> 00:30:42.980 of a thousand, making a total of the thousand 00:30:42.980 --> 00:30:43.550 people. 00:30:43.550 --> 00:30:45.480 That'll be the steady state. 00:30:45.480 --> 00:30:48.390 That's really all I need to know at infinity. 00:30:48.390 --> 00:30:50.160 But if I want to know what's happened 00:30:50.160 --> 00:30:53.280 after just a finite number like 100 steps, 00:30:53.280 --> 00:30:55.380 I'd better find this eigenvector. 00:30:55.380 --> 00:30:57.840 So can I -- can I look at -- 00:30:57.840 --> 00:31:00.130 I'll subtract point seven time -- ti- 00:31:00.130 --> 00:31:06.250 from the diagonal and I'll get that and I'll look at the null 00:31:06.250 --> 00:31:11.280 space of that one and I -- and this is going to give me x2, 00:31:11.280 --> 00:31:13.540 now, and what is it? 00:31:13.540 --> 00:31:16.380 So what's in the null space of -- that's certainly singular, 00:31:16.380 --> 00:31:20.920 so I know my calculation is right, and -- 00:31:20.920 --> 00:31:24.290 one and minus one. 00:31:24.290 --> 00:31:26.240 One and minus one. 00:31:26.240 --> 00:31:30.580 So I'm prepared now to write down 00:31:30.580 --> 00:31:32.470 the solution after 100 time 00:31:32.470 --> 00:31:33.180 steps. 00:31:33.180 --> 00:31:35.670 The -- the populations after 100 time steps, 00:31:35.670 --> 00:31:36.420 right? 00:31:36.420 --> 00:31:39.290 Can -- can we remember the point one -- the -- 00:31:39.290 --> 00:31:43.100 the one with this two one eigenvector and the point seven 00:31:43.100 --> 00:31:44.930 with the minus one one eigenvector. 00:31:44.930 --> 00:31:46.160 So I'll -- let me -- 00:31:46.160 --> 00:31:48.960 I'll just write it above here. 00:31:48.960 --> 00:31:53.810 u after k steps is some multiple of one 00:31:53.810 --> 00:31:57.610 to the k times the two one eigenvector 00:31:57.610 --> 00:32:04.230 and some multiple of point seven to the k times the minus one 00:32:04.230 --> 00:32:07.430 one eigenvector. 00:32:07.430 --> 00:32:09.910 Right? 00:32:09.910 --> 00:32:11.290 That's -- I -- 00:32:11.290 --> 00:32:15.010 this is how I take -- how powers of a matrix work. 00:32:15.010 --> 00:32:21.360 When I apply those powers to a u0, what I -- so it's u0, 00:32:21.360 --> 00:32:26.390 which was zero a thousand -- 00:32:26.390 --> 00:32:29.220 that has to be corrected k=0. 00:32:29.220 --> 00:32:35.720 So I'm plugging in k=0 and I get c1 times two one and c2 times 00:32:35.720 --> 00:32:38.640 minus one one. 00:32:38.640 --> 00:32:44.990 Two equations, two constants, certainly 00:32:44.990 --> 00:32:49.710 independent eigenvectors, so there's a solution 00:32:49.710 --> 00:32:51.770 and you see what it is? 00:32:51.770 --> 00:32:56.450 Let's see, I guess we already figured that c1 was a thousand 00:32:56.450 --> 00:32:59.390 over three, I think -- did we think that had to be a thousand 00:32:59.390 --> 00:33:02.200 over three? 00:33:02.200 --> 00:33:05.840 And maybe c2 would be -- 00:33:05.840 --> 00:33:07.920 excuse me, let -- get an eraser -- 00:33:07.920 --> 00:33:08.930 we can -- 00:33:08.930 --> 00:33:11.330 I just -- I think we've -- get it here. 00:33:11.330 --> 00:33:15.590 c2, we want to get a zero here, so maybe we 00:33:15.590 --> 00:33:21.400 need plus two thousand over three. 00:33:21.400 --> 00:33:22.840 I think that has to work. 00:33:22.840 --> 00:33:26.010 Two times a thousand over three minus two thousand 00:33:26.010 --> 00:33:29.160 over three, that'll give us the zero and a thousand 00:33:29.160 --> 00:33:32.020 over three and the two thousand over three will give us 00:33:32.020 --> 00:33:33.290 three thousand over three, 00:33:33.290 --> 00:33:34.160 the thousand. 00:33:34.160 --> 00:33:37.820 So this is what we approach -- 00:33:37.820 --> 00:33:42.000 this part, with the point seven to the k-th power 00:33:42.000 --> 00:33:45.050 is the part that's disappearing. 00:33:45.050 --> 00:33:48.270 That's -- that's Markov matrices. 00:33:48.270 --> 00:33:48.770 Okay. 00:33:48.770 --> 00:33:52.610 That's an example of where they come from, 00:33:52.610 --> 00:34:00.220 from modeling movement of people with no gain or loss, 00:34:00.220 --> 00:34:03.690 with total -- total count conserved. 00:34:03.690 --> 00:34:04.440 Okay. 00:34:04.440 --> 00:34:07.260 I -- just if I can add one more comment, 00:34:07.260 --> 00:34:11.550 because you'll see Markov matrices in electrical 00:34:11.550 --> 00:34:16.949 engineering courses and often you'll see them -- sorry, 00:34:16.949 --> 00:34:19.679 here's my little comment. 00:34:19.679 --> 00:34:22.280 Sometimes -- in a lot of applications they prefer 00:34:22.280 --> 00:34:25.590 to work with row vectors. 00:34:25.590 --> 00:34:29.480 So they -- instead of -- this was natural for us, right? 00:34:29.480 --> 00:34:32.429 For all the eigenvectors to be column vectors. 00:34:32.429 --> 00:34:37.139 So our columns added to one in the Markov matrix. 00:34:37.139 --> 00:34:39.889 Just so you don't think, well, what -- 00:34:39.889 --> 00:34:41.820 what's going on? 00:34:41.820 --> 00:34:48.170 If we work with row vectors and we multiply vector times matrix 00:34:48.170 --> 00:34:51.179 -- so we're multiplying from the left -- 00:34:51.179 --> 00:34:55.679 then it'll be the then we'll be using the transpose of -- 00:34:55.679 --> 00:35:00.100 of this matrix and it'll be the rows that add to one. 00:35:00.100 --> 00:35:05.390 So in other textbooks, you'll see -- instead of col- 00:35:05.390 --> 00:35:08.200 columns adding to one, you'll see rows add to one. 00:35:08.200 --> 00:35:09.320 Okay. 00:35:09.320 --> 00:35:10.410 Fine. 00:35:10.410 --> 00:35:13.030 Okay, that's what I wanted to say about Markov, 00:35:13.030 --> 00:35:17.560 now I want to say something about projections and even 00:35:17.560 --> 00:35:22.190 leading in -- a little into Fourier series. 00:35:22.190 --> 00:35:24.960 Because -- but before any Fourier stuff, 00:35:24.960 --> 00:35:28.150 let me make a comment about projections. 00:35:28.150 --> 00:35:33.310 This -- so this is a comment about projections onto -- 00:35:33.310 --> 00:35:36.526 with an orthonormal basis. 00:35:42.420 --> 00:35:49.770 So, of course, the basis vectors are q1 up to qn. 00:35:49.770 --> 00:35:50.860 Okay. 00:35:50.860 --> 00:35:52.330 I have a vector b. 00:35:52.330 --> 00:35:58.700 Let -- let me imagine -- let me imagine this is a basis. 00:35:58.700 --> 00:36:00.640 Let -- let's say I'm in n by n. 00:36:00.640 --> 00:36:06.660 I'm -- I've got, eh, n orthonormal vectors, 00:36:06.660 --> 00:36:09.350 I'm in n dimensional space so they're a complete -- 00:36:09.350 --> 00:36:11.110 they're a basis -- 00:36:11.110 --> 00:36:16.170 any vector v could be expanded in this basis. 00:36:16.170 --> 00:36:22.050 So any vector v is some combination, some amount of q1 00:36:22.050 --> 00:36:28.060 plus some amount of q2 plus some amount of qn. 00:36:28.060 --> 00:36:35.220 So -- so any v. 00:36:35.220 --> 00:36:41.740 I just want you to tell me what those amounts are. 00:36:41.740 --> 00:36:46.510 What are x1 -- what's x1, for example? 00:36:46.510 --> 00:36:49.980 So I'm looking for the expansion. 00:36:49.980 --> 00:36:51.900 This is -- this is really our projection. 00:36:51.900 --> 00:36:56.330 I could -- I could really use the word expansion. 00:36:56.330 --> 00:37:01.450 I'm expanding the vector in the basis. 00:37:01.450 --> 00:37:05.770 And the special thing about the basis is that it's orthonormal. 00:37:05.770 --> 00:37:10.470 So that should give me a special formula for the answer, 00:37:10.470 --> 00:37:12.220 for the coefficients. 00:37:12.220 --> 00:37:14.010 So how do I get x1? 00:37:14.010 --> 00:37:15.510 What -- what's a formula for x1? 00:37:18.250 --> 00:37:23.040 I could -- I can go through the projection -- 00:37:23.040 --> 00:37:26.940 the Q transpose Q, all that -- 00:37:26.940 --> 00:37:31.450 normal equations, but -- 00:37:31.450 --> 00:37:32.140 and I'll get -- 00:37:32.140 --> 00:37:33.960 I'll come out with this nice answer 00:37:33.960 --> 00:37:36.060 that I think I can see right away. 00:37:36.060 --> 00:37:38.370 How can I pick -- 00:37:38.370 --> 00:37:42.920 get a hold of x1 and get these other x-s out of the equation? 00:37:42.920 --> 00:37:47.460 So how can I get a nice, simple formula for x1? 00:37:47.460 --> 00:37:50.751 And then we want to see, sure, we knew that all the time. 00:37:50.751 --> 00:37:51.250 Okay. 00:37:51.250 --> 00:37:52.610 So what's x1? 00:37:52.610 --> 00:37:59.020 The good way is take the inner product of everything with q1. 00:37:59.020 --> 00:38:02.430 Take the inner product of that whole equation, every term, 00:38:02.430 --> 00:38:03.850 with q1. 00:38:03.850 --> 00:38:08.370 What will happen to that last term? 00:38:08.370 --> 00:38:11.560 The inner product -- when -- if I take the dot product with q1 00:38:11.560 --> 00:38:13.970 I get zero, right? 00:38:13.970 --> 00:38:17.160 Because this basis was orthonormal. 00:38:17.160 --> 00:38:20.330 If I take the dot product with q2 I get zero. 00:38:20.330 --> 00:38:24.570 If I take the dot product with q1 I get one. 00:38:24.570 --> 00:38:28.740 So that tells me what x1 is. q1 transpose 00:38:28.740 --> 00:38:31.250 v, that's taking the dot product, 00:38:31.250 --> 00:38:41.060 is x1 times q1 transpose q1 plus a bunch of zeroes. 00:38:41.060 --> 00:38:45.330 And this is a one, so I can forget that. 00:38:45.330 --> 00:38:47.820 I get x1 immediately. 00:38:47.820 --> 00:38:49.840 So -- do you see what I'm saying -- 00:38:49.840 --> 00:38:53.070 is that I have an orthonormal basis, 00:38:53.070 --> 00:38:58.760 then the coefficient that I need for each basis vector is 00:38:58.760 --> 00:38:59.340 a cinch to 00:38:59.340 --> 00:39:00.070 find. 00:39:00.070 --> 00:39:02.310 Let me -- let me just -- 00:39:02.310 --> 00:39:05.580 I have to put this into matrix language, too, 00:39:05.580 --> 00:39:07.350 so you'll see it there also. 00:39:07.350 --> 00:39:10.690 If I write that first equation in matrix language, what -- 00:39:10.690 --> 00:39:12.010 what is it? 00:39:12.010 --> 00:39:13.870 I'm writing -- in matrix language, 00:39:13.870 --> 00:39:19.020 this equation says I'm taking these columns -- are -- 00:39:19.020 --> 00:39:20.620 are you guys good at this now? 00:39:20.620 --> 00:39:29.180 I'm taking those columns times the Xs and getting V, right? 00:39:29.180 --> 00:39:30.440 That's the matrix form. 00:39:30.440 --> 00:39:37.464 Okay, that's the matrix Q. Qx is v. 00:39:37.464 --> 00:39:39.005 What's the solution to that equation? 00:39:41.570 --> 00:39:44.920 It's -- of course, it's x equal Q inverse v. 00:39:44.920 --> 00:39:51.000 So x is Q inverse v, but what's the point? 00:39:51.000 --> 00:39:54.060 Q inverse in this case is going to -- is simple. 00:39:54.060 --> 00:39:58.220 I don't have to work to invert this matrix Q, 00:39:58.220 --> 00:40:03.110 because of the fact that the -- these columns are orthonormal, 00:40:03.110 --> 00:40:05.110 I know the inverse to that. 00:40:05.110 --> 00:40:10.070 And it is Q transpose. 00:40:10.070 --> 00:40:15.050 When you see a Q, a square matrix with that letter Q, 00:40:15.050 --> 00:40:17.090 the -- that just triggers -- 00:40:17.090 --> 00:40:19.670 Q inverse is the same as Q transpose. 00:40:19.670 --> 00:40:22.100 So the first component, then -- 00:40:22.100 --> 00:40:25.970 the first component of x is the first row times 00:40:25.970 --> 00:40:29.550 v, and what's that? 00:40:29.550 --> 00:40:32.650 The first component of this answer 00:40:32.650 --> 00:40:36.570 is the first row of Q transpose. 00:40:36.570 --> 00:40:42.110 That's just -- that's just q1 transpose times v. 00:40:42.110 --> 00:40:46.320 So that's what we concluded here, too. 00:40:46.320 --> 00:40:46.820 Okay. 00:40:49.590 --> 00:40:55.090 So -- so nothing Fourier here. 00:40:55.090 --> 00:40:59.650 The -- the key ingredient here was that the q-s are 00:40:59.650 --> 00:41:01.460 orthonormal. 00:41:01.460 --> 00:41:04.900 And now that's what Fourier series are built on. 00:41:04.900 --> 00:41:08.520 So now, in the remaining time, let 00:41:08.520 --> 00:41:11.760 me say something about Fourier series. 00:41:11.760 --> 00:41:12.400 Okay. 00:41:12.400 --> 00:41:20.120 So Fourier series is -- 00:41:20.120 --> 00:41:25.150 well, we've got a function f of x. 00:41:25.150 --> 00:41:28.160 And we want to write it as a combination of -- 00:41:28.160 --> 00:41:30.900 maybe it has a constant term. 00:41:30.900 --> 00:41:34.910 And then it has some cos(x) in it. 00:41:34.910 --> 00:41:38.120 And it has some sin(x) in it. 00:41:38.120 --> 00:41:42.330 And it has some cos(2x) in it. 00:41:42.330 --> 00:41:45.230 And a -- and some sin(2x), and forever. 00:41:50.780 --> 00:41:54.600 So what's -- what's the difference between this type 00:41:54.600 --> 00:41:56.900 problem and the one above it? 00:41:56.900 --> 00:42:02.090 This one's infinite, but the key property 00:42:02.090 --> 00:42:06.110 of things being orthogonal is still 00:42:06.110 --> 00:42:09.790 true for sines and cosines, so it's the property that 00:42:09.790 --> 00:42:11.170 makes Fourier series work. 00:42:11.170 --> 00:42:12.860 So that's called a Fourier series. 00:42:12.860 --> 00:42:14.620 Better write his name up. 00:42:14.620 --> 00:42:15.530 Fourier series. 00:42:22.170 --> 00:42:25.960 So it was Joseph Fourier who realized that, hey, I 00:42:25.960 --> 00:42:29.870 could work in function space. 00:42:29.870 --> 00:42:33.970 Instead of a vector v, I could have a function f of x. 00:42:33.970 --> 00:42:38.590 Instead of orthogonal vectors, q1, q2 , q3, 00:42:38.590 --> 00:42:42.180 I could have orthogonal functions, the constant, 00:42:42.180 --> 00:42:45.370 the cos(x), the sin(x), the s- cos(2x), 00:42:45.370 --> 00:42:47.450 but infinitely many of them. 00:42:47.450 --> 00:42:49.930 I need infinitely many, because my space 00:42:49.930 --> 00:42:52.440 is infinite dimensional. 00:42:52.440 --> 00:42:56.970 So this is, like, the moment in which we leave finite 00:42:56.970 --> 00:43:00.680 dimensional vector spaces and go to infinite dimensional vector 00:43:00.680 --> 00:43:03.090 spaces and our basis -- 00:43:03.090 --> 00:43:07.250 so the vectors are now functions -- 00:43:07.250 --> 00:43:09.650 and of course, there are so many functions that it's -- 00:43:09.650 --> 00:43:13.620 that we've got an infin- infinite dimensional space -- 00:43:13.620 --> 00:43:17.320 and the basis vectors are functions, too. 00:43:17.320 --> 00:43:25.400 a0, the constant function one -- so my basis is one cos(x), 00:43:25.400 --> 00:43:29.190 sin(x), cos(2x), sin(2x) and so on. 00:43:29.190 --> 00:43:32.710 And the reason Fourier series is a success 00:43:32.710 --> 00:43:35.170 is that those are orthogonal. 00:43:35.170 --> 00:43:35.760 Okay. 00:43:35.760 --> 00:43:37.190 Now what do I mean by orthogonal? 00:43:40.110 --> 00:43:44.490 I know what it means for two vectors to be orthogonal -- 00:43:44.490 --> 00:43:46.870 y transpose x equals zero, right? 00:43:46.870 --> 00:43:48.600 Dot product equals zero. 00:43:48.600 --> 00:43:52.070 But what's the dot product of functions? 00:43:52.070 --> 00:43:55.340 I'm claiming that whatever it is, the dot product -- 00:43:55.340 --> 00:43:59.840 or we would more likely use the word inner product of, say, 00:43:59.840 --> 00:44:02.430 cos(x) with sin(x) is zero. 00:44:02.430 --> 00:44:06.340 And cos(x) with cos(2x), also zero. 00:44:06.340 --> 00:44:08.780 So I -- let me tell you what I mean by that, 00:44:08.780 --> 00:44:10.410 by that dot product. 00:44:10.410 --> 00:44:12.850 Well, how do I compute a dot product? 00:44:12.850 --> 00:44:16.970 So, let's just remember for vectors v trans- 00:44:16.970 --> 00:44:23.050 v transpose w for vectors, so this was vectors, 00:44:23.050 --> 00:44:30.145 v transpose w was v1w1 +...+vnwn. 00:44:33.510 --> 00:44:34.140 Okay. 00:44:34.140 --> 00:44:34.750 Now functions. 00:44:40.100 --> 00:44:42.720 Now I have two functions, let's call them f and g. 00:44:45.260 --> 00:44:46.700 What's with them now? 00:44:46.700 --> 00:44:49.730 The vectors had n components, but the functions 00:44:49.730 --> 00:44:53.060 have a whole, like, continuum. 00:44:53.060 --> 00:44:55.640 To graph the function, I just don't have n points, 00:44:55.640 --> 00:44:57.780 I've got this whole graph. 00:44:57.780 --> 00:44:59.780 So I have functions -- 00:44:59.780 --> 00:45:01.590 I'm really trying to ask you what's 00:45:01.590 --> 00:45:03.840 the inner product of this function 00:45:03.840 --> 00:45:05.170 f with another function 00:45:05.170 --> 00:45:06.130 g? 00:45:06.130 --> 00:45:11.870 And I want to make it parallel to this the best I can. 00:45:11.870 --> 00:45:20.080 So the best parallel is to multiply f (x) times g(x) 00:45:20.080 --> 00:45:23.060 at every x -- 00:45:23.060 --> 00:45:25.120 and here I just had n multiplications, 00:45:25.120 --> 00:45:28.330 but here I'm going to have a whole range of x-s, 00:45:28.330 --> 00:45:31.980 and here I added the results. 00:45:31.980 --> 00:45:34.600 What do I do here? 00:45:34.600 --> 00:45:38.400 So what's the analog of addition when you have -- 00:45:38.400 --> 00:45:40.350 when you're in a continuum? 00:45:40.350 --> 00:45:41.830 It's integration. 00:45:41.830 --> 00:45:47.630 So that the -- the dot product of two functions will be 00:45:47.630 --> 00:45:49.525 the integral of those functions, dx. 00:45:52.240 --> 00:45:55.010 Now I have to say -- say, well, what are the limits 00:45:55.010 --> 00:45:56.800 of integration? 00:45:56.800 --> 00:46:03.980 And for this Fourier series, this function f(x) -- 00:46:03.980 --> 00:46:08.000 if I'm going to have -- if that right hand side is going to be 00:46:08.000 --> 00:46:11.920 f(x), that function that I'm seeing on the right, 00:46:11.920 --> 00:46:16.130 all those sines and cosines, they're all periodic, with -- 00:46:16.130 --> 00:46:18.210 with period two pi. 00:46:18.210 --> 00:46:21.650 So -- so that's what f(x) had better be. 00:46:21.650 --> 00:46:24.870 So I'll integrate from zero to two pi. 00:46:24.870 --> 00:46:29.160 My -- all -- everything -- is on the interval zero two pi now, 00:46:29.160 --> 00:46:33.690 because if I'm going to use these sines and cosines, 00:46:33.690 --> 00:46:39.950 then f(x) is equal to f(x+2pi). 00:46:39.950 --> 00:46:42.428 This is periodic -- 00:46:45.180 --> 00:46:48.250 periodic functions. 00:46:48.250 --> 00:46:49.800 Okay. 00:46:49.800 --> 00:46:52.770 So now I know what -- 00:46:52.770 --> 00:46:56.120 I've got all the right words now. 00:46:56.120 --> 00:47:00.490 I've got a vector space, but the vectors are functions. 00:47:00.490 --> 00:47:05.450 I've got inner products and -- and the inner product gives 00:47:05.450 --> 00:47:07.760 a number, all right. 00:47:07.760 --> 00:47:12.260 It just happens to be an integral instead of a sum. 00:47:12.260 --> 00:47:15.230 I've got -- and that -- then I have the idea of orthogonality 00:47:15.230 --> 00:47:15.860 -- 00:47:15.860 --> 00:47:18.470 because, actually, just -- let's just check. 00:47:18.470 --> 00:47:21.910 Orthogonality -- if I take the integral -- s- I -- 00:47:21.910 --> 00:47:25.890 let me do sin(x) times cos(x) -- 00:47:25.890 --> 00:47:31.106 sin(x) times cos(x) dx from zero to two pi -- 00:47:34.680 --> 00:47:36.700 I think we get zero. 00:47:36.700 --> 00:47:40.670 That's the differential of that, so it would be one half 00:47:40.670 --> 00:47:42.890 sine x squared, was that right? 00:47:47.220 --> 00:47:50.100 Between zero and two pi -- 00:47:50.100 --> 00:47:52.910 and, of course, we get zero. 00:47:52.910 --> 00:47:58.150 And the same would be true with a little more -- 00:47:58.150 --> 00:48:02.430 some trig identities to help us out -- of every other pair. 00:48:02.430 --> 00:48:05.650 So we have now an orthonormal infinite 00:48:05.650 --> 00:48:10.160 basis for function space, and all we want to do 00:48:10.160 --> 00:48:12.980 is express a function in that 00:48:12.980 --> 00:48:14.090 basis. 00:48:14.090 --> 00:48:20.120 And so I -- the end of my lecture is, okay, what is a1? 00:48:20.120 --> 00:48:24.210 What's the coefficient -- how much cos(x) is there 00:48:24.210 --> 00:48:30.230 in a function compared to the other harmonics? 00:48:30.230 --> 00:48:32.940 How much constant is in that function? 00:48:32.940 --> 00:48:35.310 That'll -- that would be an easy question. 00:48:35.310 --> 00:48:39.110 The answer a0 will come out to be the average value of f. 00:48:39.110 --> 00:48:40.620 That's the amount of the constant 00:48:40.620 --> 00:48:42.630 that's in there, its average value. 00:48:42.630 --> 00:48:45.350 But let's take a1 as more typical. 00:48:45.350 --> 00:48:48.170 How will I get -- here's the end of the lecture, then -- 00:48:48.170 --> 00:48:49.400 how do I get a1? 00:48:52.070 --> 00:48:54.720 The first Fourier coefficient. 00:48:54.720 --> 00:48:56.350 Okay. 00:48:56.350 --> 00:48:59.790 I do just as I did in the vector case. 00:48:59.790 --> 00:49:03.850 I take the inner product of everything with cos(x) 00:49:03.850 --> 00:49:07.010 Take the inner product of everything with cos(x). 00:49:07.010 --> 00:49:08.800 Then on the left -- 00:49:08.800 --> 00:49:13.990 on the left I have -- the inner product is the integral of f(x) 00:49:13.990 --> 00:49:15.365 times cos(x) cx. 00:49:18.740 --> 00:49:22.080 And on the right, what do I have? 00:49:22.080 --> 00:49:24.970 When I -- so what I -- when I say take the inner product with 00:49:24.970 --> 00:49:28.850 cos(x), let me put it in ordinary calculus words. 00:49:28.850 --> 00:49:32.610 Multiply by cos(x) and integrate. 00:49:32.610 --> 00:49:34.540 That's what inner products are. 00:49:34.540 --> 00:49:36.940 So if I multiply that whole thing by cos(x) 00:49:36.940 --> 00:49:40.990 and I integrate, I get a whole lot of zeroes. 00:49:40.990 --> 00:49:45.830 The only thing that survives is that term. 00:49:45.830 --> 00:49:47.250 All the others disappear. 00:49:47.250 --> 00:49:53.950 So -- and that term is a1 times the integral of cos(x) squared 00:49:53.950 --> 00:50:01.580 dx zero to 2pi equals -- so this was the left side and this is 00:50:01.580 --> 00:50:04.840 all that's left on the right-hand side. 00:50:04.840 --> 00:50:09.700 And this is not zero of course, because it's the length 00:50:09.700 --> 00:50:13.570 of the function squared, it's the inner product with itself, 00:50:13.570 --> 00:50:18.050 and -- and a simple calculation gives that answer to be pi. 00:50:18.050 --> 00:50:23.360 So that's an easy integral and it turns out to be pi, 00:50:23.360 --> 00:50:31.900 so that a1 is one over pi times there -- times this integral. 00:50:31.900 --> 00:50:35.490 So there is, actually -- that's Euler's famous formula 00:50:35.490 --> 00:50:39.110 for the -- or maybe Fourier found it -- 00:50:39.110 --> 00:50:41.405 for the coefficients in a Fourier series. 00:50:43.980 --> 00:50:47.940 And you see that it's exactly an expansion 00:50:47.940 --> 00:50:50.790 in an orthonormal basis. 00:50:50.790 --> 00:50:51.540 Okay, thanks. 00:50:51.540 --> 00:50:56.660 So I'll do a quiz review on Monday and then the quiz itself 00:50:56.660 --> 00:50:59.200 in Walker on Wednesday. 00:50:59.200 --> 00:51:00.590 Okay, see you Monday. 00:51:00.590 --> 00:51:02.140 Thanks.