WEBVTT

00:00:06.870 --> 00:00:09.060
PROFESSOR: Hey, we're back.

00:00:09.060 --> 00:00:11.870
Today we're going to do a
singular value decomposition

00:00:11.870 --> 00:00:13.510
question.

00:00:13.510 --> 00:00:16.630
The problem is really
simple to state:

00:00:16.630 --> 00:00:19.970
find the singular value
decomposition of this matrix

00:00:19.970 --> 00:00:25.150
C equals [5, 5; -1, 7].

00:00:25.150 --> 00:00:28.680
Hit pause, try it yourself,
I'll be back in a minute

00:00:28.680 --> 00:00:29.920
and we can do it together.

00:00:40.360 --> 00:00:44.400
All right, we're back,
now let's do it together.

00:00:44.400 --> 00:00:46.050
Now, I know Professor
Strang has done

00:00:46.050 --> 00:00:50.600
a couple of these in lecture,
but as he pointed out there,

00:00:50.600 --> 00:00:52.430
it's really easy
to make a mistake,

00:00:52.430 --> 00:00:59.460
so you can never do enough
examples of finding the SVD.

00:00:59.460 --> 00:01:02.690
So, what does the SVD look like?

00:01:02.690 --> 00:01:05.190
What do we want to end up with?

00:01:05.190 --> 00:01:16.620
Well, we want a decomposition
C equals U sigma V transpose.

00:01:16.620 --> 00:01:20.620
U and V are going to be
orthogonal matrices, that

00:01:20.620 --> 00:01:25.690
is, their columns
are orthonormal sets.

00:01:25.690 --> 00:01:28.760
Sigma is going to
be a diagonal matrix

00:01:28.760 --> 00:01:31.690
with non-negative entries.

00:01:31.690 --> 00:01:32.610
OK, good.

00:01:32.610 --> 00:01:34.930
So now, how do we find
this decomposition?

00:01:34.930 --> 00:01:39.640
Well, we need two equations, OK?

00:01:39.640 --> 00:01:52.070
One is C transpose C is equal
to V, sigma transpose, sigma,

00:01:52.070 --> 00:01:54.260
V transpose.

00:01:54.260 --> 00:01:57.390
And you get this just by
plugging in C transpose C

00:01:57.390 --> 00:02:00.940
here and noticing that U
transpose U is 1, since U

00:02:00.940 --> 00:02:02.752
is an orthogonal matrix.

00:02:02.752 --> 00:02:04.200
Okay.

00:02:04.200 --> 00:02:08.620
And the second equation is just
noticing that V transpose is V

00:02:08.620 --> 00:02:11.640
inverse, and moving it to the
other side of the equation,

00:02:11.640 --> 00:02:15.930
which is C*V equals U*sigma.

00:02:15.930 --> 00:02:18.400
OK, so these are
the two equations

00:02:18.400 --> 00:02:24.470
we need to use to find
V, sigma, and U. OK,

00:02:24.470 --> 00:02:28.040
so let's start
with the first one.

00:02:28.040 --> 00:02:33.070
Let's compute C transpose
C. So C transpose C

00:02:33.070 --> 00:02:47.220
is that-- Well, if
you compute, we'll

00:02:47.220 --> 00:03:00.300
get a 26, an 18, an
18, and a 74, great.

00:03:00.300 --> 00:03:03.100
Now, what you notice
about this equation

00:03:03.100 --> 00:03:06.680
is this is just a
diagonalization of C transpose

00:03:06.680 --> 00:03:10.460
C. So we need to find
the eigenvalues-- those

00:03:10.460 --> 00:03:13.780
will be the entries
of sigma transpose

00:03:13.780 --> 00:03:16.160
sigma-- and the
eigenvectors which will be

00:03:16.160 --> 00:03:19.910
the columns of a V. Okay, good.

00:03:22.490 --> 00:03:24.810
So how do we find those?

00:03:24.810 --> 00:03:30.690
Well, we look at the determinant
of C transpose C minus lambda

00:03:30.690 --> 00:03:35.730
times the identity,
which is the determinant

00:03:35.730 --> 00:03:48.710
of 26 minus lambda, 18, 18,
and 74-- 74 minus lambda,

00:03:48.710 --> 00:03:49.320
thank you.

00:03:55.520 --> 00:03:59.860
Good, OK, and what
is that polynomial?

00:03:59.860 --> 00:04:06.460
Well, we get a lambda squared,
now the 26 plus 74 is 100,

00:04:06.460 --> 00:04:09.130
so minus 100*lambda.

00:04:09.130 --> 00:04:14.310
And I'll let you do 26 times 74
minus 18 squared on your own,

00:04:14.310 --> 00:04:22.100
but you'll see you get 1,600,
and this easily factors

00:04:22.100 --> 00:04:25.080
as lambda minus 20
times lambda minus 80.

00:04:27.880 --> 00:04:31.550
So the eigenvalues
are 20 and 80.

00:04:31.550 --> 00:04:33.150
Now what are the eigenvectors?

00:04:33.150 --> 00:04:39.850
Well, you take C transpose C
minus 20 times the identity,

00:04:39.850 --> 00:04:47.486
and you get 6, 18, 18 and 54.

00:04:50.400 --> 00:04:54.550
To find the eigenvector
with eigenvalue 20,

00:04:54.550 --> 00:04:58.940
we need to find a vector in
the null space of this matrix.

00:04:58.940 --> 00:05:02.650
Note that the second
column is three times

00:05:02.650 --> 00:05:08.430
the first column, so our
first vector, v_1, we can just

00:05:08.430 --> 00:05:16.220
take that to be, well, we
could take it to be [-3, 1],

00:05:16.220 --> 00:05:21.040
but we'd like it to be
a unit vector, right?

00:05:21.040 --> 00:05:23.960
Remember the columns of
v should be unit vectors

00:05:23.960 --> 00:05:25.600
because they're orthonormal.

00:05:25.600 --> 00:05:28.699
So 3 squared plus
1 squared is 10,

00:05:28.699 --> 00:05:30.490
we have to divide by
the square root of 10.

00:05:34.260 --> 00:05:43.580
OK, similarly, we do C transpose
C minus 80 times the identity,

00:05:43.580 --> 00:05:56.400
we'll get -54, 18; 18,
and -6, and similarly

00:05:56.400 --> 00:06:05.000
we can find that v_2 will
be 1 over square root of 10,

00:06:05.000 --> 00:06:08.360
3 over the square root of 10.

00:06:08.360 --> 00:06:11.550
Great, OK, so what
information do we have now?

00:06:11.550 --> 00:06:15.070
we have our V matrix, which
is just made up of these two

00:06:15.070 --> 00:06:18.720
columns, and we actually
have our sigma matrix too,

00:06:18.720 --> 00:06:23.670
because the squares of the
diagonal entries of sigma

00:06:23.670 --> 00:06:26.250
are 20 and 80.

00:06:26.250 --> 00:06:33.130
Good, so let's write those
down, write down what we have.

00:06:33.130 --> 00:06:40.640
So we have V-- I just
add these vectors

00:06:40.640 --> 00:06:43.120
and make them the
columns of my matrix.

00:06:43.120 --> 00:06:48.770
Square root of 10, 1
over square root of 10;

00:06:48.770 --> 00:06:55.570
1 over square root of 10,
3 over square root of 10.

00:06:55.570 --> 00:07:02.490
And sigma, this is just the
square roots of 20 and 80,

00:07:02.490 --> 00:07:08.085
which is just 2 root 5 and
4 root 5 along the diagonal.

00:07:11.200 --> 00:07:14.500
Squeezing it in here, I hope
you all can see these two.

00:07:17.104 --> 00:07:21.830
Good, so these are two of the
three parts of my singular

00:07:21.830 --> 00:07:24.940
value decomposition.

00:07:24.940 --> 00:07:27.190
The last thing I
need to find is u,

00:07:27.190 --> 00:07:31.600
and for that I need to use this
second equation right here.

00:07:31.600 --> 00:07:37.470
So you need to multiply
C times V, okay so So

00:07:37.470 --> 00:07:47.550
c is [5, 5; -1, 7],
let's multiply it by V,

00:07:47.550 --> 00:07:54.380
1 over root 10, 3 over
square root of 10.

00:07:54.380 --> 00:07:56.960
What do we get?

00:07:56.960 --> 00:08:01.920
Well, I'll let you
work out the details,

00:08:01.920 --> 00:08:05.000
but it's not hard here.

00:08:05.000 --> 00:08:09.650
You get -10 over root 10, which
is just negative square root

00:08:09.650 --> 00:08:12.180
of 10 here.

00:08:12.180 --> 00:08:32.940
Then I just get 2 square
root of 10, and then I get--

00:08:32.940 --> 00:08:44.760
1 is 2 square root of 10 and--

00:08:44.760 --> 00:08:49.730
I think I made an error here.

00:08:49.730 --> 00:08:55.350
Give me a second to look
through my computation again.

00:08:55.350 --> 00:08:58.300
AUDIENCE: [INAUDIBLE]

00:08:58.300 --> 00:09:03.410
PROFESSOR: The (2, 1) entry
should be-- oh, yes, thank you.

00:09:03.410 --> 00:09:05.660
The (2, 1) entry should
be the square root of 10.

00:09:05.660 --> 00:09:20.980
Good, yes, that's what I was
hoping, yes, because we get--

00:09:20.980 --> 00:09:24.380
Yes, I did it in
the wrong order,

00:09:24.380 --> 00:09:28.460
right, so your recitation
instructor should

00:09:28.460 --> 00:09:33.970
know how to multiply matrices,
great, yes, thank you.

00:09:33.970 --> 00:09:37.580
You multiply this first, then
this, then this, and then this,

00:09:37.580 --> 00:09:44.030
and if you do it correctly
you will get this matrix here.

00:09:44.030 --> 00:09:45.510
Good, great.

00:09:45.510 --> 00:09:49.980
So now I'd like this
to be my U matrix,

00:09:49.980 --> 00:09:55.990
but it's actually U times sigma,
so I need to make these entries

00:09:55.990 --> 00:09:57.570
unit length.

00:09:57.570 --> 00:10:08.260
OK, so I get -1 over root 2, 1
over root 2, 1 over root 2, 1

00:10:08.260 --> 00:10:12.120
over root 2, times
my sigma matrix

00:10:12.120 --> 00:10:17.740
here, which is, remember,
2 square root of 5,

00:10:17.740 --> 00:10:22.130
4 square root of 5,
and these constants

00:10:22.130 --> 00:10:25.390
are just what I needed to
divide these columns by in order

00:10:25.390 --> 00:10:27.840
to make them unit vectors.

00:10:27.840 --> 00:10:33.630
So now, here's my U matrix,
1 over square root of 2,

00:10:33.630 --> 00:10:38.250
-1 over square root of 2;
1 over square root of 2,

00:10:38.250 --> 00:10:41.160
1 over square root of 2, good.

00:10:41.160 --> 00:10:46.710
So now I have all three
matrices, U, V, and sigma

00:10:46.710 --> 00:10:49.340
and despite some little
errors here and there,

00:10:49.340 --> 00:10:51.950
I think this is actually right.

00:10:51.950 --> 00:10:54.340
You should go check it
yourself, because if you're

00:10:54.340 --> 00:10:59.370
at all like me, you've screwed
up several times by now.

00:10:59.370 --> 00:11:02.190
But anyway, this is
a good illustration

00:11:02.190 --> 00:11:05.120
of how to find the singular
value decomposition.

00:11:05.120 --> 00:11:07.880
Recall that you're
looking for U sigma V

00:11:07.880 --> 00:11:11.990
transpose where u and v
are orthogonal matrices,

00:11:11.990 --> 00:11:16.410
and sigma is diagonal
with non-negative entries.

00:11:16.410 --> 00:11:21.290
And you find it using
these two equations,

00:11:21.290 --> 00:11:26.540
you compute C transpose C,
that's V sigma transpose sigma

00:11:26.540 --> 00:11:32.340
times V transpose, and you
also have C*V is U*sigma.

00:11:32.340 --> 00:11:35.100
I hope this was a
helpful illustration.