Consider a shop inventory which lists unit prices and quantities for each of the products they carry. For example, if the store has 32 small storage boxes at $4.99 each, 18 medium-sized boxes at $7.99 each, and 14 large boxes at $9.99 each, then the inventory's price vector and quantity vector are
The total value of the boxes in stock is
This operation—multiplying two vectors' entries in pairs and summing—arises often in applications of linear algebra and is also foundational in the theory of linear algebra.
Definition The dot product of two vectors in is defined by
Example If and then ⋅ + ⋅ + ⋅ + ⋅ = 100.
One of the most algebraically useful features of the dot product is its linearity (which may be checked using the definition):
The dot product also has two fundamental connections to geometry. The first is the identity
for all vectors .Let's see how this identity can work in conjunction with linearity of the dot product.
Exercise Show that for all vectors and in .
Solution.Using linearity of the dot product, we get
as required.
The second connection between geometry and the dot product pertains to angle. If is the angle between two vectors and (when they are situated so that their tails coincide), then
It follows that if and only if and meet at a angle. We say that two vectors and which satisfy are orthogonal.
Exercise In natural language processing, one basic way to compare a finite number of text documents is to use vectorized word counts.Suppose the documents have a combined total of distinct words, which are arranged in some order. Each document is then associated with a vector of length whose $i$th entry indicates the number of times the th word occurs in the associated document.
One way to measure similarity between two documents is to take the dot product of the associated unit vectors: If two documents and have associated vectors and respectively, their similarity is defined by
By the dot product cosine formula, we have for any two documents and Documents with no words in common are associated with orthogonal vectors and thus have similarity. If two documents have similarity their associated vectors are scalar multiples of each other, meaning that they have the same words and that the words appear in the same proportions.
The vectorized word count similarity between the sentences
"The rain in Spain falls mainly in the plain"
"The plain lane in Spain is mainly a pain"
is .
Solution.Listing the words in the order the, in, rain, Spain, falls, mainly, plain, lane, pain, is, a, the two vectorized word counts are and .Substituting into the definition of , we get a similarity of approximately 0.647.
Exercise Let be a list of orthogonal non-zero vectors, that is, for all suppose that whenever .Show that this list is linearly independent.
Solution.Suppose, for the sake of contradiction, that the vectors are linearly .Then one of the vectors can be written as a linear combination of the others. Suppose is such a vector. Then there exists a list of weights such that
Then
Since , this implies that is zero. Repeating this for all vectors we see that .Thus is also zero (since it's a linear combination of the other vectors, with all zero weights), and that contradicts the fact that .
The same reasoning tells us that none of the vectors in the list can be equal to a linear combination of the others. Therefore the vectors must be linearly .
The following exercise illustrates another way of calculating matrix products. We will call it the matrix product dot formula:
Exercise Let and Consider the matrix whose $(i,j)$th entry is equal to the dot product of the $i$th row of and the $j$th column of .Show that , and use this fact to work out the full product .
Solution.By the product column rule, the first column of is , where is the first of .Therefore, the first entry of that column is .This is the dot product of the first row of and the first column of .The same reasoning applies to the other entries.
Calculating all eight such dot products, we find that
Block multiplication
A block matrix is a matrix defined using smaller matrices which are called blocks. For example, suppose that
Then the block matrix defined in terms of the blocks ,,, and is
The advantage of writing a matrix in block form is that we can formally carry out the matrix multiplication dot formula, treating the blocks as matrix entries, and we get the correct result (in block form). For example,
if and are block matrices with blocks ,,,,,,, blocks .We call this the block matrix product formula.
Exercise Verify the matrix product block formula above with
Solution.We have
while
So the block matrix product formula checks out.
Exercise Show that if is a matrix whose columns are and is a matrix whose rows are , then
Solution.This follows directly from the block matrix product formula by writing is a block matrix with its columns as blocks and with its rows as blocks.
English