Essential Math for DataScience — Linear Algebra

Row Vector, Column Vector, Dot Product

---

Linear Algebra for Data Science

A good understanding of linear algebra is much needed for understanding many machine learning algorithms. 

In this article, we will learn about vectors, the dot product of vectors, and the geometric representation of the dot product in detail.

Table of Content

1. How point is represented as a vector?
2. Distance between point and origin
3. Distance between two points
4. Row Vector, Column Vector, Matrix
5. Mathematical Operations on Vector
6. Dot Product between two vectors
7. Geometric Interpretation of the Dot Product

---

How a Point is Represented as a Vector?

Let us consider a point P in 2-D space.

Coordinates of Point P is (2,3). The first number tells how far to walk along the x-axis from the origin and then the second number tells how far to walk along the x2 axis after that.

[I have mentioned axis as x1,x2 instead of x,y. In this way, axis representation will be easy for n-dimensional space]

The number of elements in the vector represents the dimensionality of the vector. 2 is the x1 component and 3 is the x2 component of the vector.

In 3-D space and n-D space

Likewise, A Point in three-dimensional space can be represented by a vector of size 3.

Example:

Similarly for n-Dimensional space, the point will be represented by a vector of size n. [n components]

Distance between the point and origin

Let’s calculate the distance of the point from the origin.

1. In 2-D Space.

The distance of the point from the origin is calculated by using the Pythagoras theorem. 

Example. Distance between Point (a,b) from the origin is 

d=sqrt(a²+b² )

2. In 3-D and n-D space

Likewise, we can extend the formula to 3-D space and n-D space.

Distance between two points

Let’s calculate the distance between two points P and Q.

By using Pythagoras Theorem, we can calculate the distance between these two points.

In 3-D Space and n-D Space

Likewise, we can extend the formula to 3-D space and n-D space.

---

Row Vector, Column Vector, and Matrix

Row Vector

A vector having only one row is known as a row vector.

Column Vector

A vector having only one column is known as a column vector

The default orientation of vectors is column vectors.

Transpose

The transpose of a row vector is a column vector.

Likewise, the transpose of a column vector is a row vector.

Matrix

A matrix with m rows and n columns is known as m*n matrix.

---

Mathematical Operations on Vector

Addition of two vectors

To add two vectors, we have to add the corresponding components. It’s a component-wise addition.

Example:

Given two vectors A and B, let’s calculate A+B

Multiplication

Multiplication of vectors can be done in two ways.

1. dot product 
2. cross product 

[cross product is not used much in DataScience when compared to the dot product. So we will look into dot product only in detail]

Dot Product of two vectors

The dot product is defined as the component-wise product of the two vectors.
The dot product of two vectors returns a scalar value.

Geometric Interpretation of the Dot Product

Let’s understand the geometric representation of the dot product.

If a and b are two vectors, the angle between the two vectors is θ, then the dot product is written as 

a.b=‖a‖‖b‖Cos θ

‖a‖ → Length of vector a [Distance of a from the origin]

‖b‖ → Length of vector b[Distance of b from the origin]

[In the above section, we have calculated the distance of the point from the origin. 

Using this formula, we can calculate the length of the vector 

Given components of vector a (a1,a2), vector b(b1,b2) → a.b can be written as 

a.b=a1b1+a2b2
We also know that

a.b=‖a‖‖b‖Cos θ

Hence a.b =a1b1+a2b2 =‖a‖‖b‖Cos θ

From the dot product of the two vectors, we can compute the angle between them.

Two vectors are perpendicular

If two vectors are perpendicular to each other, their dot product is always zero.

Dot Product in Numpy

In python, NumPy provides a function to compute the dot product of vectors.

In NumPy, a vector in n-dimensional space is represented as an n-dimensional array.

Example.

import numpy as np
a = np.array([1,2])
b = np.array([3,4])
print (np.dot(a,b))
#Output: 11

Conclusion:

In this article, I have covered vectors, the dot product of vectors, and the geometric representation of vectors. 

Key takeaways

The dot product of vectors will result in a scalar value.

The dot product of two vectors that are perpendicular to each other is zero.

The dot product of two vectors also used to find the angle between the two vectors.

Happy Learning!

Reference