Essential Math for DataScience — Linear Algebra

Row Vector, Column Vector, Dot Product

Photo by Andrea Piacquadio from Pexels

Linear Algebra for Data Science

A good understanding of linear algebra is much needed for understanding many machine learning algorithms. 

In this article, we will learn about vectors, the dot product of vectors, and the geometric representation of the dot product in detail.

Table of Content

  1. How point is represented as a vector?
  2. Distance between point and origin
  3. Distance between two points
  4. Row Vector, Column Vector, Matrix
  5. Mathematical Operations on Vector
  6. Dot Product between two vectors
  7. Geometric Interpretation of the Dot Product

How a Point is Represented as a Vector?

Let us consider a point P in 2-D space.

Point P in 2-D space[Image by Author]

Coordinates of Point P is (2,3). The first number tells how far to walk along the x-axis from the origin and then the second number tells how far to walk along the x2 axis after that.

[I have mentioned axis as x1,x2 instead of x,y. In this way, axis representation will be easy for n-dimensional space]

Vector Representation [Image by Author]

The number of elements in the vector represents the dimensionality of the vector. 2 is the x1 component and 3 is the x2 component of the vector.

In 3-D space and n-D space

Likewise, A Point in three-dimensional space can be represented by a vector of size 3.

Example:

Vector representation of Point in 3-D space

Similarly for n-Dimensional space, the point will be represented by a vector of size n. [n components]

Distance between the point and origin

Let’s calculate the distance of the point from the origin.

  1. In 2-D Space.
Image by Author

The distance of the point from the origin is calculated by using the Pythagoras theorem. 

Example. Distance between Point (a,b) from the origin is 

d=sqrt(a²+b² )

Image by Author

2. In 3-D and n-D space

Likewise, we can extend the formula to 3-D space and n-D space.

Distance between point and origin [Image by Author]

Distance between two points

Let’s calculate the distance between two points P and Q.

Image by Author

By using Pythagoras Theorem, we can calculate the distance between these two points.

Image by Author

In 3-D Space and n-D Space

Likewise, we can extend the formula to 3-D space and n-D space.


Row Vector, Column Vector, and Matrix

Row Vector

A vector having only one row is known as a row vector.

Row Vector Representation [Image by Author]

Column Vector

A vector having only one column is known as a column vector

Column Vector Representation [Image by Author]

The default orientation of vectors is column vectors.

Transpose

The transpose of a row vector is a column vector.

Transpose [Image by Author]

Likewise, the transpose of a column vector is a row vector.

Matrix

A matrix with m rows and n columns is known as m*n matrix.


Mathematical Operations on Vector

Addition of two vectors

To add two vectors, we have to add the corresponding components. It’s a component-wise addition.

Example:

Given two vectors A and B, let’s calculate A+B

Addition of two vectors A+B

Multiplication

Multiplication of vectors can be done in two ways.

  1. dot product 
  2. cross product 

[cross product is not used much in DataScience when compared to the dot product. So we will look into dot product only in detail]

Dot Product of two vectors

The dot product is defined as the component-wise product of the two vectors.
The dot product of two vectors returns a scalar value.

Dot product [Image by Author]

Geometric Interpretation of the Dot Product

Let’s understand the geometric representation of the dot product.

If a and b are two vectors, the angle between the two vectors is θ, then the dot product is written as 

a.b=‖a‖‖b‖Cos θ

‖a‖ → Length of vector a [Distance of a from the origin]

‖b‖ → Length of vector b[Distance of b from the origin]

[In the above section, we have calculated the distance of the point from the origin. 

distance formula 

Using this formula, we can calculate the length of the vector 

Geometric Representation of dot product [Image by Author]

Given components of vector a (a1,a2), vector b(b1,b2) → a.b can be written as 

a.b=a1b1+a2b2
We also know that

a.b=‖a‖‖b‖Cos θ

Hence a.b =a1b1+a2b2 =‖a‖‖b‖Cos θ

From the dot product of the two vectors, we can compute the angle between them.

Two vectors are perpendicular

If two vectors are perpendicular to each other, their dot product is always zero.

Image by Author

Dot Product in Numpy

In python, NumPy provides a function to compute the dot product of vectors.

In NumPy, a vector in n-dimensional space is represented as an n-dimensional array.

Example.

import numpy as np
a = np.array([1,2])
b = np.array([3,4])
print (np.dot(a,b))
#Output: 11

Conclusion:

In this article, I have covered vectors, the dot product of vectors, and the geometric representation of vectors. 

Key takeaways

The dot product of vectors will result in a scalar value.

The dot product of two vectors that are perpendicular to each other is zero.

The dot product of two vectors also used to find the angle between the two vectors.

Happy Learning!

Reference

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s