Linear algebra is the backbone of anything related to AI, or its subsets like Machine Learning, Deep Learning and Neural Networks, etc. Artificial intelligence is about processing and analyzing huge amount of data. To perform operations on such huge data, we need a optimal data structure.
And Linear algera provides us with such data structures like Matrices and Vectors. These data structures can be used to represent large datasets and perform operations on them.
In Artificial intelligence, you fit your model to the dataset you have. Fitting a dataset to a model means, refining the models parameters (feature weights) with comparing the model output with the actual output.
To represent those datasets and models, we use matrix and vectors. These two data structures are used to represent array of data with many dimensions. Each dimension representing a feature in a dataset.
Data Representation and Operation is the key in AI and Linear Algebra provides us with the methods to do that. Data Structures like Matrices and Vectors holds the data. And these structures have special operation like matrix multiplication, inverse calculation, determinant calculation, transpose, linear transformations, etc
Image Classifiers, Speech Recognition, Natural Language Processing are some of the applications of Neural Networks. And Neural networks is nothing but a matrix of neurons connected to each other with weights.
Neural Network is a layers of neurons connected to each other with weights. The process of refining these weights is called training. And training is done by comparing the model (layers of neurons (with weights)) output with the actual output.
Each layer of neuron is represented by a matrix. And the weights connecting the neurons are represented by the matrix elements.
Have you ever noticed that you getting recommendation on Youtube Videos, Tiktok videos. How do they know what you like? How do they know what you will like?
This is all because of a Recommender System. Recommender System is a system that recommends you the items based on your past behavior.
Recommender System in Linear Algebra is very simple algebraic operation. It uses matrix factorization to recommend you the items. There is matrix of users and items. Each elements in the matrix represents the rating (or feedback) of the user for the item.
It performs decomposition of the matrix into two matrices. One matrix represents the users and other matrix represents the items. And then it predicts the rating of the user for the item by multiplying the user matrix with the item matrix.
Its amazing how simple linear algebra operation predicts the items you will like. Feels like pure magic.
Computers don't see images like we human do. For computers images are just a matrix of pixel values. Each pixel value represents the intensity of the color.
Using those matrix, the computer recoginizes the objects in the image. A computer never sees image as a whole, it cannot see the image. They just read the pixel values and perform operations on them.
Models like Convolutional Neural Networks are used to process the images. These models are nothing but a matrix of neurons connected to each other with weights. And these weights are refined by comparing the model output with the actual output.
Application of Linear Algebra in Image Processing are:
Principal Component Analysis (PCA) is the process of reducing the dimensionality of the data.
So what does it mean to reduce the dimensionality of the data? It means to reduce the number of features in the dataset.
In more simpler term, for example I have a ecommerce site, and user buys a products. I have a dataset of user behavior or features (how much user buys, what user buys, when he logged in, what he searched, etc).
Now, I want to know what are the important features (user behavior) that are influencing the user to buy the product. So, I used PCA to discard the features that are not important and keep the features that are more important.
For example, the users time of login may not be important feature (keyword: "May"), but the product he buys may be important feature (keyword: "May").
PCA uses linear algebra to reduce the dimensionality of the data. It uses matrix operations like eigenvalues and eigenvectors to reduce the dimensionality of the data.
Single Value Decomposition is also a technique used to reduce the dimensionality of the data.
It uses linear algebra to decompose the feature matrix into different matrices to reduce the dimensionality of the data by keeping the important features.
Single Value Decomposition is used in Image Compression, Data Compression, and in Recommender Systems.
As you can see, there is no single way of using linear algebra in AI. There are different techniques and method that uses Linear Algebra to solve the problem.
Beginner in AI, may find it difficult to step into AI, because there is no single way to solve the problem. One blogs teaches image compression using SVD, where as other blog reaches image compression using PCA. so, its important to understand these are different techniques to solve the same problem.
However, having knowledge of them is important, but you don't have to master them all at once.
To understand them all at once, you need to have a good understanding of Linear Algebra.
As you can see almost everything in AI is related to Linear Algebra. Because AI is nothing but a mathematical model that learns from the data. If you want to get into AI, you need to have a good understanding of Linear Algebra.
Linear Algebra is not the easiest subject to learn, but its not that hard either. With practice and understanding, you can master it. With introduction to LLMs and Libraries like TensorFlow, Pytorch, Keras, etc, you don't have to understand the Linear Algebra to get into AI.
But if you have a good understanding of Linear Algebra, you are in unfair advantage. You can understand the models better, you can understand how data is being tranformed and processed. You can understand the bottlenecks of the model and so much more.
