🎓 Theory

📄️ AST

AST stands for Abstract Syntax Tree. It is a tree representation of code.

When picking the next token, instead of considering the probability of the next

To train with QLoRA, one needs to pick a lot of parameters

It is one of the GPT API of OpenAI. The documentation can be found here:

An embedding model is a model that is designed to convert text to

Classification results can be presented as a confusion matrix:

Deep learning is the area of machine learning that deals with

To train a network, you need to reduce the error the network makes. This

There is three term that we use interchangeably but are subtly different.

MPS stands for Metal Performance Shader. It's the name of the shader language

A matrix is a 2d table of number. It has a size called the dimension. An example

Neural networks are a family of algorithm that are able to learn from examples.

PCA stands for principal component analysis. It's a way to convert vectors with

Consider the following function from $\mathbb$ to $\mathbb{R}$.

It's a prompt given to the model to indicate how to behave with the user.

Source

The rank of a Matrix of size $n \times n$ is an integer

Computers are famously good with numbers. But they are bad when dealing with

Training a Neural network means changing its

Source

When using Embedding models to vectorize words

A vector are elements that can be added together and scaled by a constant. It is