Course 10, lesson 91 of 100, Adults
Vectors, matrices and tensors
The language of neural networks
Like I’m 5
AI thinks in lists of numbers. A list is a vector, a grid is a matrix, and a stack of grids is a tensor. Everything an AI sees becomes these numbers.
The big idea
A vector is an ordered list of numbers, like [0.2, -1.3, 0.7], and can represent a word, a pixel colour or a user's taste. The dot product multiplies matching entries and adds them up, giving a measure of how aligned two vectors are.
A neural network layer is mostly a matrix multiplication: output = W·x + b, followed by a non-linear function. Tensors generalise this to many dimensions, like a batch of colour images (batch × height × width × channels). GPUs are built to do these multiplications in parallel.
Examples
- Word vectors: 'king' − 'man' + 'woman' lands near 'queen' in classic embeddings.
- Images: A 224×224 colour photo is a 224×224×3 tensor.
- Dot product: [1, 2]·[3, 4] = 1×3 + 2×4 = 11.
How it works
- Represent inputs as vectors or tensors.
- Multiply by weight matrices and add biases.
- Apply a non-linear function and pass the result to the next layer.
Check your understanding
- What is [1, 2] · [3, 4]?
- Options: 11; 10; 7.
Answer: 11. 1×3 + 2×4 = 3 + 8 = 11. - What does a dense neural network layer mainly compute?
- Options: A matrix multiplication plus a bias, then a non-linearity; A sorting algorithm; A random guess.
Answer: A matrix multiplication plus a bias, then a non-linearity. Layers are W·x + b followed by an activation such as ReLU.
Remember
Neural networks are built from vectors, matrices and tensors, multiplied together at scale.
Talk about it
How could you describe your favourite food as a vector of numbers?
Go deeper
Cosine similarity is the dot product of normalised vectors. Without non-linear activations, stacked linear layers would collapse into a single linear map, which is why ReLU, GELU and similar functions matter.