COMP5318 — Week 1 Supplement
Mathematical Foundations
How Python tables become vectors and matrices, how reductions over axes work, and which statistics and linear-algebra operations keep appearing throughout the rest of the course.
axis=0 aggregates down rows and returns one value per feature.From notebook tables to matrices and vectors
The Week 1 notebook is not mathematically heavy, but it quietly establishes the representation used everywhere else in the unit: arrays, shapes, slices, and tabular columns as features. If this layer is shaky, later formulas feel harder than they actually are.
One sample
A single example is a feature vector \(x=(x_1,\dots,x_d)\in\mathbb{R}^d\).
Whole dataset
Stack samples as rows to get \(X \in \mathbb{R}^{n \times d}\). Row \(i\) is sample \(i\); column \(j\) is feature \(j\).
NumPy shape
X.shape == (n, d). Reshaping changes how values are interpreted, not how many values exist.
Pandas view
A DataFrame is the same idea with labels: rows are observations, columns are named variables.
For a 2D array with shape \((n,d)\), axis=0 means “reduce over rows” and return one number per column, while axis=1 means “reduce over columns” and return one number per row.
The formulas you should already be comfortable with
Feature-wise statistics
For one feature column \(x_1,\dots,x_n\):
Sample mean: \[\bar{x}=\frac{1}{n}\sum_{i=1}^n x_i\]
Sample variance: \[s^2=\frac{1}{n-1}\sum_{i=1}^n(x_i-\bar{x})^2\]
Standard deviation: \(s=\sqrt{s^2}\)
Relationships between two features
For centred features \(a\) and \(b\):
Sample covariance: \[\operatorname{cov}(a,b)=\frac{1}{n-1}\sum_{i=1}^n(a_i-\bar{a})(b_i-\bar{b})\]
Correlation: \[\operatorname{corr}(a,b)=\frac{\operatorname{cov}(a,b)}{s_a s_b}\]
Covariance depends on units; correlation is unitless and always lies between \(-1\) and \(1\).
Dot product and linear scoring
\[ a \cdot b = a^\top b = \sum_{j=1}^d a_j b_j \]
Later models use this as \(w^\top x+b\), where \(w\) contains feature weights and \(b\) is the intercept or bias.
Worked Example 1: axes, means, and sample variance
Take the Week 1 style data matrix
\[ X=\begin{bmatrix} 2 & 1\\ 4 & 3\\ 6 & 5 \end{bmatrix} \]
Column-wise reductions
Each column is a feature, so
\[ \bar{x}_{\text{col}}=\frac{1}{3} \begin{bmatrix} 2+4+6,\ 1+3+5 \end{bmatrix} = \begin{bmatrix} 4,\ 3 \end{bmatrix} \]
In NumPy this is np.mean(X, axis=0).
Sample variance per feature
For feature 1, deviations are \((-2,0,2)\), so
\[ s_1^2=\frac{(-2)^2+0^2+2^2}{3-1}=\frac{8}{2}=4 \]
For feature 2, deviations are also \((-2,0,2)\), so \(s_2^2=4\).
Therefore the feature-wise sample variances are \([4,4]\).
Worked Example 2: covariance and a linear score
Covariance and correlation
Using the same two feature columns \(a=(2,4,6)\) and \(b=(1,3,5)\), the means are \(\bar{a}=4\) and \(\bar{b}=3\). So both centred columns are \((-2,0,2)\).
\[ \operatorname{cov}(a,b)=\frac{(-2)(-2)+0\cdot0+2\cdot2}{2}=\frac{8}{2}=4 \]
Since \(s_a=s_b=2\), the correlation is \(4/(2 \cdot 2)=1\). These two features move perfectly together.
Dot product and model score
Let \(w=(1.5,-0.5,2)\), \(x=(2,0,1)\), and \(b=-1\). Then
\[ w^\top x+b=(1.5)(2)+(-0.5)(0)+(2)(1)-1=3+0+2-1=4 \]
That scalar \(4\) is the model score. Later weeks turn this same pattern into regression predictions, logistic probabilities, and SVM decision values.
What to carry into the next chapters
Keep straight
- Rows are examples and columns are features unless the code explicitly says otherwise.
axis=0gives one result per feature;axis=1gives one result per example.- Variance needs a denominator choice: \(n\) for population style, \(n-1\) for sample estimation.
- The dot product is the reusable building block for linear machine-learning models.
Typical confusion
- Mixing up shape \((n,d)\) with \((d,n)\).
- Taking row means when the question asks for feature means.
- Treating covariance and correlation as interchangeable.
- Forgetting the intercept \(b\) when evaluating a linear score.
np.mean(X, axis=0) return?