Get started
Cheatsheet

AI Practice

The numpy patterns behind the AI Practice labs: shapes, stable numerics, vectorised maths, gradients and seeded randomness.

Practice this path

Arrays and shapes

  • x = np.array([[1.0, 2.0], [3.0, 4.0]]); x.shape

    Build an array and read its shape, here (2, 2)

  • x.reshape(2, 4)  |  x.reshape(-1)  |  x[:, None]

    Change the shape; -1 lets numpy work out one size; None adds an axis

  • x.sum(axis=1, keepdims=True)  |  x.max(axis=-1)

    Reduce along an axis; keepdims keeps the axis so the result still broadcasts

  • x[np.arange(len(y)), y]

    Pick one value per row, for example the true-class score for each label

  • np.where(mask, a, b)  |  x[x > 0]

    Choose between two arrays by a condition; filter by a boolean mask

  • np.zeros_like(x)  |  np.eye(4)[labels]

    Zeros of the same shape; one-hot rows from integer labels

Stable numerics

  • z = x - x.max(axis=1, keepdims=True); e = np.exp(z)

    Subtract the row maximum before exp so large values cannot overflow

  • np.log(np.exp(z).sum(axis=1))

    A log-sum-exp (with z already shifted); use it instead of log(softmax)

  • np.logaddexp(0, -z)

    A stable log(1 + exp(-z)), the softplus used in logistic and focal losses

  • 0.5 * (1 + np.tanh(0.5 * z))

    A sigmoid that does not overflow for large negative z

  • np.divide(a, b, out=np.zeros_like(a), where=b > 0)

    Divide only where it is safe and leave zeros elsewhere

  • with np.errstate(all="ignore"): ...

    Silence overflow warnings while you test a step size that blows up

Vectorised maths

  • x @ W.T  |  np.einsum("bd,cd->bc", a, b)

    Matrix products, written two ways

  • np.linalg.norm(m, axis=1, keepdims=True)

    The length of every row

  • sims = (a / na) @ (b / nb).T

    All pairwise cosine similarities from one matrix product, with rows normalised first

  • np.argsort(-sims, axis=1, kind="stable")[:, :k]

    The k best per row; a stable sort keeps ties in index order

  • np.take_along_axis(sims, idx, axis=1)

    Read the scores at those indices, row by row

  • np.linalg.eigvalsh(h).max()

    The largest eigenvalue of a symmetric matrix, for example a loss curvature

Losses and gradients

  • np.mean((X @ w + b - y) ** 2)

    Mean squared error for a linear model

  • 2 / N * X.T @ (X @ w + b - y)

    Its gradient with respect to the weights

  • (p - one_hot) / len(y)

    The gradient of mean softmax cross-entropy with respect to the logits

  • dh = d2 @ W2.T * h * (1 - h)

    Backpropagate one sigmoid layer: the next delta times the weights times the slope

  • (f(x + e) - f(x - e)) / (2 * e)

    A finite-difference check of a gradient, with e around 1e-6

  • w = w - lr * grad

    One gradient descent step; make a new array rather than editing in place

Randomness and results

  • rng = np.random.default_rng(0)

    A seeded generator: the same numbers every run, so results can be checked

  • rng.normal(size=(100, 3))  |  rng.random(x.shape)  |  rng.integers(n)

    Normal samples, uniform samples in [0, 1), a random integer

  • rng.permutation(n)

    A shuffled order of 0..n-1

  • np.savez("model.npz", W=W, b=b)  |  d = np.load("model.npz"); d["W"]

    Save and load named arrays

  • np.allclose(a, b, atol=1e-6)

    Compare floats with a tolerance instead of ==

  • python3 -W ignore script.py > answers.txt

    Run a script and keep what it prints, for the labs that grade an answers file

Cheatsheet for the AI Practice path · Deboistech · All cheatsheets