Notation I use

Vectors, matrices and scalars

Symbol Meaning
$\Ab$ matrix
$\xb$ vector
$x$ scalar

Approximation, ground truth, etc

This is pretty universal; it applies vectors/matrices/etc.

Symbol Meaning
$\xb$ A general variable, typically used in $\arg \max$
$\xb^\star$ The ground truth.
$\xhatb$ An approximation for ground truth.

I represent the gradient or vector derivative with $\grad{f}$ for some function $f(x)$.

This is analogous to the derivative of one variable; all my calculus-level intuition for one variable carries over. Also, it helps to have some gradient vector identities.

Norms

$\ell_p$ norms are defined to be

$$\norm{\xb}_p := \parens{\sum x_i^p}^{1/p}$$

This means that $\norm{\xb}_2^2 = \sum x_i^2$ and $\norm{\xb}_1 = \sum \abs{x_i}$

Conventions

I’ll typically call the output of some system $\yb$ or $\bb$. I’ll typically call the input $\xb$ (or $\wb$ for weights). I’ll typically call a linear system $\Ab$ (or $\Xb$, typically when using weights).

I’ll also use $x := 3$ to mean “$x$ is defined to be $3$”.

Mathematical sentences

Formal mathematics is written in complete sentences but using symbols. The following symbol have the follow meanings:

• $\forall$ or “for all” (and in latex \forall). i.e., “for all $x$, $x \ge 0$ or $x < 0$
• $\in$ or “in” (and in latex \in). i.e., $x \in [0, 1]$ if $x = 0.5$