Notation I use

Vectors, matrices and scalars

Symbol	Meaning
$\Ab$	matrix
$\xb$	vector
$x$	scalar

Approximation, ground truth, etc

This is pretty universal; it applies vectors/matrices/etc.

Symbol	Meaning
$\xb$	A general variable, typically used in $\arg \max$
$\xb^\star$	The ground truth.
$\xhatb$	An approximation for ground truth.

Gradient

I represent the gradient or vector derivative with $\grad{f}$ for some function $f(x)$.

This is analogous to the derivative of one variable; all my calculus-level intuition for one variable carries over. Also, it helps to have some gradient vector identities.

Norms

$\ell_p$ norms are defined to be

$$ \norm{\xb}_p := \parens{\sum x_i^p}^{1/p} $$

This means that $\norm{\xb}_2^2 = \sum x_i^2$ and $\norm{\xb}_1 = \sum \abs{x_i}$

Conventions

I’ll typically call the output of some system $\yb$ or $\bb$. I’ll typically call the input $\xb$ (or $\wb$ for weights). I’ll typically call a linear system $\Ab$ (or $\Xb$, typically when using weights).

I’ll also use $x := 3$ to mean “$x$ is defined to be $3$”.

Mathematical sentences

Formal mathematics is written in complete sentences but using symbols. The following symbol have the follow meanings:

$\forall$ or “for all” (and in latex \forall). i.e., “for all $x$, $x \ge 0$ or $x < 0$
$\in$ or “in” (and in latex \in). i.e., $x \in [0, 1]$ if $x = 0.5$