Linearity

An operation is linear if it satisfies two conditions:

1. Additivity: $L(x+y) = L(x) + L(y)$
2. Homogeneity (Scaling): $L(a * x) = a * L(x)$

Some linear operations are:

• Addition, Subtraction, Multiplication, Division
• Matrix Multiplication
• Dot Product
• Convolution
• Affine Transform*
• Average Pooling

Affine transform is not actually linear because of the bias but most of the time it's considered linear. ?? This is because an affine transformation in an $n$ dimensional space is equivalent to a linear transformation in an $n+1$ dimensional space (using homogeneous coordinates) ??