There’s something called “jamais vu” which is the opposite of “deja vu” and is the feeling of experiencing something familiar but suddenly find it overwhelmingly confusing and new. You may have experienced this when repeating a word over and over again in your head and suddenly finding that word actually makes no sense…“roof”, “roof”, “roof”…?
I recently had this feeling when looking at the equation of a line.
\[y=mx + b\]or my new favorite form
\[ax + by + c = 0\]The $y = mx + b$ form feels intuitive, there’s two things you can change, the slope and the y-intercept so all good there. So why write it like $ax + by + c = 0$ ?
$ax + by + c = 0$ is equivalent to $\langle a, b \rangle \cdot \langle x, y \rangle + c = 0$. Now this reveals some more observations.
For starters, the vector $\langle a, b \rangle$ is orthogonal to the line. That’s already a useful bit of information, but why is this the case?
To answer this question, I had to re-think about the dot product. Calculating the dot product is simple enough. Given two vectors, the dot product is the following:
\[\langle a, b \rangle \cdot \langle c, d \rangle = ac + bd\]But in understanding the meaning of the dot product, I had to go back to the 1D case of the dot product: scalar multiplication.
\[2 * 3 = 6\]One way of thinking about this multiplication problem is as a stretch of the number line. Picture the number $3$ sitting on a number line. Multiplying by $2$ stretches that line by a factor of two, so the point that was at $3$ slides out to $6$. So $2 * 3$ becomes a matter of identifying $3$ on an axis scaled by $2$ or vice versa. Either way, this results in 6.
Our 2D case adds complexity. When you multiply a vector like
\[\langle 2, 2 \rangle \cdot \langle 6, 6 \rangle\]Grant Sanderson from 3Blue1Brown illustrates this 2 dimensional case very well in his video here. I have to keep coming back to it every once in a while when I see matrices.
Another factor that gets introduced when we increase dimensions is the introduction of the idea “cosine similarity”.
Here’s another way to calculate the dot product
\[\vec{a} \cdot \vec{b} = |\vec{a}||\vec{b}|cos(\theta)\]The setup is just two vectors leaving the origin: $\vec{a}$ at angle $\alpha$ from the x-axis, and $\vec{b}$ at angle $\beta$. The angle between them is the difference, $\theta = \beta - \alpha$. This can be mathematically verified as follows. Here’s a proof overview for the 2D case from mathematics stack exchange.
The transition I find most “hand-wavy” in this proof is between these two lines
\[\\ = ab(cos(\alpha)cos(\beta) + sin(\alpha)sin(\beta)) \\ = ab \cdot cos(\beta - \alpha)\]That step is the cosine angle-difference identity,
\[cos(\beta - \alpha) = cos(\beta)cos(\alpha) + sin(\beta)sin(\alpha)\]Naming it doesn’t explain it though. $cos(\beta - \alpha)$ is the length of the shadow that a unit vector at angle $\beta$ casts onto the direction pointing at angle $\alpha$. A unit vector projected onto a direction $\theta$ away from it keeps a fraction $cos(\theta)$ of its length, so that shadow has length $cos(\beta - \alpha)$.
Projection is linear, so I can chop the vector into pieces, project each piece, and add the results. Split the unit vector at $\beta$ into its horizontal and vertical parts:
- its horizontal part has length $cos(\beta)$, pointing along the x-axis,
- its vertical part has length $sin(\beta)$, pointing along the y-axis.
Now project each of those onto the direction at angle $\alpha$:
- the x-axis sits at angle $\alpha$ from that direction, so one unit of “x” survives as $cos(\alpha)$,
- the y-axis sits at angle $90^\circ - \alpha$ from it, so one unit of “y” survives as $cos(90^\circ - \alpha) = sin(\alpha)$.
Add the two contributions back up:
\[\underbrace{cos(\beta)}_{\text{x part}} cos(\alpha) + \underbrace{sin(\beta)}_{\text{y part}} sin(\alpha) = cos(\beta - \alpha)\]The identity is just “break the vector into components, project each, and sum.” Multiplying matching components and adding them is the same shape as $ac + bd$, which is why the trig form and the coordinate form of the dot product agree.
The identity shows up here because an N dimensional vector $\vec{a}$, usually written in terms of its value on each dimension axis $\vec{a} = \langle a_0, a_1, a_2, …, a_n \rangle$, can also be written by its magnitude and angle. In 2D:
\[\vec{a} = |\vec{a}| \langle cos(\alpha), sin(\alpha) \rangle, \quad \vec{b} = |\vec{b}| \langle cos(\beta), sin(\beta) \rangle\]Take the dot product of those two forms and you get
\[\vec{a} \cdot \vec{b} = |\vec{a}||\vec{b}|(cos(\alpha)cos(\beta) + sin(\alpha)sin(\beta)) = |\vec{a}||\vec{b}|cos(\beta - \alpha)\]which is the geometric formula from before, with $\theta = \beta - \alpha$.
Back to the line
This is enough to see why $\langle a, b \rangle$ is orthogonal to the line $ax + by + c = 0$.
Take any two points $P_1$ and $P_2$ that sit on the line. Both satisfy the equation:
\[\langle a, b \rangle \cdot P_1 + c = 0 \quad \text{and} \quad \langle a, b \rangle \cdot P_2 + c = 0\]Subtract one from the other and the $c$ cancels:
\[\langle a, b \rangle \cdot (P_2 - P_1) = 0\]But $P_2 - P_1$ is a vector that points along the line. So $\langle a, b \rangle$ has a zero dot product with the direction of the line, and by the cosine formula we just built, a zero dot product between two nonzero vectors means $cos(\theta) = 0$, so $\theta = 90^\circ$.
So $\langle a, b \rangle$ is the normal vector, and $ax + by + c = 0$ is the set of all points whose displacement along the line has no component in the $\langle a, b \rangle$ direction. The jamais vu wears off, and the equation reads normally again.