Let’s say we want to find MLE parameters \(\theta\) for a conditional Gaussian with constant variance. That is:
\begin{equation} p\qty(y_{i} | x_{i}) = \mathcal{N} \qty(y_{i}|f_{\theta } \qty(x_{i}), \sigma^{2}) \end{equation}
and we have a corresponding dataset: \(\qty(x_1, y_1), …, \qty(x_{m}, y_{m})\).
where:
\begin{align} \hat{\theta} &= \arg\max_{\theta} \sum_{i=1}^{m} \log p\qty(y_{i}|x_{i}) \\ &= \arg\max_{\theta} \sum_{i=1}^{m} \log \mathcal{N} \qty(y_{i}| f_{\theta} \qty(x_{i}), \sigma^{2}) \\ &= \arg\max_{\theta } \sum_{i=1}^{m} \log \frac{1}{\sqrt{{2 \pi \sigma^{2}}}} \exp \qty(- \frac{\qty(y_{i}- f_{\theta }\qty(x_{i}))^{2}}{2\sigma^{2}}) \end{align}
taking the $log $ of an $exp $, and removing constants (since they don’t affect optimization), this gives us:
\begin{equation} \arg\max_{\theta} \sum_{i=1}^{m} - \qty(y_{i} - f_{\theta} (x_{i}))^{2} \end{equation}
which is the…
\begin{equation} \arg\min_{\theta} \sum_{i=1}^{m} \qty(y_{i} - f_{\theta} (x_{i}))^{2} \end{equation}