Four key points of robustness of machine learning models
- Well-Possessedness: a problem is ill-posed if small changes in the inputs lead to large changes in the outputs, implying that any source of error would dominate the results; you can well-posedness
- Condition Number: an algorithm is ill-conditioned if small changes in the inputs lead to large changes in the output; large condition number is bad (our system is “sensitive”) and small condition number are good (insensitive); if the relative input/output change is identical, then condition number is 1
- Stability: an algorithm is stable if it could complete itself in a meaningful way—i.e. don’t create numerically explosive results
- Accuracy: we want low size of error (i.e. our answer should be close to the solution)