Take-Home Messages

  • Ultimate goal of statistics is data analysis. E.g., PageRank, handwritten digit recognization, Netflix matrix completion, ...

    Statistics is partly empirical and partly mathematical. It is now almost entirely computational.
    Kenneth Lange

Statisticians used to .. Now we spend all time ...

  • Two essential skills for modern statisticians: programming and computational algorithms.

    Data Scientist (n.): Person who is better at statistics than any software engineer and better at software engineering than any statistician.
    Josh Willis on Twitter

  • Numerical linear algebra

    • building blocks of most computing we do.
    • Use standard (and good) libraries (BLAS, LAPACK, ...) as much as possible!
    • Sparse linear algebra and iterative solvers such as conjugate gradient (CG) methods are critical for exploiting structure in big data.

  • Optimization

    • Convex programming (LS, LP, QP, GP, SOCP, SDP). Download and study Stephen Boyd's book, watch lecture vides or take UCLA EE236B by Vandenberghe, familiarize yourself with the good optimization softwares. Convex programming is becoming a technology, just like least squares (LS). Browse the documentation of cvx and Convex.jl to see which functions are implemented.

    • Generic nonlinear optimization tools: Newton, Gauss-Newton, quasi-Newton, (nonlinear) conjugate gradient, ...

    • Optimization tools developed by statisticians: Fisher scoring, EM, MM, ... Take UCLA Biomath 210 by Kenneth Lange for a thorough study of MM algorithms.