Scipy Lectures Notes Basic Introduction to Scipy – cluster & optimize modules Function Optimization With SciPy – Jason Brownlee Stanford class – Introduction to Scientific Python Scipy jac = ‘cs’ – using complex derivative which gives more accurate results than other numerical methods (2-point and 3-point). In order to understand how complex derivative works, … Read more
Fundamental algorithms for scientific computing in Python: Scipy SciPy User Guide Scipy Optimize Scipy minimize: algorithms scipy.optimize.least_squares: lsq Optimization and root finding: scipy Three examples of nonlinear least-squares fitting in Python with SciPy: examples Scipy Lecture Notes: notes Python scipy.optimize.leastsq() Examples minimize(method=’BFGS’): BFGS Broyden–Fletcher–Goldfarb–Shanno algorithm: wikipedia Limited-memory BFGS: wikipedia Large-scale Bound-constrained Optimization: L-BFGS-B Newton’s method in … Read more
In order to clearly understand the algorithms behind ML, it is important to have clear numerical analysis understanding. First a good summary of gradient descent methods are explained in this paper: An overview of gradient descent optimization algorithms – arxiv Then, Michel Bierlaire from the EPFL in Switzerland wrote a good book on optimization, and … Read more