FitBenchmarking: an open source Python package comparing data fitting software

What's Changed FitBenchmarking v0.2.0 is the result of 10 months of work, and we've introduced a lot of changes to the FitBenchmarking package. This release now requires Python 3.7.1 or greater. New Features Support for different cost functions was added; see below Improved support for different jacobians; see below Better handling of conflicting options More descriptions of the methods in docs Box constraints supported Licences of the software used documented Minimizers are collected into types Second derivatives (Hessians) supported, and can be compared Added an option to select algorithms by type Results directory can be set from the command line A maximum runtime can be selected for each solver Parsers Mantid's Crystal Field objective is supported Native parser has been refactored IVP parser has been added Fitting Software iminuit>2.0 is supported levmar supported Matlab supported (basic, and curve_fitting, statistics and optimization toolboxes) horace supported (Matlab package) scipy_go supported gradient_free_optimizers supported bumps support has been updated to version 0.9.0 Cost functions NLLS and WeightedNLLSs cost functions replace the defaults HellingerNLLS cost function Poisson cost function added Multiple cost functions can be compared at once Jacobians numdifftools Jacobians have been added Analytic Jacobians added for NIST and SIF file formats Solver default Jacobians have been added Multiple Jacobians can be compared at once Tables the generation of these has been refactored Failed fits are highlighted in the tables Customizable colourmaps in the tables Header row and leading column of results table is frozen Problem descriptions displayed in fitting report Accuracy and runtime displayed in fitting report Problem summary page has been added Data sets more SASView examples added Data Assimilation (IVP) examples added CrystalField examples added Global optimization SIF file examples added Breaking Changes Support is dropped for iminuit<2.0 lm-scipy-no-jac option under scipy_ls in [MINIMIZERS] has been removed (this is now available under the solver default jacobian) results_dir has moved from [PLOTTING] into [OUTPUT] in the options file use_errors has been removed as an option, and this functionality is controlled by the choice of cost function The Jacobian method SciPyFD is no longer supported; this has been renamed to scipy colour_scale option in [PLOTTING] is no longer supported; this is replaced by the colour_map bumps minimizers name changes: lm-bumps -> scipy-leastsq and mp -> lm-bumps Full Changelog: https://github.com/fitbenchmarking/fitbenchmarking/compare/v0.1.5...v0.2.