A Brief Tour of the Home Page on WWW Statistical Experiment Archives: Benchmark Descriptions and Posted Solutions to NP-hard Problems

More than a thousand mathematical problems arising in engineering and science have been shown to be NP-hard. Problems of practical size that are NP-hard can only be solved by devising polynomial-time heuristics, with no guarantee whatsoever on the quality of the solution. Extensive experimentation and comparative analysis is required before we can decide on a `better heuristic'. Web archives on this site support the design of experiments to evaluate performance of algorithms that solve NP-hard problem. The purpose of this report is to give the reader a brief navigational tour of the current directories, address the challenges goals and objectives of these archives, contrast current approaches to the design and reporting of benchmarking experiments with alternatives that are based on proven principles from statistics, provide a number of illustrative, tutorial-level examples, and motivate the reader to participate in collaborative, peer-reviewed web-based benchmarking experiments duri..