Statistical Methods to Compare Algorithmic Performance

This article introduces alternative techniques to compare algorithmic performance. The first approach is a controlled experiment where samples of orders are split into pairs and executed using different algorithms over the same time periods. This is appropriate for algorithms that use static trading parameters such as VWAP and percentage of volume (POV) strategies. The second approach is a two sample experiment where orders are executed over different time periods using different algorithms. This is appropriate for those algorithms with dynamic trading strategies and those that adapt to changing market conditions such as implementation shortfall and ultra-aggressive algorithms. These techniques will assist in determining differences across algorithmic performance. They are derived from non-parametric methods commonly used in biostatistics to compare the effectiveness of new drugs and medical treatment procedures