Reliability of multivariate causality measures for neural data

In the past decade several multivariate causality measures based on Granger causality have been suggested to assess directionality of neural signals. To date, however, a detailed evaluation of the reliability of these measures is largely missing. We systematically evaluated the performance of five different causality measures (squared partial directed coherence (sPDC), partial directed coherence (PDC), directed transfer function (DTF), direct directed transfer function (dDTF) and transfer function) depending upon data length, noise level, coupling strength, and model order and performed simulations based on four different neural data recording procedures (magnetoencephalography, electroencephalography, electromyography, intraoperative local field potentials). Moreover, we analyzed the effect of two common numerical methods to determine the significance of the particular causality measure (random permutation and the leave one out method (LOOM)). The simulations showed the sPDC combined with the LOOM to be the most reliable and robust choice for assessing directionality in neural data. While DTF and H by construction were unable to distinguish between direct and indirect connections, the dDTF also failed this test. Finally, we applied the causality measures to a real data set. This showed the usefulness of our simulation results for practical applications in order to draw correct inferences and distinguish between conflicting evidence obtained with different causality measures