Experts can reliably identify interictal epileptiform discharges (IEDs) in routine electroencephalograms (EEGs) in an effort to diagnose and manage patients with either suspected or established epilepsy, a study in JAMA Neurology suggests.

Despite the availability of commercial automated IED detection software, it is unclear how well the software compares with human experts. In this study, investigators performed a large prospective analysis to assess the reliability of clinical neurophysiology experts in scoring IEDs using a large number of EEGs, IEDs, and experts.

A total of 1051 consecutive noninvasive scalp EEG recordings performed at a single center between 2012 and 2016 were included. Phase 1 of the study included 9 experts who independently identified candidate IEDs among EEGs reported in the medical record to contain at least 1 IED (n=991). The total number of IED candidates marked was 87,636.

In phase 2 of the study, candidate IEDs were clustered into groups based on distinct morphologic features, which reduced the total to 12,602. Negative controls consisted of 660 waveforms, including 11 random samples from 60 randomly selected EEGs designated as free of IEDs (totaling 13,262 candidate IEDs). All candidates were independently scored by 8 experts as either IEDs or non-IEDs.

The primary outcome included the percentage of agreement and beyond-chance agreement for each individual IED (IED-wise interrater reliability ) as well as whether an EEG contained an IED (EEG-wise IRR). Secondary outcome included the correlation between numbers of IEDs identified by experts across cases.

During phase 1, potential IEDs were each marked by 9 experts in a median of 65 (interquartile range, 28-332) EEGs. There was fair expert IRR for the 13,262 individually annotated IED candidates, represented by a mean percentage of agreement of 72.4% (95% CI, 67.0%-77.8%) and mean κ of 48.7% (95% CI, 37.3%-60.1%). Comparatively, there was substantial EEG-wise IRR, represented by a mean percentage of agreement of 80.9% (95% CI, 76.2%-85.7%) and mean κ of 69.4% (95% CI, 60.3%-78.5%).

The majority of the experts that scored EEGs were trained at the same institution, which may represent a potential limitation of this study.

“Our results establish robust estimates for the reliability of experts for identifying IEDs in routine EEG recordings and provide a basis for evaluating automated IED detection systems,” concluded the researchers. Moreover, they believe that these findings “present a standard for how well an automated IED detection system must perform to be considered comparable in skill to a human expert.”

For more articles like this please sign up for our eTips by liking us on Facebook and giving us your email for our Newsletter.

Reference

Jing J, Herlopian A, Karakis I, et al. Interrater reliability of experts in identifying interictal epileptiform discharges in electroencephalograms . JAMA Neurol. doi: 10.1001/jamaneurol.2019.3531