Removing Electrode Pops

Initialize NeuroAnalyzer
using NeuroAnalyzer
using CairoMakie
CairoMakie.activate!(type = "png")
using DataFrames
using Statistics
NeuroAnalyzer.progress_bar = false
NeuroAnalyzer.Random.seed!(42)

# Makie figure and axis settings matching NeuroAnalyzer plots
fig_size = (900, 450)
ax_style = (titlesize = 18,
            xlabelsize = 18,
            ylabelsize = 18,
            xticklabelsize = 12,
            yticklabelsize = 12,
            xticks = LinearTicks(10),
            xminorticksvisible = true,
            xminorticks = IntervalsBetween(10),
            xautolimitmargin = (0, 0),
            yautolimitmargin = (0.1, 0.1))

An electrode pop is an artifact caused by a sudden change of the electrode-skin contact: a loose electrode, drying gel, cable movement or pressure on the electrode. The half-cell potential at the electrode changes abruptly and then slowly returns to its previous value as the contact stabilizes. In the recording, this appears as:

  • a step: an almost instantaneous jump of the signal (often tens to thousands of μV), usually in a single channel
  • a recovery: an exponential return to the previous level over tens to hundreds of milliseconds

Pops are larger and faster than any brain activity and, if not removed, distort amplitude measures, spectra (the step has a broad 1/f spectrum) and ICA decompositions.

A pop differs from other fast artifacts:

  • spikes (e.g. a single bad sample) return to the previous level immediately
  • eye blinks and movements rise and fall gradually
  • amplifier saturation or a lost electrode produces a flat segment that does not recover exponentially

If a pop affects all channels at the same time, its source is usually the reference (or ground) electrode.

Method

remove_pops() models a pop starting at sample \(t_0\) as a step followed by an exponential decay:

\[ \mathrm{pop}(t) = a \, e^{-(t - t_0) / \tau}, \quad t \ge t_0 \]

where \(a\) is the pop amplitude and \(\tau\) is the recovery time constant (after \(\tau\) the pop has decayed to 37%, after \(3\tau\) to 5% of its amplitude).

t_m = collect(0:(1 / 256):(1 - 1 / 256))
plot_signal(t_m, 300 .* exp.(-t_m ./ 0.15);
            ylabel = "Amplitude [μV]",
            title = "Pop model: a = 300 μV, τ = 0.15 s")

The algorithm works on each channel separately:

  1. Candidates: the first difference of the signal (the sample-to-sample change) is converted to a robust z-score (using the median and the MAD calculated within window seconds, so that changes of the signal amplitude during the recording do not matter). Jumps with z > z (default: 10) are candidates; within r samples only the largest jump is kept.
  2. Step test: the mean level after the jump must differ from the level before by at least half of the jump. Single-sample spikes, which return immediately, are rejected.
  3. Model fit: \(a\) and \(\tau\) are fitted (least squares) to the r samples after the jump, relative to the level before the jump. For each \(\tau\) the best \(a\) is calculated analytically; \(\tau\) is found by a grid search refined by golden-section search.
  4. Acceptance: a candidate is accepted as a pop if it is large (\(|a| \ge\) a_min × robust SD of the signal, default: 5) and the model explains the signal (\(R^2 \ge\) r2_min over the first \(3\tau\), default: 0.8). This rejects blinks, muscle activity and other fast activity that is not a step with an exponential recovery.
  5. Repair: the fitted model is subtracted from the signal (from the pop to the next pop or the end of the signal). Because only the artifact is subtracted, the underlying EEG is preserved: there is no interpolation, smoothing or zeroing.

Parameters:

Parameter Default Meaning
z 10 detection threshold (robust z-score of the first difference)
a_min 5 minimum pop amplitude relative to the robust SD of the signal
r2_min 0.8 minimum goodness of fit of the exponential model
r sr(obj) ÷ 2 fitting segment (samples) after the jump; also the minimum distance between pops
window 10 window (seconds) for the robust statistics

Synthetic data

A 10 Hz rhythm (20 μV) with pink noise and two pops: +300 μV with τ = 40 samples (156 ms) at 4.68 s and −200 μV with τ = 20 samples (78 ms) at 7.42 s. The second channel contains the same signal without pops.

fs = 256
t = collect(0:(1 / fs):(10 - 1 / fs))
n = length(t)
eeg_signal = generate_sine(10, t, 20) .+ generate_noise(n, 5; type = :pink)
pop = zeros(n)
pop[1200:end] .+= 300 .* exp.(-(0:(n - 1200)) ./ 40)
pop[1900:end] .-= 200 .* exp.(-(0:(n - 1900)) ./ 20)

syn = create_object(; data_type = "eeg")
syn.header.recording[:sampling_rate] = fs
add_channel!(syn;
             data = reshape(vcat((eeg_signal .+ pop)', eeg_signal'), 2, n, 1),
             label = ["Pop", "Clean"],
             type = ["eeg", "eeg"],
             unit = ["μV", "μV"])
create_time!(syn; fs = fs)
NeuroAnalyzer.plot(syn; ch = "all", gui = false)

Detect and repair the pops:

syn_clean, pops = remove_pops(syn; ch = "all")
pops
2×5 DataFrame
Row channel sample time amplitude tau
String Int64 Float64 Float64 Float64
1 Pop 1200 4.68359 303.306 0.168853
2 Pop 1900 7.41797 -186.954 0.0738814

Both pops are found at the correct samples, with amplitudes and time constants (in seconds) close to the true values; no pop is found in the clean channel.

Original (blue) and repaired (black) signal:

NeuroAnalyzer.plot(syn, syn_clean;
                   ch = "Pop",
                   seg = (4, 9),
                   gui = false)

The difference between the repaired signal and the original EEG (without pops) shows that only the artifact was removed:

residual = syn_clean.data[1, :, 1] .- eeg_signal
plot_signal(t, residual;
            ylabel = "Residual [μV]",
            title = "Repaired signal - original EEG")
println("RMS of the residual: ",
        round(sqrt(mean(abs2, residual)); digits = 2), " μV")
RMS of the residual: 2.09 μV

Real EEG

Pops are best detected after removing the DC offset, the power line noise and slow drifts: the power line noise increases the sample-to-sample changes and slow drifts make the level before and after the pop uncertain.

eeg = import_edf("files/eeg/eeg.edf")
remove_dc!(eeg; ch = "eeg")
remove_powerline!(eeg; ch = "eeg", pl_frq = 50)
NeuroAnalyzer.filter!(eeg;
                      ch = "eeg",
                      fprototype = :butterworth,
                      ftype = :hp,
                      cutoff = 0.5,
                      order = 4)
NeuroAnalyzer.filter!(eeg;
                      ch = "eeg",
                      fprototype = :fir,
                      ftype = :lp,
                      cutoff = 45,
                      bw = 2)

A known pop in real EEG

To check the repair on real background activity, a pop of 150 μV with τ = 0.1 s is added to channel O1 at 30 s:

eeg_pop = deepcopy(eeg)
o1 = get_channel(eeg; ch = "O1")[1]
p0 = 30 * sr(eeg) + 1
n_eeg = signal_len(eeg)
tau = 0.1 * sr(eeg)
eeg_pop.data[o1, p0:end, 1] .+= 150 .* exp.(-(0:(n_eeg - p0)) ./ tau)
eeg_rep, pops_o1 = remove_pops(eeg_pop; ch = "O1")
pops_o1
3×5 DataFrame
Row channel sample time amplitude tau
String Int64 Float64 Float64 Float64
1 O1 2329 9.09375 222.904 0.581053
2 O1 7681 30.0 153.665 0.0903361
3 O1 16008 62.5273 94.8454 0.547638

The added pop (at 30 s) is found with its amplitude and time constant; two other pops are already present in this recording (see below).

NeuroAnalyzer.plot(eeg_pop, eeg_rep;
                   ch = "O1",
                   seg = (28, 33),
                   gui = false)

Repaired (black) and original EEG without the added pop (blue): the background activity is preserved. A small difference (up to about 10 μV) remains during the first 0.4 s after the pop, because the fitted amplitude and time constant (154 μV, 0.09 s) differ slightly from the true ones (150 μV, 0.1 s).

NeuroAnalyzer.plot(eeg, eeg_rep;
                   ch = "O1",
                   seg = (29.5, 31.5),
                   gui = false)

Pops in the recording

Detection only:

pops_eeg = remove_pops(eeg; ch = "eeg", repair = false)
first(pops_eeg, 10)
10×5 DataFrame
Row channel sample time amplitude tau
String Int64 Float64 Float64 Float64
1 Fp1 2329 9.09375 228.816 0.568452
2 Fp1 16008 62.5273 91.0071 0.534361
3 Fp1 38836 151.699 -271.958 0.734351
4 Fp1 113439 443.117 -34.0759 0.10784
5 Fp2 15716 61.3867 -5297.05 1.08378
6 Fp2 28936 113.027 -21.4553 0.807897
7 F3 2329 9.09375 226.639 0.584838
8 F3 16008 62.5273 88.0053 0.624537
9 F3 167257 653.344 19.6573 0.0914836
10 F3 167987 656.195 17.8553 0.0244234

Number of channels with a pop at the same time:

n_ch = combine(groupby(pops_eeg, :time), nrow => :channels)
sort(n_ch, :channels; rev = true)
22×2 DataFrame
Row time channels
Float64 Int64
1 9.09375 17
2 62.5273 14
3 62.5234 3
4 1204.38 2
5 151.699 1
6 443.117 1
7 61.3867 1
8 113.027 1
9 653.344 1
10 656.195 1
11 9.08984 1
12 248.969 1
13 135.988 1
14 151.695 1
15 646.742 1
16 1192.64 1
17 355.523 1
18 382.008 1
19 680.371 1
20 148.648 1
21 361.559 1
22 1183.93 1

Most pops occur in almost all channels at the same time (9.1 s and 62.5 s): these are pops of the reference electrode. The remaining pops are single-channel pops.

eeg_clean, _ = remove_pops(eeg; ch = "eeg")
NeuroAnalyzer.plot(eeg, eeg_clean;
                   ch = "eeg",
                   seg = (5, 15),
                   gui = false)

Limitations

At 61.4 s, the raw signal of Fp2 drops by about 5.5 mV and stays clipped at about −5 mV for about 1 s (the amplifier saturates), then recovers over about 2 s. After high-pass filtering, the clipped segment becomes a large slow wave, which the two-pass filter also spreads backward in time (the ramp before 61.4 s). This is not a pop: remove_pops() removes the drop at 61.4 s as a pop of −5.3 mV, but the rest of the artifact does not follow the exponential model, and the “repaired” signal is not better than the original:

# the signal plots scale channels robustly, so the -5 mV step would be
# off scale; plotted in μV instead
fp2 = get_channel(eeg; ch = "Fp2")[1]
idx = round(Int, 59 * sr(eeg)):round(Int, 67 * sr(eeg))
fig = Figure(size = fig_size)
ax = Axis(fig[1, 1];
          ax_style...,
          title = "Fp2",
          xlabel = "Time [s]",
          ylabel = "Amplitude [μV]")
lines!(ax, eeg.time_pts[idx], eeg.data[fp2, idx, 1];
       color = (:blue, 0.5),
       label = "original")
lines!(ax, eeg.time_pts[idx], eeg_clean.data[fp2, idx, 1];
       color = :black,
       label = "repaired")
axislegend(ax; position = :rb)
fig

Such segments should be inspected and rejected (or the channel interpolated). A large step that does not recover at all within r samples (the fitted τ reaches its upper limit) is not treated as a pop: it is not repaired and remove_pops() shows a warning with its sample number.

Options

Lower detection thresholds (more, smaller pops) and a longer fitting segment (pops with slower recovery):

eeg_clean, pops = remove_pops(eeg;
                              ch = "eeg",
                              z = 8,
                              a_min = 3,
                              r = sr(eeg))

Accept pops that follow the exponential model less closely:

eeg_clean, pops = remove_pops(eeg;
                              ch = "eeg",
                              r2_min = 0.6)

In-place repair (returns the detected pops):

pops = remove_pops!(eeg; ch = "eeg")

remove_pops() and remove_pops!() must be applied to a continuous (non-epoched) signal.

Tip: Remove the DC offset, the power line noise and slow drifts (high-pass filter) before removing pops.

Tip: Always inspect the detected pops (plot(obj, obj_clean)): pops of the reference electrode appear in all channels, saturated or flat segments should be rejected instead of repaired.

Tip: If one electrode pops repeatedly, the channel should be marked as bad and interpolated.