Table of Contents
I’m swimming in reaction time data at the moment. My usual approach to analyzing reaction times in cognitive psychology is through some sort of normal random effects model of the log or inverse RT. This is really only good for picking up effects on the mean of the RT distribution, but we’re increasingly finding evidence that certain manipulations preferentially affect the upper or lower quantiles, and so we really need a more fine grained analysis than the usual “mean differences” approach.
Diffusion models (see e.g. Voss, Nagler & Lerche, 2013) are an extremely successful approach to the analysis of reaction time data, but they make specific assumptions about the cognitive processes underlying responses that aren’t really compatible with the kinds of conflict tasks that we use, so they can’t help us here. A second-best approach is to model the RT distribution directly, while avoiding any assumptions about the processes that generated the distribution.
The ex-Gaussian distribution
Theoretically, the ex-Gaussian distribution is the convolution of a normal distribution with an exponential distribution (equivalently, an ex-Gaussian random variable is the sum of a normal random variable with an exponential random variable), and is known to provide a very good fit to many kinds of empirical reaction time data. It is a three parameter distribution with parameters and , which control the mean and variance of the normal component, and a parameter which controls the rate of the exponential component (the right-skew of the distribution). Note that by convention it is often parametrized with . A few examples:
If you care, its density function is given by
The ex-Gaussian distribution can be fit straightforwardly by maximum likelihood, but it is known to be extremely sensitive to outliers (which usually means extremely long RT’s) and to require a very large sample. Outlier removal is tricky here, since the RT distribution is potentially very long-tailed, and so any approach that involves simply throwing away “extreme” observations is likely to throw away useful information.
Dealing with outliers by mixture modelling
An alternative to identifying specific observations as outliers is to assume instead that response times might be drawn from one of two distributions: An ex-Gaussian distribution of true responses, and a distribution of “contaminant” responses which are assumed to have been generated by some process other than normal decision making (the participant was distracted, they anticipated the stimulus and responded early, etc). These contaminant responses might be fast or slow, so for the sake of simplicity we assume that they are uniformly distributed (i.e. they can occur with any speed). The result looks something like this:
This is called a mixture model. We assume that the observed distribution of RT’s can be written as a weighted mixture of an ex-Gaussian and a uniform distribution. Intuitively, we can think of it as saying that an individual response comes from an distribution with probability , and a uniform distribution with probability . Our model can then be written
where is the smallest observed reaction time, and is the largest. This lets us fit an ex-Gaussian distribution to the data, while allowing for the fact that a few observations may not “fit” into the ex-Gaussian model.
Fitting the model
The downloads section includes Stan and R code for fitting the mixture model to a single participant. It also includes a file for fitting the model to simulated response times with outliers. I’m currently working on a hierarchical version of the model, to accomodate multiple participants, which will be uploaded when completed.
I’ve never actually seen this kind of model done before in a Bayesian way, so the priors should probably be played around with. The upper and lower limits for the uniform distribution of contaminants are not estimated; we simply define them to be the minimum and maximum of the data. All response times are assumed to be measured in seconds, so we choose priors that allow for a reasonable range of values
The mixture weights are trickier. I originally didn’t bother specifying a prior at all, but the model kept finding mixture weights on the order of for the contaminant distribution, which just seems unrealistic. I ended up sticking a fairly harsh beta prior
on the contaminant distribution weight to force it down to a more reasonable level around . An alternative is to not estimate the mixture weights at all, and just fix them at something like , but I wanted allow them to very in the future hierarchical model, so I settled for a strongly informative prior.
I’ve drawn observations from an ex-Gaussian distribution with parameters , , and , along with 10 uniformly distributed contaminant responses between and seconds. The simulated data look like this:
Normally, the rightmost observations would be tagged as outliers and deleted, but let’s see how the model fares. Posterior distributions for the model parameters are plotted below
The estimates of and are close to dead-on. The model seems to overestimate (the length of the tail), which may suggest that the contaminants are still exerting some influence, but it’s probably just a property of the ex-gaussian distribution itself. Maximum likelihood is known to overestimate in the absence of a huge sample, so that’s probably what’s happening here. I don’t think there’s any way of fixing this with the prior, since large values of are perfectly plausible in empirical data. I’m hoping that the shrinkage that comes from fitting a hierarchical version to many participants will improve the estimate a little. Code is on the downloads page.
Voss, A., Nagler, M., & Lerche, V. (2013). Diffusion models in experimental psychology: A practical introduction. Experimental psychology, 60(6), 385.