Functional magnetic resonance imaging (fMRI) using blood oxygenation contrast has rapidly spread into many application areas. In this paper, a new statistical model is used to evaluate the reliability of fMRI activation in a finger opposition motor paradigm for both within‐session and between‐session data and in a working memory paradigm for between‐session data. A slice prescription procedure for between‐session reproducibility is introduced. Estimates are made for the probabilities of correctly and falsely classifying voxels as active or inactive and receiver operator characteristic curves are generated. In the motor paradigm, estimated between‐session reliability was found to be somewhat reduced relative to within‐session reliability; however, this includes additional sources of variation and may not reflect intrinsically lower reliability. After matching false‐positive classification probabilities, between‐session reliability was found to be nearly identical for both motor and cognitive activation paradigms.