Cognitive control is required to regulate interactions between brain regions to produce effective, purposeful behavior. We used functional magnetic resonance imaging to investigate the nature of these interactions and the role of prefrontal cortex (PFC) in cognitive control as subjects switched between simple face and word categorization tasks. Face and word stimuli were used because previous research has shown them to activate distinguishable cortical regions, allowing us to measure levels of activity in task-selective brain regions during task switching. We found that activity in brain regions selective for the currently irrelevant task predicted the behavioral cost associated with switching tasks. This finding supports the theory that between-task competition is a critical determinant of behavior. Task switching was also associated with increased activity in a network of regions implicated in cognitive control, including lateral PFC and parietal cortex. Within this network of regions, we observed dissociations between task-selective and general purpose mechanisms. These findings provide support for theories that propose a control hierarchy comprising regions responsible for maintaining task-specific information about rules or goals, and regions involved in the coordination of these goals.