In the present study we report a double dissociation between right and left medial temporal lobe damage in the modulation of fear responses to different types of stimuli. We found that right unilateral temporal lobectomy (RTL) patients, in contrast to control subjects and left temporal lobectomy (LTL) patients, failed to show potentiated startle while viewing negative pictures. However, the opposite pattern of impairment was observed during a stimulus that patients had been told signaled the possibility of shock. Control subjects and RTL patients showed potentiated startle while LTL patients failed to show potentiated startle. We hypothesize that the right medial temporal lobe modulates fear responses while viewing emotional pictures, which involves exposure to (emotional) visual information and is consistent with the emotional processing traditionally ascribed to the right hemisphere. In contrast, the left medial temporal lobe modulates fear responses when those responses are the result of a linguistic/cognitive representation acquired through language, which, like other verbally mediated material, generally involves the left hemisphere. Additional evidence from case studies suggests that, within the medial temporal lobe, the amygdala is responsible for this modulation.