The s allele, of the 5-HTTLPR genotype has been associated with a negative effect on memory function, however very few studies have examined the impact of this gene polymorphism in older age. This is important given the functional impact of memory decline in ageing, and given that serotonin declines with age. The present study examined the effect of the 5-HTTLPR genotype (s/s and s/l versus l/l genotypes) on episodic and working memory function across age. This effect was also examined separately across males and females. A total of 298 female and 140 male participants were recruited as part of The Healthy Brian Project with a mean age 60.35 years (range 49-80). Participants were administered tests of visual and verbal episodic and working memory as well as being genotyped for the 5-HTTLPR gene. Moderation analysis revealed decline in verbal episodic memory across age was differentially predicted by the 5-HTTLPR genotype, with this effect being moderated by sex. There was no significant moderation of 5-HTTLPR on visual episodic or working memory measures across age. These findings suggest serotonin influences verbal episodic memory specifically and has a differential effect in women and men across age.