Objective To assess the effect of physical activity on cognitive domains in persons with dementia or mild cognitive impairment. Design An overview of systematic reviews and meta-analyses of randomized controlled trials were performed. Methods A literature search was performed in PubMed, Scopus, and Cochrane Database of Systematic Reviews databases up to February 2020. Data about the change in cognitive domains after physical activity intervention was extracted and plotted. Results We included 11 meta-analyses in this overview. The most frequent type of physical activity for the intervention group was the aerobic exercise with a duration between 6 and 78 weeks. Global cognition was the most common form of assessing the cognitive function, follow of executive function, delayed recall, attention, and verbal fluency. We found a positive effect of physical activity on global cognition, executive function, and delayed recall, but no effect on verbal fluency, attention, and immediate recall. Conclusion Physical activity shows a positive effect on cognition in people with dementia and mild cognitive impairment, especially when it was assessed as global cognition. Positive effects on executive function and memory were also shown. This result confirms the relevance of physical activity in the treatment of persons with cognitive impairment.