Chronic low back pain (CLBP) affects nearly 20-25% of the population older than 65 years, and it is currently the main cause of disability both in the developed and developing countries. It is crucial to reach an optimal management of this condition in older patients to improve their quality of life. This review evaluates the effectiveness of physical activity (PA) to improve disability and pain in older people with non-specific CLBP. The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines were used to improve the reporting of the review. Individual risk of bias of single studies was assessed using Rob 2 tool and ROBINS-I tool. The quality of evidence assessment was performed using GRADE analysis only in articles that presents full data. The articles were searched in different web portals (Medline, Scopus, CINAHL, EMBASE, and CENTRAL). All the articles reported respect the following inclusion criteria: patients > 65 years old who underwent physical activities for the treatment of CLBP. A total of 12 studies were included: 7 randomized controlled trials (RCT), 3 non-randomized controlled trials (NRCT), 1 pre and post intervention study (PPIS), and 1 case series (CS). The studies showed high heterogeneity in terms of study design, interventions, and outcome variables. In general, post-treatment data showed a trend in the improvement for disability and pain. However, considering the low quality of evidence of the studies, the high risk of bias, the languages limitations, the lack of significant results of some studies, and the lack of literature on this argument, further studies are necessary to improve the evidences on the topic.