Background Chronic low back pain (CLBP) is a common and often disabling musculoskeletal condition. Yoga has been proven to be an effective therapy for chronic low back pain. However, there are still controversies about the effects of yoga at different follow-up periods and compared with other physical therapy exercises. Objective To critically compare the effects of yoga for patients with chronic low back pain on pain, disability, quality of life with non-exercise (e.g. usual care, education), physical therapy exercise. Methods This study was registered in PROSPERO, and the registration number was CRD42020159865. Randomized controlled trials (RCTs) of online databases included PubMed, Web of Science, Cochrane Central Register of Controlled Trials, Embase which evaluated effects of yoga for patients with chronic low back pain on pain, disability, and quality of life were searched from inception time to November 1, 2019. Studies were eligible if they assessed at least one important outcome, namely pain, back-specific disability, quality of life. The Cochrane risk of bias tool was used to assess the methodological quality of includedrandomized controlled trials. The continuous outcomes were analyzed by calculating the mean difference (MD) or standardized mean difference (SMD) with 95% confidence intervals (CI) according to whether combining outcomes measured on different scales or not. Results A total of 18 randomized controlled trials were included in this meta-analysis. Yoga could significantly reduce pain at 4 to 8 weeks (MD = -0.83, 95% CI = -1.19 to -0.48, p<0.00001, I2 = 0%), 3 months (MD = -0.43, 95% CI = -0.64 to -0.23, p<0.0001, I2 = 0%), 6 to 7 months (MD = -0.56, 95% CI = -1.02 to -0.11, p = 0.02, I2 = 50%), and was not significant in 12 months (MD = -0.52, 95% CI = -1.64 to 0.59, p = 0.36, I2 = 87%) compared with non-exercise. Yoga was better than non-exercise on disability at 4 to 8 weeks (SMD = -0.30, 95% CI = -0.51 to -0.10, p = 0.003, I2 = 0%), 3 months (SMD = -0.31, 95% CI = -0.45 to -0.18, p<0.00001, I2 = 30%), 6 months (SMD = -0.38, 95% CI = -0.53 to -0.23, p<0.00001, I2 = 0%), 12 months (SMD = -0.33, 95% CI = -0.54 to -0.12, p = 0.002, I2 = 9%). There was no significant difference on pain, disability compared with physical therapy exercise group. Furthermore, it suggested that there was a non-significant difference on physical and mental quality of life between yoga and any other interventions. Conclusion This meta-analysis provided evidence from very low to moderate investigating the effectiveness of yoga for chronic low back pain patients at different time points. Yoga might decrease pain from short term to intermediate term and improve functional disability status from short term to long term compared with non-exercise (e.g. usual care, education). Yoga had the same effect on pain and disability as any other exercise or physical therapy. Yoga might not improve the physical and mental quality of life based on the result of a merging.