Osteoporosis is a major health problem in post-menopausal women (PMW). Exercise training is considered a cost-effective strategy to prevent osteoporosis in middle aged-older people. The purpose of this study is to summarize the effect of exercise on BMD among PMW. A comprehensive search of electronic databases was conducted through PubMed, Scopus, Web of Science, Cochrane, Science Direct, Eric, ProQuest, and Primo. BMD changes (standardized mean differences: SMD) of the lumbar spine (LS) femoral neck (FN) and/or total hip were considered as outcome measures. After subgroup categorization, statistical methods were used to combine data and compare subgroups. Seventy-five studies were included. The pooled number of participants was 5,300 (intervention group: n = 2,901, control group: n = 2,399). The pooled estimate of random effect analysis was SMD = 0.37, 95%-CI: 0.25–0.50, SMD = 0.33, 95%-CI: 0.23–0.43, and SMD = 0.40, 95%-CI: 0.28–0.51 for LS, FN, and total Hip-BMD, respectively. In the present meta-analysis, there was a significant (p < 0.001), but rather low effect (SMD = 0.33–0.40) of exercise on BMD at LS and proximal femur. A large variation among the single study findings was observed, with highly effective studies but also studies that trigger significant negative results. These findings can be largely attributed to differences among the exercise protocols of the studies. Findings suggest that the true effect of exercise on BMD is diluted by a considerable amount of studies with inadequate exercise protocols.