This meta-analysis aimed to comprehensively assess the effects of long-term air pollution exposure on the risk of type 2 diabetes mellitus (T2DM). Studies were selected from three electronic databases. Random- or fixed-effect model was used to obtain the pooled hazard ratios (HRs) and corresponding 95% confidential intervals (CIs). Stratified analyses by regions of the studies and length of follow-up were conducted to assess the effects in different subgroups. Sensitivity analyses by omitted studies one by one, as well as adjusting certain confounding factors, were also conducted. The search resulted in 1878 studies, among which 16 studies with 18 cohorts were included. The incidence of T2DM was significantly associated with 10 μg/m3 increase of PM2.5 (overall HR = 1.11, 95% CI: 1.03, 1.19) and PM10 (overall HR = 1.12, 95% CI: 1.01, 1.23) exposure. Stratified analyses confirmed that PM2.5 was significantly associated with increased T2DM incidence in American countries but not European countries. The results in the long follow-up subgroup also confirmed that exposure of PM2.5 and PM10 was associated with increased T2DM incidence. Interestingly, educational level and gender could potentially affect the impacts of PM10 and PM2.5 on T2DM incidence. The findings show long-term exposure to PM2.5, and PM10 could significantly increase the incidence of T2DM, especially in cohorts with long follow-up time.