River systems are important components of earths surface landforms, which represent the complex interactions of tectonics, climate change, ecology and landscape formation. The movements of sediments determine the modes of river systems evolution. The sediment grain size on river bed was often used to characterize the direction of river channel adjustment. For most rivers, the grain size of river sediments has a downstream fining trend. However, the controls and mechanisms of downstream fining remain unclear. In this study, we comprehensively analyzed the distribution of downstream grain size fining and coarsening, and the controls of abrasion, selective transport, weathering and lateral input on downstream grain size change in previously reported investigations. The abrasion is significantly dependent on the lithology of the watershed, and abrasion- easy particles produce a strong mass loss. Although the impact of abrasion on the downstream grain size fining was limited, the mechanical abrasion caused by splitting and chipping in upstream of the river cannot be ignored. The abrasion rate from tumbling mill and abrasion tank experiments are usually 10 to 100 times lower than that from field observations. In contrast to abrasion, the hydraulic selective transport or sorting received wide concerns. It shows that hydraulic selective transport may play an important role in the grain size downstream fining. In addition, some studies also discussed the comprehensive influencing factors of the river bed grain size downstream change, and found that the grain size of sediments in some river sections exhibit downstream coarsening. Downstream coarsening was generally observed in colluvial channels with steep slopes in mountainous areas due to massive input of coarse sediments from tributaries, river bank, debris flow, glacial feeding and artificial river regulation. We found that in most mountain rivers,the grain size of river sediments was controlled by multiple factors. More field work and flume experiments are needed to reveal the driving mechanisms of grain size change of river sediment.