The China marginal sea is one of the most active areas of material and energy exchange in the sea—land—atmosphere. The reconstruction of marine ecological environment evolution is helpful for the comprehensive understanding of the coupling effects of human activity and climatic change. Biomarkers in marine sediments are effective means of qualitative/quantitative insights into the evolution of marine ecosystems, it could provide a realistic approach to predicting the evolution of marine ecosystems under the influence of human activities. In this study, the characteristics of commonly used marine sedimentary biomarkers from archaea, bacteria and eukaryote are summarized in terms of their marine/terrestrial origin, chemical structure, distribution pattern and function. Besides, the application principles and relative merits of different biomarkers in reconstructing marine temperature, pH, redox and phytoplankton evolution are generalized. The shortages of using multiple biomarkers in reconstructing the evolution of ecological environments in the China marginal sea are also summarized. Overall, the studies using biomarkers reconstructing the ecological evolution of the China marginal sea mostly stayed at the level of qualitative description. The interpretation of biomarker indicators may be misguided due to insufficient knowledge of the biogenic derivation of the biomarkers and the adaptive mechanisms, which in turn affects the accuracy of marine environmental reconstruction results. Besides, effective biomarker indicators have not been screened to indicate multiple important marine environmental parameters. Future research on marine ecosystem reconstruction should be further implemented by clarifying the biological sources combing the new technology and new method, integrating big data analysis methods to build correction formulas with more accurate correspondence for quantitative reconstruction and accurate prediction, and screening novel biomarkers in combination with functional microbial communities.