Quantitative Evaluation and Evolution Characteristics of Consistency Level of Black Soil Conservation Policy, Taking China’s Black Soil Protection Policy as an Example

Abstract

Scientifically evaluating China’s black soil protection policy and exploring its evolution characteristics are crucial for promoting high-quality and efficient black soil protection projects worldwide. Utilizing the Policy Modeling Consistency (PMC) index model, the effectiveness of China’s black soil protection policy system is assessed. Building upon this evaluation, the research delves into the evolution process of the black soil protection policy system, examining its dynamic characteristics across different government levels. Through the application of text mining technology and visual analysis techniques, the study explores four dimensions: keyword co-occurrence, vertical synergy calculation, policy theme identification, and policy tool analysis. The results show the following: (1) The PMC index showed a phased growth trend, and various indicators gradually improved and tended to be stable. (2) Government policies at the same level exhibit progressive evolution patterns in terms of policy keywords and themes. (3) Across different levels of government, there are hierarchical lag evolution rules observed in policy keywords and provincial policy tools. Additionally, policy keywords display vertical synergy evolution patterns. In conclusion, future black soil conservation policies need to reduce hierarchical delays, enhance the coverage and comprehensiveness of policy content, and establish a sound policy feedback mechanism. The conclusions of the study are significant for improving management approaches throughout the black soil zones of the world.

1. Introduction

Black soil is one of the most valuable soil resources on Earth, containing biodiversity and significant amounts of soil organic carbon [1]. It is recognized worldwide as the best land for agriculture and is key to climate action, sustainable and resilient livelihoods, and food security. Since 2017, the FAO, through its Global Soil Partnership, has been actively engaged in the conservation and sustainable management of black soil. It highlights the benefits, challenges, and opportunities of black soil and recommends actions to support a sustainable future for black soil. While countries have started to prioritize the protection of black soil [2], the FAO highlights the significant challenge posed by the absence of effective policies dedicated to black soil preservation. This deficiency not only jeopardizes the quality and health of black soil but also threatens global food security. The FAO underscores the critical importance of governance at both national and subnational levels in safeguarding black soil [3].

As one of the three black soil areas in the world, China has always regarded black soil as China’s vegetable basket [4]. In 2022, they produced 18.3% of rice, 33.4% of corn, and 51.8% of soybeans [5]. Only China has a national law in place to protect, conserve, and encourage sustainable management of black soils now. It is hoped that the examples of effective policies can serve as an inspiration for the adoption of improved management approaches throughout the black soil zones of the world [3]. Therefore, it is necessary to study the policy of black soil protection in China.

It is found that the existing research on black soil protection mainly focuses on the soil characteristics of black soil and the methods and measures of black soil protection. Scholars mainly evaluated black soil from soil quality [6,7], soil productivity [8], and other aspects, and found that there are problems such as soil organic matter decline and soil erosion [9,10,11] in black soil at present. Given these problems, scholars studied and proposed land use strategies such as three-line defense control mode, pyramid control mode, minimum development, and maximum protection control mode [12], as well as black soil protection technologies such as straw returning to field [13] and conservation tillage [14]. For black soil protection policy, scholars mainly use differences-in-differences and other methods to evaluate the effects of black soil protection policy [15,16], including economic effects and ecological effects.

There are few studies on the policy system of black soil protection; some scholars have carried out preliminary studies on it, such as evaluating the resilience of cultivated land ecosystem in black soil areas under the background of China’s cultivated land protection policy [17]. However, cultivated land protection policy pays attention to the protection of the quantity–quality–ecology trinity [18], while black soil protection policy pays more attention to solving the core problems such as soil management and sustainable management, which cannot be equated, so there is still a lack of research on black soil protection policy. As the first country in the world to legislate on black soil protection, the consistency and evolution law of China’s black soil protection policy system has not yet formed a scientific and systematic quantitative evaluation and research system, which makes it difficult to provide scientific reference basis for subsequent adjustment and optimization of policy measures, judgment of future trends, and guidance of policy application, and also makes it difficult to provide reference significance for soil management in black soil areas all over the world.

The PMC index model and text mining method can be used to quantitatively study the consistency and evolution characteristics of policy texts. However, there are still several points to be discussed: First, the PMC index model is a method to evaluate the internal consistency of policy by taking the policy text itself as the research object. Compared with the existing DID method for ex-post evaluation of policy effects, the PMC index model focuses more on the policy content itself, and can also analyze the advantages and disadvantages of the policy [19]. Secondly, quantitative text analysis has become a common method for studying policy texts. Quantitative text analysis methods can effectively extract information, recognize emotions, and analyze policy features by focusing on both the external characteristics and content of policy documents [20]. However, when quantitative text analysis is used to study the text content of policy system, most of them only pay attention to the focus of policy or the use of policy tools [21], and do not include keywords, policy themes, policy tools, etc., in a unified research framework to study the characteristics of policy evolution. Thirdly, most of the existing studies focus on the whole policy system, revealing the evolution characteristics of the whole policy system, and lack of hierarchy to explore the evolution characteristics of the same level and the relationship between different levels. Black soil protection policy includes not only the governance at the national level but also the protection at all levels below the national level [3]. There are differences in the evolution rules between policies at the same level and between policies at different levels. Therefore, it is necessary to explore the evolution rules of black soil protection policies at different levels from a multidimensional perspective.

To sum up, to make up for the shortcomings of the existing research, this study takes the black soil protection policy as the research object, uses the PMC index model to evaluate the effectiveness of the black soil protection policy system, divides the evolution stages based on the fluctuation law of policy consistency, and integrates social network analysis, vertical synergy measurement, confusion measurement, text similarity, and other methods. From the four dimensions of keyword co-occurrence, vertical synergy calculation, policy theme identification, and policy tool analysis, this paper constructs an analysis framework of the evolution law of China’s black soil protection policy system and uses a variety of visualization technologies to reveal the evolution characteristics and development law of the black soil protection policy system. The contributions of this paper are as follows: Firstly, different from the evaluation of policy effect afterward, this paper evaluates the effectiveness of black soil protection policy itself in advance, explores the evolution law, and divides the evolution stage of policy based on the fluctuation law of policy effectiveness. Secondly, integrating keyword co-occurrence, policy theme identification, and policy tool analysis into the same research framework enables a comprehensive exploration of the evolution of policy systems across multiple dimensions, using various methods in text quantitative analysis. Thirdly, it not only discusses the evolution characteristics of policies at the same level of government but also reveals the relationship and evolution rules between policies issued by different levels of government. The conclusions can provide a new research paradigm and theoretical basis for the research of the black soil protection policy system, and provide a theoretical basis and practical reference for optimizing and perfecting black soil protection policy and promoting global black soil protection.

2. Materials and Methods

According to the policy connotation, this paper uses the policy consistency index model to study the effectiveness of black land protection policy, and divides the policy evolution stage. On this basis, from the same level, using text mining, co-word analysis, theme recognition, and other methods to study the evolution of policy keywords and policy themes, policy keywords represent the content focus of policy text, and extracting policy keywords can sort out the policy focus at different levels of government development [22]. Through the mining and analysis of policy themes, we can clearly find the change of policy core direction. At different levels [23], vertical synergy degree calculation and text similarity calculation are used to study the evolution law of policy keywords and policy tools, quantify the vertical synergy of black soil protection policy, and test the synergy degree between policies at different levels [24]. By mining the types and quantities of policy tools, the implementation subjects of policies at different development stages can be clarified [22]. The overall research idea of this paper is shown in Figure 1.

2.1. Research Methods

2.1.1. PMC-Index Model

The evaluation of policy effectiveness is to evaluate the effectiveness of the policy, which can evaluate and analyze the policy content itself, to find the problems existing in the policy, and timely modify and improve it. The policy consistency index model is suitable for the evaluation of policy effectiveness. Its advantage lies in that the variable setting is based on the comprehensive analysis and mining of the policy text, and the assignment is strictly based on the policy content, which can avoid artificial subjectivity and improve the accuracy of policy quantitative evaluation. Therefore, this paper uses the PMC index model to evaluate the effectiveness of the black soil protection policy.

The evaluation index variable setting is one of the key links of this study, which directly affects the rationality and practicability of the policy evaluation results. Following the hypothesis of “Omnia Mobilis” [25], combining the actual characteristics of black soil protection policies and the analysis conclusions of policy text mining in the previous chapter, and referring to the relevant research of existing scholars [26,27,28,29,30,31], 9 primary variables and 34 secondary variables are finally determined in this paper, as shown in

Table 1. Variables of black soil protection policy.

Main-Variables	Sub-Variables	Main-Variables	Sub-Variables
Policy type (X1)	Laws and regulations	Policy tool (X6)	Command-and-control tools
	Plans, programs		Economic incentive tools
	Opinions, measures		Public participation tools
	Decision		Promotional facilitation tools
Policy release agency (X2)	State Council	Policy area (X7)	Economy
	Ministry of Agriculture and Rural Affairs		Society
	Other departments		Political
Policy receptor (X3)	Government		Technology
	Agribusiness	Implementation guarantees (X8)	Supervision and management
	Farmers		Information promotion
	Rural collective economic organizations		Public participation
	New agricultural business entities		Typical demonstration
Policy function (X4)	Cultivated land quantity protection	Policy evaluation (X9)	Guiding ideology
	Restore the quality of arable land		Basic guidelines
	Continuous use of black soil resources		Main objective
Policy timeliness (X5)	Long term		Goal decomposition
	Medium term
	Short term

.

The PMC index is calculated as follows: Firstly, the corresponding values of the secondary variables are obtained by keyword search on the specific contents of the policy text, as shown in Formula (1); secondly, the corresponding secondary variable values under each primary variable are added and divided by the number of secondary variables, to obtain the corresponding primary variable values, as shown in Formula (2); finally, the PMC index score of each policy sample is obtained by adding the primary variable values corresponding to each policy text and dividing by the number of primary variables, as shown in Formula (3).

$$X~[0,1]$$

(1)

$$X_t=\frac{{\displaystyle \sum _{i=1}^n{X}_{ti}}}{Nt}\hspace{1em}\hspace{1em}t=1,2,3,4,5\dots ,10$$

(2)

$$PMC={\displaystyle \sum _{t=1}^k{X}_t}$$

(3)

PMC-Surface represents the value of PMC-Index in the form of a stereoscopic image, which can visually show the advantages and disadvantages of the black soil protection policy. PMC surfaces usually depend on the results of PMC matrices. Considering the symmetry and balance requirements of the matrix [25,32], the PMC matrix is a matrix with 3 rows and 3 columns, and the matrix formula is as follows:

$$PMCsurface=\left(\begin{array}{ccc}{X}_1& X_2& X_3\\ X_4& X_5& X_6\\ X_7& X_8& X_9\end{array}\right)$$

(4)

2.1.2. Quantitative Analysis Method of Policy Text

The policy text on black soil protection has the characteristics of a large number of texts, complex content, and long length. The traditional text analysis method is no longer applicable, and the text mining technology and visual analysis method in the field of information science provide effective tools for studying the dynamic evolution law of black soil protection in China. Among them, text mining is a method that combines machine learning and information retrieval technology to discover and extract implicit knowledge that meets user information needs from large-scale text data [33]; visual analysis can clearly and intuitively present the results of text mining. Therefore, this study comprehensively applies co-word analysis technology, vertical synergy calculation technology, LDA topic modeling and similarity calculation technology in text mining technology, Gephi visualization, LDAvis visualization, and ThemeRiver visualization technology to try to evaluate the policies related to black soil protection before quantification. Specific study methods are described as follows:

(1) Research methods of policy keyword recognition: Policy keywords represent the content focus of policy text, and extracting policy keywords can sort out the policy focus of governments at different levels of development. ROSTCM6 (Version 5.8.0.603) text mining software can mine the selected black soil protection policy text and comb the policy keywords in each policy text. On this basis, Gephi visual view technology can carry out co-word analysis, and then obtain a keyword co-occurrence network.

(2) Vertical synergy means that policy outputs are consistent with the original views of decisionmakers [34]. We classify government policy texts through policymakers to form a database of vertical collaborative assessments. Vertical synergy is calculated according to the proportion of keywords in total word frequency [24], as follows:

$$Sxyi=\frac{{\displaystyle \sum _{x=1}^{n}axyi}}{{\displaystyle \sum _{x=1}^{\infty }axyi}}$$

(5)

$$Zxyi=\frac{{\displaystyle \prod _{y=1}^{3}Sxyi}}{{\displaystyle \prod _{y=1}^{3}Sxy}}$$

(6)

where x represents the keywords of the policy, y represents a policy dimension, i represents the policy promulgator, axyi represents the total frequency of the policy keywords of dimension y of the policy promulgator i; Sxyi represents the intensity of policy of dimension y of policy promulgator i; Sxy represents the intensity of the policy for dimension y in all policy promulgators; and Zxyi represents the synergy degree of policy promulgator i.

(3) The research method of policy theme identification: The policy theme represents the core direction of policy text, and identifying the policy theme provides auxiliary support for studying the law of policy change. Firstly, we use the method of confusion degree measurement to determine the number of policy topics as 4, taking confusion degree as the optimal evaluation model. Secondly, the intensity of the policy theme is calculated to reveal the keywords that constitute the theme. Finally, the calculation results are visualized by the LDAvis visualization method. The calculation formula is as follows:

$$perplexity(D)=\exp (-\frac{{\displaystyle \sum \log p(w|d)}}{{\displaystyle {\sum }_{d=1}^M}{W}_d})$$

(7)

$$T_i=\frac{{\displaystyle \sum _{d=1}^M{P}_{id}}}{M}$$

(8)

where M represents the total number of documents, W_d represents the number of words in the dth document, and the lower the confusion, the better the clustering effect. By analyzing the topic intensity of the policy, we can further understand the relative component of each topic in the policy corpus. The calculation formula is shown in Formula (8). T_i denotes the topic strength of the ith word, and P_id denotes the probability of the ith word in the dth document.

(4) Research methods identified by policy tools represent the methods adopted by the government to achieve the established policy goals. Mining policy tools can understand the government’s solution direction to the black soil problem. Text similarity computing technology can mine policy tools included in policy text, and on this basis, ThemeRiver visualization technology can display the mining results. This paper divides policy tools into four types: command and control, economic incentives, information disclosure, and propaganda guidance [35].

2.2. Data Source

The research samples were mainly derived from the national, provincial, and municipal black soil protection policy documents issued from 2004 to 2023, and the text data were preprocessed. The sources of policy text data mainly include (1) official websites of government departments issuing black soil protection policies such as the Ministry of Agriculture and Rural Affairs and provincial agricultural and rural departments; (2) policy big data platforms such as “Beidafabao database” and “Bailu Think Tank”; (3) search engines such as Google and Baidu, which supplement the collected policies. To sum up, a total of 277 copies of black soil protection policy texts were collected. To ensure the relevance of the final adopted documents, excluding documents with low relevance to black soil protection, the number of effective policy texts finally selected was 220, including 36 at the national level, 67 at the provincial level, and 117 at the municipal level. Some of the texts of China’s black soil protection policies are shown in

Table 2. Text of China’s black soil protection policy (excerpt).

Release Time	Policy Name	Policy Type	Synopsis
2005	Comprehensive control plan for soil erosion in the black soil area of Northeast China	Planning	Carry out soil erosion control in the black soil area
2017	Outline of Black Soil Conservation Planning in Northeast China (2017–2030)	Planning	It is necessary to take effective measures to protect the black soil
2021	Implementation Plan of the National Black Soil Protection Project (2021–2025)	Scheme	The implementation content and zoning priorities of the national black soil protection project are clarified
2021	Technical Guidelines for Conservation Tillage Action Plan for Black Soil in Northeast China	Guidelines	Promote the four northeastern provinces to do a good job in conservation tillage in 2023
2022	Black Soil Protection Law of the People’s Republic of China	Law	Provide a strong legal guarantee for the protection of black soil

.

3. Quantitative Evaluation of Policy Consistency of China’s Black Soil Protection

3.1. Results of Policy Consistency Evaluation and Classification of Evolution Stages

Based on the PMC knowledge framework, this paper evaluates the effectiveness of black land protection policy samples collected from 2004 to 2023 at national and provincial levels and analyzes the dynamic mining of issuing subject dimension and policy measure dimension. The PMC index model is used to establish multi-input and output tables of variables, and PMC index results of selected policy texts are obtained. The time-series trends in policy effectiveness levels for each year are shown in Figure 2 and Figure 3.

As shown in Figure 2 and Figure 3, the PMC index of black land protection policy shows an overall fluctuation growth trend. According to the change trend of effectiveness evaluation of national black land protection policy, it can be seen that the policy consistency index periodically rises sharply to peak value and then tends to stabilize. The sharp increase in policy effectiveness is due to the large number of issuance of landmark policies and supporting policies. The aggregation of landmark policies and supporting policies will greatly increase the policy effectiveness to peak value, but with the passage of time, the reduction in supporting policies will gradually reduce the policy effectiveness. Therefore, according to the year when the effectiveness of national policies rises rapidly and then rises to the peak, this paper divides the black land protection policy into four development stages: 2004–2012: exploration initial period; 2013–2016: initial formation period; 2017–2021: policy development period; and 2021–now: policy refinement period.

3.2. The Degree of Policy Equilibrium Analysis

Policy equilibrium can be analyzed by the concavity of the PMC surface, which can be measured by using the ratio of the highest peak point value of the surface to the lowest slot point value [36]. A visualization of the PMC surface’s overall score versus the scores for each stage is shown in Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8. During the initial exploration phase, the concavity of the PMC surface is at its maximum (Figure 4), indicating a lack of policy equilibrium. Among them, the lowest point is the policy receptor dimension, the PMC index score is only 0.2, and the highest point is the policy instrument dimension, the PMC index score is 0.75. It can be seen that in the initial stage of black soil protection, policy tools are used more, but the policy system only covers the single receptor of government. With time, the concavity of the PMC surface tends to smooth, and the equilibrium degree of policy has been gradually optimized and perfected. The policy equilibrium degree is the highest in the policy refinement period. This is because PMC index scores of policy receptors, policy tools, and policy fields are gradually increasing and finally reaching equilibrium, especially the dimension of policy receptors, with the largest increase, from the PMC index score of 0.2 in the initial exploration period to PMC index score of 1 in the policy refinement period. This is because, with the improvement of black soil protection policies, policies have added agricultural enterprises, farmers, rural collective economic organizations, and new agricultural management to policy receptors. That is to say, there are corresponding black soil protection policies for different subjects. These receptors play an important role in the protection of black soil, such as the need to promote agricultural enterprises to accelerate the reduction and efficiency of chemical fertilizers and pesticides, reducing the pollution of black soil; the need to promote farmers to strengthen training and study, improving their quality level; the need for rural collective economic organizations to allocate high-powered machinery, straw returning machinery, and no-tillage seeder, promoting straw returning work.

It is worth noting that PMC index scores of policy area indicators continue to increase over time, and PMC index scores reach full marks in policy refinement periods. Policy areas are those areas of social development that policies will affect [37]. A more comprehensive policy system should cover economic, social, political, and technical aspects [38]. In the initial period of exploration, the black soil policy only covered the technical aspect, the primary formation period of the policy increased the economic aspect, and in the refinement period of the policy, it covered all four aspects. This is because black soil protection is a complex and systematic project involving rural development, farmers’ welfare, and agricultural technology, which should not only consider economic effects such as ensuring comprehensive grain production capacity, but also social effects such as farmers’ income, and political factors. For example, the effectiveness of black soil protection has become an important indicator for performance evaluation and political promotion of local government officials in China. Technical aspects are also very important, such as the promotion of conservation tillage and other technologies. To sum up, after entering the policy refinement period, the policy areas covered by China’s black soil protection policy have been relatively comprehensive.

4. Research on the Evolution of Black Soil Conservation Policy in China

According to the quantitative analysis of PMC consistency, this paper divides the policy evolution stages into four stages: the initial exploration stage, the primary formation stage, the policy development stage, and the policy refinement stage according to the year when the policy consistency rises rapidly and then rises to the peak. Based on sorting out the policies of each stage, the evolution process of black soil protection policy was summarized and analyzed.

4.1. Exploring the Initial Period (2004–2012)

Exploring the initial period is the stage when the protection of black soil received initial attention, and the main body of policy issuance is mainly a single subject. In 2002, the Premier of the State Council issued important instructions twice, demanding that the prevention and control of soil erosion in black soil areas be placed on the agenda. In 2003, the central budget allocated special funds to support the four northeastern provinces and regions to carry out a three-year pilot project for comprehensive prevention and control of soil erosion in the black soil region of northeast China. During the development period of the system of construction occupation and farmland quality management and protection (2004–2011), black soil protection was first proposed by the Ministry of Agriculture of China, who began to pay attention to the management and protection of black soil quality. However, policies and measures on the protection of black soil quality still need to be further strengthened, and the ecological protection of black soil has not been fully paid attention to. In 2004, the Ministry of Natural Resources reported that China’s black soil was suffering from serious damage and erosion, and experts called for the urgent need to formulate policies and regulations to protect the black soil. In 2005, the Ministry of Agriculture proposed, for the first time, the establishment of a long-term mechanism for the rational use and protection of black soil.

Up to 2006, the area of soil and water loss in the black soil area of Northeast China accounted for 27.0% of the total area of the whole region. Given this problem, the Comprehensive Prevention and Control Plan of Soil and Water Loss in the Black Soil Area of Northeast China proposed the measures of using both gully and slope control, increasing ground vegetation, rational development, and so on to comprehensively prevent and control the soil and water loss in the black soil area of Northeast China. The trend of soil erosion has been alleviated, but the situation is still grim. In the following five years, although the state in the notice, opinions, outline, and other documents proposed to diffuse hillock areas, hilly areas, and other different characteristics of the black soil area of soil and water loss prevention and control, the black soil problem still remains for the prevention and control of soil and water loss. To better guarantee the economic and social development during the Twelfth Five-Year Plan period, and ensure that the quantity of cultivated land in China is stable and the quality does not decline, in 2012, the Ministry of Land and Resources issued the Notice on Improving the Level of Cultivated Land Protection and Comprehensively Strengthening the Construction and Management of Cultivated Land Quality. As China’s cultivated land protection enters the transformation and improvement stage, integrating quantity, quality, and ecology, the northeast black soil region is also strengthening engineering measures to improve the regional water and soil conservation capacity.

4.2. Primary Formation Period (2013–2016)

The preliminary formation period is the stage of starting the first batch of black soil protection and utilization pilot projects. The black soil protection technology was still in the experimental stage, and the coordination trend of black soil protection policy departments was initially formed. In 2013, climate issues adversely affected natural ecosystems and socioeconomic systems, and the black soil problem was raised again. The Notice on Printing and Distributing the National Adaptation Strategy to Climate Change, jointly issued by the National Development and Reform Commission and the Ministry of Agriculture and nine other departments, with Jilin Province as the first pilot, proposed to carry out the pilot demonstration project of black soil protection and governance adaptation in Jilin’s main grain-producing areas. Subsequently, in 2015, the policy-issuing department joined the Ministry of Science and Technology and the Ministry of Environmental Protection to jointly issue the National Agricultural Sustainable Development Plan (2015–2030) to promote the development of the pilot demonstration area for sustainable development of black soil protection; on this basis, the agricultural machinery purchase subsidy policy issued by Jilin Province proposed black soil protection and issued subsidies for the purchase of agricultural machinery implementing black soil conservation tillage technology. Subsequently, the utilization efficiency of agricultural resources was significantly improved in 2016, and Jilin Province promulgated the Jilin Province Agricultural Sustainable Development Plan (2016–2030) to promote the scale expansion process of the black soil protection and utilization pilot.

4.3. Policy Development Period (2017–2020)

During the policy development period, the mainstream technology of black soil protection made obvious progress and began to be popularized and applied in Northeast China. The Ministry of Agriculture and Rural Affairs launched the second batch of pilot projects for black soil protection and utilization, and the policy of black soil protection issued by multiple departments in coordination, had approached maturity. In 2017, China’s understanding of black soil problems was deeper, that is, in the past 60 years, the organic matter content of the black soil tillage layer soil had decreased by 1/3, and in some areas by 50%. The National Development and Reform Commission, together with the Ministry of Agriculture, the Ministry of Finance, the Ministry of Water Resources, the Ministry of Land and Resources, and the Ministry of Environmental Protection, issued the Outline of Northeast black soil Protection Planning (2017–2030). Under the condition of clarifying the importance and urgency of black soil protection, they proposed the overall requirements, key tasks, technical modes, and safeguard measures for black soil protection.

In 2020, the Ministry of Water Resources proposed that the area of soil and water loss in the northeast black soil accounted for 19.86% of the total land area, and the control of soil and water loss has achieved success, but the problems of “thinning” of denudation loss, “thinning” of soil organic matter and nutrient element attenuation, and “hardening” of soil structure change and water storage capacity decline have not been completely solved. At the same time, Northeast China’s conservation tillage technology has made significant progress, and the overall technical model of the shape, of the key equipment passed, has been in the appropriate area to promote the comprehensive application of the foundation. Accordingly, the Ministry of Agriculture and Rural Affairs and the Ministry of Finance jointly issued the Action Plan for Conservation Tillage of Northeast black soil (2020–2025), to accelerate the popularization and application of conservation tillage, making conservation tillage the mainstream agricultural tillage technology in the suitable areas of Northeast China, realizing the stable improvement of cultivated land quality and comprehensive agricultural production capacity, and enhancing ecological, economic, and social benefits. During this period, based on the locally cultivated land characteristics, the provincial and municipal governments of the four provinces in Northeast China began to promulgate policies to explore technologies and modes more conducive to their development.

4.4. Policy Refinement Period (from 2021 to Now)

In the policy refinement period, multiple departments jointly issued several policies to promote black soil protection as a national project. In 2021, the cultivated land quality of the four provinces (autonomous regions) in Northeast China increased by 0.29 grades compared with five years ago. However, since the black soil cultivated land that has implemented comprehensive treatment measures is still dominated by pilot and demonstration areas, the protected area accounts for a relatively low proportion, and the degradation trend of black soil cultivated land has not been effectively curbed. The Ministry of Agriculture and Rural Affairs, together with six other departments, issued the Implementation Plan of the National black soil Protection Project (2021–2025). The implementation contents of black soil protection were proposed from the aspects of monitoring and evaluation systems, farmland infrastructure, technology, environment, etc., and black soil protection was formally proposed as a national project. In 2021, the Ministry of Agriculture and Rural Affairs reissued the Technical Guidelines for the Action Plan for Conservation Tillage of Northeast black soil in 2021 to promote the quality improvement and expansion of conservation tillage in 2021, realize the standardized application of conservation tillage technology, and continuously enhance the soil protection effect.

In 2022, the Law of the People’s Republic of China on the Protection of black soil was formally implemented, which is the first law in China and the only law in the world to legislate to protect black soil at the national level, which marks that the protection of black soil in China has officially stepped into the track of legalization. In addition, the provinces and municipalities in Northeast China have issued a large number of detailed policies at the provincial and municipal levels.

5. China’s Black Soil Protection Policy Analysis of the Results of the Dynamic Evolution of the Law

5.1. China’s Black Soil Protection Policy Analysis of the Key Words

5.1.1. Co-Occurrence Network Analysis of Policy Keywords

Through the analysis of the co-occurrence and evolution of keywords in the policies issued by governments at different levels in different periods, we can sort out the key points of policies at different levels of development of governments, and provide auxiliary support for policy research and formulation. The co-occurrence network of policy keywords at different stages of the evolution of national black soil protection policies is shown in Figure 9, Figure 10, Figure 11 and Figure 12.

It can be seen from Figure 9 that the “soil” point degree centrality is the highest in the initial exploration period, indicating that the frequency of “soil” appearing in the same policy document with other keywords is the highest, indicating that the black soil protection policy at this stage focuses on the problem of water and soil loss. “Soil”, “erosion”, “prevention”, and “engineering” are at the core of the co-occurrence network, and the edge weights are higher. In this period, the state made great efforts to develop soil erosion control projects, fertile soil projects, and irrigation area reconstruction projects to improve the level of intensive land use, and to establish a long-term mechanism for the rational use and protection of black soil.

As can be seen from Figure 10, the keywords “protection” and “ecology” in the primary formation period have the highest influence on the social network. The protection of black soil has been promoted from solving the problem of soil erosion to improving the ecological environment, improving soil quality, increasing infrastructure, and other agricultural protection. The emergence of keywords such as “pilot” and “governance” is precisely because the report of the 18th National Congress of the Communist Party of China points out that the construction of ecological civilization should be vigorously promoted under the increasingly tight constraints of global resources and environment. Under the key task of protecting cultivated land resources and promoting the sustainable utilization of farmland, strengthening the protection of black soil in Northeast China and improving the land quality has become an indispensable link. This also reflects that the central government’s policy focus on black soil protection at this stage is to “carry out pilot projects” for black soil protection. In 2013, the central government increased capital investment to carry out pilot projects for black soil protection in Jilin Province, striving to explore effective measures to control soil erosion and restore soil fertility.

It can be seen from Figure 10 that the keywords “black soil”, “protection”, and “technology” have a high degree of centrality during the policy development period, which just shows that in order to accelerate the implementation of the strategy of “storing grain in the land and storing grain in technology” and strengthen the research on black soil protection technology, the state focuses on the technical mode. The keywords “farming”, “promote”, and “northeast” reflect that the pilot work of black soil protection has been extended from Jilin Province to the whole northeast region.

It can be seen from Figure 12 that “conservation tillage” has the highest influence in the keyword network during the policy refinement period, which shows that with the gradual improvement of black soil protection work, conservation tillage technology has become the mainstream tillage technology for black soil protection in Northeast China. Keywords such as “quality”, “technology”, “monitoring”, and “effect” indicate that the black soil protection technology is relatively mature, and the policy focus has been changed to how to promote and implement the black soil protection technology. The next work’s focus is to strengthen the monitoring and evaluation of the quality of black soil cultivated land. Through strict supervision and assessment, centralized and contiguous promotion is realized, and a new pattern of sustainable development in which the black soil is protected during utilization and promoted by protection is formed.

5.1.2. Hierarchical Evolution Analysis of Policy Keywords

The main body of black soil protection policy includes three levels of government at the national, provincial, and municipal levels, and the policy evolution characteristics of the same level of government and different levels of government are also different. To explore the evolution characteristics of the association between the same level and different levels of policies at different development stages, the high-frequency keywords at three levels and four development stages were comprehensively analyzed, as shown in

Table 3. Keywords at national, provincial, and municipal levels in different periods.

	Explore the Initial Period	Primary Formation Period	Policy Development Period	Policy Refinement Period
National Policy Keywords	“soil” “erosion” “project” “cultivated land” “ecology” “prevention”	“protect” “environment” “ecology” “pilot project”	“black soil” “technology” “protect” “northeast” “soil” “cultivated land”	“farming” “protective” “technology“ “conservation tillage” “monitor” “black soil”
Provincial Policy Keywords		“farmland” “ecology” “cultivated land” “protect”	“operation” “straw” “technology” “black soil” “protect” “quality”	“black soil” “protective” “cultivated land” “soil” “monitor” “technology”
Municipal Policy Keywords			“farming” “technology” “soil” “straw” “quality” “promote”	“protective” “black soil” “technology” “farming” “monitor” “assess”

. It can be seen from the table that there is a law of “progressive evolution” in China’s black soil protection policy from the same level. The progressive evolution process is mainly reflected in the policy content. In the early stage of policy development, the keywords of policy are “prevention” and “pilot”, which reflect the singularity of black soil protection and utilization technology and the locality of the implementation area. In the follow-up development, more black soil degradation problems have been discovered, and the technical model of black soil protection and utilization has been continuously innovated and enhanced. In the later stage of policy development, keywords such as “technology”, “monitoring”, and “northeast” indicate that the black soil protection and utilization technology has mature and supporting technical measures, and the protection scale has been extended from pilot projects in some regions to the whole northeast region.

From the different levels, there are “level lag” and “level transmission” in the early and late development of China’s black soil protection policy. Among them, the lag of hierarchy is mainly reflected in the hierarchy and policy content of the main body of policy issuance. From the perspective of the main body level of policy issuance, most of the policies issued at the initial stage of exploration are limited to the national level, while the provincial and municipal levels begin to promulgate relevant policies at the primary formation stage and policy development stage, respectively. From the perspective of policy content, in the early stage of policy development, the policy keywords “soil” and “ecology” have obvious “hierarchical lag” evolution characteristics; these focuses are the first attention of the central government; local governments will gradually pay attention to them in the subsequent development. In the later stage of policy development, the “hierarchical lag” gradually weakened, gradually showing the evolution characteristics of “hierarchical transmission”, and keywords such as “technology” and “monitoring” appeared in different levels of policy in the same period. To sum up, the evolution law of black soil protection policy is mainly reflected in the same level, and its evolution law conforms to “progressive evolution”. Between different levels, the early and late stages of policy development show the evolution characteristics of “hierarchical lag” and “hierarchical transmission”, respectively.

On the basis of studying the evolution characteristics of different levels of government, this paper uses the change of high-frequency word proportion to reflect the change of policy concern and further quantifies the vertical synergy between central black soil protection policy and local government policy, that is, the consistency degree between different levels of policy. On the basis of classifying and merging all the strategy texts, the first 40 high-frequency words in the merged strategy texts are selected as the keywords of the strategy type. Using Equations (5) and (6) to quantify the degree of vertical policy synergy, the results are shown in Figure 13, Figure 14, Figure 15 and Figure 16. The vertical synergy degree of central government is the highest in the initial formation period and the overall stage, and the vertical synergy degree of Heilongjiang Province is the highest in the policy development period and the policy refinement period.

As shown in Figure 13, the vertical coordination degree of the central government and local governments at all stages is ranked as follows: Central Government > Jilin Province > Heilongjiang Province > Inner Mongolia Autonomous Region > Liaoning Province. The vertical synergy degree at the national level is 0.8, which is higher than that of the local government. This shows that, compared with local governments, the central government pays more attention to the consistency of policy intentions between upper and lower governments when formulating policies. Second, there is a big gap in the vertical synergy between local governments. The highest vertical synergy degree of Jilin Province is 0.6, which is higher than that of other provinces. This is because Jilin Province set up the first pilot project of black soil protection in China, and later formed a “pear tree model” suitable for black soil protection in Northeast China through exploration. Black soil protection was carried out earlier and policy consistency was strong. On the basis of fully implementing and understanding the purpose of central policy in provincial policies, municipal policies followed its strategic layout.

5.2. Analysis of the Policy Theme of Black Soil Protection in China

Through the excavation and analysis of the policy theme of black soil protection, we can find the change in policy core direction. Figure 17 shows the LDA model recognition results of the national black soil protection policy theme. The red and blue bars on the right side of Figure 17 represent the correlation degree between the word and the theme and the probability of the theme, respectively. The left side figure shows that a total of four themes are identified. To sum up, we extracted the most closely related words and summarized the theme contents of the national black soil protection policy as follows: theme 1 (agricultural protection area), theme 2 (black soil construction), theme 3 (technology promotion), and theme 4 (strengthening ecology). According to the calculation principle of the topic model, the closer the distance, the stronger the relevance. Therefore, as can be seen from Figure 13, theme 1 has the strongest degree of association with theme 2 and the weakest degree of association with theme 4. It can be seen that the protected area is the core of the black soil protection project; compared with this, strengthening the ecological environment accounts for less in the black soil protection project and is in a subsidiary position.

Further identifying the theme of black soil protection policy in different stages, we find that the theme of black soil protection policy in China has the law of “stage evolution”, which is manifested in the core content of black soil protection gradually changing from single to comprehensive. The policy theme in the initial period of exploration is “ecological environment improvement”. Under the background of lack of mature black soil protection technology, it focuses on controlling ecological problems such as soil erosion and desertification of black soil, giving priority to ensuring the quantity of black soil, laying a foundation for the protection and improvement of black soil. The policy themes of the primary formation period are “pilot protection areas”, “strengthening soil quality”, and “black soil protection demonstration areas”. The establishment of pilot and demonstration areas has become a key link in the protection of black soil in this period. Under the background of exploring the most effective methods suitable for black soil protection, the demonstration areas have a demonstration and guidance role, which provides strong support for the comprehensive promotion of black soil protection models.

The policy themes In the policy development period are “conservation technology”, “farming mode”, and “straw returning to the field”. Through the implementation of conservation tillage in the pilot area, and the simultaneous implementation of grain and bean rotation, deep tillage, straw return, and other technical measures, a number of black soil protection and utilization technical models and operation mechanisms suitable for Northeast China were explored and summarized. The policy themes of the policy refinement period are “black soil quality”, “supervision and evaluation”, and “technology promotion”. The establishment of a supervision and evaluation system has become an important task at present. Under the background of mature black soil protection technology and large-scale popularization, perfecting the monitoring index system and evaluation technology of black soil quality can assist in the effective implementation of the black soil protection project.

In order to explore the dynamic evolution characteristics of the policy themes of black soil protection in different development stages of China, this paper comprehensively analyzes the policy themes of four development stages and finds that the policy themes of black soil protection in China have the law of “stage evolution”, which is manifested in the gradual change of the content of black soil protection from single to comprehensive. The policy theme developed from ecological problem management to exploring suitable technical mode, then combined new farming technology with protection mode, finally improved the supervision feedback mechanism on the basis of comprehensively promoting black soil protection work, and gradually diversified and integrated the content of the black soil protection policy system. Therefore, it shows that the policy theme development of black soil protection policy conforms to the characteristics of stage evolution.

5.3. An Analysis of Policy Tools for China’s Black Soil Protection Policy

By mining the types and quantities of policy tools, we can clarify the implementation subjects of policies at different stages of development, and provide auxiliary support for policy research and formulation. The evolution trend of the number of different policy instruments adopted by governments at all levels in the past 20 years is shown in Figure 18. Figure 18 shows that the use of policy instruments is more balanced at all levels of government. This is due to the wide range of subjects implementing the black soil protection policy, including farmers, enterprises, agricultural cooperatives, and government agencies. For example, farmers, the main application subjects of black soil protection technology, have a low overall cultural level, have less cognition of the necessity of black soil protection and the effectiveness of black soil protection technology, and are more suitable for propaganda and guidance policy tools to popularize science among farmers. For the main enterprises of policy regulation and control, some industrial and mining enterprises pollute and destroy the land, and government departments need to control their emission standards, so it is suitable to use command-and-control tools; for the agricultural machinery cooperatives, which are the main body of technology implementation of black soil protection, they are encouraged to purchase and use agricultural machinery equipment conducive to black soil protection, and economic incentive tools are used to purchase agricultural machinery subsidies. Therefore, the use of the four policy tools by governments at all levels has the characteristics of balance and universality.

The use of policy instruments is characterized by weak volatility at the national level and strong volatility at the local level. In the early stage, local governments were in a weak position, mostly implementing the black soil protection policies issued by the central government, but in recent years, local governments have gradually occupied a dominant position in the formulation and implementation of black soil protection policies. In the initial period of exploration, the government began to realize the problem of black soil protection, so the number of policy tools was relatively small during this period. In the primary formation period, the pilot work enabled more government departments to participate in the black soil protection work, so the number of national-level policy tools began to increase, and local governments in pilot areas also issued relevant documents to ensure the implementation of the pilot work, so the number of local policy tools also began to increase. During the policy development period, the state issued policies to explore and popularize the protection technologies and protection models suitable for black soil, and the use of policy tools did not change greatly. At the same time, local governments invested less in exploring new technologies and models, so the number of policy tools available to local governments began to decrease. During the policy refinement period, the black soil Protection Project was formally implemented as a national project, and the number of national policy tools did not fluctuate greatly. At the same time, local governments began to issue black soil protection policies intensively, and local governments gradually occupied a dominant position in the formulation and implementation of black soil protection policies.

We explore the change trend of the use of policy tools at different levels, and display the use of black soil protection policy tools at different periods and levels in the form of thematic river diagrams. The evolution trend of policy tools is shown in Figure 19, Figure 20 and Figure 21. It can be seen from Figure 19, Figure 20 and Figure 21 that the changing trend of national-level policy tools is to increase first and then stabilize. The number of four policy tools has increased slowly since 2015. During this period, the black soil protection policy system has initially taken shape. The state mainly uses command-and-control tools and other three types of policy tools to assist government departments in promoting black soil protection. In 2017, China attached great importance to the protection of black soil in Northeast China and clearly proposed to take effective measures to protect black soil. Therefore, during this period, the intensity of issuing policies was strengthened, and the number of policy instruments reached a peak, and then overall tended to be stable. The changing trend of provincial policy tools was increasing and had the characteristics of hierarchical lag. After the relevant policies were promulgated by the state, the provincial government did not respond immediately. Until 2017, the number of provincial policy instruments began to grow slowly, with command-and-control instruments as the main types of instruments, supplemented by the other three types of policy instruments, and lagging behind the use of national-level policy instruments by two years. In 2016, in response to national policies, some cities issued detailed policies based on the current situation of the city, and the number of municipal policy instruments increased. While the state searched for new technologies and models for black soil protection, municipal governments did not respond, and the number of policy tools began to decrease. From 2019 to 2021, the protection of black soil was rapidly promoted, and municipal governments actively responded to this by formulating policies consistent with their own regional characteristics and current situation.

6. Research Conclusions

This study used the policy consistency index model to evaluate the effectiveness of the black soil protection policy and comb the evolution process of black soil protection policy in China. At the same time, using quantitative analysis methods such as text mining and text measurement, the dynamic evolution characteristics of black soil protection policy at the same level and different levels were studied from four levels: policy keywords, vertical synergy, policy themes, and policy tools. The following conclusions are drawn:

(1) According to the results of policy effectiveness evaluation, PMC index scores show a phased growth trend as a whole, and each index gradually improves and tends to stabilize. In the early stage of the black soil protection policy, the internal consistency level is relatively low. After the gradual development and refinement of policies in each stage, the consistency index of policies shows a phased upward trend. According to the characteristics of the PMC index rising to the peak and tending to be stable, the evolution stage of black soil protection policy can be divided into an exploration initial period, preliminary formation period, policy development period, and policy refinement period.

(2) From the perspective of the evolution law of policies at the same level, there are progressive evolution characteristics in both policy keywords and policy themes. On the one hand, the logical main line of policy keyword evolution is mainly reflected in the degree of understanding of the black soil problem and the way technological innovation promotes black soil protection and utilization. That is to say, the change in the policy of black soil protection in Northeast China is a process of continuous practice, exploration, and renewal of black soil protection technology, and also a process of policy evolution promoted by agricultural technology progress. On the other hand, the development process of policy theme shows that the content of black soil protection policy gradually changes from single to comprehensive. The policy theme ranges from exploring soil erosion control in the initial period to exploring suitable new models in the initial formation period, then combining new farming techniques with conservation models in the development period, and finally improving the regulatory feedback mechanism in the policy refinement period. Generally speaking, the contents of the black soil protection policy system are gradually diversified and integrated.

(3) From the evolution law of different levels of policies, there is a hierarchical lag evolution law between policy keywords and provincial policy tools, among which policy keywords also have a vertical synergy evolution law. First of all, the lag of policy keyword hierarchy is mainly reflected in the hierarchy and policy content of policy-issuing subjects. Most of the policies issued in the initial stage of exploration are limited to the national level, while the provincial and municipal governments begin to issue relevant policies in the initial formation stage and policy development stage, respectively. In the early stage of policy development, the keywords “soil” and “ecology” have obvious characteristics of “hierarchical lag” evolution; these focuses have the first attention of the central government, and the local government will gradually pay attention to them in the subsequent development. Secondly, after the promulgation of relevant policies by the state, the response time of provincial governments lags behind, and the number of provincial policy instruments begins to grow slowly in 2017. The types of instruments used are mainly command–control instruments, supplemented by other three types of policy instruments, and lag behind the use of national-level policy instruments by two years. Finally, the vertical coordination degree of central government is higher than that of provincial governments in the process of formulating black soil protection policy.

By reviewing the policy development process, evaluating its effectiveness, and exploring its evolution rules, we find that the black soil protection policy highlights the problems that need to be optimized, such as imperfect incentive measures and imperfect feedback mechanisms. Based on this, this paper proposes the following perfect countermeasures: Shorten the policy delay. Black soil protection policy is a project that needs department coordination, multiparticipation, and linkage to shorten the time delay caused by policy diffusion and implementation among different levels of government as much as possible, and better guarantee the effect of black soil protection policy. In this regard, we can improve the transmission mechanism of policies, improve the implementation mechanism of relevant supporting policies, strengthen policy interaction among different levels of government, and exert efforts to give full play to the incentive and traction role of national policies and provincial policies to municipal, district, and county governments, so as to improve the efficiency of policy implementation and strengthen the continuity and coherence of policies. Strengthen the correlation and coupling between policies, and pay attention to the division of labor, cooperation, and overall consideration among various departments.

Specifically speaking, the adjustment of black soil protection policy should be promoted with the systematic concept, considered as a whole, and the coordination and internal connection among policies such as good governance, repair, good use, and good maintenance of black soil should be emphasized, so as to better grasp the evolution direction of black soil protection policy and provide a guarantee for more efficient promotion of black soil protection work. Strengthen the coverage and comprehensiveness of the policy content: black soil protection involves many aspects such as economy, society, politics, and technology. Expand the coverage of black soil protection policy, improve the comprehensiveness of black soil protection policy, and provide protection to black soil from many aspects, so as to better guarantee national food security.

In general, this paper makes an overall overview and objective evaluation of the policy content text. The research conclusions can provide a new research paradigm and theoretical basis for the research of the black soil protection policy system, and provide a theoretical basis and practical reference for the subsequent optimization and perfection of black soil protection policy to promote the northeast region to solve the bottleneck problem of black soil protection. However, there are still some limitations and deficiencies in the research, such as the following: the PMC index model has limitations in variable selection, universality is still low, some variables cannot adapt to other policies, and the universality of the policy evaluation system can be continued to be strengthened in the future.