Location via proxy:   [ UP ]  
[Report a bug]   [Manage cookies]                
Skip to main content
Ross Nehm
  • Stony Brook University (SUNY)
    Center for Science and Math Education &
    Department of Ecology & Evolution
    650 Life Sciences Building
    Stony Brook, NY 11794-5233
  • 631-632-7247

Ross Nehm

  • The Nehm Lab studies science learning, with a focus on biological concepts such as natural selection and evolution. W... moreedit
Automated computerized scoring systems (ACSSs) are being increasingly used to analyze text in many educational settings. Nevertheless, the impact of misspelled words (MSW) on scoring accuracy remains to be investigated in many domains,... more
Automated computerized scoring systems (ACSSs) are being increasingly used to analyze text in many educational settings. Nevertheless, the impact of misspelled words (MSW) on scoring accuracy remains to be investigated in many domains, particularly jargon-rich disciplines such as the life sciences. Empirical studies confirm that MSW are a pervasive feature of human-generated text and that despite improvements, spell-check and auto-replace programs continue to be characterized by significant errors. Our study explored four research questions relating to MSW and text-based computer assessments: (1) Do English language learners (ELLs) produce equivalent magnitudes and types of spelling errors as non-ELLs? (2) To what degree do MSW impact concept-specific computer scoring rules? (3) What impact do MSW have on computer scoring accuracy? and (4) Are MSW more likely to impact false-positive or false-negative feedback to students? We found that although ELLs produced twice as many MSW as non-ELLs, MSW were relatively uncommon in our corpora. The MSW in the corpora were found to be important features of the computer scoring models. Although MSW did not significantly or meaningfully impact computer scoring efficacy across nine different computer scoring models, MSW had a greater impact on the scoring algorithms for naïve ideas than key concepts. Linguistic and concept redundancy in student responses explains the weak connection between MSW and scoring accuracy. Lastly, we found that MSW tend to have a greater impact on false-positive feedback. We discuss the implications of these findings for the development of next-generation science assessments.
ABSTRACT
The growing importance of genomics and bioinformatics methods and paradigms in biology has been accompanied by an explosion of new curricula and pedagogies. An important question to ask about these educational innovations is whether they... more
The growing importance of genomics and bioinformatics methods and paradigms in biology has been accompanied by an explosion of new curricula and pedagogies. An important question to ask about these educational innovations is whether they are having a meaningful impact on students’ knowledge, attitudes, or skills. Although assessments are necessary tools for answering this question, their outputs are dependent on their quality. Our study 1) reviews the central importance of reliability and construct validity evidence in the development and evaluation of science assessments and 2) examines the extent to which published assessments in genomics and bioinformatics education (GBE) have been developed using such evidence. We identified 95 GBE articles (out of 226) that contained claims of knowledge increases, affective changes, or skill acquisition. We found that 1) the purpose of most of these studies was to assess summative learning gains associated with curricular change at the undergra...
This critical review examines the challenges and opportunities facing the field of Biology Education Research (BER). Ongoing disciplinary fragmentation is identified as a force working in opposition to the development of unifying... more
This critical review examines the challenges and opportunities facing the field of Biology Education Research (BER). Ongoing disciplinary fragmentation is identified as a force working in opposition to the development of unifying conceptual frameworks for living systems and for understanding student thinking about living systems. A review of Concept Inventory (CI) research is used to illustrate how the absence of conceptual frameworks can complicate attempts to uncover student thinking about living systems and efforts to guide biology instruction. The review identifies possible starting points for the development of integrative cognitive and disciplinary frameworks for BER. First, relevant insights from developmental and cognitive psychology are reviewed and their connections are drawn to biology education. Second, prior theoretical work by biologists is highlighted as a starting point for re-integrating biology using discipline-focused frameworks. Specifically, three interdependent...
Additional file 1: Table S1. Mean GeDI item measures (measure) and standard error (SE) by vignette rotation position.
Additional file 2. Item difficulties, and weighted (infit) and unweighted (outfit) MNSQ fit statistics for the MATE facts and MATE credibility dimensions.
Additional file 1. Item and person separation reliabilities for the MATE facts and MATE credibility dimensions.
Concept inventories, consisting of multiple-choice questions designed around common student misconceptions, are designed to reveal student thinking. However, students often have complex, heterogeneous ideas about scientific concepts.... more
Concept inventories, consisting of multiple-choice questions designed around common student misconceptions, are designed to reveal student thinking. However, students often have complex, heterogeneous ideas about scientific concepts. Constructed-response assessments, in which students must create their own answer, may better reveal students ’ thinking, but are time- and resource-intensive to evaluate. This report describes the initial meeting of a National Science Foundation– funded cross-institutional collaboration of interdisciplinary science, technology, engineering, and mathematics (STEM) education researchers interested in exploring the use of automated text analy-sis to evaluate constructed-response assessments. Participants at the meeting shared existing work on lexical analysis and concept inventories, participated in technology demonstrations and work-shops, and discussed research goals. We are seeking interested collaborators to join our research community.
College science instructors need continuous professional development (PD) to meet the call to evidence-based practice. New PD efforts need to focus on the nuanced blend of factors that influence instructors’ teaching practices. We used... more
College science instructors need continuous professional development (PD) to meet the call to evidence-based practice. New PD efforts need to focus on the nuanced blend of factors that influence instructors’ teaching practices. We used persona methodology to describe the diversity among instructors who were participating in a long-term PD initiative. Persona methodology originates from ethnography. It takes data from product users and compiles those data in the form of fictional characters. Personas facilitate user-centered design. We identified four personas among our participants: Emma the Expert views herself as the subject-matter expert in the classroom and values her hard-earned excellence in lecturing. Ray the Relater relates to students and focuses on their points of view about innovative pedagogies. Carmen the Coach coaches her students by setting goals for them and helping them develop skill in scientific practices. Beth the Burdened owns the responsibility for her students...
Helping faculty develop high-quality instruction that positively affects student learning can be complicated by time limitations, a lack of resources, and inexperience using student data to make iterative improvements. We describe a... more
Helping faculty develop high-quality instruction that positively affects student learning can be complicated by time limitations, a lack of resources, and inexperience using student data to make iterative improvements. We describe a community of 16 faculty from five institutions who overcame these challenges and collaboratively designed, taught, iteratively revised, and published an instructional unit about the potential effect of mutations on DNA replication, transcription, and translation. The unit was taught to more than 2000 students in 18 courses, and student performance improved from preassessment to postassessment in every classroom. This increase occurred even though faculty varied in their instructional practices when they were teaching identical materials. We present information on how this faculty group was organized and facilitated, how members used student data to positively affect learning, and how they increased their use of active-learning instructional practices in ...
We conducted a study of 19 biology instructors participating in small, local groups at six research-intensive universities connected to the Automated Analysis of Constructed Response (AACR) project (www.msu.edu/∼aacr). Our aim was to... more
We conducted a study of 19 biology instructors participating in small, local groups at six research-intensive universities connected to the Automated Analysis of Constructed Response (AACR) project (www.msu.edu/∼aacr). Our aim was to uncover participants' motivation to persist in a long-term teaching professional development effort, a topic that is understudied in discipline-based educational research. We interviewed each participant twice over a 2-year period and conducted qualitative analyses on the data, using expectancy-value theory as a framework for considering motivation. Our analyses revealed that motivation among instructors was high due to their enjoyment of the AACR groups. The high level of motivation is further explained by the fact that AACR groups facilitated instructor involvement with the larger AACR project. We also found that group dynamics encouraged persistence; instructors thought they might never talk with colleagues about teaching in the absence of AACR g...
Understanding sources of performance bias in science assessment provides important insights into whether science curricula and/or assessments are valid representations of student abilities. Research investigating assessment bias due to... more
Understanding sources of performance bias in science assessment provides important insights into whether science curricula and/or assessments are valid representations of student abilities. Research investigating assessment bias due to factors such as instrument structure, participant characteristics, and item types are well documented across a variety of disciplines. However, the relationships among these factors are unclear for tasks evaluating understanding through performance on scientific practices, such as explanation. Using item-response theory (Rasch analysis), we evaluated differences in performance by gender on a constructed-response (CR) assessment about natural selection (ACORNS). Three isomorphic item strands of the instrument were administered to a sample of undergraduate biology majors and nonmajors (Group 1: n = 662 [female = 51.6%]; G2: n = 184 [female = 55.9%]; G3: n = 642 [female = 55.1%]). Overall, our results identify relationships between item features and perf...
EvoGrader is a free, online, on-demand formative assessment service designed for use in undergraduate biology classrooms. EvoGrader’s web portal is powered by Amazon’s Elastic Cloud and run with LightSIDE Lab’s open-source... more
EvoGrader is a free, online, on-demand formative assessment service designed for use in undergraduate biology classrooms. EvoGrader’s web portal is powered by Amazon’s Elastic Cloud and run with LightSIDE Lab’s open-source machine-learning tools. The EvoGrader web portal allows biology instructors to upload a response file (.csv) containing unlimited numbers of evolutionary explanations written in response to 86 different ACORNS (Assessing COntextual Reasoning about Natural Selection) instrument items. The system automatically analyzes the responses and provides detailed information about the scientific and naive concepts contained within each student’s response, as well as overall student (and sample) reasoning model types. Graphs and visual models provided by EvoGrader summarize class-level responses; downloadable files of raw scores (in .csv format) are also provided for more detailed analyses. Although the computational machinery that EvoGrader employs is complex, using the syst...
ABSTRACT A large body of work has been devoted to reducing assessment biases that distort inferences about students' science understanding, particularly in multiple-choice instruments (MCI). Constructed-response instruments... more
ABSTRACT A large body of work has been devoted to reducing assessment biases that distort inferences about students' science understanding, particularly in multiple-choice instruments (MCI). Constructed-response instruments (CRI), however, have invited much less scrutiny, perhaps because of their reputation for avoiding many of the documented biases of MCIs. In this study we explored whether known biases of MCIs—specifically item sequencing and surface feature effects—were also apparent in a CRI designed to assess students' understand-ing of evolutionary change using written explanation (Assessment of COntextual Reasoning about Natural Selection [ACORNS]). We used three versions of the ACORNS CRI to investigate different aspects of assessment structure and their corresponding effect on infer-ences about student understanding. Our results identified several sources of (and solutions to) assessment bias in this practice-focused CRI. First, along the instrument item sequence, items with similar surface features produced greater sequencing effects than sequences of items with dissimilar surface features. Second, a counterbalanced design (i.e., Latin Square) mitigated this bias at the population level of analysis. Third, ACORNS response scores were highly correlated with student verbosity, despite verbosity being an intrinsically trivial aspect of explanation quality. Our results suggest that as assessments in science education shift toward the measurement of scientific practices (e.g., explanation), it is critical that biases inherent in these types of assessments be investigated empirically.
Research Interests:
Background: Although a large body of work in science education has established the pervasive problem of science teachers' alternative conceptions about evolution, knowledge deficits, and anti-evolutionary attitudes, only a handful of... more
Background: Although a large body of work in science education has established the pervasive problem of science teachers' alternative conceptions about evolution, knowledge deficits, and anti-evolutionary attitudes, only a handful of interventions have explored the mitigation of these issues using professional development (PD) workshops, and not a single study to our knowledge has investigated if positive outcomes are sustained long after program completion. The central aim of our study was to investigate the long-term consequences of an intensive, short-term professional development program on teachers' knowledge of evolution, acceptance of evolution, and knowledge of the nature of science (NOS). Methods: Program efficacy was examined using a pre-post, delayed post-test design linked to quantitative measures of teacher knowledge, performance (explanatory competence), and acceptance using published instruments shown to generate reliable and valid inferences. Results: Our study is the first to report sustained large effect sizes for both knowledge of evolution, NOS, and acceptance change ~1.5 years after program completion. Concordant with other measures, teacher self-reports indicated that the PD program had lasting effects. Conclusions: Our study suggests that short-term PD built using specific research-based principles can have lasting impacts on teachers' evolutionary knowledge and acceptance. Because evidence of sustained knowledge and belief change is prerequisite to downstream classroom studies (e.g., impacts on student learning), retention of evolutionary knowledge improvements and acceptance change emerge as central, but previously unstudied, components of teacher evolution PD.
Research Interests:
Understanding sources of performance bias in science assessment provides important insights into whether science curricula and/or assessments are valid representations of student abilities. Research investigating assessment bias due to... more
Understanding sources of performance bias in science assessment provides important insights into whether science curricula and/or assessments are valid representations of student abilities. Research investigating assessment bias due to factors such as instrument structure, participant characteristics, and item types are well documented across a variety of disciplines. However, the relationships among these factors are unclear for tasks evaluating understanding through performance on scientific practices, such as explanation. Using item-response theory (Rasch analysis), we evaluated differences in performance by gender on a constructed-response (CR) assessment about natural selection (ACORNS). Three isomorphic item strands of the instrument were administered to a sample of undergraduate biology majors and nonmajors (Group 1: n = 662 [female = 51.6%]; G2: n = 184 [female = 55.9%]; G3: n = 642 [female = 55.1%]). Overall, our results identify relationships between item features and performance by gender; however, the effect is small in the majority of cases, suggesting that males and females tend to incorporate similar concepts into their CR explanations. These results highlight the importance of examining gender effects on performance in written assessment tasks in biology.
Research Interests:
Research Interests:
Our study investigates the challenges introduced by students’ use of lexically ambiguous language in evolutionary explanations. Specifically, we examined students’ meaning of five key terms incorporated into their written evolutionary... more
Our study investigates the challenges introduced by students’ use of lexically ambiguous language in evolutionary explanations. Specifically, we examined students’ meaning of five key terms incorporated into their written evolutionary explanations: pressure, select, adapt, need, and must. We utilized a new technological tool known as the Assessment Cascade System (ACS) to investigate the frequency with which biology majors spontaneously used lexically ambiguous language in evolutionary explanations, as well as their definitions and explanations of what they meant when they used such terms. Three categories of language were identified and examined in this study: terms with Dual Ambiguity, Incompatible Ambiguity, and Unintended Ambiguity. In the sample of 1282 initial evolutionary explanations, 81 % of students spontaneously incorporated lexically ambiguous language at least once. Furthermore, the majority of these initial responses were judged to be inaccurate from a scientific point of view. While not significantly related to gender, age, or reading/writing ability, students’ use of contextually appropriate evolutionary language (pressure and adapt) was significantly associated with academic performance in biology. Comparisons of initial responses to follow-up responses demonstrated that the majority of student explanations were not reinterpreted after consideration of the follow-up response; nevertheless, a sizeable minority was interpreted differently. Most cases of interpretation change were a consequence of resolving initially ambiguous responses, rather than a change of accuracy, resulting in an increased understanding of students’ evolutionary explanations. We discuss a series of implications of lexical ambiguity for evolution education.
Considerable research has focused on differences in expert and novice problem representation and performance within physics, chemistry, and genetics. Here, we examine whether models of problem solving based on this work are useful within... more
Considerable research has focused on differences in expert and novice problem representation and performance within physics, chemistry, and genetics. Here, we examine whether models of problem solving based on this work are useful within the domain of evolutionary biology. We utilized card sort tasks, interviews, and paper-and-pencil tests to: (1) delineate problem categorization rules, (2) quantify problem solving success, and (3) measure the relationships between the composition, structure, and coherence of problem solutions. We found that experts and novices perceived different item features to be of significance in card sort tasks, and that sensitivity to item surface features was adversely associated with problem solving success. As in other science domains, evolutionary problem representation and problem solving performance were tightly coupled. Explanatory coherence and the absence of cognitive biases were distinguishing features of evolutionary expertise. We discuss the implications of these findings for biology teaching and learning.
Our study explored the prospects and limitations of using machine-learning software to score introductory biology students’ written explanations of evolutionary change. We investigated three research questions: 1) Do scoring models built... more
Our study explored the prospects and limitations of using machine-learning software to score introductory biology students’ written explanations of evolutionary change. We investigated three research questions: 1) Do scoring models built using student responses at one university function effectively at another university? 2) How many human-scored student responses are needed to build scoring models suitable for cross-institutional application? 3) What factors limit computer-scoring efficacy, and how can these factors be mitigated? To answer these questions, two biology experts scored a corpus of 2556 short-answer explanations (from biology majors and nonmajors) at two universities for the presence or absence of five key concepts of evolution. Human- and computer-generated scores were compared using kappa agreement statistics. We found that machine-learning software was capable in most cases of accurately evaluating the degree of scientific sophistication in undergraduate majors’ and nonmajors’ written explanations of evolutionary change. In cases in which the software did not perform at the benchmark of “near-perfect” agreement (kappa > 0.80), we located the causes of poor performance and identified a series of strategies for their mitigation. Machine-learning software holds promise as an assessment tool for use in undergraduate biology education, but like most assessment tools, it is also characterized by limitations.

And 30 more

... Rysanek Rivera, Jermaine Lawson, Maria Harvey, and Ross H. Nehm ... Upper panel: location of NMB samples 15966–15809 in the Gurabo Formation, along with the geographic location of each sample along the banks of the Rio Gurabo (see... more
... Rysanek Rivera, Jermaine Lawson, Maria Harvey, and Ross H. Nehm ... Upper panel: location of NMB samples 15966–15809 in the Gurabo Formation, along with the geographic location of each sample along the banks of the Rio Gurabo (see Saunders et al., 1986 for a ...
For nearly 30 years the Dominican Republic Neogene has served as a productive research system for exploring a broad array of palaeobiological topics, including speciation (e.g., Cheetham, 1986, 1987; Nehm and Geary, 1994; Nehm, 2005),... more
For nearly 30 years the Dominican Republic Neogene has served as a productive research system for exploring a broad array of palaeobiological topics, including speciation (e.g., Cheetham, 1986, 1987; Nehm and Geary, 1994; Nehm, 2005), intraspecific morphological variation (e.g., Anderson, 1994, 1996; Foster, 1986; Nehm, 2001), palaeoecological reconstruction (e.g., Vokes, 1989; Costa et al., 2001), and faunal turnover (e.g., Budd
... The description and study of ontogenetic variability is an essential component of systematic and evolutionary research (Raup and Stanley, 1978:55), yet its explicit consideration is often absent from contemporary paleobiological... more
... The description and study of ontogenetic variability is an essential component of systematic and evolutionary research (Raup and Stanley, 1978:55), yet its explicit consideration is often absent from contemporary paleobiological studies of molluscs. ...
Page 179. Linking Macroevolutionary 6 Pattern and Developmental Process in Marginellid Gastropods Ross H. NEHM Introduction What processes generate, maintain, and constrain the production of mor-phological disparity through geologic time?... more
Page 179. Linking Macroevolutionary 6 Pattern and Developmental Process in Marginellid Gastropods Ross H. NEHM Introduction What processes generate, maintain, and constrain the production of mor-phological disparity through geologic time? ...
The richly fossiliferous Neogene stratigraphic sections of the Dominican Republic serve as one of only a few geological research systems in the world where morphological stasis and punctuated speciation have been investigated in multiple... more
The richly fossiliferous Neogene stratigraphic sections of the Dominican Republic serve as one of only a few geological research systems in the world where morphological stasis and punctuated speciation have been investigated in multiple lineages. This research system provides unprecedented opportunities for comparative studies of evolutionary stasis and change and their environmental and ecological contexts. In this volume, a diverse group of geologists and paleobiologists collectively focus their attention on this research system, providing an updated geological framework and a series of novel studies of evolutionary stasis and change among different lineages and associated ecological communities. This collection of studies illustrates the immense potential of collaborative, multidisciplinary, and field-based paleobiological research for studies of macroevolutionary change in the fossil record.
... The Books. Genetics of original sin: The impact of natural selection on the future of humanity.Ross H. Nehm. Article first published online: 30 MAR 2011. DOI: 10.1002/sce.20454. ... More content like this. Find more content: like this... more
... The Books. Genetics of original sin: The impact of natural selection on the future of humanity.Ross H. Nehm. Article first published online: 30 MAR 2011. DOI: 10.1002/sce.20454. ... More content like this. Find more content: like this article. Find more content written by: Ross H. Nehm ...
Although scientists view evolution as an indisputable feature of the natural world, most Americans simply do not believe that it occurs, or they reject naturalistic explanations for biotic change. Empirical studies have revealed that... more
Although scientists view evolution as an indisputable feature of the natural world, most Americans simply do not believe that it occurs, or they reject naturalistic explanations for biotic change. Empirical studies have revealed that students and teachers often know ...
The deluge of scholarly titles produced in association with the 150th anniversary of the publication of Darwin's On the Origin of Species has left many evolution enthusiasts struggling to keep up with the literature. Many popular... more
The deluge of scholarly titles produced in association with the 150th anniversary of the publication of Darwin's On the Origin of Species has left many evolution enthusiasts struggling to keep up with the literature. Many popular works also continue to be produced ...
... complexes (Budd and Klaus) and geminate species pairs (Marko and Jackson) to genera (Cheetham et al., Roopnarine, Tschudin) to families and superfamilies (Beu, Hottinger, Nehm, Vermeij, Schneider and Carter) to orders (Aguilera and... more
... complexes (Budd and Klaus) and geminate species pairs (Marko and Jackson) to genera (Cheetham et al., Roopnarine, Tschudin) to families and superfamilies (Beu, Hottinger, Nehm, Vermeij, Schneider and Carter) to orders (Aguilera and Aguilera, Donovan and Veltkamp ...
This study investigated whether an increase in secondary science teacher knowledge about evolution and the nature of science gained from completing a 14-week graduate-level evolution intervention was associated with greater advocacy for... more
This study investigated whether an increase in secondary science teacher knowledge about evolution and the nature of science gained from completing a 14-week graduate-level evolution intervention was associated with greater advocacy for teaching evolution in ...
A marine and coastal resource education project for secondary science teachers, "NYC Marine Life", was developed for a cohort of pre-certified, in-service secondary science teachers. The project had five overarching goals: (1) to increase... more
A marine and coastal resource education project for secondary science teachers, "NYC Marine Life", was developed for a cohort of pre-certified, in-service secondary science teachers. The project had five overarching goals: (1) to increase teacher knowledge of the biodiversity and marine environments that occur throughout the New York City region; (2) to increase teacher use of local coastal and marine examples when teaching required components of the secondary science curriculum (e.g., the NYS Living Environment); (3) to assess the impact of "NYC Marine Life" on the percentage of teachers who introduce their secondary students to coastal habitats; and (4) to determine how learning about local natural history and marine biology affected teachers' cognitive conceptualizations of the city and their "sense of place". Nineteen pre-certified secondary science teachers participated in the 13-week intervention. This cohort consisted largely of teaching fellows originally from outside of NYC. Pre-and post-course instruments, along with transcribed oral interviews, were employed to assess program goals. On the pre-course instrument 100% of teachers self-reported having limited knowledge of the marine resources of New York City and 61% self-reported having no knowledge of this topic at all. Pre/post intervention analyses indicated that (1) teacher self-reported knowledge and measured knowledge of the biology, ecology, and geology of the marine resources of the NYC region increased significantly; (2) teachers' planned use of local coastal and marine resource examples in their classroom curricula increased significantly; and (3) the percentage of teachers who intended to make use of the field increased significantly. "Sense of place" also emerged as a component of planned pedagogical change; several teachers thought that students needed to be better connected with their local environment. Constraints imparted by the NY Regents test, the lack of marine biology in the Living Environment curriculum, the difficulty of accessing the field, and student unfamiliarity with marine biodiversity were reported as explanations for why teachers would not change their curricula and pedagogy.
Although macroevolution has been the subject of sustained attention in the history and philosophy of science (HPS) community, only in recent years have science educators begun to more fully engage with the topic. This chapter first... more
Although macroevolution has been the subject of sustained attention in the history and philosophy of science (HPS) community, only in recent years have science educators begun to more fully engage with the topic. This chapter first explores how science educators have conceptualised macroevolution and how their perspectives align with the views from HPS. Second, it illustrates how science educators’ limited engagement with HPS scholarship on macroevolution has influenced construct delineation, measurement instrument development, and educational arguments about which aspects of macroevolution are most important for students to learn. Third, it discusses how scientific debates about the causal factors responsible for macroevolutionary patterns have been exploited by creationists and have impacted the teaching of evolution. Finally, it emphasizes that the rich perspectives that HPS has to offer on the important topic of macroevolution have yet to be integrated into science education scholarship.