More Related Content
Similar to buku report tentang analisis TIMSS 2023.pdf
Similar to buku report tentang analisis TIMSS 2023.pdf (20)
Recently uploaded
Recently uploaded (20)
buku report tentang analisis TIMSS 2023.pdf
- 1. TIMSS 2023 Progress Report Matthias von Davier & Ann Kennedy 64th Annual Meeting of the IEA General Assembly September 23-28, 2023
- 2. Overview of the TIMSS 2023 Schedule Frameworks and instrument development 2021 2022 Field Test and finalizing instruments 2023 Data collection and processing 2024 Data analysis and reporting
- 3. The 8th cycle of TIMSS completes the transition to a fully digital assessment. 56 countries and six benchmarking systems are participating in the 4th- grade assessment, while 42 countries and five benchmarking systems are participating in the 8th- grade assessment. Nearly all countries are participating in the digital assessment, a paper- based version featuring only trend items is available. TIMSS 2023 provides almost three decades of trend data in mathematics and science. A bird’s eye view of TIMSS 2023
- 4. TIMSS 2023 Participants Administered TIMSS at both Grade 4 and Grade 8 Australia Azerbaijan Albania (4) Armenia (4) Austria (8) Bahrain Belgium (Flemish) (4) Belgium (French) (4) Bosnia and Herzegovina (4) Bulgaria (4) Brazil Canada (4) Chile Chinese Taipei Cote D’Ivoire (8) Czech Republic Denmark (4) England Finland France Georgia Germany (4) Hong Kong SAR Hungary Iran, Islamic Rep. of Iraq (4) Ireland Israel (8) Italy Japan Jordan Kazakhstan Korea, Rep. of Kosovo (4) Kuwait Latvia (4) Lithuania Macao SAR (4) Malaysia (8) Malta (8) Montenegro (4) Morocco Netherlands (4) New Zealand North Macedonia (4) Norway Oman Palestine (8) Poland (4) Portugal Qatar Romania Saudi Arabia Serbia (4) Singapore Slovak Republic (4) Slovenia (4) South Africa Spain (4) Sweden Türkiye United Arab Emirates United States Uzbekistan Benchmarking Participants Ontario, Canada Quebec, Canada Kurdistan, Iraq Abu Dhabi, UAE Dubai, UAE
- 5. TIMSS 2023 Participants Administered TIMSS at both Grade 4 and Grade 8 Australia Azerbaijan Albania (4) Armenia (4) Bahrain Belgium (Flemish) (4) Belgium (French) (4) Bosnia and Herzegovina (4) Bulgaria (4) Brazil Canada (4) Chile Chinese Taipei Czech Republic Denmark (4) England Finland France Georgia Germany (4) Hong Kong SAR Hungary Iran, Islamic Rep. of Iraq (4) Ireland Italy Japan Jordan Kazakhstan Korea, Rep. of Kosovo (4) Kuwait Latvia (4) Lithuania Macao SAR (4) Montenegro (4) Morocco Netherlands (4) New Zealand North Macedonia (4) Norway Oman Poland (4) Portugal Qatar Romania Saudi Arabia Serbia (4) Singapore Slovak Republic (4) Slovenia (4) South Africa Spain (4) Sweden Türkiye United Arab Emirates United States Uzbekistan Benchmarking Participants Ontario, Canada Quebec, Canada Kurdistan, Iraq Abu Dhabi, UAE Dubai, UAE – Grade 4
- 6. – Grade 8 TIMSS 2023 Participants Administered TIMSS at both Grade 4 and Grade 8 Australia Azerbaijan Austria (8) Bahrain Brazil Chile Chinese Taipei Cote D’Ivoire (8) Czech Republic England Finland France Georgia Hong Kong SAR Hungary Iran, Islamic Rep. of Ireland Israel (8) Italy Japan Jordan Kazakhstan Korea, Rep. of Kuwait Lithuania Malaysia (8) Malta (8) Morocco New Zealand Norway Oman Palestine (8) Portugal Qatar Romania Saudi Arabia Singapore South Africa Sweden Türkiye United Arab Emirates United States Uzbekistan Benchmarking Participants Ontario, Canada Quebec, Canada Kurdistan, Iraq Abu Dhabi, UAE Dubai, UAE
- 7. – Paper Assessment TIMSS 2023 Participants Administered TIMSS at both Grade 4 and Grade 8 Bulgaria (4) Cote D’Ivoire (8) Iran, Islamic Rep. of Iraq (4) Morocco South Africa Benchmarking Participants Ontario, Canada Quebec, Canada Kurdistan, Iraq Abu Dhabi, UAE Dubai, UAE
- 8. • The TIMSS 4th and 8th grade questionnaires share many common context variables. These can be used to reveal shifts in students' attitudes towards math and science over time. • A study funded by the American Educational Research Association (Braun, von Davier, & Chen, 2023) utilizes this pseudo-longitudinal design to predict average achievement changes using shifts in attitudes and other contextual factors. Some Advantages of Participation in TIMSS at both the 4th and 8th grade:
- 9. • Instrument Translation for (Main) Data Collection, Layout, and Functionality Verification • Survey Operations, Sampling, and Data Management Activities • TIMSS 2023 Data Collection • International Quality Control Monitoring Program • Scoring Training and Scoring • Automated Scoring • Analysis and Report Planning TIMSS 2023 Activities and Accomplishments since September 2022
- 10. • Quality Assurance Through Verification • Multiple Language-Specific Versions • Linguistic and Functional Review • Organized Translation and Layout Checks ON TRACK AND COMPLETED Instrument Translation for Main Data Collection, Layout, and Functionality Verification
- 11. • Guidance through Survey Operation Procedures • Comprehensive Coverage – sampling, – test preparation – quality control – data management • Paper-based Assessment Support • Collaborative Sampling Approach ON TRACK AND COMPLETED Survey Operations, Sampling, and Data Management Activities
- 12. TIMSS 2023 Main Data Collection Instruments – Fully Digital • Grade 4: 167 mathematics and 174 science items • Grade 8: 183 mathematics and 202 science items • Student, home, teacher, and school questionnaires Country Participation* Grade 4 54 countries 5 benchmarking participants 90 different country/language versions Grade 8 42 countries 3 benchmarking participants 66 different country/language versions
- 13. • Northern Hemisphere Data Collection began March 2023 • Data Collection Largely on Track • Southern Hemisphere Data Collection began August 2023 • Timely Data Submission on the Horizon • Effective Collaboration and Resolution TIMSS 2023 Main Data Collection ON TRACK AND NEARLY COMPLETE
- 14. • Quality Control for Consistency • Independent Monitoring System • Transition of Responsibility to IEA August 2022 • Efficient Monitoring Process and Collaboration • Active Observations Underway ON SCHEDULE AND COMPLETED (NH) OR UNDERWAY (SH) TIMSS 2023 Quality Control Monitoring
- 15. • Multiple Item Types in TIMSS Assessment Scoring • Scoring Guide Development with Partners • Comprehensive Scoring Training and Support • Continuous Improvement and Updates • Online Scoring System Utilization TIMSS 2023 Scoring Training and Scoring ON TRACK AND NEARLY COMPLETE
- 16. Digital Assessment Benefits: Digital format stores real-time student responses, reducing manual entry and errors, allowing for efficient, reliable, and comparable automated scoring of many items. TIMSS 2023 Digital Assessment Automation
- 17. • Rising Trend in Machine- Supported Scoring • Crucial Field Test Data For Training • Validating Human Scoring by Experts and Machines • Enhancing Scoring Efficiency TIMSS 2023 Automated Scoring ON TRACK TO SUPPORT CONTINUOUS IMPROVEMENT
- 18. TIMSS 2023: Engaging & Interactive Problem Solving and Inquiry Tasks • The ‘building’ is an example of engaging, multi-step, interactive tasks • This TIMSS 2019 task was released and data will be made available for research • TIMSS 2023 will contain the next generation of interactive tasks
- 19. TIMSS 2023: AI Supported Scoring Research Automated Scoring of Graphical Open Responses Incorrect Responses Correct Responses
- 20. TIMSS 2023 Automated Scoring Machine scoring improves processes and data quality – In 4th-grade mathematics, very few human-scored items remain. Most are machine-scored – In science, we were able to switch several items from human to (more objective, reliable, and comparable) machine scoring – AI-based machine scoring will be used to QC scoring variability across countries for selected human-scored items • The goal is to implement a score discrepancy resolution system • Experts will review scoring disagreement between human and AI
- 21. Number of Items Validated Using AI or Rule Based Automated Scoring TIMSS 2023 Automated Scoring
- 22. • Scale Specifications and Context Scaling • Achievement Scaling and Linking to TIMSS 2019 • International Results Report • Methods and Procedures Report TIMSS 2023 Analysis and Reporting IN PREPARATION AND ON TRACK
- 23. • Multiple Secondary Reports in Planning • Curriculum Questionnaires and Encyclopedia • Enhancing Data Visualization and Web Presence TIMSS 2023 Analysis and Reporting IN PREPARATION AND ON TRACK
- 24. • Survey Operations Procedures Units 4–7; Oct 2022-Mar 2023 • Environmental Attitudes and Behaviors Framework; Nov 3, 2022 • Project Management Team meeting (online); Nov 10-11, 2022 • Context Questionnaires and OSS; Sep 29, 2022 • IQCM Training hosted by IEA (virtual); Jan 23-24, 2023 • 6th NRC Meeting in Prague - scoring training; Feb 27, 2023-Mar 02, 2023 • Northern Hemisphere countries conduct data collection; Feb-Jun, 2023 • Curriculum Questionnaires, Encyclopedia outline, and Test Curriculum Matching Analysis spreadsheets posted to countries; Apr 13, 2023 • Southern Hemisphere countries begin data collection; Aug 2023 • Translation & layout verification complete for all countries; Sep 2023 TIMSS 2023 Activities, Meetings, and Milestones since September 2022
- 25. • Project Management Team meeting; October 17-18,2023 • Posting unweighted data almanacs and IEA Hamburg posts national Version 1.0 data for Northern Hemisphere countries; November 22, 2023 • Data collection to be complete; December 2023 • Human-scoring to be complete in Southern Hemisphere countries; January 2024 • 7th NRC meeting to review draft report outline & exhibits; March 17-22, 2024 • Project Management Team meeting; March/April 2024 • 4th Science and Mathematics Item Review Committee meeting; June 12-14, 2024 • 8th NRC meeting (virtual) to review draft text and exhibits for the international report; July 28-August 02, 2024 • Project Management Team meeting; September 2024 Upcoming Activities and Meetings for TIMSS 2023
- 26. TIMSS 2023 Progress Report Matthias von Davier & Ann Kennedy 64th Annual Meeting of the IEA General Assembly September 23-28, 2023