Joint modeling of product data and process data for psychometric analysis, item response theory modeling, computer-based testing, psychometric research and analysis for large-scale high-stakes tests.

I am a Professor at the University of Maryland (UMD), College Park specializing in educational measurement and psychometrics in large-scale assessment, and also Director of the Maryland Assessment Research Center (MARC). I received my Ph.D. from Florida State University in Measurement, Statistics, and Evaluation. Prior to joining the faculty in Measurement, Statistics, and Evaluation at UMD, I worked as a psychometrician at Harcourt Assessment on different state assessment programs.

The overarching goal of my methodological research is to improve the practice of educational and psychological assessment and develop solutions to emerging psychometric challenges. Many of these are due to the use of more complex innovative assessment formats. I believe that ultimately this work will promote the use of assessment data for cognitive diagnosis to facilitate learning and instruction. I have successfully established a coherent program of research by integrating item response theory (IRT) models, cognitive diagnostic models (CDM), and machine learning methods with multilevel and finite mixture modeling approaches with an aim to exert a positive and lasting effect on measurement practice in large-scale assessments and for cognitive diagnosis purposes.

My research agenda can be summarized into five general categories: methodological research on local dependence due to the use of complex innovative testlets, modeling classification and cognitive diagnosis, Bayesian model parameter estimation, research on computer-based testing, and practical psychometric issues in large-scale tests. My methodological research has been recognized by two national awards, academic work including edited books, book chapters, refereed journal papers, and national and international invited and refereed presentations, and research grants and contracts. I proposed a multilevel testlet model for mixed-format tests that won the 2014 Bradley Hanson Award for Contributions to Educational Measurement by the National Council on Measurement in Education (NCME) and my work with colleagues on Process Data for Educational and Psychological Measurement won the 2023 NCME Annual Award.

I have participated in a variety of professional service activities. I serve on the Technical Advisory Committee (TAC) for the Maryland state testing programs. I have also served on the Research and Psychometric committee for the PARCC consortium testing program representing Maryland and on the Research Advisory Committee for the Prince George Public School of Maryland. Further, my research expertise has been recognized by being voted Chair-Elect for the Rasch Special Interest Group for the AERA, Co-Chair, and Chair of the AERA Division D2 program, Co-Chair and Chair of the AERA Division D Significant Contribution to Educational Measurement and Research Methodology Award Committee and serving on the Early Career Award committee. As a co-chair, I hosted the 2023 International Meeting of the Psychometric Society (IMPS) on the UMD campus.

I have served on the editorial board for the American Educational Research Journal - Teaching, Learning, and Human Development section, Educational Measurement: Issues and Practices, Measurement: Interdisciplinary Research and Perspectives, and the Methods in Psychology journals. I am also on the Editorial Board of the Springer book series: Methodology of Educational Measurement and Assessment. In addition, I served as a reviewer for national conferences, journals, and books on large-scale assessments. I also served on review panels for the National Science Foundation. I served as a guest editor for two special topics on “Process Data in Educational and Psychological Measurement” and “Cognitive Diagnostic Assessment for Learning” in Frontiers in Psychology, and two issues on "Machine Learning and Deep Learning in Assessment" for the Journal of Psychological Test and Assessment Modeling. I co-organized several MARC annual conferences and co-edited books on different cutting-edge topics in assessment including technology-enhanced innovative assessment, the applications of artificial intelligence in assessment, and enhancing effective instruction and learning using assessment data: theory and practice.

2003               The Spaan Fellowship, Funded research in Second or Foreign Language Testing, University of Michigan, Ann Arbor , MI.

2002               The Lenke Psychometric Fellowship, Harcourt Educational Measurement, San Antonio, TX.

1998-2001    University Fellowship, Florida State University, Tallahassee, FL.

2023 The Annual Award for Exceptional Achievement in Educational Measurement by the National Council on Measurement in Education.

2014 The Bradley Hanson Award for Contributions to Educational Measurement by the National Council on Measurement in Education.
2010 The American Educational Research Association Research Grant sponsored by the National Science Foundation.
2005 The Revere Award for Customer Focus, Harcourt Assessment, Inc., San Antonio, TX.

Books Edited (Selected)

  1. Jiao, H., & Lissitz, R. W. (2020). Applications of artificial intelligence to assessment. Charlotte, NC: Information Age Publisher.
  2. Jiao, H., Lissitz, R. W., & Van Wie*, A. (2018, Eds.). Data analytics and psychometrics: Informing assessment practices. Charlotte, NC: Information Age Publisher.
  3. Jiao, H., & Lissitz, R. W. (2017, Eds.). Technology enhanced innovative assessment: Development, modeling, and scoring from an interdisciplinary perspective. Charlotte, NC: Information Age Publisher.
  4. Jiao, H., & Lissitz, R. W. (2017, Eds.). Test fairness in the new generation of large-scale assessment. Charlotte, NC: Information Age Publisher.
  5. Jiao, H., & Lissitz, R. W. (2015, Eds.). The next generation of testing: Common core standards, Smarter-Balanced, PARCC, and the nationwide testing movement. Charlotte, NC: Information Age Publisher.
  6. Lissitz, R. W., & Jiao, H. (2014, Eds.). Value added modeling and growth modeling with particular application to teacher and school effectiveness. Charlotte, NC: Information Age Publisher.
  7. Lissitz, R. W., & Jiao, H. (2012, Eds.). Computers and their impact on state assessment: Recent history and predictions for the future. Charlotte, NC: Information Age Publisher.

Chapters in Books (Selected)

  1. Jiao, H., Liao*, D., & Zhan*, P. (2019). Utilizing process data for cognitive diagnosis. In M. von Davier & Y. Lee (Eds.), Handbook of Diagnostic Classification Models.
  2. He, Q., Liao*, D., & Jiao, H. (2019). Clustering behavioral patterns using process data in PIAAC problem-solving items. In B. P. Veldkamp & C. Sluijter (Eds.). Theoretical and Practical Advances in Computer-Based Educational Measurement. Springer. Methodology of Educational Measurement and Assessment (book series), Springer. https://doi.org/10.1007/978-3-030-18480-3_10.
  3. Jiao, H., & Li*, C. (2018). Progress in International Reading Literacy Study (PIRLS) data. In The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation. Thousand Oaks, CA: Sage.
  4. Jiao, H., & Liao*, D. (2018). Testlet response theory. In The SAGE Encyclopedia of Educational Research, Measurement, and Evaluation. Thousand Oaks, CA: Sage.
  5. Jiao, H., Lissitz, R. W., & Zhan*, P. (2017). Calibrating innovative items embedded in multiple contexts. In H. Jiao & R.W. Lissitz (Eds.), Technology-enhanced innovative assessment: Development, modeling, scoring from an interdisciplinary perspective. Charlotte, NC: Information Age Publishing.
  6. Jiao, H., Kamata, A., & Xie, C. (2015). A multilevel cross-classified testlet model for complex item and person clustering in item response modeling. In J. Harring, L. Stapleton, & S. Beretvas (Eds.), Advances in multilevel modeling for educational research: Addressing practical issues found in real-world applications (pp.139-161). Charlotte, NC: Information Age Publishing.
  7. Luo, Y., Jiao, H., & Lissitz, R. W. (2015). An empirical study of the impact of the choice of persistence model in value-added modeling upon teacher effect estimates. In L. A. van der Ark, D. Bolt, W.-C. Wang, J. A. Douglas & S.-M. Chow (Eds.), Quantitative psychology research (pp.133-143). Springer, Switzerland.
  8. Jiao, H., & Lissitz, R. W. (2014). Direct modeling of student growth with multilevel and mixture extensions. In R. W. Lissitz & H. Jiao (Eds.), Value added modeling and growth modeling with particular application to teacher and school effectiveness (pp.293-306). Charlotte, NC: Information Age Publisher.
  9. Jiao, H., & Chen*, Y.-F. (2014). Differential item and testlet functioning. In A. Kunnan (Ed.),The Companion to Language Assessments (pp.1282-1300). John Wiley & Sons, Inc.
  10. Chen*, Y.-F., & Jiao, H. (2014). Does model misspecification lead to spurious latent classes? An evaluation of model comparison indices. In R. E. Millsap et al. (Eds.), New development in quantitative psychology, Springer Proceedings in Mathematics & Statistics, 66, DOI 10.1007/978-1-4614-9348-8_22, Springer Science +Business Media, New York.
  11. Jiao, H., & Lissitz, R. W. (2012). Computer-based testing in K-12 state assessments: An Introduction. In R. W. Lissitz & H. Jiao (Ed.), Computers and their impact on state assessment: Recent history and predictions for the future (pp. 1-21). Charlotte, NC: Information Age Publisher.
  12. Templin, J. & Jiao, H. (2011). Applying model-based approaches to identify performance categories. In G. Cizek (Ed.), Setting performance standards: foundations, methods, and innovations (pp. 379-397). New York, NY: Routlege.
  13. Jiao, H., Wang, S., Kamata, A. (2007). Modeling local item dependence with the hierarchical generalized linear model. In E. V. Smith & R. M. Smith (Eds.), Rasch Measurement: Advanced and Specialized Applications. JAM press.
  14. Jiao, H. (2004). Evaluating the Dimensionality of the Michigan English Language Assessment Battery. Spaan Fellow Working Papers in Second or Foreign Language Assessment: Volume 2 (pp. 27-52). University of Michigan, Ann Arbor, MI.

Articles in Refereed Journals (Selected)

  1. Liao*, M. & Jiao, H. (2022). Modeling multiple problem-solving strategies and strategy shift in cognitive diagnosis for growth. British Journal of Mathematical and Statistical Psychology.
  2. Qiao*, X. & Jiao, H. (2022). Explanatory cognitive diagnostic modeling incorporating response times. Journal of Educational Measurement. DOI: 10.1111/jedm.12306.
  3. Jiao, H, & Liao, M. (2021). Testlet response theory. Educational Measurement: Issues and Practices.
  4. Liao, M., Patton, J., Yan, R., & Jiao, H. (2020). Mining process data to detect aberrant test takers. Measurement: Interdisciplinary Research and Perspectives.
  5. Jiao, H. & Lissitz, R. (2020). What hath the Coronavirus brought to Assessment? Unprecedented Challenges in Educational Assessment in 2020 and Years to Come. Educational Measurement: Issue and Practices. https://doi.org/10.1111/emip.12363.
  6. Liao*, D., He, Q., & Jiao, H. (2019). Mapping background variables with sequential patterns in problem solving environments: An investigation on US adults’ employment status in PIAAC. Frontiers in Psychology. https://doi.org/10.3389/fpsyg.2019.00646.
  7. Zhan*, P., Jiao, H., Liao*, D. & Li, F (2019). A longitudinal higher-order diagnostic classification model. Journal of Educational and Behavioral Statistics. Advanced Online Publication. URL https://doi.org/10.3102/1076998619827593.
  8. Zhan*, P., Ma, W., Jiao, H., & Ding, S. (2019). A sequential higher-order latent structural model for hierarchical attributes in cognitive diagnostic assessments. Applied Psychological Measurement. Advanced Online Publication. URL https://doi.org/10.1177/0146621619832935.
  9. Zhan*, P., Jiao, H., Man, K, & Wang, L. (2019). Using JAGS for Bayesian cognitive diagnosis modeling: A tutorial. Journal of Educational and Behavioral Statistics. Advanced Online Publication. URL https://doi.org/10.3102/1076998619826040
  10. Zhan*, P., Wang, W.-C., Jiao, H., & Bian, Y. (2018). The probabilistic-inputs, noisy conjunctive models for cognitive diagnosis. Frontiers in Psychology. URL https://doi.org/10.3389/fpsyg.2018.00997.
  11. Zhan*, P., Jiao, H., Liao*, M., & Bian, Y. (2018). Bayesian DINA modeling incorporating within-item characteristics dependency. Applied Psychological Measurement. 43, 143–158. https://doi.org/10.1177/0146621618781594
  12. Qiao*, X., & Jiao, H. (2018). Comparing data mining techniques in analyzing process data: A case study on PISA 2012 problem-solving items. Frontiers in Psychology.
  13. Zhan*, P. Jiao, H., & Liao*, D. (2017). Cognitive diagnosis modeling incorporating item response times. British Journal of Mathematical and Statistical Psychology. doi: 10.1111/bmsp.12114
  14. Luo, Y., & Jiao, H. (2017). Using the Stan program for Bayesian item response theory. Educational and Psychological Measurement. DOI: 10.1177/0013164417693666
  15. Li*, T., Xie*, C., & Jiao, H. (2016). Assessing fit of alternative unidimensional polytomous item response models using posterior predictive model checking. Psychological Methods.
  16. Li*, T., Jiao, H., & Macready, G. (2015). Different approaches to covariate inclusion in the mixture Rasch model. Educational and Psychological Measurement. DOI: 10.1177/0013164415610380
  17. Jiao, H., & Zhang*, Y. (2015). Polytomous multilevel testlet models for testlet-based assessments with complex sampling designs. British Journal of Mathematical and Statistical Psychology, 1, 65-83. Online first，DOI:10.1111/bmsp.12035.
  18. Wolfe, E., Song, T. W., & Jiao, H. (2015). Features of difficult-to-score essays. Assessing Writing.27, 1-10.
  19. Wolfe, E. W., Jiao, H., & Song, T. (2015). A family of rater accuracy models. Journal of Applied Measurement. 16
  20. Chen*, Y.-F. & Jiao, H. (2014). Exploring the utility of background and cognitive variables in explaining latent differential item functioning: An example of the PISA 2009 reading assessment. Educational Assessment.19, 77-96.
  21. Jiao, H., Wang, S., & He, W. (2013). Estimation methods for one-parameter testlet models. Journal of Educational Measurement, 50, 186-203.
  22. Li*, Y., Jiao, H., & Lissitz, R.W. (2012). Applying multidimensional IRT models in validating test dimensionality: An example of K-12 large-scale science assessment. Journal of Applied Testing Technology, vol. 13, n2.
  23. Jiao, H., Macready, G., Liu*, J., & Cho*, Y. (2012). A mixture Rasch model based computerized adaptive test for latent class identification. Applied Psychological Measurement, 36, 469-493.
  24. Jiao, H., Kamata, A., Wang, S., & Jin, Y. (2012). A multilevel testlet model for dual local dependence. Journal of Educational Measurement, 49, 82-100.
  25. Jiao, H., Liu*, J., Haynie, K., Woo, A., & Gorham, J. (2012). Comparison between dichotomous and polytomous scoring of innovative items in a large-scale computerized adaptive test. Educational and Psychological Measurement, 72, 493 - 509.
  26. Jiao, H., Lissitz, B., Macready, G., Wang, S., & Liang*, S. (2011). Exploring levels of performance using the Mixture Rasch Model for standard setting. Psychological Testing and Assessment Modeling, 53, 499-522.
  27. Jiao, H., & Wang, S. (2010). A multifaceted approach to investigating the equivalence between computer-based and paper-and-pencil assessments: An example of Reading Diagnostics. International Journal of Learning Technology, 5, 264-288.
  28. Wang, S., & Jiao, H. (2009). Construct equivalence across grades in a vertical scale for a K-12 large-scale reading assessment. Educational and Psychological Measurement, 69, 760-777.
  29. Wang, S., Jiao, H., Young, M. J., Brooks, T., & Olson, J. (2008). Comparability of computer- based and paper-and-pencil testing in K-12 reading assessments: A meta-analysis of testing mode effects. Educational and Psychological Measurement, 68(1), 5-24.
  30. Wang, S., Jiao, H., Young, M. J., Brooks, T., & Olson, J. (2007). A meta-analysis of testing mode effects in Grade K-12 Mathematics Tests. Educational and Psychological Measurement, 67(2), 219-238.

  31. Jiao, H., Wang, S., & Kamata, A. (2005). Modeling local item dependence with the hierarchical generalized linear model. Journal of Applied Measurement, 6(3), 311-321.

     

External Funding

Funding agency: Maryland State Department of Education 
Title: Psychometric research and analysis for Maryland state assessment programs.

Funding agency: The Council of Chief State School Officers

Title: Applying sampling weights in Kindergarten Readiness Assessment

Funding agency: The Partnership for Assessment of Readiness for College and Careers, Inc.
Title: Investigating New York City students’ performance on and experience with the 2015 PARCC pilot tests.
Funding agency: Management Systems International
Title: Alignment Study for University Readiness Test in Egypt.
Funding agency: National Council on Measurement in Education
Title: A multilevel testlet model for mixed-format tests
Funding agency: American Educational Research Association/National Science Foundation #DRL-0941014
Title: Latent differential item functioning analysis for testlet-based assessments
Funding agency: National Council of State Boards of Nursing, Joint Research Committee
CO-PI with PI: Kathleen C. Haynie
Title: A Partial Credit Modeling Study of NCLEX Innovative Items
Funding agency: University of Michigan, Ann Arbor
Title: Evaluating the dimensionality of the Michigan English Language Assessment Battery

 

COURSES TAUGHT 

Graduate Courses

Course Title
Instrumentation
Applied Measurement: Issues and Practices
Classification and cognitive diagnosis
Computerized adaptive testing
Psychometrics in large-scale assessment
Quantitative methods I
Modern measurement theory-Item response Theory