The research team for this project is an outgrowth of the PI Forum on Peer Assessment sponsored by the NSF TUES program.  Its members come from a wide variety of academic academic fields.  All have extensive experience in peer review. The project team will be assisted and guided through monthly online meetings with a distinguished three-member Advisory Board. 

Principal Investigators

david-tinapple David Tinnaple (PI), is an Assistant Professor at the School of Arts, Media and Engineering at Arizona State University. He has research experience in interface design for domains where cognitive overload is common, such as healthcare, air-traffic control, and intelligence analysis.  He developed the CritViz [cite, cite] peer-review system to support real-time peer critique of projects in the arts.

Related Publications: 

  • Tinapple, D., Olson, L., & Sadauskas, J. (2013). CritViz: Web-Based Software Supporting Peer Critique in Large Creative Classrooms. Bulletin of the IEEE Technical Committee on Learning Technology, 15(1), 29.
  • Sadauskas, J., Tinapple, D., Olson, L., & Atkinson, R. (2013, June). CritViz: A Network Peer Critique Structure for Large Classrooms. In World Conference on Educational Multimedia, Hypermedia and Telecommunications (Vol. 2013, No. 1, pp. 1437-1445).
  • Tinapple, D., & Woods, D. (2003, October). Message overload from the inbox to intelligence analysis: how spam and blogs point to new tools. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 47, No. 3, pp. 419-423).
gehringer Dr. Ed Gehringer (Lead PI) is a Associate Professor of Computer Science and Computer Engineering at NC State University.  He is the originator of the Expertiza system, which uses peer review to manage projects in which teams of students work on small parts of a large project, such as a wiki textbook.

Related Publications: 

  • Lakshmi Ramachandran, Balaraman Ravindran, and Edward F. Gehringer, “Determining review coverage by extracting topic sentences using a graph-based clustering approach,” EDM 2013: Sixth International Conference on Educational Data Mining, Memphis, July 6–9, 2013
  • Lakshmi Ramachandran and Edward F. Gehringer, “A word-order based graph representation for relevance identification,” CIKM ’12: Proceedings f the 21st ACM International Conference on Information and Knowledge Management, Maui, Hawaii, October 2012, pp. 2327–2330.
  • Lakshmi Ramachandran and Edward F. Gehringer, “Automatic quality assessment for peer reviews of student work,” American Society for Engineering Education, San Antonio, TX, June 0–13, 2012, paper #2012-4776.
  • Lakshmi Ramachandran and Edward F. Gehringer, “Determining Degree of Relevance Between Reviews Using a Graph-Based Text Representation,” Proc. 23rd IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2011), Boca Raton, FL Nov. 7-9, 2011, pp. 442–445.
  • Lakshmi Ramachandran and Edward F. Gehringer, Automated assessment of review quality using latent semantic analysis, 11th IEEE International Conference on Learning Technologies, Athens, GA, July 6-8, 2011.
E_Ford_photo_2009 Dr. Eric Ford (PI) is a Professor at Johns Hopkins University. Prior to that, he was the Forsyth Medical Center Distinguished Professor of Healthcare at the University of North Carolina Greensboro. He is the founder of the Mobius SLIP project, which began as a vehicle for peer-reviewing contributions to case studies.

Related Publications: 

  • Ford, E.W. Wells, R. and Reeves, T.C. (2014). Strategy and Structure: A Learning Perspective and Analysis. Business and Economics Research. Vol. 4(1), pp. 1-14. doi:10.5296/ber.v4i1.4602.
  • Reeves, T.R., Huerta, T.R., and Ford, E.W. (2011). Co-Creating Management Education: Moving Toward Emergent Education in a Complex World. International Journal of Information and Operations Management Education. Vol. 4(3-4). pp. 265-283. doi: 10.1504/IJIOME.2011.044615.
  • Ford, E.W., Huerta, T.R., Diana, M.L., and Campbell, C.R. (2009). A Student Journaling Exercise: Preparing for Competency-based Curricula. The Journal of Health Administration Education. 26: 3, pp. 193-201.
  • Reeves, T.C., Ford, E.W., Duncan, W. J. and Ginter, P.M. (2005). Communication Clarity in Strategic Management Date Sources. Strategic Organizations. Volume 3, pp. 243-278.
  • Richardson, W.D., Ford, E.W., Slovensky, D. J., and Menachemi, N. (2001). The Information Superhighway: Are Management Faculty Still Parked in the Driveway? Journal of Education for Business. Volume 77 (1), pp. 57-61.
jkidd-340x222 Dr. Jennifer Kidd (PI) is a Lecturer in the Darden College of Education at Old Dominion University.  She has written extensively on student-authored wiki textbooks, and has used several different online peer-review systems to review student contributions to them.

Related Publications: 

  • Kidd, J. & O’Shea, P. (under review). Students write the textbook: A quasi-experimental study of an open educational pedagogy. Journal of Interactive Technology and Pedagogy.
  • O’Shea, P. & Kidd, J. (2011). Crowdsourced Grading: Exploring the Validity and Effects of Student-Authored and Student-Evaluated Textbooks. In S. Barton et al. (Eds.), Proceedings of Global Learn 2011 (pp. 1266-1271).
  • Kidd, J., O’Shea, P., Allen, D. & Tamashiro, R. (2011). Student-Authored Textbooks: The Future or Futile?. In M. Koehler & P. Mishra (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2011 (pp. 3274-3279). Chesapeake, VA: AACE.
  • O’Shea, P., Baker, P., & Kidd, J. (2008). Let A Thousand Wikibooks Bloom. The Chronicle of Higher Education, 55(14), A29. Available at http://chronicle.com/free/v55/i14/14a02902.htm
  • O’Shea, P., Kidd, J., Baker, P., Kaufman J., Xiao, Y., & Allen D. [2009, April]. Traditional Textbook vs Student-authored Wikibook: A Quasi-experimental Study of Student Academic Outcomes. Paper presented at annual meeting of American Educational Research Association, San Diego, CA
Luca de Alfaro Dr. Luca de Alfaro (PI) is a Professor of Computer Science at the University of California-Santa Cruz.  His research interest in crowdsourcing led to the peer-review system CrowdGrader. He also works in the areas of reputation systems for e-commerce and collaboration, crowdsourcing, game theory, and formal methods for system design and verification.

Related Publications:

  • L. de Alfaro, M. Shavlovsky. CrowdGrader: Crowdsourcing the Evaluation of Homework Assignments. SIGCSE 2014. ACM Press, 2014.
  • L. de Alfaro, M. Shavlovsky. CrowdGrader: Crowdsourcing the Evaluation of Homework Assignments. Technical report UCSC-SOE-13-11, August 2013. http://arxiv.org/abs/1308.5273
  • V. Polychronopoulos, L. de Alfaro, J. Davis, H. Garcia-Molina, N. Polyzotis. Human-Powered Top-k Lists. In Proceedings of the 16th International Workshop of the Web and Databases (WebDB), 2013.
  • B.T. Adler, L. de Alfaro, A. Kulshrestra, I. Pye. Reputation Systems for Open Collaboration. Communications of the ACM 54.8 (2011): 81-87.

Researchers

OLYMPUS DIGITAL CAMERA Dr. Dmytro Babik, is a Visiting Professor at James Madison University. He has ten years of academic and industry experience in finance and risk analysis, including four years in banking. Four years of entrepreneurship; co-founder and managing partner of Social Learning Solutions/CTASIT (creator of MobiusSLIP.com). His research interests include solving complex open-ended problems in peer-based environments; social learning and peer review analytics and visualization; social computing in open innovation, group decision support and knowledge management.

Related Publications: 

ferry
Dr. Ferry Pramudianto is a Senior Research Engineer at Computer Science Department in NC State University. He has more than seven years of experience in European projects, during which he has led three multinational teams, organized technology transfer workshops, and held presentations in international conferences, as well as for the European Commission. His main research area includes Peer Assessment, Learning Analytics, Service-Oriented Architecture, Model Driven Development, and the Internet of Things.
Related Publications: 
  • Pramudianto, F., Pulman, J., Jahn, M., Avila, B., & Jarke, M. (2014, August). Extending Semantic Device Discovery with Synonym of Terms. InEmbedded and Ubiquitous Computing (EUC), 2014 12th IEEE International Conference on (pp. 81-88). IEEE.
  • Pramudianto, F., Kamienski, C. A., Souto, E., Borelli, F., Gomes, L. L., Sadok, D., & Jarke, M. (2014, December). IoT Link: An Internet of Things Prototyping Toolkit. In 11th Intl Conf on Ubiquitous Intelligence and Computing, 2014 IEEE
  • Lima, W. S., Souto, E., Rocha, T., Pazzi, R. W., & Pramudianto, F. (2015, July). User activity recognition for energy saving in smart home environment. In Computers and Communication (ISCC), 2015 IEEE Symposium on (pp. 751-757). IEEE.
  • Conzon, D., Brizzi, P., Kasinathan, P., Pastrone, C., Pramudianto, F., & Cultrona, P. (2015, February). Industrial application development exploiting IoT vision and model driven programming. In Intelligence in Next Generation Networks (ICIN), 2015 18th International Conference on (pp. 168-175). IEEE.
rachel Rachel Jones is a graduate student in the Mental Health Counseling Program at Old Dominion University. She earned her bachelor’s degree in psychology from James Madison University. She is a graduate assistant for the Department of Teaching and Learning at Old Dominion University.

Related Publications: 

ysong8 Yang Song is a Ph.D. candidate at NC State University. His research is mainly on educational data analysis, educational data mining, systems and algorithms for computer-supported education. As a member of the PeerLogic project, he also works on designing a data sharing mechanism which facilitates data sharing between different educational peer-review systems. In his spare time, he is also a fisherman and photographer.

Related Publications: 
Zhewei Zhewei Hu is a Ph.D. candidate at Computer Science Department at NC State University. He earned his Bachelor in educational technology from Zhejiang Normal University.  In his master thesis, he developed a “pluggable” reputation web service to quantify the reliability and validity of each peer reviewer. He is also one of the main developers of Expertiza, a peer-review web application. His main research area includes Peer Assessment, Reputation Calculation, and Web Application Development.

Related Publications: 

Advisory Board

gilber1 Steven W. Gilbert is the founder and CEO of Teaching, Learning, and Technology (TLT) Group, an independent nonprofit organization affiliated with the American Association for Higher Education (AAHE).  He had worked ten years with EDUCOM, where he served as vice president. He is a frequent guest speaker, consultant, author, and keynote speaker at campus events and conferences, having visited more than 500 colleges and universities since 1994
Arlene_Photo Dr. Arlene Russell is Senior Lecturer in the Department of Chemistry and Biochemistry at UCLA.  She is the director of the Calibrated Peer Review® project, which is by far the most widely used peer-review system in American higher education, having been used by 250,000 students at 1400 institutions. 
schunn_c Dr. Christian Schunn is Professor of Psychology, Learning Sciences, and Intelligent Systems at the University of Pittsburgh Learning Resource Design Center.  He is the originator of the SWoRD project, which has been used by approximately 27,000 students at 150 institutions, and is one of the leaders in research on effective peer review [xiong 2012; cho 2007; wooley 2008; Nelson 2009; Falakmasir 2013].

 

Former Staff

person_placeholder  Julia Morris is a Ph.D. candidate in the Darden College of Education at Old Dominion University.
ravi Ravi Kumar Yadav is a master student at NC State University.Related Publications:

  • Yadav, R. K., & Gehringer, E. F. (2016). Metrics for Automated Review Classification: What Review Data Show. In State-of-the-Art and Future Directions of Smart Learning (pp. 333-340). Springer Singapore.
  • Yadav, R. K., & Gehringer, E. F. Automated Metareviewing: A Classifier Approach to Assess the Quality of Reviews.
  • Yadav, R. K. (2016). Web Services for Automated Assessment of Reviews. (Master Thesis)