Upcoming CARS Talk!

Screen Shot 2015-09-14 at 10.59.23 AM

A&M Alumni Promoted at The College Board

We were notified by some of our friends at The College Board that one of our accomplished alumni was recently promoted in her role as a Psychometrician. See below some of the wonderful things folks over at The College Board had to say about Pamela’s accomplishments!

“It is with great pleasure that I would like to announce that Pamela Kaliski has been promoted from Associate Psychometrician to Psychometrician. Pamela has accepted increased responsibilities over the past year with her work on AP and, in particular, her role leading the rSAT and PSAT Concordance work. She is an exemplar employee with excellent time and project management skills which contribute to her ability to be very productive and successful at the many tasks she manages as part of her workload. Pamela is particularly adept at mentoring junior staff and working with other team members. She is a very valuable member of the Standard Setting Team and Psychometrics. Please join me in congratulating Pamela on her well-deserved promotion.” – Deanna Morgan, Senior Director of Psychometric – Standard Setting 

Congratulations to Pamela on this great achievement!

Quantitative Psychology Student Wins Best Research Paper Award at Conference

Liz Pyburn, a student in the Quantitative Psychology Concentration of the Psychological Sciences M.A. program, received the competitive Best Paper by a Graduate StudentAward from the Northeastern Educational Research Association (NERA). Liz presented this empirical research at NERA in October 2014.  Her paper, along with numerous other papers, was subsequently reviewed for quality and contribution to the field by scholars attending the conference. Liz will be formally recognized for her award-winning research at the NERA 2015 conference. Below is a brief summary of her praise-worthy work, which was supervised by her advisor and co-author Dr. Jeanne Horst.

Liz is now a 1st year doctoral student in the Assessment & Measurement PhD program at JMU.

Pyburn, E., Johnston, M., Horst, S.J., & Hathcoat, J. (2014, Oct).  A psychometric evaluation of Miami University Diversity Awareness Subscales. Paper presented at the annual meeting of the Northeastern Educational Research Association, Rocky Hill, CT.


Abstract: Scores from two subscales of the Miami University Diversity Awareness Scale (Learning/Knowledge and Intercultural Interaction) were systematically evaluated using data collected from 2,332 sophomore undergraduate students. The study was conducted in four stages. Item wording was carefully evaluated (Stage One). A series of factor analytic studies on full and reduced sets of items were conducted using random, independent subsamples (Stages Two and Three). There was support for three factors (Cultural Awareness, Personal Awareness, and Openness to Diversity) when analyzing the reduced set of items. Configural and metric invariance across gender was also supported (Stage Four), providing evidence for similar properties of the scores for males and females. Implications and suggestions for use of and future development of the scale are addressed.

A whole lot of Doctors are in the building today!

Last week was a very eventful and exciting week here in CARS. We had 4 dissertation defenses in a row. We’d like to congratulate Megan Rodgers Good, Jerusha Gerstner, Matthew Swain, and Rory Lazowski; who are now all able to put the letters Dr. before their names! Below are some pictures celebrating this special occasion.

Jerusha Gerstner presenting during her dissertation defense on Friday, April 3rd.

Jerusha Gerstner presenting during her dissertation defense on Friday, April 3rd.

Dr. Jerusha Gerstner and her dissertation committee. From left to right: Dr. Jeanne Horst, Dr. Monica Erbacher, Dr. Chris Coleman, Dr. Debbi Bandalos (Chair), and Dr. Dena Pastor

Dr. Jerusha Gerstner and her dissertation committee. From left to right: Dr. Jeanne Horst, Dr. Monica Erbacher, Dr. Chris Coleman, Dr. Debbi Bandalos (Chair), and Dr. Dena Pastor

Megan Rodgers Good presenting during her dissertation defense on Wednesday, April 1st.

Megan Rodgers Good presenting during her dissertation defense on Wednesday, April 1st.

Dr. Lazowski and his committee. From left to right: Dr. Robin Anderson, Dr. Dena Pastor (Chair), Dr. Debbi Bandalos, (on the screen) Dr. Chris Hulleman

Dr. Lazowski and his committee. From left to right: Dr. Robin Anderson, Dr. Dena Pastor (Chair), Dr. Debbi Bandalos, (on the screen) Dr. Chris Hulleman

March 2015 Progress Report


Progress toward achieving the CARS mission during the month of March was evidenced by the following activities and achievements:

Schubert, C., Baller, S., Ott, Walter, K., Sapp, L., Swanson, M., & Jacovidis, J. (2015, March). Developing a health sciences information literacy assessment for undergraduates. Poster presented at the 2015 Association of College and Research Libraries Conference, Portland, OR.


Horst, S.J., Ghant, W.A., & Whetstone, D.H. (2015). Enhancing assessment through the use of mixed methods. Assessment Update, 27, 4-14. Doi: 10.1002/au

Pastor, D.A. & Finney, S.J. (2015). Quantitative psychology. In J.G. Irons & R.L Miller (Eds.) Academic Advising: A handbook for advisors and students volume 2: A guide to the sub-disciplines. Retrieved from the Society for the Teaching of Psychology website: http://teachpsych.org/ebooks/academinc-advising-2015-vol2

Sessoms, J. & Finney, S.J. (in Press). Measuring and modeling change in examinee effort on low-stakes tests across testing occasions. International Journal of Testing.


  • SASS has been reviewing the website constructed by the Student Affairs Assessment Council. They have discussed avenues which through which SASS may be more integrated on this website as a resource for student affairs practitioners. 1st year Doctoral student Kristen Smith has been critical in summarizing this feedback. Feedback to the SAAC website has also been submitted.
  • John Hathcoat and 1st year doctoral student Courtney Sanders developed a SACSCOC workshop proposal. This proposal aims to demystify the concepts of reliability and validity in the instrument selection/design process. This workshop may be critical for not only assessment at the program level but also for selecting performance indicators of institutional effectiveness.
  • 1st year doctoral student Kristen Smith and Tia Mann from the Substance Abuse Prevention program at JMU had a presentation accepted for the annual conference of the Association for the Assessment of Learning in Higher Education that will take place in June 2015. This presentation trains assessment practitioners on implementation fidelity.
  • The results from the STEM Competition have been disseminated for 2015. This year’s competition ended in a tie between Physics and Biology. Congrats to the winning seniors.
  • John Hathcoat and members of the Cluster 2 committee are in the process of developing training materials for establishing criteria to evaluate hand written responses to video. They are in the process of pursuing funding for summer training and coding. As of now, a Q-sort has been constructed for each prompt. Faculty will be trained on how to use the Q-sort to generate ideas for scoring criteria for a rubric. Most of the training materials have been developed, though additional material will need to be created for the audio. Two teams of faculty will likely meet. The first team will use the Q-sort to provide insight into the initial stages of a rubric whereas a second team will review audio recordings of each focus group to inform possible scoring criteria. A plan has also been developed to ensure that Cluster 2 will demonstrate student learning outcomes by Spring 2020.
  • 3rd year Doctoral student Matt Swain traveled to Radford College on March 30th to present to their staff about Assessment Day practices at JMU. Matt served as a GA on the Assessment Day team for two of the three years of his Doctoral program, and in his second year led the Assessment Day GA team.
  • Quantitative Psychology MA student Liz Pyburn successfully defended her thesis titled “Persons can speak louder than variables: Person-centered analyses and the prediction of student success” on Friday, March 6th. Her committee consisted of her advisor, Jeanne Horst (Chair), Monica Erbacher, and Keston Fulcher. Congratulations to Liz!
  • Jeanne Horst and John Hathcoat nominated first year doctoral student Kristen Smith for the JMU Student Employee of the year. Kristen won runner up for her leadership of the Student Affairs Assessment Support Services. Thanks for your hard work, Kristen! Glad that you could be recognized for this.
  • Quantitative Psychology MA student Heather Harris successfully defended her thesis titled “Propensity score matching in higher education assessment” on Wednesday, March 25th. Her committee consisted of Jeanne Horst (Chair), Monica Erbacher, and Keston Fulcher. Congrats to Heather! What an accomplishment!
  • Cluster One representatives met with Dr. Carol Hurney, Executive Director of the Center for Faculty innovation, to discuss assessment report writing. The meeting focused on strategies for report writing that would facilitate meaningful consumption of assessment results. Some of the suggestions included sharing assessment instruments with affiliated programs, framing results by research question (rather than by objective), and discussing the results in light of the more meta-skills or attitudes addressed (rather than in light of specific item writing). Further, Dr. Hurney encouraged Cluster One to ‘tell stories’ with their data, rather than simply reporting it.
  • The APT online submission system is officially open for 2015 APT submissions. All assessment coordinators for academic degree and certificate programs received email notification of this on March 10th.
  • On March 10th, CARS hosted an item writing workshop for the core section of the Information Literacy Test for seniors. Librarian Kathy Clarke led the session where 30 new items were reviewed and revised. There were four graduate students and four faculty members present to assist with the item critiques and revisions. Pizza was enjoyed by all!
  • Assessment Day makeup sessions were held throughout the month of March. There were multiple proctored sessions held in the Assessment & Testing Center for students who were unable to attend Assessment Day. 241 students were invited to attend one of the four scheduled sessions that took place in the evenings and were proctored by CARS GAs. Some of the sessions had to be rescheduled due to snow days, and the team finally wrapped up the last session as of March 20th. After the four sessions were held, any students remaining were directed to go to the Assessment & Testing Center to complete their assessment on a walk-in basis. These students also had holds placed on their accounts that would only be removed upon completion of the necessary assessments. We placed 119 holds, which is a greater number than usual, but we believe that this may have been influenced by the snow cancellations. Students are taking their assessments every day, and we continue to remove holds as they are completed.


  • Laura Williams successfully defended her dissertation titled “The Effect of Examinee Motivation on Value-Added Estimates” on March 17th, 2015. Members of Laura’s committee included Donna Sundre (Chair), Monica Erbacher, and Jeanne Horst. Congratulations to Laura on this accomplishment; this makes her our 44th graduate of the Assessment & Measurement PhD program.
  • Although they were both accepted to other programs, we are thrilled to announce that both Heather Harris and Liz Pyburn have selected to stay at JMU to complete their PhD in our Assessment & Measurement program. Heather & Liz will be graduating from the Psychological Sciences MA program with a concentration in Quantitative Psychology in May. Both Heather and Liz have served external GA positions doing assessment work in the offices of Residence Life and Career and Academic Planning while they were enrolled in the MA program. We are so glad they chose our program to continue on for their PhD and we look forward to having them in CARS for another 3 years!
  • CARS continued their search for a new faculty colleague. We brought in three wonderful candidates for three day campus visits filled with interviews and presentations. We were very pleased with the quality of the applicant pool, and our expectations were met with the candidates. An offer has been extended and accepted and we are very excited to bring Ms. Allison Ames, who will be graduating from the University of North Carolina – Greensboro to JMU to join our faculty. We are very grateful to a hardworking, dedicated search committee co-chaired by Debbi Bandalos and John Hathcoat. Deborah Kipps-Vaughn and David Chase also served on the search committee.
  • 3rd year doctoral student Megan Rodgers Good has accepted a position as the Director of Academic Assessment at Auburn University. She will be the chief advisor to the Provost on all matters to assessment. Megan graduates in May and will begin her new position at Auburn on July 13th. Congratulations to Megan on this new and exciting opportunity. We look forward to seeing where her career takes her!


  •  Emily Hoole from the Center for Creative Leadership (CCL) in Greensboro, North Carolina traveled to JMU to deliver a CARS Talk titled “Assessment for Development: Assessment and Measurement Applications in the Leadership Development Field.” Dr. Hoole, a graduate of JMU’s Assessment & Measurement PhD program provided a very stimulating talk pertaining to the CCL, the Assessment for Development field, and new frontiers in assessment of leadership effectiveness. Emily, who serves as Group Director of Global Research and Evaluation is scheduled to being a three-year stint in Singapore in summer 2015.
  • On March 11th, CARS served as host to a team of five visitors from Osaka University. The team was comprised of Tatsuo Kawashima, Professor; Yukiki Abe, Associate Professor; Kazuya Yamaguchi, Professor; Hiroki Minamioka; and Junko Seno, Administrative Staff. The morning was spent with two General Education cluster coordinators, the Associate Vice Provost, two cluster assessment liaisons, and the Director and Associate Director of CARS. Morning discussions focused on JMU’s general education history, philosophy and framework. The afternoon meeting focused on assessment practice at JMU with Keston Fulcher, Donna Sundre, and Dena Pastor serving as facilitators. It was a very full day. We also found time to share some southern foods at lunch, provide a tour, and shop at the JMU bookstore.

Upcoming Dissertation Defense Announcements

With the arrival of spring also comes a season of dissertation defenses for our graduating PhD students. Below is a schedule of defenses that will be held in the coming days/weeks. Dissertation defenses for doctoral candidates are open to the public.

Wednesday, April 1st, 2015 

Megan Rodgers Good will be defending her dissertation titled “Improving Student Learning in Higher Education: A Mixed Methods Study” from 1:00 – 2:00 P.M.

Thursday, April 2nd, 2015

Matthew Swain will be defending his dissertation titled “The Effects of a Planned Missingness Design on Examinee Motivation and Psychometric Quality” from 3:00 – 4:00 P.M.

Friday, April 3rd, 2015 

Jerusha Gerstner will be defending her dissertation titled “Addressing Serial-Order and Negative-Keying Effects: A Mixed-Methods Study” from 10:00 – 11:00 A.M.

Rory A. Lazowski will be defending his dissertation titled “A Meta-Analytic Tutorial and a Narrative Review on Motivation Interventions in Education” from 1:00 – 3:00 P.M.

Thursday, April 9th, 2015 

Mandalyn Swanson will be defending her dissertation titled “Extending an IRT Mixture Model to detect Random Responders on Non-Cognitive Polytomously Scored Assessments” from 3:00 – 4:00 P.M.

All defense presentations will take place in Anthony-Seeger Hall, Room 9. The presentations will be 45 minutes long, with a 15 minute question and answer session. Committees will then deliberate and discuss with the candidates; this portion of the defense is closed to the public.

We hope you can come and join us in celebrating the culmination of hard-work and research these students have accomplished over their three years in our doctoral program.

Dissertation Defense for Laura Williams

We are pleased to announce that third-year doctoral student Laura Williams successfully defended her dissertation on Tuesday, March 17th, 2015. Laura’s dissertation was titled “”The Effect of Examinee motivation on Value-Added Estimates”. Laura’s committee was very pleased with her presentation and her work. Her committee members were Dr. Donna Sundre (chair), Dr. Jeanne Horst, and Dr. Monica Erbacher. Congratulations to Laura! We wish her continued luck while she searches for jobs in the area. Laura will graduate as the 44th graduate of the Assessment & Measurement PhD program.

Dr. Laura Williams and her advisor & committee chair, Dr. Donna Sundre.

Dr. Laura Williams and her advisor & committee chair, Dr. Donna Sundre.


Get every new post delivered to your Inbox.