CARS Talk – December 11th

We are pleased to announce that 2011 graduate of the Ph.D. program, Javarro Russell, will be coming back to campus to deliver a CARS talk on Friday, December 11th from 12:00-1:30 PM.

All are invited to attend.

Screen Shot 2015-12-04 at 12.35.07 PM

CARS Hosts Alumni Panel

Six alumni from the assessment and measurement Ph.D. program at JMU gathered to discuss how they landed their jobs, their careers, and their most-used knowledge and skills acquired while at the university. Drs. Jerusha Gerstner (’15), Andrew Jones (’09), Pamela Kaliski (’09), Ross Markle (’10), Laura Williams (’15), and John Willse (’02) all participated in the panel.

Alumni Panelists: (Left to right) Andrew Jones, Pamela Kaliski, Ross Markle, Laura Williams, John Willse, Jerusha Gerstner

Alumni Panelists: (Left to right) Andrew Jones, Pamela Kaliski, Ross Markle, Laura Williams, John Willse, Jerusha Gerstner

The panelists hold a variety of jobs in the field: Jerusha is the manager of Talent Management and Analytic Solutions at Marriot International, Andrew is a Research Scientist with the American Board of Internal Medicine, Pamela is a Psychometrician for The College Board, Ross is a Senior Research and Assessment Advisor in the Higher Education Division of the Educational Testing Service, Laura is the Assessment Program Manager with Western Governors University, and John is the Associate Chair and Associate Professor of Educational Research Methodology as well as the Program Director of the Office of Assessment, Evaluation, and Research Services at the University of North Carolina at Greensboro.

During the discussion, students and visitors asked questions about job searching, ranging from how the panelists knew what they wanted to do in the field to how they found out about their first job opportunities. Many of the panelists credited their mentors in the program, internship experiences, and networks as being instrumental in the job search. Specific skills used in each position were also mentioned, such as being good technical communicators and having the ability learn new techniques independently. The group also discussed the importance of being a published researcher. Areas of focus, such as simulations and standard setting, were also mentioned, but all agreed that strengthening areas of weakness will help students looking for a job after they graduate.


Current students in the assessment and measurement program were able to get a sense of the typical work environment, workday, and career paths to which the skills they acquired from the program could lead them. They were also able to learn how to search for jobs via LISTSERVs and how to figure out which jobs would be right for them; the jobs that they would most enjoy based off of their own experiences through the program.

All in attendance - Current students, alumni, prospective students, and faculty.

All in attendance – Current students, alumni, prospective students, and faculty.

Upcoming CARS Talk!

Screen Shot 2015-09-14 at 10.59.23 AM

A&M Alumni Promoted at The College Board

We were notified by some of our friends at The College Board that one of our accomplished alumni was recently promoted in her role as a Psychometrician. See below some of the wonderful things folks over at The College Board had to say about Pamela’s accomplishments!

“It is with great pleasure that I would like to announce that Pamela Kaliski has been promoted from Associate Psychometrician to Psychometrician. Pamela has accepted increased responsibilities over the past year with her work on AP and, in particular, her role leading the rSAT and PSAT Concordance work. She is an exemplar employee with excellent time and project management skills which contribute to her ability to be very productive and successful at the many tasks she manages as part of her workload. Pamela is particularly adept at mentoring junior staff and working with other team members. She is a very valuable member of the Standard Setting Team and Psychometrics. Please join me in congratulating Pamela on her well-deserved promotion.” – Deanna Morgan, Senior Director of Psychometric – Standard Setting 

Congratulations to Pamela on this great achievement!

Quantitative Psychology Student Wins Best Research Paper Award at Conference

Liz Pyburn, a student in the Quantitative Psychology Concentration of the Psychological Sciences M.A. program, received the competitive Best Paper by a Graduate StudentAward from the Northeastern Educational Research Association (NERA). Liz presented this empirical research at NERA in October 2014.  Her paper, along with numerous other papers, was subsequently reviewed for quality and contribution to the field by scholars attending the conference. Liz will be formally recognized for her award-winning research at the NERA 2015 conference. Below is a brief summary of her praise-worthy work, which was supervised by her advisor and co-author Dr. Jeanne Horst.

Liz is now a 1st year doctoral student in the Assessment & Measurement PhD program at JMU.

Pyburn, E., Johnston, M., Horst, S.J., & Hathcoat, J. (2014, Oct).  A psychometric evaluation of Miami University Diversity Awareness Subscales. Paper presented at the annual meeting of the Northeastern Educational Research Association, Rocky Hill, CT.


Abstract: Scores from two subscales of the Miami University Diversity Awareness Scale (Learning/Knowledge and Intercultural Interaction) were systematically evaluated using data collected from 2,332 sophomore undergraduate students. The study was conducted in four stages. Item wording was carefully evaluated (Stage One). A series of factor analytic studies on full and reduced sets of items were conducted using random, independent subsamples (Stages Two and Three). There was support for three factors (Cultural Awareness, Personal Awareness, and Openness to Diversity) when analyzing the reduced set of items. Configural and metric invariance across gender was also supported (Stage Four), providing evidence for similar properties of the scores for males and females. Implications and suggestions for use of and future development of the scale are addressed.

A whole lot of Doctors are in the building today!

Last week was a very eventful and exciting week here in CARS. We had 4 dissertation defenses in a row. We’d like to congratulate Megan Rodgers Good, Jerusha Gerstner, Matthew Swain, and Rory Lazowski; who are now all able to put the letters Dr. before their names! Below are some pictures celebrating this special occasion.

Jerusha Gerstner presenting during her dissertation defense on Friday, April 3rd.

Jerusha Gerstner presenting during her dissertation defense on Friday, April 3rd.

Dr. Jerusha Gerstner and her dissertation committee. From left to right: Dr. Jeanne Horst, Dr. Monica Erbacher, Dr. Chris Coleman, Dr. Debbi Bandalos (Chair), and Dr. Dena Pastor

Dr. Jerusha Gerstner and her dissertation committee. From left to right: Dr. Jeanne Horst, Dr. Monica Erbacher, Dr. Chris Coleman, Dr. Debbi Bandalos (Chair), and Dr. Dena Pastor

Megan Rodgers Good presenting during her dissertation defense on Wednesday, April 1st.

Megan Rodgers Good presenting during her dissertation defense on Wednesday, April 1st.

Dr. Lazowski and his committee. From left to right: Dr. Robin Anderson, Dr. Dena Pastor (Chair), Dr. Debbi Bandalos, (on the screen) Dr. Chris Hulleman

Dr. Lazowski and his committee. From left to right: Dr. Robin Anderson, Dr. Dena Pastor (Chair), Dr. Debbi Bandalos, (on the screen) Dr. Chris Hulleman

March 2015 Progress Report


Progress toward achieving the CARS mission during the month of March was evidenced by the following activities and achievements:

Schubert, C., Baller, S., Ott, Walter, K., Sapp, L., Swanson, M., & Jacovidis, J. (2015, March). Developing a health sciences information literacy assessment for undergraduates. Poster presented at the 2015 Association of College and Research Libraries Conference, Portland, OR.


Horst, S.J., Ghant, W.A., & Whetstone, D.H. (2015). Enhancing assessment through the use of mixed methods. Assessment Update, 27, 4-14. Doi: 10.1002/au

Pastor, D.A. & Finney, S.J. (2015). Quantitative psychology. In J.G. Irons & R.L Miller (Eds.) Academic Advising: A handbook for advisors and students volume 2: A guide to the sub-disciplines. Retrieved from the Society for the Teaching of Psychology website:

Sessoms, J. & Finney, S.J. (in Press). Measuring and modeling change in examinee effort on low-stakes tests across testing occasions. International Journal of Testing.


  • SASS has been reviewing the website constructed by the Student Affairs Assessment Council. They have discussed avenues which through which SASS may be more integrated on this website as a resource for student affairs practitioners. 1st year Doctoral student Kristen Smith has been critical in summarizing this feedback. Feedback to the SAAC website has also been submitted.
  • John Hathcoat and 1st year doctoral student Courtney Sanders developed a SACSCOC workshop proposal. This proposal aims to demystify the concepts of reliability and validity in the instrument selection/design process. This workshop may be critical for not only assessment at the program level but also for selecting performance indicators of institutional effectiveness.
  • 1st year doctoral student Kristen Smith and Tia Mann from the Substance Abuse Prevention program at JMU had a presentation accepted for the annual conference of the Association for the Assessment of Learning in Higher Education that will take place in June 2015. This presentation trains assessment practitioners on implementation fidelity.
  • The results from the STEM Competition have been disseminated for 2015. This year’s competition ended in a tie between Physics and Biology. Congrats to the winning seniors.
  • John Hathcoat and members of the Cluster 2 committee are in the process of developing training materials for establishing criteria to evaluate hand written responses to video. They are in the process of pursuing funding for summer training and coding. As of now, a Q-sort has been constructed for each prompt. Faculty will be trained on how to use the Q-sort to generate ideas for scoring criteria for a rubric. Most of the training materials have been developed, though additional material will need to be created for the audio. Two teams of faculty will likely meet. The first team will use the Q-sort to provide insight into the initial stages of a rubric whereas a second team will review audio recordings of each focus group to inform possible scoring criteria. A plan has also been developed to ensure that Cluster 2 will demonstrate student learning outcomes by Spring 2020.
  • 3rd year Doctoral student Matt Swain traveled to Radford College on March 30th to present to their staff about Assessment Day practices at JMU. Matt served as a GA on the Assessment Day team for two of the three years of his Doctoral program, and in his second year led the Assessment Day GA team.
  • Quantitative Psychology MA student Liz Pyburn successfully defended her thesis titled “Persons can speak louder than variables: Person-centered analyses and the prediction of student success” on Friday, March 6th. Her committee consisted of her advisor, Jeanne Horst (Chair), Monica Erbacher, and Keston Fulcher. Congratulations to Liz!
  • Jeanne Horst and John Hathcoat nominated first year doctoral student Kristen Smith for the JMU Student Employee of the year. Kristen won runner up for her leadership of the Student Affairs Assessment Support Services. Thanks for your hard work, Kristen! Glad that you could be recognized for this.
  • Quantitative Psychology MA student Heather Harris successfully defended her thesis titled “Propensity score matching in higher education assessment” on Wednesday, March 25th. Her committee consisted of Jeanne Horst (Chair), Monica Erbacher, and Keston Fulcher. Congrats to Heather! What an accomplishment!
  • Cluster One representatives met with Dr. Carol Hurney, Executive Director of the Center for Faculty innovation, to discuss assessment report writing. The meeting focused on strategies for report writing that would facilitate meaningful consumption of assessment results. Some of the suggestions included sharing assessment instruments with affiliated programs, framing results by research question (rather than by objective), and discussing the results in light of the more meta-skills or attitudes addressed (rather than in light of specific item writing). Further, Dr. Hurney encouraged Cluster One to ‘tell stories’ with their data, rather than simply reporting it.
  • The APT online submission system is officially open for 2015 APT submissions. All assessment coordinators for academic degree and certificate programs received email notification of this on March 10th.
  • On March 10th, CARS hosted an item writing workshop for the core section of the Information Literacy Test for seniors. Librarian Kathy Clarke led the session where 30 new items were reviewed and revised. There were four graduate students and four faculty members present to assist with the item critiques and revisions. Pizza was enjoyed by all!
  • Assessment Day makeup sessions were held throughout the month of March. There were multiple proctored sessions held in the Assessment & Testing Center for students who were unable to attend Assessment Day. 241 students were invited to attend one of the four scheduled sessions that took place in the evenings and were proctored by CARS GAs. Some of the sessions had to be rescheduled due to snow days, and the team finally wrapped up the last session as of March 20th. After the four sessions were held, any students remaining were directed to go to the Assessment & Testing Center to complete their assessment on a walk-in basis. These students also had holds placed on their accounts that would only be removed upon completion of the necessary assessments. We placed 119 holds, which is a greater number than usual, but we believe that this may have been influenced by the snow cancellations. Students are taking their assessments every day, and we continue to remove holds as they are completed.


  • Laura Williams successfully defended her dissertation titled “The Effect of Examinee Motivation on Value-Added Estimates” on March 17th, 2015. Members of Laura’s committee included Donna Sundre (Chair), Monica Erbacher, and Jeanne Horst. Congratulations to Laura on this accomplishment; this makes her our 44th graduate of the Assessment & Measurement PhD program.
  • Although they were both accepted to other programs, we are thrilled to announce that both Heather Harris and Liz Pyburn have selected to stay at JMU to complete their PhD in our Assessment & Measurement program. Heather & Liz will be graduating from the Psychological Sciences MA program with a concentration in Quantitative Psychology in May. Both Heather and Liz have served external GA positions doing assessment work in the offices of Residence Life and Career and Academic Planning while they were enrolled in the MA program. We are so glad they chose our program to continue on for their PhD and we look forward to having them in CARS for another 3 years!
  • CARS continued their search for a new faculty colleague. We brought in three wonderful candidates for three day campus visits filled with interviews and presentations. We were very pleased with the quality of the applicant pool, and our expectations were met with the candidates. An offer has been extended and accepted and we are very excited to bring Ms. Allison Ames, who will be graduating from the University of North Carolina – Greensboro to JMU to join our faculty. We are very grateful to a hardworking, dedicated search committee co-chaired by Debbi Bandalos and John Hathcoat. Deborah Kipps-Vaughn and David Chase also served on the search committee.
  • 3rd year doctoral student Megan Rodgers Good has accepted a position as the Director of Academic Assessment at Auburn University. She will be the chief advisor to the Provost on all matters to assessment. Megan graduates in May and will begin her new position at Auburn on July 13th. Congratulations to Megan on this new and exciting opportunity. We look forward to seeing where her career takes her!


  •  Emily Hoole from the Center for Creative Leadership (CCL) in Greensboro, North Carolina traveled to JMU to deliver a CARS Talk titled “Assessment for Development: Assessment and Measurement Applications in the Leadership Development Field.” Dr. Hoole, a graduate of JMU’s Assessment & Measurement PhD program provided a very stimulating talk pertaining to the CCL, the Assessment for Development field, and new frontiers in assessment of leadership effectiveness. Emily, who serves as Group Director of Global Research and Evaluation is scheduled to being a three-year stint in Singapore in summer 2015.
  • On March 11th, CARS served as host to a team of five visitors from Osaka University. The team was comprised of Tatsuo Kawashima, Professor; Yukiki Abe, Associate Professor; Kazuya Yamaguchi, Professor; Hiroki Minamioka; and Junko Seno, Administrative Staff. The morning was spent with two General Education cluster coordinators, the Associate Vice Provost, two cluster assessment liaisons, and the Director and Associate Director of CARS. Morning discussions focused on JMU’s general education history, philosophy and framework. The afternoon meeting focused on assessment practice at JMU with Keston Fulcher, Donna Sundre, and Dena Pastor serving as facilitators. It was a very full day. We also found time to share some southern foods at lunch, provide a tour, and shop at the JMU bookstore.

Get every new post delivered to your Inbox.