A&M Alumni Promoted at The College Board

We were notified by some of our friends at The College Board that one of our accomplished alumni was recently promoted in her role as a Psychometrician. See below some of the wonderful things folks over at The College Board had to say about Pamela’s accomplishments!

“It is with great pleasure that I would like to announce that Pamela Kaliski has been promoted from Associate Psychometrician to Psychometrician. Pamela has accepted increased responsibilities over the past year with her work on AP and, in particular, her role leading the rSAT and PSAT Concordance work. She is an exemplar employee with excellent time and project management skills which contribute to her ability to be very productive and successful at the many tasks she manages as part of her workload. Pamela is particularly adept at mentoring junior staff and working with other team members. She is a very valuable member of the Standard Setting Team and Psychometrics. Please join me in congratulating Pamela on her well-deserved promotion.” – Deanna Morgan, Senior Director of Psychometric – Standard Setting 

Congratulations to Pamela on this great achievement!

Quantitative Psychology Student Wins Best Research Paper Award at Conference

Liz Pyburn, a student in the Quantitative Psychology Concentration of the Psychological Sciences M.A. program, received the competitive Best Paper by a Graduate StudentAward from the Northeastern Educational Research Association (NERA). Liz presented this empirical research at NERA in October 2014.  Her paper, along with numerous other papers, was subsequently reviewed for quality and contribution to the field by scholars attending the conference. Liz will be formally recognized for her award-winning research at the NERA 2015 conference. Below is a brief summary of her praise-worthy work, which was supervised by her advisor and co-author Dr. Jeanne Horst.

Liz is now a 1st year doctoral student in the Assessment & Measurement PhD program at JMU.

Pyburn, E., Johnston, M., Horst, S.J., & Hathcoat, J. (2014, Oct).  A psychometric evaluation of Miami University Diversity Awareness Subscales. Paper presented at the annual meeting of the Northeastern Educational Research Association, Rocky Hill, CT.


Abstract: Scores from two subscales of the Miami University Diversity Awareness Scale (Learning/Knowledge and Intercultural Interaction) were systematically evaluated using data collected from 2,332 sophomore undergraduate students. The study was conducted in four stages. Item wording was carefully evaluated (Stage One). A series of factor analytic studies on full and reduced sets of items were conducted using random, independent subsamples (Stages Two and Three). There was support for three factors (Cultural Awareness, Personal Awareness, and Openness to Diversity) when analyzing the reduced set of items. Configural and metric invariance across gender was also supported (Stage Four), providing evidence for similar properties of the scores for males and females. Implications and suggestions for use of and future development of the scale are addressed.

A whole lot of Doctors are in the building today!

Last week was a very eventful and exciting week here in CARS. We had 4 dissertation defenses in a row. We’d like to congratulate Megan Rodgers Good, Jerusha Gerstner, Matthew Swain, and Rory Lazowski; who are now all able to put the letters Dr. before their names! Below are some pictures celebrating this special occasion.

Jerusha Gerstner presenting during her dissertation defense on Friday, April 3rd.

Jerusha Gerstner presenting during her dissertation defense on Friday, April 3rd.

Dr. Jerusha Gerstner and her dissertation committee. From left to right: Dr. Jeanne Horst, Dr. Monica Erbacher, Dr. Chris Coleman, Dr. Debbi Bandalos (Chair), and Dr. Dena Pastor

Dr. Jerusha Gerstner and her dissertation committee. From left to right: Dr. Jeanne Horst, Dr. Monica Erbacher, Dr. Chris Coleman, Dr. Debbi Bandalos (Chair), and Dr. Dena Pastor

Megan Rodgers Good presenting during her dissertation defense on Wednesday, April 1st.

Megan Rodgers Good presenting during her dissertation defense on Wednesday, April 1st.

Dr. Lazowski and his committee. From left to right: Dr. Robin Anderson, Dr. Dena Pastor (Chair), Dr. Debbi Bandalos, (on the screen) Dr. Chris Hulleman

Dr. Lazowski and his committee. From left to right: Dr. Robin Anderson, Dr. Dena Pastor (Chair), Dr. Debbi Bandalos, (on the screen) Dr. Chris Hulleman

March 2015 Progress Report


Progress toward achieving the CARS mission during the month of March was evidenced by the following activities and achievements:

Schubert, C., Baller, S., Ott, Walter, K., Sapp, L., Swanson, M., & Jacovidis, J. (2015, March). Developing a health sciences information literacy assessment for undergraduates. Poster presented at the 2015 Association of College and Research Libraries Conference, Portland, OR.


Horst, S.J., Ghant, W.A., & Whetstone, D.H. (2015). Enhancing assessment through the use of mixed methods. Assessment Update, 27, 4-14. Doi: 10.1002/au

Pastor, D.A. & Finney, S.J. (2015). Quantitative psychology. In J.G. Irons & R.L Miller (Eds.) Academic Advising: A handbook for advisors and students volume 2: A guide to the sub-disciplines. Retrieved from the Society for the Teaching of Psychology website: http://teachpsych.org/ebooks/academinc-advising-2015-vol2

Sessoms, J. & Finney, S.J. (in Press). Measuring and modeling change in examinee effort on low-stakes tests across testing occasions. International Journal of Testing.


  • SASS has been reviewing the website constructed by the Student Affairs Assessment Council. They have discussed avenues which through which SASS may be more integrated on this website as a resource for student affairs practitioners. 1st year Doctoral student Kristen Smith has been critical in summarizing this feedback. Feedback to the SAAC website has also been submitted.
  • John Hathcoat and 1st year doctoral student Courtney Sanders developed a SACSCOC workshop proposal. This proposal aims to demystify the concepts of reliability and validity in the instrument selection/design process. This workshop may be critical for not only assessment at the program level but also for selecting performance indicators of institutional effectiveness.
  • 1st year doctoral student Kristen Smith and Tia Mann from the Substance Abuse Prevention program at JMU had a presentation accepted for the annual conference of the Association for the Assessment of Learning in Higher Education that will take place in June 2015. This presentation trains assessment practitioners on implementation fidelity.
  • The results from the STEM Competition have been disseminated for 2015. This year’s competition ended in a tie between Physics and Biology. Congrats to the winning seniors.
  • John Hathcoat and members of the Cluster 2 committee are in the process of developing training materials for establishing criteria to evaluate hand written responses to video. They are in the process of pursuing funding for summer training and coding. As of now, a Q-sort has been constructed for each prompt. Faculty will be trained on how to use the Q-sort to generate ideas for scoring criteria for a rubric. Most of the training materials have been developed, though additional material will need to be created for the audio. Two teams of faculty will likely meet. The first team will use the Q-sort to provide insight into the initial stages of a rubric whereas a second team will review audio recordings of each focus group to inform possible scoring criteria. A plan has also been developed to ensure that Cluster 2 will demonstrate student learning outcomes by Spring 2020.
  • 3rd year Doctoral student Matt Swain traveled to Radford College on March 30th to present to their staff about Assessment Day practices at JMU. Matt served as a GA on the Assessment Day team for two of the three years of his Doctoral program, and in his second year led the Assessment Day GA team.
  • Quantitative Psychology MA student Liz Pyburn successfully defended her thesis titled “Persons can speak louder than variables: Person-centered analyses and the prediction of student success” on Friday, March 6th. Her committee consisted of her advisor, Jeanne Horst (Chair), Monica Erbacher, and Keston Fulcher. Congratulations to Liz!
  • Jeanne Horst and John Hathcoat nominated first year doctoral student Kristen Smith for the JMU Student Employee of the year. Kristen won runner up for her leadership of the Student Affairs Assessment Support Services. Thanks for your hard work, Kristen! Glad that you could be recognized for this.
  • Quantitative Psychology MA student Heather Harris successfully defended her thesis titled “Propensity score matching in higher education assessment” on Wednesday, March 25th. Her committee consisted of Jeanne Horst (Chair), Monica Erbacher, and Keston Fulcher. Congrats to Heather! What an accomplishment!
  • Cluster One representatives met with Dr. Carol Hurney, Executive Director of the Center for Faculty innovation, to discuss assessment report writing. The meeting focused on strategies for report writing that would facilitate meaningful consumption of assessment results. Some of the suggestions included sharing assessment instruments with affiliated programs, framing results by research question (rather than by objective), and discussing the results in light of the more meta-skills or attitudes addressed (rather than in light of specific item writing). Further, Dr. Hurney encouraged Cluster One to ‘tell stories’ with their data, rather than simply reporting it.
  • The APT online submission system is officially open for 2015 APT submissions. All assessment coordinators for academic degree and certificate programs received email notification of this on March 10th.
  • On March 10th, CARS hosted an item writing workshop for the core section of the Information Literacy Test for seniors. Librarian Kathy Clarke led the session where 30 new items were reviewed and revised. There were four graduate students and four faculty members present to assist with the item critiques and revisions. Pizza was enjoyed by all!
  • Assessment Day makeup sessions were held throughout the month of March. There were multiple proctored sessions held in the Assessment & Testing Center for students who were unable to attend Assessment Day. 241 students were invited to attend one of the four scheduled sessions that took place in the evenings and were proctored by CARS GAs. Some of the sessions had to be rescheduled due to snow days, and the team finally wrapped up the last session as of March 20th. After the four sessions were held, any students remaining were directed to go to the Assessment & Testing Center to complete their assessment on a walk-in basis. These students also had holds placed on their accounts that would only be removed upon completion of the necessary assessments. We placed 119 holds, which is a greater number than usual, but we believe that this may have been influenced by the snow cancellations. Students are taking their assessments every day, and we continue to remove holds as they are completed.


  • Laura Williams successfully defended her dissertation titled “The Effect of Examinee Motivation on Value-Added Estimates” on March 17th, 2015. Members of Laura’s committee included Donna Sundre (Chair), Monica Erbacher, and Jeanne Horst. Congratulations to Laura on this accomplishment; this makes her our 44th graduate of the Assessment & Measurement PhD program.
  • Although they were both accepted to other programs, we are thrilled to announce that both Heather Harris and Liz Pyburn have selected to stay at JMU to complete their PhD in our Assessment & Measurement program. Heather & Liz will be graduating from the Psychological Sciences MA program with a concentration in Quantitative Psychology in May. Both Heather and Liz have served external GA positions doing assessment work in the offices of Residence Life and Career and Academic Planning while they were enrolled in the MA program. We are so glad they chose our program to continue on for their PhD and we look forward to having them in CARS for another 3 years!
  • CARS continued their search for a new faculty colleague. We brought in three wonderful candidates for three day campus visits filled with interviews and presentations. We were very pleased with the quality of the applicant pool, and our expectations were met with the candidates. An offer has been extended and accepted and we are very excited to bring Ms. Allison Ames, who will be graduating from the University of North Carolina – Greensboro to JMU to join our faculty. We are very grateful to a hardworking, dedicated search committee co-chaired by Debbi Bandalos and John Hathcoat. Deborah Kipps-Vaughn and David Chase also served on the search committee.
  • 3rd year doctoral student Megan Rodgers Good has accepted a position as the Director of Academic Assessment at Auburn University. She will be the chief advisor to the Provost on all matters to assessment. Megan graduates in May and will begin her new position at Auburn on July 13th. Congratulations to Megan on this new and exciting opportunity. We look forward to seeing where her career takes her!


  •  Emily Hoole from the Center for Creative Leadership (CCL) in Greensboro, North Carolina traveled to JMU to deliver a CARS Talk titled “Assessment for Development: Assessment and Measurement Applications in the Leadership Development Field.” Dr. Hoole, a graduate of JMU’s Assessment & Measurement PhD program provided a very stimulating talk pertaining to the CCL, the Assessment for Development field, and new frontiers in assessment of leadership effectiveness. Emily, who serves as Group Director of Global Research and Evaluation is scheduled to being a three-year stint in Singapore in summer 2015.
  • On March 11th, CARS served as host to a team of five visitors from Osaka University. The team was comprised of Tatsuo Kawashima, Professor; Yukiki Abe, Associate Professor; Kazuya Yamaguchi, Professor; Hiroki Minamioka; and Junko Seno, Administrative Staff. The morning was spent with two General Education cluster coordinators, the Associate Vice Provost, two cluster assessment liaisons, and the Director and Associate Director of CARS. Morning discussions focused on JMU’s general education history, philosophy and framework. The afternoon meeting focused on assessment practice at JMU with Keston Fulcher, Donna Sundre, and Dena Pastor serving as facilitators. It was a very full day. We also found time to share some southern foods at lunch, provide a tour, and shop at the JMU bookstore.

Upcoming Dissertation Defense Announcements

With the arrival of spring also comes a season of dissertation defenses for our graduating PhD students. Below is a schedule of defenses that will be held in the coming days/weeks. Dissertation defenses for doctoral candidates are open to the public.

Wednesday, April 1st, 2015 

Megan Rodgers Good will be defending her dissertation titled “Improving Student Learning in Higher Education: A Mixed Methods Study” from 1:00 – 2:00 P.M.

Thursday, April 2nd, 2015

Matthew Swain will be defending his dissertation titled “The Effects of a Planned Missingness Design on Examinee Motivation and Psychometric Quality” from 3:00 – 4:00 P.M.

Friday, April 3rd, 2015 

Jerusha Gerstner will be defending her dissertation titled “Addressing Serial-Order and Negative-Keying Effects: A Mixed-Methods Study” from 10:00 – 11:00 A.M.

Rory A. Lazowski will be defending his dissertation titled “A Meta-Analytic Tutorial and a Narrative Review on Motivation Interventions in Education” from 1:00 – 3:00 P.M.

Thursday, April 9th, 2015 

Mandalyn Swanson will be defending her dissertation titled “Extending an IRT Mixture Model to detect Random Responders on Non-Cognitive Polytomously Scored Assessments” from 3:00 – 4:00 P.M.

All defense presentations will take place in Anthony-Seeger Hall, Room 9. The presentations will be 45 minutes long, with a 15 minute question and answer session. Committees will then deliberate and discuss with the candidates; this portion of the defense is closed to the public.

We hope you can come and join us in celebrating the culmination of hard-work and research these students have accomplished over their three years in our doctoral program.

Dissertation Defense for Laura Williams

We are pleased to announce that third-year doctoral student Laura Williams successfully defended her dissertation on Tuesday, March 17th, 2015. Laura’s dissertation was titled “”The Effect of Examinee motivation on Value-Added Estimates”. Laura’s committee was very pleased with her presentation and her work. Her committee members were Dr. Donna Sundre (chair), Dr. Jeanne Horst, and Dr. Monica Erbacher. Congratulations to Laura! We wish her continued luck while she searches for jobs in the area. Laura will graduate as the 44th graduate of the Assessment & Measurement PhD program.

Dr. Laura Williams and her advisor & committee chair, Dr. Donna Sundre.

Dr. Laura Williams and her advisor & committee chair, Dr. Donna Sundre.

Winter in CARS

Snowy campus

It sure has been a crazy winter here in Harrisonburg. We have had many snow days, as have most of the schools in the area; but that somehow hasn’t stopped us from being productive. It has been a very fruitful winter here in the Center. Here are a few notes to highlight what our faculty and students have been up to.

  • We have been honored with hosting multiple groups of scholars from Japan; who have all come to the United States, and to JMU to look for best practices in assessment. Our first visitor, Dr. Satoko Fukahori, from the National Institute for Educational Policy Research of Japan, was able to join us for our Spring Assessment Day, where she witnessed a large part of our assessment model in action! Earlier this month Donna Sundre met with a visiting group from Osaka University. The group was very eager to learn about assessment practices at JMU, particularly regarding General Education. The CARS and University Programs folks worked together to set up a day of meetings that we hope met all of their expectations. They were also excited to enjoy some local “southern” style foods! We are so humbled by the outreach from our colleagues from Japan, and look forward to further expanding the global culture of assessment that already exists today.
  • A faculty search is currently underway here in CARS as we look for someone to fill Dr. Keston Fulcher’s position as Keston prepares to move into the Executive Director role effective July 1st. As we mentioned previously, Dr. Donna Sundre will be retiring at the conclusion of the spring semester, as she sees her last two doctoral advisees through their dissertation defenses and graduation. Keston, as you may know, has been our Associate Director for some time, as well as having a dual role like all our faculty split between teaching and assessment. At this time we are looking to fill Keston’s position in regards to the assessment/teaching role, and will select a new Associate Director at a later date. The search has been underway since late fall and we are hopeful to be able to complete the process and hire someone very soon.
  • Keston Fulcher, Laura Williams (3rd yr. Doc), and the graduate assistants in the PASS office are gearing up for a new year of APT submissions. This past year brought a new option for a program to select to complete an alternate project in place of their typical APT. Programs would have to meet special criteria to be offered this alternative, such as receiving a score of exemplar for two reporting years in a row. The idea is that the programs will take this alternative year to explore new ideas, or perhaps work on an assessment related project they have been unable to accomplish as a result of having to keep up with the APT reporting cycle. As we go into our second year offering this option, we are pleased to see that programs are taking advantage of this opportunity. We can’t wait to see how these programs will use their time. Additionally, a change to element 6A on the APT will be implemented this year, so Keston and his GA’s have been hard at work preparing for that, meeting with programs to introduce the changes. Stay tuned for more on that in our monthly progress reports!
  • CARS has been working hard to prepare for a move out of Anthony-Seeger scheduled for June. For those alumni who are reading, you may be sad to know that when you return to visit, we will no longer be in the same place geographically that you once called home for 2-3 years. As with most changes, we are sad to leave the comfort of our space, but are eager to see what the new building has to offer. We will be moving into the newly named “Lakeview Hall” where we will, in fact, have a view of the lake. A spectacular one at that! This move is an opportunity for the center to get some new furniture and spruce up our space to become a true showpiece of an office. Keston Fulcher, David Yang, Paula Love, and Sharon Sipe have been working with the university’s property coordinators to ensure that the move is as seamless as possible. We will be sure to share an update with pictures and our new address when we get into our new space in June. A perfect time of year to come visit us on the lake and feed the ducks! (Don’t quote me on that, I don’t know if we are allowed to feed them.)

While this isn’t even close to a full list of the things that have been happening here, it’s a little glimpse into what our winter has been like! We are excited that spring has arrived, as we all prepare to head off to Chicago for the AERA/NCME conferences, and the ongoing dissertation and thesis defenses of our soon to be graduates! Keep checking back for announcements as our students finish up their time here at JMU.


Get every new post delivered to your Inbox.