Book of Interest: "Demystifying Outcomes Assessment for International Educators: A Practical Approach" by Darla Deardorff

I'm writing about a book that was published in 2015 that may be of interest to many IHEC Blog readers.  "Demystifying Outcomes Assessment for International Educators: A Practical Approach" by my colleague Darla Deardorff with Forewards by Trudy Banta and Hans de Wit is an interesting and valuable book that many in the field may wish to consider reading. 

Demystifying Outcomes Assessment for International Educators is intended for practitioners and professors in international higher education (including study abroad professionals, senior international officers, etc.) and includes chapters on Frequently Asked Questions on outcomes assessment, practical strategies for getting started, avoiding assessment pitfalls, and even a chapter for university leaders on strategic-level assessment issues.   The appendices contain useful worksheets and handouts that can be used with others, as well as practical examples, resources and insightful articles related to assessing intercultural learning outcomes.

Following is a description of the book from the back cover along with the table of contents:

This is a practical guide to learning outcomes assessment in international education for practitioners who are starting to engage with the process, as well as for those who want to improve the quality and effectiveness of existing assessment practices

Table of Contents:

Foreword 1 – Trudy Banta

Foreword 2 – Hans de Wit

Acknowledgements

Preface

Chapter 1: Framing International Education Assessment 

Chapter 2: Thirty Frequently Asked Questions on Assessment in International Practice

Chapter 3: The Unique Assessment Context of International Education

Chapter 4: Principles for Effective Assessment in International Education

Chapter 5: Approaching International Education Assessment Holistically: The Logic Model

Chapter 6: Getting Started

Chapter 7: Developing an Assessment Strategy: The Assessment Process

Chapter 8: Pitfalls to Avoid, Strategies and Lessons Learned

Chapter 9: The Leadership Role in International Education Assessment

Chapter 10: The Future of International Education Assessment 

APPENDICES

Appendix A: Handouts and Worksheets

Appendix B: Assessing Intercultural Competence as a Learning Outcome of International Education

Appendix C: Some Thoughts on Assessing Intercultural Competence

Appendix D: Checklist for Evaluating the Effectiveness of Outcomes Assessment in International Education

Appendix E: Some Assessment Tools for International Education Contexts

Appendix F: International Education Learning Outcomes Examples

Appendix G: Selected Assessment Resources for International Educations

References

Index


Disclaimer:  I received no payment nor other forms of payment for posting about Demystifying Outcomes Assessment for International Educators: A Practical Approach

Study Abroad Outcomes!

Many people know of my strong interest in research on outcomes of international educational and cultural experiences. 

Following are two photos of an outcome from my study abroad experience in Spain during my junior year in college...I met my wife while studying abroad (we were on the same study abroad program) and today we are celebrating our 21st wedding anniversary!

This obviously has nothing to do with research on outcomes of international educational and cultural experiences but for me it was the best outcome of my entire exprience!

Wedding Day - June 9, 1995

My Daughter's 8th Grade Graduation - June 7, 2016

Relationships of Assessment, Evaluation, Standards & Ethics in Education Abroad

Some time ago when I was involved more heavily with the Forum on Education Abroad's Standards of Good Practice Committee on the Standards Toolbox and as a consultant I spent much time thinking about the relationships between assessment, evaluation, standards and ethics in education abroad.  As I was traveling one day I started to doodle various configurations of these relationships and upon return home prepared them on a single document to further spur my interest in and investigation of these relationships.  At one point in time I was thinking that this would be the focus of my dissertation.  Anyways, last night I was searching for some of my documents I have hosted over on Scribd and came across the document where I turned my doodles of these relationships into more concrete ideas.  Following are just thoughts I had on this topic back in 2007 or so and I thought I would embed the document here on IHEC Blog to see what others think.
Relationships of Assessment, Evaluation, Standards & Ethics in Ed. Abroad by Comp, 2007-2009

The Consultative Service on United States Undergraduate Study Abroad

Frequent IHEC Blog readers know that I am very interested in the history of international education and in particular here in the United States.  The following is a snippet from my research notes on the history of outcomes assessment, program evaluation and standards of good practice in study abroad that I thought I would share here on IHEC Blog:


The Consultative Service on United States Undergraduate Study Abroad was a coordinating venture of the Association of American Colleges, the Council on Student Travel, the Experiment in International Living, and the Institute of International Education.  It was formed in the spring of 1963 with a grant by the Carnegie Corporation of New York.   Its objectives were:  (1) to secure full and precise information on international education, (2) to coordinate the rapidly proliferating ventures of United States educational institutions for undergraduate study abroad, (3) to ease communication and cooperation between institutions, and (4) to establish the basis for determining standards which will evaluate undergraduate study abroad programs.

Institute of International Education:  Undergraduate Study Abroad, Report of the Consulate Service on U.S. Undergraduate Study Abroad (New York:  The Institute, 1964).

Speakman, C.E.  (1966).  International Exchange in Education.  New York:  Center for Applied Research in Education.  pp. 71-72

New Book In The Spotlight: “Transformations at the Edge of the World: Forming Global Christians through the Study Abroad Experience”

From time to time I highlight/review books on IHEC Blog and today I wanted to highlight a new book I received from (2010) Abilene Christian University Press that many IHEC Blog readers may find of interest.  Transformations at the Edge of the World: Forming Global Christians through the Study Abroad Experience is edited by Ronald J. Morgan and Cynthia Toms Smedley and is 272 pages in length.  Here is a description of Transformations at the Edge of the World from the website:

“For the editors of this volume, "the edge of the world" is neither a place of physical remoteness where cell phone coverage runs out nor a region of economic marginality. Instead, it is a place of personal transformations, the point of convergence between the individual student's spiritual development and her commitment to global engagement. The formation of such students requires a holistic approach to education, one that connects knowledge with action and heart with head. In the study abroad context, holistic education emphasizes ethical formation, cultivation of empathy, the creation of deep communities, and commitment to social responsibility.


This collection of essays is characterized by the diversity of its voices. The contributors represent a broad range of faith-based institutions and a wide array of academic disciplines. The study abroad programs they describe, scattered across five continents including North America, reflect a variety of models. Gathered into this pioneering collection are essays that theorize about the conjunction of spiritual formation and global engagement, share practices that are already bearing fruit, and identify potential areas of growth and improvement for the future.”


What I like about Transformations at the Edge of the World is that it focuses on an aspect of study abroad [spirituality] that is quite often neglected in the literature base of our field.  I found Transformations at the Edge of the World to be an excellent read and I highly recommend it to all colleagues and researchers in the field and not just those working at Christian Colleges, Universities and organizations.  Additionally, Transformations at the Edge of the World is very reasonably priced at $24.99 and you can order it here.  Following is an abbreviated Table of Contents:

Table of Contents

Foreword—Ken Bussema (Vice President of the Council for Christian Colleges and

Universities)

Introduction—Cynthia Toms Smedley (University of Notre Dame, Notre Dame, Indiana;

Center for Social Concerns)

Part I:

The Journey Inward

Chapter One: Doors to Transformation— Janine Paden Morgan (Abilene Christian University, Abilene, Texas; ACU in Oxford)

Chapter Two: Students Abroad as Tourists and Pilgrims— John D. Barbour (St. Olaf College, Northfield, Minnesota; semester in Asia)

Chapter Three: Reflection as a Means of Discovery: Where Is God in the Experience?— Andrea Smith Shappell (University of Notre Dame, Notre Dame, Indiana; Center for Social Concerns)

Chapter Four: Seizing the “God Appointments” When There Is Cultural Disorientation in a Study Abroad Program— Lon Fendall (George Fox University, Newberg, Oregon; semester in Bolivia-Paraguay)

Part II:

Inward Journey to Outward Living: Community Teacher

Chapter Five: New Monasticism Meets Renaissance Bottega: Gordon College’s Semester Program in Orvieto, Italy— JohnSkillen (Gordon College, Wenham, Massachusetts; semester in Orvieto, Italy)

Chapter Six: An Intentional Roman Catholic Community: Integrating Faith, Reason, and Service at the Heart of the Church— Don Briel (University of St. Thomas, St. Paul, Minnesota; semester in Rome)

Part III:

Coming Face to Face with the Social Other: Bridging Intercommunal Divides

Chapter Seven: “With Open Eyes”: Cultivating World Christians through Intercultural Awareness— Laura Montgomery and Mary Docter (Westmont College, Santa Barbara, California; Westmont in Mexico)

Chapter Eight: Who Is My Neighbor? Forming Kingdom People in a World of Conflict— Ronald J. Morgan (Abilene Christian University, Abilene, Texas; ACU in Oxford)

Chapter Nine: Middle Eastern Mirrors for the Children of Empire— David Holt (Council for Christian Colleges and Universities [CCCU] Middle East Studies Program, Cairo, Egypt)

Chapter Ten: San Francisco Urban Program: Encountering America’s Future-Tense— Scott McClelland, Karen Andrews, and Brad Berky (Westmont College, Santa Barbara, California; San Francisco Urban Program)

Part IV:

TheYear of the Lord’s Good Favor: Cultivating Solidarity with the Global Poor

Chapter Eleven: Learning from Slums: Study and Service in Solidarity with the World’s Urban Poor— Richard Slimbach (Azusa Pacific University, Azusa, California; Global Studies Program, India, Philippines, and Haiti)

Chapter Twelve: The Phenomenology of a Christian Environmental Study Abroad Program— Chris Elisara (Creation Care Study Program, San Diego, California; Belize, New Zealand, and Samoa)

Chapter Thirteen: The Study Service Term: An Alignment of a Religious Tradition with an Academic Program— Thomas J. Meyers (Goshen College, Goshen, Indiana; Study Service Term, primarily working in developing countries)

Conclusion: Toward Successful Transformations— Ronald J. Morgan (Abilene Christian University, Abilene, Texas; ACU in Oxford)

Appendix: Liturgies for Study Abroad— Janine Paden Morgan (Abilene Christian University, Abilene, Texas; ACU in Oxford)

A Thought on the Problems I see with Researching Outcomes from Study Abroad

Many IHEC Blog readers and colleagues know that I am very interested in the scholarly side of the field of international education with particular research interests focusing on U.S. students studying abroad. While my research interests have been changing over the last several years I’m still very interested in the micro level outcomes that study abroad programs have on students. The following two articles published in International Education Forum (former journal of AIEA) in 1997 not only sparked my interest in the scholarly side of the field but put forth a framework that as guided my thinking since.

Stimpfl, Joseph R., and David Engberg. “What to Know Before You Go: Creating a Comparison for Research on Study Abroad Programs,” International Education Forum, 17 no. 1 (1997): 7-21.

Stimpfl, Joseph R. and David Engberg. "Comparing Apples to Apples: An Integrated Approach to Study Abroad Program Assessment," International Education Forum, 17 no. 2 (1997): 97-109.

In my opinion, it is a flawed research design to compare study abroad programs without calling attention to the difference between the programs. Study abroad programs share some characteristics but there are huge differences between programs. Can we really compare the student experiences on a six to twelve month homestay immersion program in China with courses taught in Chinese by local faculty/instructors to the student experiences of a three to six month program in Spain with courses taught in English by home institution faculty in our research/assessment activities?

There is not a clear set of categories that allow us to relate study abroad programs. Other research and work has been done in this area, most notably the work Lilli Engle and John Engle on study abroad levels and classification of program types, but nothing is currently being utilized in the field and research.

A question that comes to my mind when I read research studies on study abroad is:

- How do the factors that affect change relate across study abroad programs?

What are your thoughts on this?

Photo credit: Dano

Point/Counterpoint on the Value of Study Abroad

In a Sept. 21st commentary piece entitled “Study Abroad Is Often Not All It Should Be,” John F. Burness, visiting professor in the DeWitt Wallace Center for Media and Democracy at the Sanford School of Public Policy at Duke University, expresses his concerns about the value of study abroad. It’s a good read with valid points.

In a recent, Nov. 2nd, Letter to the Editor entitled “Education Abroad Is More Important Than Ever For U.S. Students,” Joshua S. McKeown, Director of International Education and Programs at State University of New York at Oswego, counters Burness’ claims and is also a good read with valid points.

You can access the two articles by clicking on the links below:

“Study Abroad Is Often Not All It Should Be” by John F. Burness

“Education Abroad Is More Important Than Ever For U.S. Students” by Joshua S. McKeown

Please take a moment to contribute to the discussion in the comments section of the two Chronicle articles or here on IHEC Blog.

You will need a subscription to The Chronicle to access the articles (although many college and university libraries (and some organizations) subscribe to The Chronicle so you may be able to access the articles via your campus or organizational network.

“Who Are You Now” ~ Study Abroad Alumni Survey by The Foundation for Asia Pacific

The Foundation for Asia Pacific Education is surveying study abroad alumni from the United States and Canada to learn more about the impact study abroad has had one’s life today. All study abroad alumni (meaning it doesn’t matter when you studied abroad) can take the survey so please alert any study abroad alumni you know to this research project and if you studied abroad please try to fit 15 minutes into your schedule to complete the survey. Additionally, all survey participants will receive a copy of the results via e-mail.

If you have difficulty accessing the survey via the button above you can also link to the survey here: http://asia-pacificfoundation.org/survey2009/index.htm

I really like the work that The Foundation for Asia Pacific Education is doing (in addition to conducting this survey) and I previously posted to IHEC Blog about them which you can read here.

Is Your International Education Data Ready for a Change in Institutional/Organizational Leadership?

During a meeting at work this morning I was reminded of how a change in institutional/organizational (and perhaps divisional) leadership can change the way you are requested to collect and report data about your international education programming efforts. For example, a few years ago the institution I work for, The University of Chicago, saw a change in University President from a Musicologist to a Mathematician and then a change in University Provost from an Historian to a Physicist. This change in leadership from backgrounds in the Humanities and non-quantitative Social Sciences to backgrounds in the very quantitative Physical Sciences no doubt brought about changes in how the University was to collect, analyze and report data. To be sure, the Musicologist and the Historian valued, wanted and needed the quantitative data to make informed decisions just as the Mathematician and the Physicist no doubt value, want and need qualitative data to make informed decisions. These disciplines, however, vary greatly in their methodological and data analysis approaches and it is these differences we should be aware of when collecting, analyzing and reporting our international education data.

My question is this: Are you and your data collection efforts ready for an institutional/organizational shift in leadership? Thoughts?

Growth of Research on U.S. Students Abroad

One of my main interests/activities in the field of international education has been compiling annotated bibliographies on a variety of topics and primarily on U.S. students studying abroad. All of my bibliographies are available on my consulting website here. Five years ago I decided to count and chart the number of research based articles, reports, books and presentations per decade found on the three main annotated bibliographies focusing on research on U.S. students abroad which you can access here. The following chart shows the steady increase in research based literature on U.S. students abroad from 1950 through 2003.

I’m slowly working on updates to my bibliography work in this area as well as creating new bibliographies including one on international education and public diplomacy/soft power. I’m also researching and searching for more international education literature from the late 1800’s up to the start of World War II. I’m finding some very interesting pieces from this time period and it is largely unknown to both colleagues and researchers in the field that I communicate with. Additionally, I believe that there are still many pieces of literature from the 1950’s and 1960’s (and possibly into the 1970’s) that are waiting to be discovered. I plan to eventually update the chart above as I discover new resources but it will be in several years. For now, I stand by my review and count as documented in the chart above!

On a related note…back on July 24, 2008 I posted to IHEC Blog on what I believe to be the first research study on outcomes of study abroad conducted by Roxana Holden and published in 1934. Holden analyzed statistics and statements from alumni participants of the first ten years of the Junior Year Abroad programs. You can read more of my post entitled “First Research Study on Outcomes of Study Abroad” here.

If anyone knows of any older research or avenues I should pursue in my quest to locate historical literature on international education please contact me...I want to hear from you!

Notes:
[1] Data for the chart was compiled from a review of the three main research bibliographies on study abroad (U.S. students). Weaver, Henry D. (Ed.). Research on U.S. Students Abroad: A Bibliography with Abstracts. (Council on International Educational Exchange; Education Abroad Program, University of California; Institute of International Education; and National Association of Foreign Student Affairs, 1989); Chao, Maureen. (Ed.) Research on U.S. Students Abroad, Volume II, A Bibliography with Abstracts 1989–2000. (NAFSA: Association of International Educators/SECUSSA (Section on U.S. Students Abroad) 2001); and, Comp, David. (Ed.). Research on U.S. Students Abroad, Volume III, 2001 to 2003 with Updates to the 1989 and Volume II Editions. (Loyola Marymount University, 2005).
[2] The 2000-2003 total includes research identified through May, 2003. A conservative estimate is that by the end of 2009 there may well be over 1,000 research-based articles, reports, books and presentations on U.S. study abroad.

Current Research Projects in Education Abroad Roundtable at Forum on Education Abroad Conference

Next week Wednesday to Friday (Feb. 18th to 20th) I’ll be attending the annual conference of the Forum on Education Abroad in Portland, Oregon. On Thursday, February 19th I’ll be assisting with the “Research Roundtable: Current Research Projects in Education Abroad” session starting at 3:45pm. I’m looking forward to learning what research is going on in the field as well as providing a venue for researchers in the field to network.

During the first round of conference sessions at 8:45am on Thursday, February 19th I plan to attend the “Exchange Participants as Citizen Diplomats: Best Practices” presentation by Jed Willard (The Public Diplomacy Collaborative at the Ash Institute for Democratic Governance and Innovation at the Harvard Kennedy School ~ more to come on this effort in a future post) and Danielle Scugoza (Penn Abroad Office at the University of Pennsylvania).

Looking forward to seeing many colleagues again in Portland.

First Research Study on Outcomes of Study Abroad

During my research efforts I came across what I believe to be the first research study on the outcomes of study abroad.

The article by Roxana Holden was published in The Modern Language Journal in 1934. Holden analyzes statistics and statements from alumni participants of the first ten years of the Junior Year Abroad Programs (which first started at the University of Delaware in 1923). To be sure, there is much literature on U.S. students studying/educated abroad that pre-date this 1934 article. Some of the literature from the 1920’s and 1930’s provides anecdotal evidence of the “value” of study abroad or provides information and quotes of opinion from U.S. based faculty and administrators on the outcomes of study abroad but the Holden article is the first I’ve found that discusses outcomes based on actual data analysis.

The article is available via JSTOR and I’ve provided a citation below:

Holden, Roxana. (1934, November). "Ten Years of Undergraduate Study Abroad". The Modern Language Journal, 19 (2), 117-122.

Resources from NAFSA conference sessions

During the NAFSA: Association of International Educators conference last week in D.C. I was a panelist at three separate sessions and chaired the special research seminar on global student mobility. The plan is for all of the power point slides and other handouts to be made avialable online. As the session materials or other information about the sessions become available I will post messages with information on where to access the documents. Following is what is currently available:

Assessment Toolbox for International Educators
Presenters: David Comp, University of Chicago
Darla Deardorff, Duke University
Elaine Meyer-Lee, Saint Mary's College
Lee Glover Sternberg, James Madison University
Victor Savicki, Western Oregon University
http://www.saintmarys.edu/~cwil/php/intercultural.learning/IILOutcomes.php

and

Data and Research on U.S. Multicultural Students in Study Abroad
Presenters: David Comp, University of Chicago
Gayle Woodruff, University of Minnesota
http://insidehighered.com/news/2008/05/29/nafsa

OECD International Initiative on Assessing Higher Education Learning Outcomes

In April 2007, the OECD began an initiative to develop an international assessment of higher education learning outcomes. You can access the OECD website that that has more information on the initiative including a 49 page .pdf working paper here. You can read an interesting article on this initiative from the September 19, 2007 insidehighered.com here.

Recently, ACE sent a letter to OECD outlining concerns with this international assessment initiative. You can access this letter as well as important links and documents on the ACE website here.