Why Imperfection in a Study Abroad Evaluation is Ideal

Abroad101 has many many reviews from students. The study abroad reviews are meant for study abroad students, their schools and future students to analyze the program they went on but also to help the student reflect and understand what they learned from their time abroad. Future students and parents find reviews very helpful when trying to decide which program to pick for an upcoming study abroad trip. With this in mind the quality and variety of the review comments and ratings becomes very important.

Our guest blogger, Missy Gluckmann wrote an article about this concept. It is reposted for you here.

Education Abroad Evaluations: Why Imperfection is Ideal

Truman students jumping high in the gold coast australiaThe culture of higher education is to want to measure.  We want to know not only how people have been changed by education abroad experiences, but how they FEEL about them too.  Assessment has grown exponentially in our field and we sometimes get stuck on the metrics over the human experience – at least that is my opinion. But this post is about evaluations, so let’s talk about them.

I recently had a conversation with Mark Shay, CEO of Abroad101.com, about evaluations and what they really mean in our field these days.  Let me back up though and share some history; Mark and I have worked in international education since the 1990s. While we did not work “directly” together on a specific project back then, I clearly remember the early days of his educational entrepreneur ventures at studyabroad.com.  Back in the day (wait, am I really old enough to say that – yup, apparently I am!), this was “the tool” for finding education abroad program information. Today, he is leading Abroad101.com which is described as follows:

“ With more than 20,000 reviews of 3,800 programs, Abroad101 is the leading student review site for study abroad programs. It provides a free service for universities and students to rate, review and rank student experiences in study abroad programs.”

It is a timely business idea, as education abroad offices are tasked with creating evaluations, measuring “success” and increasingly looking to student satisfaction in this game of extreme competition that has evolved in higher education and sadly, has trickled down to the education abroad world.  Ironically, small education offices often don’t have much time to think about evaluations, and those that do often don’t have much time to make meaning of the data or simply use it to pull “sounds bites” for marketing.

The “bigger players” tend to evaluate, sometimes not allowing students to receive their credits until a mandatory evaluation is completed.  I find this to be a challenge, as I believe that time to properly reflect on an experience abroad is more important than keeping to a timetable. Withholding grades, in my opinion, borders on unethical.

Mark and I had a chat about the direction of his company’s website now that he is “officially” on board as CEO.  We got into an important and sticky topic – why do so many programs come up so very high on ratings?  Why are there so many “perfect” ratings in education abroad reviews?

We know that education abroad experiences, like many things in life, cannot reach perfection or close to it. The lack of funding to train faculty to encourage participants to do what Dr. Anthony Ogden describes as “getting students off the veranda”, is evidenced by the countless “island” experiences that we witness on faculty led academic courses abroad. We see comments by students about “how transformed” they are, yet how does a perfect five star rating illustrate how a program’s design has positively impacted that young mind and the abroad experience? This is where the rating (“five out of five stars”) of a program can diverge from the actual review of the details of a program – and the “devil is in the details”!

What is “perfection” in education abroad?  Can we truly be “perfect” at anything in life?  Does a “perfect” score mean we are doing all that we can to create an optimal learning experience for young minds? Or does it mean we are not setting a high enough bar or that students don’t have high expectations – or even enough life experience to have realistic expectations across cultures? Does a lack of maturity prevent students from providing deep and thoughtful critical analysis?

More precarious is that there may be pressure from program providers and administrators to keep the scores ‘high’ – as some education abroad departments are embracing the competitive nature of higher education admissions’ philosophy, transferring the demand to perform to those going through re-entry by offering an incentive to them to complete a positive evaluation for a prize of some sort (bookstore bucks, access to academic records, etc).

Another challenge is that students haven’t been given much guidance in HOW to actually review a program so that information can be gleaned that will actually help improve the program design. I recentlywrote a guest blog post for abroad101.com about this subject.

During our chat, Mark and I quickly agreed that giving (or coaching your students to give) a “perfect” score on a program is not useful to those who are researching future participation in an experience abroad.  Giving someone a score of five on a site like abroad101.com is not Abroad101_logo_minus_com_2like jumping on a site like yelp.com and letting someone know that you thought a particular meal in a new restaurant was “spot on” and to your liking on a given day.  Education abroad is so much more complex than one random stop for brunch.  It requires that we are critical and thoughtful in our feedback and that we specifically disclose the imperfections to provide accountability and necessary education for others who are considering these programs.

Let’s consider an example: A student in Rome completed an evaluation and ranked it “five stars” (perfect score), yet she indicated that she would not return abroad with the same program.  When reading her evaluation further, she illustrated concerns about program administration, yet still rated the program with a perfect score.

Confused? So am I.

My guess is that someone in her study abroad office failed to take her to task on the incongruous feedback. It would have been so much more helpful is she had actually taken her comments on the program administration and carefully weighed them against the scale, providing a more realistic overall rating score, perhaps a three out of five.  This would then prompt future readers to consider the program weaknesses, which really are opportunities to address issues and improve processes and outcomes.  These types of scores are the ones that create REAL dialogue about program design and delivery of service to an academic sojourner.  They also open up the door for discussion about partnership between universities and third party providers (who I prefer to call third party partners – as that is what the relationship should be based on – partnership…but I digress).

I have seen a similar example for a third party provider in Tanzania. The student rates the program a five yet consistently ranks it lower in all subcategories, including a “one star” on the academic experience with this commentary:

The academics were overall pretty terrible, whenever the professors did actually show up they usually did not even teach material that was relevant.” Regarding housing, she says “The dorms are pretty terrible and my roommate didn’t even live here so I was by myself. The safety on campus is not great and you can’t walk alone after dark. It’s about a 15 minute walk to class and a 30 minute bus ride to nightlife. There is also no access to kitchens….and that was a big problem.”

It begs us to ask how THAT evaluation was overlooked by the home university and permitted to land on the site in such condition.

A score of five indicates to others that a program is stellar. Perfect. Wouldn’t change a thing. Except that in some cases, there IS much that needs to change.  Sadly, we miss the opportunity to get to the fine detail about what those components in need of work are when we see imperfection as failure.

Had the students rated the programs with more accuracy (e.g. “not perfect”), it would encourage not only administrators but prospective students to more carefully consider the actual “review.” A score in the “high fours” is much more revealing to someone seeking information on a program.  It offers a positive endorsement of the overall program yet provides specific details of what to expect, what could change to improve the experience and what to consider when making the important decision of who to go abroad with. Its value is priceless when compared to the ubiquitous “five”, as it actually provides insight into the little nuances that are so important (e.g. what to pack that was not mentioned in pre-departure materials/orientation) and the bigger ticket items (e.g. feedback on where budget improvements can be adjusted by a provider to allow students to more consistently engage in activities throughout a period abroad, vs. blowing all of their money during the first two weeks).

I realize that someone who rates a five may be someone who feels they had a positive, “life changing” overall experience abroad and wants to communicate that to the masses, but as Katy Rosenbaum from the Melibee team stated so eloquently, “I think it’s safe to say that a great time does not necessarily equal a great program.”  Frankly, we all know of less than immersive education abroad programs that are highly rated.

Perhaps this quote by Iain Thomas sums it up best when it comes to education abroad evals:

“But life isn’t something that should be edited. Life shouldn’t be cut. The only way you’ll ever discover what it truly means to be alive and human is by sharing the full experience of what it means to be human and each blemish and freckle that comes with it.”

Give me imperfection over a perfect score any day.  It is those blemishes and freckles that will inform and make the evolving world of free evaluation services in education abroad a truly meaningful tool for all of us.

 

Missyheadshot2.jpegAbout the Author: Missy Gluckmann is the founder of Melibee Global. You can learn more about why she built Melibee Global and her background here.  

 

 

 

Top 5 Reasons to Study Abroad with CAPA International Education


This guest post is written by Jessica, a college student, blogger, and recent CAPA International Education Alumnus.  Jessica spent six weeks studying International Marketing in London, England with CAPA during the summer session, and it was one of the most fulfilling experiences of her life.  Read on to see her Top 5 reasons to study abroad with Capa International Education.

Continue reading

Abroad101 Advisors Newsletter – Fall 2015

Abroad101-reviews-logo-web

We know it’s a hectic time of year, so we thank you for taking a few minutes to think about broader issues like using Abroad101 to advance study abroad on your campus.  As we begin our 9th year of serving American college study abroad, we continue to enhance the features, improve the directory content and bring innovation to the field.  Here’s how:

Abroad101 is Free Software

The Abroad101 software is online, “cloud based”, so you never to worry about updates or backing up your data.  Abroad101 is program evaluation software that gives your students the means to tell their story and share it with the world.  Unlike websites that grab quick sound byte testimonials, Abroad101’s “re-entry in a box” is designed to draw an insightful student expose of their time abroad.  When students take the time and effort to produce good work it becomes a point of pride AND social media.  As its own webpage you have now given the student a lasting testimonial of their time away and the resulting transformation.  These evaluations are great for future advising and orientations and great for the returning students.  The service is offered free to American colleges and universities, supported by advertising from program providers and foreign universities eager to reach future students.

Got Graphs? – Study Abroad Reporting

Abroad101 provides each university pie chart graphs showing how their students compare to Abroad101 averages.  The graphs show overall satisfaction with the education abroad experience plus ratings on Academics, Program Administration, Housing, Food, Social Integration, Health Care, Safety, and Cost of Living.  Take a screen shot, copy and paste these into presentation slides and impress your Provost!

Re-entry in 37 Questions (or more)

The standard evaluation/review is 37 questions, some Yes/No answers, some ratings and some open-ended essay prompts. The questions are designed and sequenced to guide a student through the entirety of their experience away, helping them be reflective, offer constructive criticism, list accomplishments and offer forward thinking statements.  The standard set allows for an apples-to-apples comparison both on the front end (website for future students, parents) and back-end (data for downloading and analyzing).  If you need more data points, Abroad101 allows additional “University Added Questions.”  These custom questions are not displayed to the public and can help you justify the Abroad101 platform as your official program evaluation tool and help to consolidate your student surveying. 

Reviews of Exchange Programs

Abroad101’s evaluation tool works great with third-party providers, faculty-led and exchange programs.  Each foreign university has a catch-all “Direct Enrollment & Exchange” listing where students from all over the country will submit reviews of direct enrollment, consortia exchanges or bi-lateral exchange programs, all through this one listing.  It’s a great place for your students to see the benefits of these programs and for students to share those with the world when they return.

The Abroad101 Pre-approved/Recommended Programs Feature

For universities who use a pre-approved programs list or recommend programs to their students, Abroad101 offers an easy way to show those choices to students searching on the site.  When a student logs in to Abroad101, they will see your recommendations highlighted with an approved icon.  They also have the option to view only the pre-approved programs in any directory or only view programs that have been reviewed by students from their home university.  The pre-approved programs are easy to manage using your account dashboard.  If you’d prefer we maintain them for you, simply email support@abroad101.com and identify the source of your approved programs list (web links, spreadsheet or other document) and our team will take care of this for you.

Summer and Faculty-led Programs

Abroad101 has over 4,500 faculty-led/university-run programs listed. Some universities use Abroad101 to manage those program evaluations/reviews and to draw future students.  We welcome the submission of programs that are either open to outside students or restricted to your own.   Managing these requires a free “providers” account on Abroad101, which you can request from support@abroad101.com.   You’ll still get the benefit of the apples-to-apples comparison of the resulting reviews, plus be able to engage students from other institutions. 

Abroad101 is a Free Directory, now with “Search by Subject”

Released earlier in the year, the Abroad101 Search by Subject directories are filling out nicely as the program providers update their listings and add their course areas.  This is a great tool for advisors as students who are looking for courses outside their major can come to Abroad101 and search from the more than 350 subject areas to find a program just for them: semester, summer or even shorter term: http://www.studyabroad101.com/subjects

Free Software from AbroadOffice

If you are looking for a cloud-based study abroad enrollment management system, we would invite you to consider AbroadOffice, free software from our sister company.  You use Abroad101 for the program evaluations, and AbroadOffice for the rest!  Learn more about AbroadOffice and schedule a web demo by visiting The Global Learning Cloud

Don’t Worry, We’re Here to help

Our company slogan remains: Fast. Free. Easy. We like to think that the system is self-explanatory and our processes are intuitive, but just in case, we’re here to help.  If you’d like a demo of the system or assistance in setting it up, don’t hesitate to contact us.  Support is Free too.

For those that don’t connect with us online, we look forward to meeting you at these upcoming conferences later in the semester: IIE Generation Study Abroad Summit, NAFSA Region XI, VIII and V.

 

Happy Fall!

Mark Shay

CEO

Abroad101

+1-610-357-4648

mark@abroad101.com

http://www.StudyAbroad101.com

 

Cool Program of the Week – Jazz in Paris

France-Paris-UCsandiego

Cool Program of the Week loves all the stories coming in from summer’s abroad.  Summer is such a good time to study and do cool things – like Jazz in Paris:

http://www.studyabroad101.com/programs/uc-san-diego-global-seminars-paris-jazz-in-paris

What is this program about?

We spend the summer following in their footsteps to understand how Jazz developed and why it was so enthusiastically received by the Parisians in the 1920s. In addition to historical lectures, films, and live music in class, we will capture the flavor of the period with excursions to the nightclubs, restaurants, concert venues, and cafes owned and frequented by these African American pioneers abroad.

The ability to speak French or read music is not necessary—a passion to learn about American jazz in Paris is required!

Student Review

“I gained confidence in following my dreams and turning them into reality. The experience was the best world exposure I ever had.”  Read the full review

 

Five Reasons to Consider a Long-term Study Abroad Program

Review_30573_Photo__f9d7So you want to see the world, and you’re enticed by all the short-term programs out there. A few weeks in Sweden, a stop in Germany, and a long weekend in Belgium. Sounds like a good deal: you get to see lots of places in a short amount of time, right? Well, it’s a good start, but I’d like to propose a deeper adventure. I wouldn’t want you to miss out on some life-changing opportunities. As someone who spent a year studying at a university in Toulouse, France, I can tell you from firsthand experience what you will gain if you decide to go the distance and chose an immersion program:

Make Lasting Friendships

Relationships take time to cultivate. You may begin to make friends on a summer program, but if you stay for a year, you’ll have a chance to make lasting friendships not only with other study abroad students, but also students from the host country. Doing class projects with French students and living in a student residence gave me the opportunity to meet American, international and French students with whom I studied, cooked meals and even went to Barcelona and Dublin. Who knows, you could even develop a romantic connection that inspires you to stay for even longer!

Cultivate a Deeper Understanding of the Language

What if you accidentally told your host family that you were pregnant when you meant to say you were full? Well, spending longer in your destination will sharpen your language skills to the point where you won’t be making silly mistakes like that anymore. Going to class everyday, speaking to classmates, neighbors, and your local grocery clerk will put you on the fast track to fluency. Your improved language ability will improve your grades, communication skills, and make you more qualified for jobs that list bilingual as an asset or requirement.

Really Get to Know the Culture

How many kisses do you give when greeting someone in Normandy? (4!) When do you use informal and formal parts of speech? Which words are OK to shorten? Which fingers do you hold up when you mean 2? While some of this can be learned beforehand, most cultural competency is learned by repetitive practice. If you’re only in a place for a few weeks, how will you get to know its people and customs? Long-term programs give participants the opportunity to become aware of stereotypes and biases they may have subconsciously been subscribing to before living in the country. You will also learn about beliefs held by host country natives about Americans! By developing a deeper understanding of the host culture, you will be able to communicate better with people from all countries because you will learn how to put your “American” beliefs aside in order to really understand where someone else is coming from. This skill comes in very handy for networking, international business, and simply communicating with people different from ourselves.

Make Progress Towards your Degree

I was a French major, so for me it was easy to find classes that counted towards my major in France. However, you can make progress towards your major, minor or general requirements while studying abroad. If you plan carefully ahead of time by making sure the host university offers the kinds of classes you want (biology in Australia, literature in England, computer science in Chile, fashion in Rome, etc.) you can stay on track for graduation or even get ahead! Talk to your department chair and study abroad advisor for more information. In addition to meeting graduation requirements, study abroad (especially long-term programs) looks impressive on a resume or grad school application and can help you stand out in a crowd.

Learn about Yourself

Aside from the practical reasons listed above, living in another country enriches your life personally as well. How do I set up electricity in my apartment? Open a bank account? Plan a trip to Argentina? Where do I go if I get sick? You learn your limits, become more independent and open-minded, develop resiliency and the ability to cope with difficult situations. Instead of someone planning everything for you and telling you where to go and what to do, you are in charge. Because of this, the victory of solving a problem such as finding an apartment or even finding your way home is that much more satisfying. Once you’ve accomplished the challenge of really living abroad, you have some bigger questions to answer: Do I want to come back and live here? Will I inspire others to take on the challenge? Where will I go next?

 

My study abroad program affected me so profoundly that I ended up working at my university’s study abroad office for three years. I then lived in Paris during my graduate studies and again while writing a book about my study abroad experience: Pas Possible: Falling in and out of Love with France, available on Amazon. I invite you to read the book if you’re curious about what it’s like do an immersion program, live in France, or both. Still not sure? Go out there and see for yourself!

Read Jessica’s study abroad review on Abroad101.com

 

Guest article by:

Jessica Pasa

French Teacher

M.A. French Studies NYU

Author of Pas Possible: Falling in and out of Love with France