International Accountability Initiatives

Assessment of Higher Education Learning Outcomes (AHELO)

The Organization for Economic Development (OECD) established the Assessment for Higher Education Learning Outcomes (AHELO) in 2007 with the goal of creating a common international system that compares learning outcomes across countries, cultures and languages. In 2010, it was fully launched as a feasibility study. Previous efforts to involve the United States in this effort were rebuffed by the Bush administration, but this year the Obama administration signed on as a full supporter of the project.

Recognizing the difficulty of comparing learning outcomes across cultures and languages, the project is still in the experimental phase. It will begin by focusing on producing three separate measures of learning outcomes: one designed to measure general skills, and two designed to measure discipline-specific skills, economics and engineering. For the general skills test, OECD will fund the creation of an international-version of the Collegiate Learning Assessment (CLA), frequently touted for its value-added approach to assessing student learning.

According to this Inside Higher Ed article, details about the roughly $12.5 million project are still being developed, but Richard Yelland, director of the project, said the experiment would aim to test about 200 students at roughly 10 institutions of diverse types in each of six countries: Finland, Italy, Japan, Kuwait, Mexico and the United States, with participation in the U.S. limited to four states: Connecticut, Massachusetts, Missouri and Pennsylvania. Students will be tested "near the end" of their either three- or four-year (depending on the country) bachelor's degree programs, and groups of experts will, by the early next year, decide whether the measures provide reliably comparable measures across the various countries.

Finally, there will be an attempt to look at the factors specific to each institution involved in the AHELO assessment—such as student body composition, institutional mission, teaching and learning resources. Details of this kind of assessment mechanism are still coming together.

Some concerns expressed by academics:

It could be extremely difficult to design one assessment tool that produces any meaningful and comparable measure across the different cultures, languages, disciplines, and institutions within OECD countries.
Internationally, and even within countries, there is a notable lack of consensus on what should be the appropriate practices and outcomes of higher education. There is no widely accepted set of skills, competencies, and attributes that are expected of higher education students.
What constitutes quality teaching and research is best debated, established, and reassessed at the institutional level through effective academic senates or councils that have meaningful representation from staff and students.
From the standardized test-driven school improvement efforts pursued by many OECD governments over the past two decades, data suggests that this type of testing does not, in fact, lead to improvements in education. Instead teachers are encouraged to "teach to the test."
AHELO could easily be transformed into a simplistic ranking or league table of institutions.
The number of countries and institutions proposed for the feasibility study is rather low, making it difficult to assess the full complexity of the different educational systems across the OECD-countries.
The subjects chosen for the feasibility study—engineering, economics, and biotechnology—might not constitute the most challenging regarding international comparability.
There is no provision for stakeholder inclusion foreseen, more than the institutions included in the OECD IMHE program—which excludes important teacher and student organizations.

The Bologna Process

Named after the Italian city where this agreement originated, the 10-year old Bologna Process' vision is to harmonize higher education across Europe so that degrees are portable and transferable across borders, encouraging student mobility. In 1999, ministers of higher education from 29 European countries decided to create a European Higher Education Area (EHEA) by 2010 to promote the European system of higher education worldwide. It now has 46 participating countries.

Principal Objectives

Adoption of a system of clear and comparable European degrees, each with a Diploma Supplement to accompany transcripts and provide institutions and prospective employers with detailed information on foreign coursework. This system involves the adoption of a "qualifications framework," which are sets of learning outcomes and competencies that a student must demonstrate in order to receive a degree at a specific level.
Employ a tuning project that is an attempt to identifies specific disciplinary skills sets that would be expected of any graduate in a particular field. That is, what are the core competencies expected of a biology, philosophy, or mathematics major.
Adoption of a system based on three cycles: bachelor's (achieved in three years of full-time study), master's (achieved in two years of full-time study) and a doctoral program.
Establishment of the European Credit Transfer System, providing a common method for transferring credit for academic work completed in a foreign institution, to promote student mobility.
Promotion of mobility by overcoming obstacles to the effective exercise of free movement, such as onerous visa requirements.
Promotion of European cooperation in quality assurance.
Promotion of common European patterns in higher education.

Source: Bologna Declaration (1999); Dept. for Education and Skills, UK (2005).

Concerns

David Robinson, associate executive director of the Canadian Association of University Teachers, mentioned in his bulletin here that some governments are using Bologna compliance as a guise under which to enact unpopular reforms. Robinson stated:

In 2006 and 2007, the Greek government claimed a series of controversial measures, including a proposed amendment to the country's constitution eliminating the prohibition on private higher education institutions, were necessary in order to meet the obligations of Bologna. Students and staff responded with nationwide strikes and protests.

Further, most institutions have received no additional funding to support the huge bureaucratic burdens that Bologna has placed on faculty and staff as they endeavor to change degree structures, programs, and curricula.

Finally, some of the greatest concern lies with qualifications frameworks — statements of learning outcomes and competencies a student has to demonstrate in order to be awarded a degree — and their heavy focus on vocational education. As Robinson describes, "The disciplinary qualifications frameworks are developed through what is called a 'tuning process' where desired student learning outcomes and competencies are identified. In other words, you attempt to define the competencies you'd expect of a philosophy, biology, or engineering graduate. The problem, in my view, is that these qualifications frameworks have focused far too much on identifying competencies that above all else try to match students to specialized labor market needs." Developing critical thinking skills and other competencies outside of the aims of the specialization and necessary for professional and personal success are not safeguarded with this structure.

Implications for U.S. Higher Education

Europe's three-year bachelor's degrees.
The degree to which American graduate schools accept foreign bachelor's degrees that take three years to complete varies widely. Frequently schools turn a blind eye to this discrepancy with U.K. degrees, but students from India, for example — which also has a three-year bachelor's degree — are often encouraged to secure a master's before applying to graduate programs in the U.S. Now that we are in a time when American institutions of higher education are competing harder than ever for a large chunk of the international student market, many Europeans could look to Australia, where a three-year bachelor's degree is the standard.

Greater focus on subject-area competencies rather than credits earned/courses completed.
Cliff Adelman, a senior associate with the Institute for Higher Education Policy (IHEP) who is known for his expertise on the Bologna Process, has been vocal about what the Bologna Process means for higher education. His vision is outlined in "The Bologna Club: What U.S. Higher Education Can Learn From European Reconstruction," which explains that the European model for making higher education "harmonized" should lead to similar efforts in the United States, with states taking the place of countries, and pushing colleges for agreements on what a bachelor's degree truly represents in various fields and what skills students should possess– otherwise, U.S. higher education risks finding that the entire world defines some of the key features of higher education in different ways, and American higher education risks being passed by. 1

"Tuning" projects.
Borrowing from the Bologna Process, three U.S. states have introduced projects aimed at "tuning" academic programs in six fields of study. Funded by the Lumina Foundation, the project will involve research and surveys of faculty, students and employers to articulate student learning outcomes and competencies for degrees in education, history, chemistry (Indiana, Utah and Minnesota), physics (Utah), and graphic design (Minnesota).

The idea is that if degrees provide evidence of student learning and competencies rather than pointing to credits earned or courses completed, then they will have greater meaning for prospective employers and graduate schools. There is concern among faculty, however, that the "tuners" who will be focused on developing learning outcomes will be less interested in more subtle aspects of learning for which professors are responsible, thereby forcing professors to turn away from encouraging students' growth as critical thinkers and good writers.

References

Adelman, Clifford. "Learning Accountability from Bologna: A Higher Education Policy Primer." Institute for Higher Education Policy, Washington, D.C.: July 2008.

"A Unified Higher Education Area in 2010: What does it mean for Europe and U.S. Higher Education?" NEA Higher Education Research Center Update: Vol. 11, No. 2, November 2005.

Robinson, David. "Bologna Process Poses Dangers & an Opportunity." CAUT/ACPPU Bulletin, Vol. 57, No. 2, February 2010.

A Worldwide Test for Higher Education? – Inside Higher Ed, September 19, 2007.

Graduate Education, Post-Bologna – Inside Higher Ed., June 4, 2007.

Making Sense of 'Bologna Degrees '- Inside Higher Ed, November 6, 2006.

Measuring Student Learning, Globally – Inside Higher Ed, January 28, 2010

On Accountability, Consider Bologna – Inside Higher Ed, July 28, 2008.

'Tuning' College Degrees – Inside Higher Ed, April 8, 2009.

Wake-Up Call for American Higher Ed – Inside Higher Ed, May 21, 2008.