Debra Humphreys on Quality

Debra Humphreys is vice president for policy and public engagement at the Association of American Colleges and Universities (AAC&U). In addition to leading advocacy and policy efforts related to issues of student success and the quality of student learning in higher education, she works to help college and university leaders and faculty members communicate with the public about the value of an engaged liberal education to prepare for the changing global economy.

AAC&U’s annual survey of employers, which Humphreys oversees, is a widely cited source of data regarding the knowledge, skills, and abilities demanded of today’s college graduates in the workplace.

Humphreys speaks widely to educators, business leaders, and policy makers about the importance of liberal and higher education to the future of America’s economic health and democratic vitality. Serving as AAC&U’s official spokesperson, she has appeared on Fox News, NBC Nightly News, and PBS’s “To the Contrary” program and has had op-eds published in USA Today and The Chronicle of Higher Education. She also serves on the editorial advisory boards of University Business, Change Magazine, and About Campus.

Humphreys earned a bachelor’s degree from Williams College (Massachusetts) and a doctorate in English from Rutgers University (New Jersey).

Debra, could you start by telling us a little bit about the goals in your policy and public engagement role at AAC&U?

AAC&U is somewhat new to the policy space in higher education. We have always been focused very intently on the teaching and learning process, the curricular design of undergraduate education, and what liberal education really means for students in terms of their educational outcomes. And so our projects and resources have traditionally been focused on helping educators and educational leaders fulfill their educational goals.

In the last 10 or 15 years, our membership and our board have realized that the policy environment is making it more and more challenging to advance a vision of quality in undergraduate education—everything from new pressures on accrediting groups to new calls for accountability at various levels in states and the federal government. And as a result of that, it became clear that while AAC&U did not want to and had no plans to become primarily a policy influencer or a lobbying group or anything like that (we’re not that, and we don’t intend to become that), we do want to work with our members and with external stakeholders to help the public, policy makers, and others understand what quality learning in undergraduate education really means and why liberal education continues to be a valuable form of education.

We also want to inform the larger public environment to help people understand how liberal education is changing—how lots of colleges and universities all across the country remain committed to liberal education but continue to evolve its format, its practices, even some of the outcomes that it’s trying to achieve for students. So we are trying to intervene where necessary in policy debates, but more often than not we’re trying to create a more informed environment where the public and policy makers can better understand what our members are doing and what quality really needs to look like.

What should everyone better understand about quality learning?

I think the public doesn’t fully understand what kinds of educational practices really will set up students to be successful over the long term. And I think policy makers, also, don’t always understand the ways in which higher education institutions contribute to economic vitality.

Right now it’s not surprising, given the economic downturn and the current challenges in terms of the funding environment, that policy makers and the public are increasingly concerned about whether the investment in higher education is paying off—specifically in terms of economic success or professional success for students.

And so the first thing the policy makers and the public need to understand is the value of the kinds of outcomes that a good, engaged liberal education can provide to students. Higher education actually hasn’t done a very good job of painting a really accurate picture of what liberal education is and why it’s important to advancing students’ capacities to function in an innovation-driven, global knowledge economy. So that’s one of the things AAC&U has tried to do. Our research, events, and engagement with the employer community and our economic research are aimed at trying to understand what’s really happening in the economy in a very broad sense, so that our institutions can do a better job of making sure the curricular pathways we’ve created are really leading students to a set of outcomes that will set them up for future success.

That said, we also very firmly believe that while it’s not surprising that everyone is focused right now on economic outcomes and workforce development issues, it’s really, really important that we continue to recognize the enormous strength and unique contribution that American higher education makes to developing an informed, educated, responsible citizenry. That has traditionally been a function of higher education in America, and both public and private colleges consider that part of their missions. It’s very hard to find a college in the United States, at least in the public sector or the not-for-profit sector, that doesn’t say that part of their mission is to help students learn what they need to learn to contribute to building our democracy. That is a uniquely American feature of higher education, and it’s one that we’re concerned about because we think it’s needed more than ever. We absolutely need more college graduates who are informed citizens. And the more we [colleges and universities] lose sight of that as part of our mission, we think the nation is going to be worse off for it.

So the message we’re trying to get out is that we do not have to sacrifice preparing students for the workforce in order to help prepare them to be informed citizens. The truth is that the broad outcomes—the broad knowledge, skills, capacities, sensibilities—that a good liberal education provides to students that will help them succeed in the workplace are precisely the same kinds of capacities that they need to be informed citizens. So it’s not an either/or proposition. But it is very important to continue to remind the public that colleges and universities have traditionally had a dual role, and it’s an enormous source of strength in our system and in our society that higher education has played that role.

What are the components of a good liberal education? And you mentioned that it’s sort of changing?

First I should say, one thing the public doesn’t fully understand is that when we use the term “liberal education,” we are talking about a broad philosophy of education that has evolved quite a lot over the years. And it can be obtained through any major pathway. So when I use the term “liberal education,” I’m not only referring to traditional liberal arts disciplines, or liberal arts and sciences disciplines, or the humanities. And often the public, when they hear the term “liberal education,” that is what they refer to. Our sense is that a good liberal education should provide to students exposure and learning in the traditional liberal arts, but that’s not all it is. What really defines it is a set of broad, crosscutting outcomes that are essential for success in this kind of economy.

It’s interesting what’s changed and what hasn’t changed. Liberal education has always sought to provide to students some broad intellectual and practical skills: analytic reasoning skills, communication skills, ethical reasoning skills, the ability to understand the cultural and historical context in which they’re working or in which a society is functioning. All of that has always been part of a good liberal education. I think what has changed are the curricular pathways to those outcomes.

Certainly as technology has changed our society, that has placed some demands on us to change how we teach as well as what we teach. Any education like a liberal education that is committed to helping students understand the world in which they’re living and working has to continuously be examining what are the most important ways to do that and what are the most important areas of knowledge that students need. I think the biggest shift that we’re seeing is also related to technology in the knowledge economy.

If you look back far enough historically, the academy traditionally was sort of the holders of knowledge, and the role of education was transmitting that knowledge to students, and then also working with scholars to create new knowledge. Now, of course, we’re in a society where knowledge is so much more available, online and in other venues, that access to knowledge has been radically democratized—which actually changes our role. And it doesn’t make us less important, it actually makes us more important. But what we need to think about doing for students has become different.

So it’s no longer about knowledge delivery; it’s about building the capacity of students to understand how to use knowledge responsibly and appropriately to solve really complex problems. And it’s not that knowledge is no longer important, but we have to think about our curriculum quite differently. So it’s not just, “Oh, we need students to be exposed to these important areas of knowledge.” Instead, we have to organize our curricula to be sure that students are getting practice really understanding what knowledge they need to know and evaluating what knowledge is useful in a particular problem-solving situation.

The stakes in some ways are even higher for the capacity-building part of our job, and the idea of transmitting static knowledge from one body to another is less important. And that’s the biggest shift we’re seeing. I think the danger is for people to consider that shift and think, “Oh, we just don’t have to worry about knowledge at all. We just teach skills.” That’s not what I’m saying. It’s sort of hard to get people to understand that in a knowledge-intensive society and a society that’s innovating so rapidly, it’s not that knowledge isn’t important, it’s that students need to know how to use knowledge and know what expertise really is all about and why it is required.

What we hear repeatedly from the employers we talk to in our surveys, in our focus groups, in our events, is that students really need to understand that the day they graduate is not the last day that they will be learning new knowledge—that it’s actually the beginning of their continual new learning, and that the most important thing a liberal education or a quality undergraduate education needs to do is ensure that students are graduating with the ability to keep learning as rapidly as possible in new settings. That’s a pretty big shift for us. And it’s one that is essential for us to get right.

So your members are working towards doing that better, and then also communicating that they’re doing it. Which is the bigger challenge, do you think?

That’s a good question! I don’t know which is easier, but the more important thing, of course, is actually doing it better and ensuring students actually have what they need to succeed. But part of my job is to try to help our members meet the communications challenge, and it’s a pretty big one.

It isn’t actually that difficult to get our own institutions and everyone in them on the same page about what the educational experience is supposed to produce for students in terms of learning. But we have to get a whole lot more serious about assessing whether students actually have achieved the outcomes at the levels we need them to. Right now in our society when we think about assessment generally, we’re still in this weird mode where too often we think multiple-choice tests and standardized tests, because that’s how we’ve always done it. But in fact, we’re in an age where that testing technology is woefully inadequate to examine whether students have achieved the kinds of complex skills and abilities we need them to have. There’s a huge mismatch between our assessment approaches and our ability to demonstrate that students have what we really mean when we say they are liberally educated graduates. So that’s a much harder challenge.

But I think the communications challenge, while it’s actually not as challenging as the educational and assessment reform challenge, is one that we probably don’t spend enough time on. We probably don’t do as well as we need to. And one of AAC&U’s jobs right now is just to do the best we can to provide our members with the tools they need to do it better. For instance, we do our employer research to help us understand what the economy demands of students, but also to help us understand how we can better communicate whether we’re providing that to students.

What are the most important findings from your employer research?

We’ve been doing various kinds of public opinion surveys and focus groups from the beginning of our LEAP initiative (Liberal Education and America’s Promise), and we’ve learned a few things that have been very consistent in all of that research. And then in other cases, things have shifted.

The main finding that has been very consistent across five national surveys of employers—people who are senior-level executives at both non-profit and for-profit companies that hire a lot of college graduates—is that employers are not looking for narrowly trained college graduates. Narrow learning is not enough in today’s world, no matter what field one’s going into. One of the key findings from our last two surveys is that more than 90 percent of the employers say a candidate’s ability to think critically, communicate clearly, and solve complex problems is more important than their particular major or field of study.

And that’s something students don’t fully understand, that whatever they chose to major in, whatever their career aspirations, those broader crosscutting outcomes are what is going to set one up for success, not only in getting the first job, but in being promoted over time. We’ve asked that question a number of times, and employers overwhelmingly (it’s about two-thirds) say that what’s most important for long-term success in the economy is really a both/and vision: You need depth of understanding in a field—if you’re going into nursing, you need nursing knowledge; if you’re going into engineering, you need knowledge of engineering—and you need broad, cross-cutting outcomes that will allow you to move with the times with applied skills and knowledge across different areas. So that’s, I would say, the most important finding.

We asked some new questions in the most recent survey we did, about what outcomes are most important and also how we’re doing—in other words, how recent college graduates that employers are actually hiring are doing on the most important outcomes. We tested 17 different learning outcomes areas, and it was very consistent in terms of the ones that people ranked as most important: written and oral communication, teamwork skills, ethical decision making, critical thinking, and the ability to apply knowledge in real-world settings. More than eight in 10 employers rated those five as very important. But we also took those same outcomes and asked, “Okay, how are recent college graduates doing on them?” And we found that employers don’t rate them very highly.

On none of the 17 outcomes that were rated did a majority of employers say that recent college graduates are well prepared. And for some of the outcomes that they think are most important, the numbers are very low. About 30 percent said they were well prepared in the area of ethical judgment and decision making. The highest we got was 37 percent saying that students are well prepared in teamwork skills. That means a majority—a fairly healthy majority—of the employers are dissatisfied with students’ capacities in these areas.

We also (for the first time, actually) did a national survey of college students in the fall and released the findings in January. There’s some good news and some bad news. The students understand that a quality undergraduate education needs to prepare them with these broad, crosscutting outcomes. The top five outcomes students considered most important were the same as the top five for employers. So they get what they need to do. But they are vastly overconfident about their own abilities in these areas.

Now, that’s not particularly surprising. I mean, there’s no reason why students should have an acute understanding of whether they’re getting to where they need to go from an employer’s point of view. And of course, we interviewed them as they were in their senior year getting close to graduation. So it’s not surprising that they said, “Yeah, I’m doing well.” And they may have made enormous progress on some of those outcomes. I think what this finding reflects, however, is the way in which the competitive, innovation-driven, global economy is demanding so much more of college-educated workers that we still have a long way to go in really advancing all students’ abilities.

If I put this research next to other educational research and the knowledge I have about what our members are doing, I think we’re in an interesting historical moment in which our very best students—those students who are getting the very best education we know how to provide—are doing phenomenally well and are getting precisely what they need to succeed. The problem is, the economy is demanding higher numbers of those people than we are currently graduating, and that we have so many internal gaps in achievement that we have not bridged.

And so we are, in fact, graduating phenomenally well-prepared students. But we’re also graduating students who we have allowed to make their way through our curricula in ways that don’t challenge them, that don’t advance their communication skills, their analytic reasoning skills, their research skills, or the things that they really need to be successful. Those are things that often require a lot of work on their part, and a lot of effort and energy that they don’t always want to expend, or they don’t have time to expend. And they haven’t fully understood that if they don’t expend that time and if they don’t really challenge themselves in college, they really are hampering their long-term success prospects.

That is a communications challenge. We have not made that clear enough to students. And we have not done enough to organize educational pathways so that students cannot avoid the kinds of educational practices that will help prepare them. We know what we’re doing. We know, actually, how to get students to these skills; we just aren’t providing it to enough of them.

For those students who have experienced the best, does it have something to do with their college being more intentional about linking career preparation with liberal education?

That’s part of it, but I don’t even think that’s the most important part of it.

If you put the employer research we’ve done next to really robust educational research on what actually works for students, what you find is a lot of it is stuff we know how to do and we’re already doing, we’re just not doing it, as you mentioned, intentionally enough or for everyone. So for instance, students who are required to do more writing and research exceed other students who are not doing those things on skills that employers value. And so some of that is fairly simple to do—we just have to get students more engaged in research, writing, analytic reasoning, etc.

I think the thing we most need to do is make sure that no matter what major a student is going through, they’re having the opportunity to do real-world application and applied project-based learning. One of the things we kept hearing from employers in our focus groups was, “You are graduating students who have a lot of technical expertise and have some knowledge, but they have trouble if a problem doesn’t look exactly like it did in the textbook.” In other words, we’re providing them with some problem-solving abilities, but we’re not giving them enough practice solving problems that are messy, that don’t have one single answer, that require them to bring together skills and abilities across disciplines to solve a problem—often in groups of people who may disagree with them.

That was something else that came out on the top of one of the questions we asked in terms of what employers are looking for. They very strongly agree that all students should have practice solving problems with people who come from backgrounds that are different from their own and have views that are different from their own. And in that regard, I think we have a long way to go. One way to do that is to get students out doing internships more connected to the workforce, but there are lots of ways we can accomplish that goal; it isn’t only internships that would be the way to go.

In terms of liberal arts colleges, in particular, I think many of them actually are doing better on a lot of these outcomes that employers really care about. But they are not doing as well on giving students the opportunity to demonstrate those skills in real-world settings, so that they can really hit the ground running and successfully make that transition from college to career.

How do the findings from your research about educational outcomes line up with what the Gallup-Purdue Index has found regarding the contributors to well-being and workplace engagement?

I was really interested in what Gallup researched and what they found. They did their research in a quite different way than we did, but I think we both have demonstrated something that’s very much in sync and reinforcing. They measured the correlation between certain kinds of educational environments and experiences and professional well-being. And one of the things they’ve found is very much what we found when we talked to employers: Students who worked on a project that took a semester or more to complete were much more likely to experience well-being in their professions later on.

We’re working with our members to try to get every single student doing what we call signature work—the kind of work that they care about, that they spend a lot of time on, that is the kind of work that gets them engaged in real-world problem solving. And so we were very interested in that finding, because when we asked employers about that as well, very large percentages of them said, “Yes, we think all students should have that kind of experience.” So that was the most important finding for me from the Gallup survey.

Another Gallup finding also has relevance in the current policy environment and in the sort of frenzy to try to find more efficient ways of educating people, and that is the finding about how important it was for students to have, in effect, a faculty mentor—a faculty member who cared about them, took an interest in them, and helped them develop their own vocational goals and their own life goals. That is something that liberal arts colleges pride themselves on. If you go to a small liberal arts college, it would be pretty challenging for you to even get very far into the experience without having faculty members really working closely with you. So that’s one of the real advantages of a liberal arts college. And the shocking thing about the Gallup survey, on the one hand they found that graduates who had those experiences were much more likely to have well-being in their professional lives—but the numbers were very, very low in terms of the graduates who reported actually having those experiences.

That is an area where I wish policy makers would focus a lot more attention. Because a lot of the new models that they are advocating are ones that eliminate the faculty-student interaction completely and basically assume that many, many students can just be thrown out into cyberspace and will learn what they need to learn and will succeed in building the portfolio of skills that will set them up for success. And there isn’t a lot of evidence that that will happen for very many students without real faculty engagement and mentorship. Whether it’s online or in person, the role of the faculty mentor is incredibly important to student success academically. And we have to figure out a way to ensure that students have that.

Do you think a hybrid model will end up prevailing? Especially since technology is enabling competency-based learning that has a built-in assessment component.

In its most ideal form (which I will say, I don’t think actually exists!), the idea of a truly competency-based program has enormous promise. Ideally you’re very, very clear about what you’re trying to get students to know and be able to do, and you have really sophisticated assessments to ensure that students know they’re on track and are demonstrating the knowledge and abilities. However, I think the current environment is one where the inflated expectations for how that method of organizing undergraduate education will produce enormous cost savings is potentially a problem.

Because one way to produce the cost savings, of course, is to do the lowest level forms of competency development and assessment. It’s quick, right? You can go online and anything that’s easy to measure with a multiple-choice test, you can move people along. But the truth—from all of the research we’ve done, Gallup has done, others have done—is that if all a graduate can do is the kind of work that can be easily assessed with a multiple-choice test, that graduate is not going to be ready for the kinds of challenges that this economy and this workplace demand.

So I think there’s enormous promise for moving to new methods of helping students develop and demonstrate the achievement of competencies, but we’re nowhere near where we need to be in terms of our own assessment approaches and capacity to really bring that to scale for a lot of students.

The other worry I have is that a lot of those programs are appropriately designed for older, returning adult students who have actually already gotten some work experience, and many of them already have prior degrees of various sorts. Those students may succeed in that environment in a much better way than your traditional-age college student who needs a lot of support, a lot of mentoring, and a lot of interaction with peers in order to achieve what they really need to achieve. So I’m cautiously optimistic about competency-based education having some impact, but I still haven’t seen any hybrid environments or online environments that have really fully replaced the enormous power of a traditional liberal arts college environment with lots of interaction with peers and faculty. We haven’t really gotten to a point where the impact of that kind of environment can be replicated online.

So I think we’re moving into a whole new ecosphere of education, and it is going to be characterized by lots of hybridity and lots of approaches in all different kinds of institutions to providing students with some online interaction and some face-to-face interaction. And the combination of those is going to shift over time, so we’ll just have to see. I hope that we’re able to create an environment where we can really test what works for whom and really pay attention to who’s getting what, or I worry the kinds of findings we had on our employer survey are going to continue to get worse, and we’re going to just graduate a lot more people with credentials but without the skills they need.

That brings us back around to measuring the skills and providing evidence of the outcomes. How does this relate to accountability and what the U.S. Department of Education is trying to do with its college ratings system?

This is a very challenging moment we’re in. I think that it’s completely reasonable for various government entities, including the federal government, to say, “Wow, higher education is incredibly important, we know we need more students to get through, and we want to enable them to make really good choices and get to institutions that really will help them.” That set of goals is completely understandable and appropriate.

The problem is there’s a huge disconnect between the data we currently have and the data we need to try to create some kind of Consumer Reports-like apparatus to help people make good decisions. And I think that staff in the Department of Education understand the limitations of the data, but they’re still trying their best to try to create a workable system. Our own sense is that it’s not something that’s going to be very valuable to do, even if we understand the impulse to do it, because we just don’t think the data is up to the task.

And the big problem is that if you go down this path with inadequate data, the public doesn’t know enough to understand that we don’t really have adequate data to rate institutions of higher education on their quality. All the data points that the Obama Administration is talking about using for their rating system are about things that are unrelated to the quality of student learning, with the highly controversial exception of how much money people make after they graduate. And that, I think, is somewhat correlated to the quality of your education, but not really. It’s much more correlated to what’s happening in the economy and which fields are in demand and therefore pay more.

Plus it has huge problems in terms of what the mission is as a higher education institution. If you’re the kind of institution that has made it your mission to educate the teachers and social workers of our society, you may be doing a fantastic job of that and students are getting very well educated, but they are going out into fields that society has decided not to pay very well. So your institution is somehow, on this sort of system, going to be rated lower on the list than an institution that has been designed to educate engineers, which right now in our society are paid a lot of money. Obviously, if you only go by that kind of data, you really know nothing about those institutions in terms of what they’re actually teaching students and what value added they’re providing in terms of learning.

So there are a lot of problems with it. I do think that the U.S. Department of Education has been listening very hard to the community as we’ve tried to explain some of the challenges and potential misrepresentation and misinformation that such a system could provide to students and their parents—they really have been listening, and they’ve been trying to figure out how to do this better. Though I’m still not a supporter of the whole effort (because I actually think it’s going to be a whole lot of time and effort that isn’t going to result in what they need), I certainly see movement in a positive direction in potentially coming up with something that has more value to it. But we’ll see.

Ultimately, I think the proposed ratings system doesn’t really get at everything we’ve been talking about. And everything we’ve been talking about is pretty important. I just can’t see a way forward in the short term to use this kind of mechanism to address any of the things we’ve been talking about, because the assessments are just not in a position—and may never be in a position—to be summarized in simple data points. And that makes it very, very difficult to imagine these two pathways sort of converging at some point.

What does give you optimism about college value being recognized?

I like to remind people that education is not something we just deliver like a pizza. It’s a way more complex process, and when it works, it provides students with something that really transforms their confidence, their way of working in the world, and their whole way of thinking about what they want to do with their lives. And so, even though I feel pretty strongly that we have to respond in very practical ways to the concerns people have around the economy, we can also remind people of the larger mission. I think that while most parents care deeply about making sure their investment in their children’s education results in positive economic outcomes, they can easily see how important a more transformative vision of higher education is for their children as well. These are not either/or things. We can do both.