How to Define and Measure Higher Ed's Value

As college costs have soared, people are paying more and more attention to the value of college. Is what you get from college worth the cost? To answer the question, there's been increasing focus on the roi, or the return on investment for students and their families.

Yeah, that's right, Michael. But as the focus on economic returns from college has heated up in recent years, there's also this recognition from some circles that value is multifaceted. The answer on ROI is a lot more complex than dollars and cents. Indeed, how we think about the value of college is anything but straightforward. So to help us through this discussion today, we're welcoming Ed Smith Lewis from the United Negro College Fund and Karen Stout from Achieving the dre.

Support for this podcast is provided by the Bill and Melinda Gates foundation, which is proud to support the work of the Post Secondary Value Commission because the question what is college worth? Deserves answers. Learn more at postsecondaryvalue.org subscribe to Future youe wherever you get your podcasts and follow us on Twitter at the Handle FutureYou podcast. And if you enjoy the podcast, please leave us a five star rating so others can discover the conversations we're having about higher education.

I'm Jeff Salingo.

And I'm Michael Horn.

So, Michael, I know that when I travel around the country talking to students and parents and counselors, increasingly a top question on their mind is, is college worth it? Now, of course, this wasn't a question I don't think a lot of people were asking 20 or 30 years ago when we were going to college. But given the focus on job outcomes for graduates, rising costs, especially as family incomes are not keeping up the amount of student debt, which we talked about on a recent episode. And then there's the focus of think tanks and foundations on the conversation, and they're bringing a lot more data to this college value question than ever before.

Yeah, that's absolutely right, Jeff. And it seems to me an important conversation, and one in which, frankly, as a field, we've started to take some big steps forward. But the notion of value is also a complex concept. So we want to go beyond the usual tropes on today's episode. And to help us with that, we have two people who have thought a lot about this question and spend a lot of their careers focused on student success. Karen Stout is the CEO of Achieving the Dream, which works with hundreds of community colleges on student success strategies. She's also President Emerita of Montgomery County Community College in Pennsylvania. Also with us today is Ed Smith Lewis, who's the vice President for Strategic Partnerships and Institutional Programs at the United Negro College Fund. Welcome to Future youe. Ed and Karen, thank you.

It's great to be here.

Absolutely. Thank you for having us.

So the value of higher education is a term that's thrown around a lot these days. So we're curious to start with understanding how each of you define it. And so let's start with you, Ed. How do you define value when you're talking about higher education?

Welcome to the deep end. Value is one of the hardest questions I think we've struggled with in higher education. Perhaps from the beginning, but definitely today. When I define the value of higher education, I think much broader than what I think we typically land on. I think it's very clear that value in one sense is the economic value that an individual receives after he or she has attained that degree and moved into the workforce. But in many ways, that definition is a bit myopic. Not only does it not look at the type of employment that a student may decide to enter in terms of his or her career. Thinking about the role of teachers or socially oriented careers that have innate value for individuals because they chose that. Think about the arts, it doesn't really account for that. More importantly, I like to think about even beyond the individual. How do we think about the context in which that individual lives, getting to the institution itself, communities, and the systems in which we operate. As we think about institutions, there's value to higher education depending on the type of institution you attend, whether it's an affinity or mission oriented institution, and the diversification it adds to our broader educational context. When you think about the communities in which these institutions operate, many of them in rural communities operate as the sole workforce provider for institutions. And how do you start to think about the value of the institution in the place in which it sits? And then most importantly for me, the systemic value that institutions provide, thinking about how they change narratives for communities. As a proponent of historically black colleges, I know that their value, the value of those institutions, stem back all the way to America's original sin and today continue to push a counter narrative that not only are their institutions themselves valuable, but the education they provide to generations of students are also valuable. So in short, it's a very complex, I think answer.

Super helpful though, to give us the round out version. And Karen, what about you? How would you define value?

Well, obviously what is considered valuable really depends on the individual and the context that they're sitting in. And that's particularly important when we're talking about community colleges and marginalized populations. You just can't determine value in a vacuum. Our movement, though, has, I think, continually tried to do that and to come up with one single data point that can never tell the whole story of the diversity of the institutions in the higher education ecosystem. And I've been part of a lot of those attempts, not necessarily to get to one data point, but a few data points that never have been able to tell the whole story, like the voluntary framework for accountability. Each of these has moved forward the field's definition of value, but have been incomplete. The pdp, the Post Secondary Data Partnership, which is beginning to look at some leading indicators to help us understand valuable. But we really need to get to what is valuable for our students, their families and the communities. And that engagement really is an intangible value of community colleges. The conversation can't be about individual value. It needs to be partially about individual value, what students get, but it has to also be connected to what improves communities economically and socially. And Ed touched on that. And of course, value is perceived and baked into our systems in terms of funding, in terms of how we price tuition and the experience, and in endowments. Right? So this is really a complex topic and I'm eager to dig into it. Today.

Defining value is one thing, but probably measuring it is another. So, Ed, you've told us that to determine value, students should ask if they see themselves reflected in a college. And when you say that, what does that really tangibly mean? And how can prospective students, for example, do that? And in turn, how can colleges themselves better help students understand that that's important and then help show them how that happens? So could you give us a little bit more tangible examples about how students should see themselves in college?

I'll start by saying I'm a first generation high school graduate. And so when I headed off to Morehouse College, getting a career was top of mind. I have to say I was looking to change the financial and economic outcomes from not only myself, but Karen, to your point, but my family and the community in which I left. And I have to tell you, almost from the first time I stepped on campus, I realized how narrow my perspective was. By the time I graduated from Morehouse, I had visited 25 different countries. I never thought about value as getting exposure to the world. I had built a set of lifelong friends who had a variety of different experiences that were separate from mine. As a first generation black male, I had assumed most black males in college were first generation black males. But I had a roommate that was a fourth generation college graduate. By the end, when I went and I learned about the diversity of people that looked like me. It changed my perspective so much that I went from wanting to be a high school teacher when I left for college to working at an investment bank, ending up in consulting, and now doing a career I could have never fathomed. And so for me, the opportunities that colleges provide for many students go well beyond that sort of single point of economic value and career outcomes. And so when I think about how students should explore colleges, and I'll tell you, I went to a private day school and I was told, hey, guy, you might not want to go to that HBCU in Atlanta because administrative struggles might be hard. I took a leap. And doing that, and I took a leap for three reasons. The first reason was the fact that I had never been to the South. And so getting away from the communities in which I knew was a plus. The second reason was I had an alumni network that reached out to me fairly aggressively to say all the reasons why I should attend that college. And then number three, and probably most importantly, and I didn't realize the value of it until well after I had graduated from the institution, is going to a school where people looked like me and I had an affinity towards allowed me to remove the need to be representative of. And so when I think of my community, I got to be myself. And so when I start to think about how do students today evaluate colleges? You have to look beyond what careers and majors do they offer, but for whom do they offer it? When you go to their website, do you feel like you could be there? Does it feel representative of you? Do the faculty and staff look like you when you look at the outcomes for the students, Are they speaking to your aspirations? And for colleges, if students do that appropriately, colleges, all you have to do is invest in a sense of belonging for the students that are on your campus. And you do that by investing in faculty and staff that look like the diverse student population you're seeking to serve. You speak to the voice and the experiences of those communities, and then you illustrate that in your website, in your promotional material, et cetera, because it's about seeing a student for more than what you have to offer them, but accepting also what that student will offer you as an institution.

So, Karen, I want to turn to you on this because you both are pointing out the benefits of moving beyond just thinking about the value of higher education in terms of economics. And obviously, we've been getting better over recent years at measuring economic returns, and so we tend to lead with that. But obviously, as you both have pointed out, there's much more to this college equation, if you will. And so how can we shift that conversation, in your opinion, to something more than just economic benefits? As we think about value, how specifically do we start to capture and measure what else is important?

You know, I don't think it's bad or wrong to start with those economic benefits, but I think the question has to be larger. And I think ED started to lead us there and it's about who's benefiting and how wide is the impact of that benefit. And for us at atd, thinking about the role of community colleges as important local institutions, and I know HBCUs play a similar role in a number of communities, that means thinking about benefits, economic, social, cultural, in terms of equity and the community. And I know, Jeff, you've written about the need to rethink what student success means, and we've been thinking about that a lot at ATD as well. And we've been thinking about that in three phases. So when the student success movements first started, there was really this important shift from access and measuring value based on who's going to college to thinking about who's completing student completion. And of course now we're shifting again, and we have been shifting rightfully. Completion is now a progression metric. And we're thinking more about student economic gains. And some research that ATD did with Strata Education a few years ago and Gallup really shows some results around looking at value from the point of return on investment as measured by student and community well being. And that's really important. I think it begins to get to whether the college experience is building that sense of belonging and that social capital that ED has referred to in his response. And this is where many of our leading community colleges are now they're thinking about their work in the context of community well being or community vibrancy. How is they measuring their success? By the way they're contributing to the economic mobility of their students and the families that are associated with those students and therefore the strengthening of their communities. And that really requires us to think differently about the indicators we're using and to move beyond graduation rates to a lot of other indicators. And it really forces us to think about community vibrancy metrics and our role in as higher ed institutions and moving the needle in our communities equitably around early childhood education, reading and math proficiency on time, high school graduation. There's a lot of those intergenerational mobility that specifically for community colleges, because we have been woven into the fabrics of our community are really, really important for us to begin to measure value and talk about value in a way that has broad public policy kind of reach. I think too, we have to acknowledge that at the heart of all of this is this question of equity and how do we measure and bring equity measures throughout every one of those indicators that I just mentioned. Anthony Carnevale's quote about how American higher education is the capstone in an education system that's a primary cause of the reproduction of race and class privilege across generations is something that's really important. So however we're going to define value, we have to disassemble the gear wheel that we have become as a sector. And we need to think about how those value indicators are going to begin to help us break down the systemic barriers. And I think the way we have traditionally defined value has not been beneficial, let's just face it, for historically black colleges and universities and community colleges.

So I want to talk a little bit about the tools that we use to communicate value. Right. During the Obama administration, the federal government launched the College Scorecard that was very focused on the ROI of college. More recently, of course, the Gates foundation, which is a supporter of future you, launched its Equitable Value Explorer, in which students can evaluate colleges on factors such as racial composition of the student body, net price of attendance, completion rates, et cetera. I'm curious to see what you think of these tools. How should they be used, how they can be improved? Because institutions you both represent, as you've mentioned already, don't always come out on top when making these comparisons using these tools. Ed, let's start with you on that.

And this goes back to, I think, where we've been in the conversation already. The idea that one metric out of context could actually help a student make a decision is problematic. I think we saw with the College Scorecard that a pure aggregate average then compared across a set of institutions with different majors graduating in different geographies, with different social opportunities and career opportunities for certain people, based off of their generational context, doesn't actually lead to a fair comparison. And so for me, I struggle with both the College Scorecard Gates Value Explorer tool, the Value Commission's work. Partially because of the single point comparison, we have to move beyond that and allow for context to matter both for where institutions are located, what students aspire to be, and where they feel most comfortable. And so until we can get tools that contextualize at first just to geographies understanding that economic outcomes vary greatly in the American south versus some of our more metropolitan coastal cities and that comparing Caltech and an extreme focus on technical education, engineers, et cetera, to institutions that are producing more socially oriented graduates, teachers, etc. Isn't a fair comparison for a student to make. And as I think about the student too, the tools aren't very intuitive or engaging for a student population that I would say is learning to understand what they should value from higher education. In many cases, these tools feel more oriented towards policymakers and decision makers at sort of different stage in life than student. And I think that's where some of the problems I have lie, that the feeling that the emotions, that the experience a student gets isn't taken into account when we're sort of pushing tools to help students make decisions. So that's really where my challenge is. The context matters so much more for a student and, and many of these tools, although are believed to be developed for students, seem more policy oriented than not.

So, Karen, how about you? What do you think about these tools?

I think you all know that achieving the dream has really one of our central tenets is around helping our institutions with data informed decision making. So data tools are really fundamental and important to the work that we do with our colleges. And I do think there are some benefits to tools that offer comparisons for, for internal continuous improvement using tools to do benchmarking against like kind of institutions so you can aspirationally see what's possible with your redesign and transformation work internally. And I think that the tools are really attempting to put data in the hands of students and families and communities and that's important. That said, like Ed, I worry when we look to create these large comparative tools and where we're only relying on the comparative data that we have, which we've already talked about, how that may or may not be shaped in a way that's comprehensive enough to understand the local context and local missions that we have. And therefore the value is limited. The Equitable Value Explorer and the larger work of the Values Commission is really, really important. I do think that this continuous movement in the field to learn and build these tools is really important and we keep evolving. But even the Explorer comes with its own warning. And I'll read the warning. This tool leverages the best available public data from the College Scorecard and other sources, but those data are incomplete. So results should be interpreted with great caution. So it's not surprising to me that our institutions don't come out on the top when making these types of comparisons unless we really are serious about digging into two of the principles in the Values Commission report that really resonate with me, and that is that equity matters and that public returns matter. Because if I think we can really go deep into those and think about new data sets and new opportunities, we may be able to create a tool that's more valuable for the students that are coming to our institutions. We really need tools that are accessible and relevant and locally oriented. Our students don't even know where to begin and they aren't looking to compare 10 or 12 colleges across the nation or even the state to figure out what the best path for them is. So localizing these tools is really, really important.

That's a great point. So let's end on this for both of you. The agenda of college leaders are clearly going to be packed in the coming years, given all the pressures bearing down on higher ed right now. So as we think about value in particular, what's one area that you hope or think they should focus on that will move the needle on a lot of the stuff that we talked about today? What's your advice for the institutional leaders out there that might be listening to us today? And let's start with you, Ed.

I think it depends on your starting point. As I think about institutions, let's consider sort of the larger, more well resourced PWIs. I think a big question on sense of belonging to and how do you ensure you're preparing for a diverse student population Is the space where you need to invest a bit more energy and resources to really be prepared for the changing student demographic that's before us and start to think about how your outcomes might shift as that population or those populations start to grow on your campus. For institutions that have been historically mission oriented, whether those are community colleges, HBCUs, women's colleges, et cetera, really trying to understand how you elevate your narrative and your value will be critical. And so what does that mean? I think in addition to investing in much of the work that ATD leads in data infrastructure, because I'll agree that real data, hard data will not go away, numerical data, quantitative data will not go away. But also investing in that qualitative data like start to gather your students stories, their narratives, how your educational opportunities has shifted, their realities, changed their trajectories, start to work more closely with your local communities to understand what impact your institution is having on your localized areas. Again, to Karen's point, that in order to show value, you have to be able to document it. And for those institutions that may have value that is differential from the economic one, how do you start to Supplement that data collection to build that narrative differently. A lot of the work we do here at United Negro College Fund is around shifting how we see these data and these outcomes. And our recent report on social mobility understands the importance of economic outcomes for students, but then takes into account the input. It talks about the access rate that leads to those economic outcomes and how we're pivoting individuals from one social strata to another. And how do we start to think about the inputs of the institution to those outcomes and then the lived experiences of the individuals going through those outcomes or the communities in which these institutions sit. And I think much of that data will be qualitative in nature and require a much bigger lift, both in terms of the infrastructure needed, but also the narrative that must be built on that in order to get others to believe in it.

So, Karen, how about you? For institutional leaders out there, or even for political leaders and others, what can they really do in the next couple of years to move the needle?

You know, I'm really seeing from a movement perspective at achieving the dream. I just want to say I agree with Ed about the qualitative look at students, but that we're really beginning to look because of any number of factors, we're peeling back and looking at programs. And this really gets to one of the principles, too, in the Values Commission report that institutions and programs matter. I think we've waited too long in our movement to understand programs, who is coming into programs and who's benefiting from the programs and our program mix. And we're really challenged as community colleges at this moment around our relevancy. That's not an easy thing to say, but I think it's because we really haven't focused strongly enough on that program mix. And by program, I also am thinking about deconstructing the degree, Jeff, and what we mean by the degree. But we do know from some recent research and the Brookings Institute research that looking at program level data that earnings and student loan repayment outcomes at more than 1200 of the community colleges show that we've really misdiagnosed our problem and we have to redesign our solutions. And we're finding that programs that enroll more underrepresented minorities, black, Hispanic, Native American students, tend to have the worst track records on average. And what's surprising is that field of study, not student demographic characteristics, explains a lot of the variation in earnings across programs. And racially minoritized students are disproportionately enrolling in the programs with lower earnings. And this is unequivocally a design problem. For us as higher education leaders, we have to get to the design problem and the design problem carries into other programs, like dual enrollment programs, all types of programs, even the way we think about transfer and transfer programs. So for community colleges, I think moving our thinking to programs and understanding the student journey into and through programs is really important at this moment. If we have relevancy, value is easier to demonstrate. When you're less relevant, you struggle with demonstrating value. I think we as higher ed leaders need to think about that.

Well, Karen, Ed, thank you so much for joining us on Future youe. Really appreciate it.

Thanks Jeff.

Thank you.

And we'll be right back.

Support for this podcast is provided by the Bill and Melinda Gates foundation, which is proud to support the work of the Post Secondary Value Commission because the question what is college worth? Deserves a answers. Learn more at postsecondaryvalue.org subscribe to Future youe wherever you get your podcasts and follow us on Twitter at the Handle Future you podcast. And if you enjoy the podcast, please leave us a 5 star rating so others can discover the conversations we're having about higher education.

Welcome back to Future youe off of one of the more nuanced conversations I think I've heard around value anywhere, Jeff, and I'm curious to start with what was your reaction to their definitions of value and how they're thinking about the topic, given the rise of people you talk to all the time, ordinary people in schools, in high schools and so forth, talking about value of higher education?

Yeah, Michael, I want to zero in on something that Ed brought up, and this is on the type of employment. When we think about outcomes, the type of employment that students are getting after college graduation. There's the paper that Chris Howard of Robert Morris University, soon to be at Arizona State University, Singleman from Burning Glass, recently did, and I wrote the foreword to it and was really focused on job outcomes for underrepresented students. And what it really shows is that many underrepresented students go into higher education and take part in majors that don't necessarily lead to the best job outcomes. Now this focus on ROI as a key metric in outcomes really does make a lot of college leaders uncomfortable. And I think money is not the only outcome from college that matters, but it's really naive, I think, to downplay its importance. And in this paper, as Chris and Matt point out, black and Hispanic students are disproportionately concentrated in majors with lower earnings, and that contributes to the higher likelihood of being underemployed after Graduation. And what's more, there's depressed earnings and underemployment can cause really a vicious cycle after graduation. Right. So it's resulting in less engaged alumni for colleges and more challenging enrollment environment for those institutions as well. Now, I realize that this is not the point Ed was making. Indeed, he was actually arguing a little bit of the opposite, that we need to lift up certain professions like teaching and social work, which don't always fare well in looking at these averages. And he's right. But then this gets to his other point about the types of institutions. And you know, should someone who's looking to get a teaching degree, for example, go to an expensive private college where they need to take out tens of thousands of dollars in loans? And maybe that private college has a graduation rate of 60 or 70% when the local regional public institution has a similar degree? And I think that gets to the value equation in terms of those jobs that Ed was talking about. I agree that we need teachers and social workers and so forth, but should they be going to the institutions where they are going to go deep in debt, Maybe they won't have necessarily job outcomes of students from other colleges. Or even if they do though, they're going to be kind of in the hole coming out of those institutions. And I think that's the differentiation that I want to see when we talk about value. Now, I know we're going to be talking about the tools in a little bit, but I would love to see more engines like Amazon or Netflix, so that when students are looking at, for example, that expensive private college with not very good graduation outcomes, they also see something pop up saying, you know, have you considered these types of institutions for that same degree also within 50 miles of your home? So we're not talking about institutions several states away. And then there was Karen's point about individual value versus what improves the community. And I don't think it's an either or as I think she set it up. I agree that we need to think more about the social good and the social value of higher education, but we shouldn't do that at the expense of the individual good. Now, maybe the measures for community colleges are different, right? Because we know that the outcomes right now that we measure community colleges by are not great, right, in terms of graduation rates and so forth. And most students are going to community colleges because they either want to transfer to a four year institution or they want to get degrees that matter in the marketplace. So rather than judge these institutions largely on the same ones that we judge four year institutions on I would like to see some sort of measure about what are we trying to get out of this two year education. If we're trying to get a two year degree that has power in the marketplace, that is currency in the marketplace, well then that's their roi. Or if you want to transfer, that's your roi. But again, I think this goes probably more to the conversation about the tools than anything else. So Michael, what about you? I'm actually more curious to get your own definition on value because I know that this is a topic you spent a lot of time on dating back to you creating a quality value index for institutions over a decade ago and more recently co founding the Education Quality Outcome Standards Board.

Yeah, Jeff, well, first up front I'll just say everything you said I agree with. I think they're really well taken points and things that people ought to pay attention to. I was taken with both Karen and Ed's answers because I think they recognize fundamentally though, that value can only be defined relative to the progress that an individual, or I suppose a stakeholder in the case of thinking about the value to a community of a college is seeking from the experience. And I wrote about this recently for Forbes and talked about how all of our metrics around college outcomes are really what I would call supply side metrics. They look at things from the perspective of institutions and what's easier to measure. And in my view that's in opposition to what I'd call demand side metrics, the outcomes to what you were talking about desired by each individual learner. And I think demand side metrics are really hard, I think, to capture because from a student perspective you have to start with understanding first. Why is someone enrolling in a college and then build from there? Are they looking to step it up like Ed was and get a great job out of the experience? Are they looking just to get away from their current reality? Maybe they don't really know what the outcome that they're seeking is in some cases. And success I think looks really different in those instances and individual stories rather than averages actually matter a great deal. But I guess I come back to where you highlighted. I'd say none of these conversations though matter if something is unaffordable from the get go and it's going to throw someone into something that they simply can't afford. So I do think the roi, the return on investment conversation is a really critical part of this value conversation because as Karen said, I think it's an important picture that we're doing a better job of filling in as sort of table stakes for the bigger conversation. But we need to address it. And as we talked about in the episode Jeff with Beth Akers and Josh Mitchell, for the student that wants to go into the arts and is in a position to afford a big price tag for little economic return, that's fine. But if you can't afford that, then that's not fine. And I think rather than have colleges shirk that reality, they ought to be able to say, okay, how do we make the programs that do have less of an economic benefit far more affordable to those individuals? Because it's not that the arts are unimportant or that there aren't other parts of value that one would get out of them, but just that if they are unaffordable from the get go, then they're unattainable in the conversation to some degree is moot. Now, a problem, of course, is that the data today we're getting about these programs, they calculate an on average cost in the denominator. And as you know, students may pay dramatically different prices for college. So an ROI calculator or something that bases off the average as opposed to what you as an individual actually pay, I do think creates some awfully misleading results, if you will, for certain schools or certain students. Now that actually leads me, though, into my next question, which gets at these tools, which is I thought both Karen and Ed raised really good points about why there's a lot of data that shows that students, and low income students in particular, don't actually really use the data sets that are out there because they aren't really relevant to the colleges that they're comparing or that they're thinking about in their decision. And you alluded to it, Jeff. So I'm curious about your thoughts. You know, you talk to a lot of students, many of whom I'm sure you found, who've said to you these tools are utterly unusable or useless. So are we seeing changes in your mind to that now or what's it going to take?

Yeah, I mean, they're absolutely right on that. That these tools aren't very intuitive or engaging, that they're geared toward many cases, policymakers or others who are really well familiar with higher education, particularly institutions themselves. And I'm also really surprised, Michael, on how many people don't know about them. I probably bring up the college scorecard at every talk I give, and people always say, what was that? You know, could you repeat that again? Could you tell us where to find it? And these are at high schools with Resources, Right. With big college counseling offices. And they still don't know about these resources that you know is a federal government website. And so I can't even imagine at high schools with even fewer resources, what they're actually using or what they know. And as I said, when it comes to these tools, as I said earlier, I wish that there was more of a nudge of a recommendation as students are finding colleges on roi, as they're looking up earnings by major, by institution, as they're looking up different sizes of institutions, whether they're doing on the college scorecard or elsewhere. I wish that there would be a little bit of a nudge to say, did you consider these other institutions? Now I realize that institutions don't want that to happen. I mean, there was enough of a fight about the college scorecard that they're not going to want anything that drives students away from a college beyond the data. In some cases, the data alone will drive students away from a college. But I think that what we do need, though, is something that gets a little bit at this recommendation piece, because that's what students really need. Something needs to help them make sense of a universe with thousands of institutions. And that's really confusing to them. And even if they're going to college within 100 miles of home, as many students do, we're still talking perhaps dozens of colleges, some that might be good choices for them, but some that might be bad choices. But it also gets to something that you said and that Ed said and that Karen said, like, what do you want out of your higher education experience? Because right now all of these tools treat students as kind of a one size fits all entity. You know, it's interesting. I bank with Wells Fargo and I mention that they're not a sponsor or anything, but I was on their website recently and they have this thing called the Intuitive Investor Questionnaire. If I wanted them to invest my money, they first asked me, you know, what are my goals? What are my risk tolerance for different things? How often do I want to use this money? There's a lot of AI type of questions in there that get at my use of money, my risk tolerance and so forth. I would love a tool that did the same thing with students. Right. What are you looking for again? If you're going to go to a two year college, do you want a transfer or do you want an associate's degree that's going to have some sort of currency in the job market? These are the questions before students ever see anything ROI displayed to them. I wish that we would ask these questions first so that we know what to display to them, put them in context. And then there was something else that they said that really resonated with me, and this was about, we treat this kind of as a national database when most students do go to college within 100 miles of home. And so they're not comparing, for example, Caltech with mit. They're really comparing regional public colleges with maybe community colleges and maybe a private or two. And so it would be better, I think, if these tools operated more exclusively on the local or statewide level and then looked at ROI at that point as well, because many of these students might want to go to school within 100 miles and they might want to stay close to home to work afterwards. And getting a bachelor's degree in, say, my hometown in northeastern Pennsylvania and then staying there, you could have a pretty good ROI on that, given the low cost of living there and the fact that very few people there have bachelor's degrees. It actually looks better, for example, if you stay there than if you were to go somewhere else and go to New York City or Washington or Boston, where that ROI might not be as good as.

Yeah. And Jeff, you actually did a good job in your book of chronicling one student, as I recall, I think in Pennsylvania, if memory serves correctly, right, who all of a sudden got down to the limit and made this decision sort of bizarrely. Right. Brought in a new choice, if I remember the story correctly. You know. Two thoughts toward that. One. Briefly, I do wonder what the role of perhaps social capital could be in this. And what I mean by that is, you know, a lot of people's search choices are informed by the people around them. They know that went to a certain school or their counselor. I wonder if some of these recommendation engines might pair you up with someone who could maybe give a couple different ideas of something to put more of a human face on it. Because some of these choices are not deeply numbers driven, analytical. But to your point, they realize, gee, Michael might benefit from knowing there's these two other schools in the region. Let's connect him with an alum who went there or looked at it or whatever. So just a thought on that. And then the second point around comparisons and what you're really comparing a school or program to. It's one of the reasons when we built the EQUOS framework, the Education Quality Outcome Standards Board framework, we really wanted to do it from a value added perspective. So asking, like, what's the increase in earnings relative to what a student from a particular background and circumstance would likely earn if they didn't enroll in this program. For some students, that would help tease out what our friend Ryan Craig always likes to say, which is a lot of the kids who go to Yale and Harvard would be just fine. And the earnings they see are really about selection bias, not the quality of the school per se. But at the same time, you and I both know that for a low income student who does go to an Ivy League school, the experience is transformational for them and their family. From an earnings perspective. I just think factoring that value add perspective, you know, the counterfactual, and then comparing like schools to like schools, I think would be a real important contribution to evolving these tools.

Jeff all right, Michael, we're going to leave the conversation there and tackle more on this value equation on a future episode. But for now we have a listener question from Elizabeth C. On LinkedIn, and she asked this, which is really pertinent, I think, to this conversation around value. She says, when so much of the best of guides include things like expenditures on athletic centers and other accoutrements, how can we dig deeper into the value of what students learn? And she emphasizes that word learn. In other words, how can we rank the value of learning students receive? And we've talked about this for so long in the US about those institutions that are actually taking students from one place coming in, and by the time they graduate, they actually could showcase their learning. But what do you think of that?

So I have two totally different takes, Jeff, and then I'm curious, yours, the cynical take on me is in some degree it doesn't matter. Like there's a black box called college. You enter it, and if it produces great outcomes because of the social networks that you get into, the experiences you have, the mentors you have, the learning, maybe it was transformational, maybe it wasn't. But if it produces that ROI and that value, broadly speaking, that you wanted out of it, I'm not sure that it matters to some degree. So that's the cynical take on it. You know, the second side of it is this, which is it's really tricky to compare learning outcomes across programs for a whole host of reasons. The Equals framework that we designed does have an area for learning outcomes. And it basically says to schools that if you claim that your value is tied to learning outcomes, put forth an objective measure and do a pre and post right assessment, essentially, or portfolio of work or something like that, to verifiably and reliably showcase the learning that your students have, and that could be you know, if it's an entrance exam into a professional field like doctors, nurses, healthcare workers, the bar exam, et cetera, then there is an objective standard that you can look at, pass rates, for example. We ought to look at that and see, you know, how did students do. Alternatively, I think schools could do a better job of creating objective assessments themselves that they think are indicative of the learning outcomes that they produce. Is citizenship something that they claim would be an outcome of your program? Certain schools would say yes. Measure voting, for example. But I'd like to see schools, and preferably bands of schools with like programs come together and create some common measures so that we could have this conversation more robustly. And so it's not defined in a top down way from say the state or federal government, which I think could get dangerous really quickly.

Yeah, I think, Michael, I don't really have much to add to that. Great answer to Elizabeth's question, except that we really measure things. And this goes back to the whole roi. We measure things that we have easy access to the data for doesn't mean we should, but we do in higher ed because we have that data available to us. And the things that we should measure, we don't because we don't have access to that data. And this is a hard one to answer as a result of that, because we don't have easy access to to the data that we need in order to measure student learning. So we'll leave it at that. And thanks to Elizabeth for asking that question on LinkedIn. Please send us your questions and if we have one and if yours appears on the episode, you'll get this great future you Tervis Tumblr sent to you. So please keep those questions coming in. And until the next episode of Future youe, stay safe and we'll see you in a couple weeks.

Sa.