Podcast: Jay Rosner: Bias in Testing and College Admissions

In today’s episode, I sit down with Jay Rosner to explore the realm of standardized testing, discuss fairness in education, and dissect the recent legal battles over affirmative action in college admissions.

TRANSCRIPT

Jay Rosner: 

It just so happens that the SAT on the math side particularly has a disparity consistent disparity over decades against girls. What happened for a long time from the 1990s through 2016, when the SAT changed was that girls consistently, on average, performed a third of a standard deviation lower than boys.

Sheila Akbar: 

Hey, everybody, welcome back to the podcast. It is mid November, somehow I know where the time went. But the rest of those college deadlines I know are bearing down on us. So if you need help, please don’t hesitate. Reach out. I have lots of free resources. I’m always happy to talk to anybody. And let’s get you the help that you need. But today, we are not here to talk about essays or college applications so much as we are going to talk about some equity issues in testing and hear a little bit more about what was happening in the process of the supreme court hearings around affirmative action. My guest today is Jay Rosner, and I’m going to do something a little different today. I’m actually gonna read their bios so you know who I’m talking to. Before we start the conversation. Jay Rozner is an admissions testing and test prep expert based in the Bay Area. He’s the Executive Director of the Princeton Review Foundation, a small nonprofit that provides heavily subsidized high quality test prep programs for organizations serving low income underrepresented minority students. His multifaceted work focuses upon fairness and admissions tests. And he speaks and writes about testing as an activist, advocate, expert, witness consultant, researcher, organizer, and lapsed lawyer. You’ll hear this in our conversation, Jay is a bit of an agitator. And I love the work that he does and the causes that he takes up. And he’s a really great voice to follow around these conversations. So take a listen. Jay, thanks so much for joining me today. My pleasure. Let’s start with talking a little bit about what you do, and your career journey, how you got there.

Jay Rosner: 

So let me start with my journey. I went to law school practice law for a while and got very interested in standardized testing, mainly through hearing Ralph Nader talk about it in his consumer advocacy activities did an investigation of ETs. And one of the factors that he talked about was misrepresentation on test preparation on coaching that ETs and the College Board, but it was mainly ETS back then made it clear in its publications that coaching and test prep simply didn’t work and was ineffective. And Ralph Nader’s book made the point that that was simply not true. Not only was it not true with regard to the SAT, which was the focus of the book, but it was illogical and anti educational, the concept that you can’t prepare for a testing experience, makes no sense. And our system is based on teachers teaching, giving tests that students study for so they’ll do better on. And so hearing Ralph Nader talk about the book got me interested in testing, and I actually shifted my whole career to test prep, and worked at the Princeton Review and on the for profit side for several years. And then starting a long time ago in 1995, ran the what was a nonprofit affiliate of the Princeton Review called the Princeton Review Foundation. And my charge was to try to get test prep information and resources to low income and underrepresented students and student groups, which I’ve been doing since 1995. So tell us what that looks like. Well, my mission is national. So I’m doing workshops, training sessions, talking to educators, talking to policymakers, talking to administrators, and talking to students. I mean, a typical thing that I would do would be to set up a test prep program for a nonprofit serving African American kids or Hispanic kids or Native American kids. And I’d come in and do a workshop for the folks at the nonprofit to orient them toward how they could support the test prep. And then often I would do an introductory presentation to the students. So I was negotiating these programs, establishing them, administrating them, supporting them, monitoring them. And I had test prep teachers teaching them.

Sheila Akbar: 

And one of our first interactions was on a panel about things that were unfair in the SAT. And I know you do give a lot of talks on which questions benefit, which populations and what kind of bias is built into these tests. So tell us about that kind of arc of your also of your career, because in addition to doing this sort of nonprofit work, I know you’re a very outspoken critic of tests.

Jay Rosner: 

Yes, I am a critic. I’m an advocate. And I’m also an activist. And you and I first met because I got interested in something you wrote about the disadvantages that girls face on the SAT. So let me start with that, because that hasn’t been a main focus of my work. But it’s been a corollary to my work. And I’ve paid attention to it over the years. It just so happens that the SAT on the math side particularly has a disparity consistent disparity over decades against girls. What happened for a long time, from the 1990s, through 2016, when the SAT changed was that girls consistently, on average, performed a third of a standard deviation lower than boys. And so we’re talking about a 30 to 35 point disparity and deficit for girls on the SAT, and interesting things have happened with regard to that one interesting tidbit as MIT I believe in was in the 1990s, although I may be a little off, specifically said that they were going to add 50 points to female applicants math, SATs scores, because they specifically wanted to diversify their student population and have better representation from females. And that helped, and they attain the kind of diversity that they wanted. And so they were able to stop doing that. Because once females realized, hey, MIT is an actual possibility for me, and they have 35 or 40% women, then they get more applicants. And so it’s it’s a virtuous cycle. And so they were able to stop that. But they were very clear. They said that at conferences. And it was interesting, because I heard them say that at conferences, they were met with Oh, and no one else said, Maybe we should do that. It was they were out there alone, doing it for that specific purpose. Referencing to a the disparity, and B, they didn’t notice any difference in student performance from females, even though they got 50 more points on the sad man. Yeah. And I know you bring these critiques up to the College Board. Have you ever gotten a response on why that disparity has been there so consistently? Well, there’s actually there’s been a change in the disparity. Remember, I said it was 30 35 points to 2016. It was about around 2016 2017 8 2018, that I noticed that disparity diminishing. And in fact, that was over a pretty short period of time, it got down to 20 points, 22 points, I mean, after consistently 30 to 35 points for decades, and a few years down to 20 something points and my theory and I asked the College Board, and they didn’t have any response that made any sense to me. My theory is that there’s a very high percentage of test takers, since then, and even more now, who do school day testing. And if you think about school day testing, you have a school where let’s say 60 or 70% of the students are college bound, and everyone has to take the test. So you have 30% of the students who aren’t interested in going to college, having to sit and fill in bubbles on an SAT for three hours. What do you think happens? And who do you in that 30%? Who might be taking it more seriously. And I’m thinking that if you take that 30% cohort, and this might be unfair stereotyping, but I think the best hypothesis that I’ve come up with that in that 30%, the 15, or 16% of girls are probably taking it a little more seriously on average, than the 15 or 16% of boys. And once you have the test taking population, I think it’s I may be mistaken, I think it may be over 50%. Now taking school day SATs, do you know whether that’s a factor? I don’t know off the top my head, it is a large number, though. It’s certainly it’s certainly 40%. So when you have a significant percentage of a significant percentage, blowing off the test, essentially, I think that can create enough difference. So that’s my hypothesis. You know, I haven’t heard anything better.

Sheila Akbar: 

Where we’re going down a rabbit hole here, but I think we should chase the rabbit a little further. Do you have a sense of those sorts of statistics when it comes to the ACT? They’re generally parallel. I glanced at the ACT every once in a while. And you know, it was the 3035 point disparity on math for females that I knew. And ACT had something comparable, I don’t remember the number. Interesting. And take you back to the MIT thing. They didn’t see a difference in performance of girls versus boys in college. So this is like clearly some kind of bias built into the test. It’s not that girls are not as good at math. There’s boys. Right?

Jay Rosner: 

And it’s only one Sheila. It’s only one of many indicators of the bias built in the past, but it is definitely one. Yes.

Sheila Akbar: 

Yeah. Yeah. And it’s very interesting to think about. All right, so Princeton Review Foundation, critic of testing. You’ve been doing some stuff around. I want to sell the LSAT these days. Tell us about that. Yes, I’ve been focused on the LSAT since August of last year, what triggered my focus is a standard 503 of the American Bar Association. Now the American Bar Association is the accreditor of law schools. And standard 503 mandates law schools to require an admissions test. So there’s no test optional law school there a creditor mandates them to require an admissions test. There was a proposal before the ABA to do away with that standard and give the schools the flexibility to be test optional, or test free and admissions. And what was a it was so appalling to me, that a group of several dozen Dean’s and I’m thinking 60 Law School Dean’s out of the 200 ABA accredited law schools submitted a comment to retain the mandate. And this was remember, we’re in the summer of 22. The entire undergraduate educational experience is test optional. And here you have the law schools, not only in a very different position, but in some ways in the opposite position with a substantial cohort of Dean’s defending the opposite position. I was stunned. And I had been working on the MCAT on issues of fairness and flexibility, all to decrease the this disparate impact of the MCAT that had been my focus by switch my focus last summer to the LSAT. So I think we all benefit from this sort of curious indignation that you get overcome by I’m really glad for the work that you do. I know also, you were following very closely, the suits against Harvard and UNC that went to the Supreme Court recently. Can you tell us how you were involved? And I know you were actually in the courtroom and heard a lot of the testimonies. I’m really interested in hearing hearing you reflect on that?

Jay Rosner: 

Yes. So it was either 2014 or 2016. I mean, it was years ago when the case was filed by SFFA. Miss named the students for fair admissions against Harvard, attacking race, conscious admissions attacking affirmative action. And in past cases, for example, in the Michigan affirmative action case, the affirmative action case against the University of Michigan minority students were permitted to intervene, because the dynamic in the courtroom is you have? Well, in all the cases, including Michigan, you had a white student. And in the Harvard case, it was an organization comprised of Asian American students. You had those people in that entity, asserting that race conscious admissions should be abolished. And the Supreme Court has decided that not abolishing race conscious admissions but constricting it in a very significant way, that the entities who suffer are the underrepresented minority students whose numbers would drop, and they’re not in the courtroom. It’s the plaintiffs whom I described the white or Asian plaintiffs, and the university, as the defendant, and the quote victims close, quote, have an adverse decision aren’t even there to talk. So most courts have permitted intervention by minority students because they would suffer the consequence, and they’re not there. What happened. So I was involved at the outset of the formation of the minority intervention. And the reason one of the reasons I was I was so involved at that early stage, which was within a couple of months of the case being filed, was I testified as an expert witness for the minority intervenors in the Michigan Law School case, which was the Grutter case, which was the landmark decision by Justice O’Connor supporting affirmative action. And so my involvement with a prior intervention, and what I saw it to be the effectiveness of that intervention, led me to get involved in the intervention in the Harvard case. So I was talking to the attorneys and talking to the student intervenors, all along the way attended nearly every day of the trial, attended the appellate argument, didn’t quite make it down to Washington to hear the Supreme Court argument, but was as involved as I could be sort of in a consultant role. Tell us about what you heard. Wow. Well, it was kind of surreal, to be in the courtroom, where Asian students would show up. But the real plaintiff was Edward Blum, who financed the whole thing. And he was in the courtroom every day. And then there were substantial cohorts of Asian American students defending affirmative action, who were also in the courtroom. So it was a wonderful coalition of the broad range of minority students in favor of affirmative action. I think the most effective day of the trial, and I clearly biased in this opinion, was the day that eight students testified, four of our students who were with minority intervenors. And then, because our students were permitted to testify, the judge got a request from a group of minority organizations at Harvard, represented by the NAACP Legal Defense Fund, our students were represented primarily by the Lawyers Committee for Civil Rights Under Law in a large public interest national law firm. So you heard at the Harvard case, if you were there for Asian American students, very compelling, interesting, testimony under oath about how they favored affirmative action. They believe that it helped them get admitted, what was written in their files supported that. That was the Asian American testimony in the room, there was not one witness from the plaintiffs who testified as to discrimination against them. There was not one file presented. There was a sort of a statistical argument that the personal rating was skewed against Asian American students. I mean, the plaintiffs were just really struggling and looking around for some argument. So on the facts of that case, and everything I heard in the courtroom and supported by the judge’s opinion and the appellate court opinion, the university won, and everybody about the Supreme Court thought that race conscious admissions was viable. It was important and that there was no discrimination against Asian American students. That was sufficient. There were there were some concerns, you know, there was concern about the disparity in the personal rating. But the elimination of race conscious admissions really was tightly connected to that. So what the Supreme Court said is, well, here’s a little bit of a problem with race. And we’re gonna go way over here and do something dramatic to resolve it, where that’s the two really aren’t connected. So, yeah, and one of the things I wanted to mention, you know, one of the things that just one of the things that got left out of the case and testing was something that really wasn’t discussed very much in the case, the disparities in the SAT and Harvard puts a lot of weight on the SAT more, it was certainly more than most schools. Another disparity that’s been talked about a lot recently, is the tip for legacies. And I just wanted to mention, you know, one of the weirdest things in the Harvard case you asked me what I heard, so I’ve told you the best thing. I told you some general stuff, that the facts I heard, just supported the use of affirmative action. The weirdest thing was Ruth Simmons, who’s this highly respected African American woman, former president of Brown University, might have been the first female minority president in the Ivy League, pathbreaking, highly respected college president, actually now went to become a president of an HBCU, Ruth Simmons gave eloquent testimony in favor of race conscious admissions, she also provided a vigorous defense of the use of legacy tips. And that was really weird.

Sheila Akbar: 

Well, I’ll tell you, what was her argument.

Jay Rosner: 

Her argument, it’s good for the school, it’s good for tradition, it’s good for it to maintain loyalty throughout the generations. It’s good for, you know, for revenue. And it was, it was unfortunate. She was Harvard’s witness. So the intervenors, who were aligned with Harvard, were reluctant to really critique that part of our testimony. And so that whole thing was pretty weird.

Sheila Akbar: 

Right, just for the listeners. I mean, I think there are two main critiques of legacy advantages in admission. One is it’s just unfair, and very clearly favors white, wealthy, alumni, children of alumni. Right. The second argument is sort of tied in with race consciousness. And it’s that well, if, before, things were more fairly done in admissions, right before we had affirmative action admissions, you know, it was white, wealthy, landowning people who went to college, and so their children are not all that diverse. And so if, you know, if we want to encourage diversity, one of the things that would actually help with that is doing away with these advantages that we give to legacy applicants. Right. Would you say that’s prep?

Jay Rosner: 

Sure. Yeah, it’s basically a set aside, which has been criticized, I mean, the Baki case, the affirmative action case against the University of California Davis Medical School back in the 70s. The critique was it was a set aside of a small number of seats for underrepresented minority students set asides are really wrong. Right. We will see admissions just a set aside for predominantly rich white students. I mean, that’s all it is. And the fact that that is only now getting the level of critique that it should have gotten starting back in the 70s, when set asides were at least alleged to have been a problem is, on the one hand, surprising. On the other hand, if you have structural racism built in, and a certain default to the wealthy that we don’t often notice or think about, then yeah, you’ll have legacy admissions until there’s a pretty massive movement to get rid of them.

Sheila Akbar: 

He’s a great example of, you know, when when I start talking about admissions with people, and we talk about the bias and tests, we talk about the problems in the review process and how much stress it creates for students and how it’s just kind of a rat race. People will say, Yeah, admissions is broken. And I always counter with well, it’s working for somebody, otherwise it wouldn’t be this way.

Jay Rosner: 

If your lens is equity, and particularly broken, well, it’s broken in a number of ways. Admissions is broken in incredibly compelling ways with regard to the debt that students have to undertake the fact that we as a society, basically don’t fund higher Education, the way that you know, dozen other countries in the world don’t Scandinavian countries, for example, you don’t take on debt that will take half to a whole lifetime to pay back to get a university education they fund University educations?

Sheila Akbar: 

Well, I will say, again, it’s certainly broken for students and their families. But that level of debt is a win for someone on the other side of that loan. Right?

Jay Rosner: 

Exactly. A win for someone on the other side of the loan. And it’s a win for someone who can afford to pay the full freight. You know, affluent families just pay the $50,000. It doesn’t have an impact on them. And they don’t have to think about the debt that other kids are on families are undertaking so so you’re right, it works for the money lenders, and it works for the people who don’t have to think about it. So you have that. And then you have the resource allocation to the universities themselves. Were the highly resourced universities, the name brands that we know the Ivy’s and the others, have gargantuan resources and reserves on their own, and then get well funded with research grants and philanthropic gifts. And you have community colleges that are struggling to fund the basic courses that the students who are attending them want. And so you have, you know, you have the crushing student debt issue. And then you have, in my view, the disparate and distorted allocation of resources, from our two year universities, to our four year brand name universities that cause all kinds of inequity issues, if we’re talking about testing, and where testing has an impact on the selective and highly selective set a couple of 100 universities. And so what we’re talking about is the issue of fair representation in the student body, at those universities. And for those of us who don’t think that that representation is fair, that underrepresented students, a shouldn’t be underrepresented, and B certainly shouldn’t be as underrepresented as they are. That’s when these issues, Sheila that you and I are primarily talking about come into play, but in a lot of ways the student debt issues, and the allocation of university resource issues are even more significant than what we’re talking about.

Sheila Akbar: 

Absolutely, yeah. And we’re all just so obsessed with whatever those 200 100 foot whatever the those, you know, schools that we always see in the media are that we’re not actually paying attention to these larger problems that are 100% solvable. If we have the political will.

Jay Rosner: 

Yes. All policy not only saw a solvable in policy fashions, but half a dozen to a dozen other countries have solved them, like what’s our problem?

Sheila Akbar: 

Right. Well, I think they’re related. The problem with that, and the problem with the highly selective, they’re related. But that’s a whole other episode. So I think we’ll leave it there. Jay, thanks so much for joining us. We’ll certainly have you back to talk more about all of this.

Jay Rosner: 

Thanks, Sheila really enjoyed it.

Sheila Akbar: 

So folks, we’re a little bit in the weeds with this type of interview. And yeah, you don’t need to know all of this in order to navigate the questions around standardized testing for your child if they’re applying to college soon. But I think it’s really important to understand this context so that you can navigate these challenges and these questions with your eyes opened, and maybe spread some of the truth around these exams, and advocate yourself for what’s fair in your own community. We’re going to go on a little break for Thanksgiving. I hope you all have a restful and meaningful holiday. And that you get to connect with people that you care about that fill your cup. I want to express my gratitude to all of you listening. We’ve only been doing this podcast for about seven or eight months. And the response from all of you has been really positive and I enjoy it and I get to talk to some really cool people. So thank you for giving me this opportunity. And we’ll see you in December.

Signet Education

Signet Education