Every late summer, U.S. News releases its annual college rankings. Are you using them, or have you used them to make college selection decisions? If so, be careful.

The College Confidential discussion forum explodes this time every year with an avalanche of frantically defended guesses as to which schools will top the rankings' lists and which schools have “gone up" or down. It's an annual rite at CC.

As you may have gathered from some of my past posts here, I am not a fan of these (or other) rankings. They're just too arbitrary and hide behind a confusing barrage of methodology rationale, especially that of U.S. News.

Today, I'd like to share information from two sources. The first is a very interesting article, 14 Reasons Why US News College Rankings are Meaningless, that presents as good an argument I've seen against the U.S.News approach.

The second is a press release from Hampshire College, in Massachusetts: “Hampshire Reports Results of No-SAT No-ACT Admissions Strategy; US News Disqualification Continues." Apparently, a college can prosper without being part of the U.S. News circus.

First, let me highlight seven of my favorite 14 Reasons for you. I'll also include a portion of the article's reasoning behind each one. You can click the link above to see all 14 plus the full explanation for each one.

1. Unstable rankings and irregular shifts continue to be a problem for US News.

People in higher ed have been complaining about these unreliable measures for over 20 years. Here is a personal letter from the president of Stanford University in 1996, directly criticizing the USNWR college rankings for their inconsistencies.

3. Flawed metrics have persisted for over a decade.

This study, all the way back in 2002, offered constructive fixes for most of the flaws in the US News rankings, with suggestions for improved methodology and examples of better metrics to adopt … the people working [at U.S. News] to improve them are not successfully doing so. In short, no. They have not fixed the problem.

4. Moving up the rankings has nothing to do with improving the student's experience.

This “success story" about Northeastern University climbing the US News rankings actually points out flaws in the system. Tangible actions and investments that would lead to more attention per student, a more diverse campus and different mix of residence/commuter students, better facilities, more resources, and new amenities didn't actually budge the ranking …

6. Failure to consider student debt load.

Student loans are an inevitable part of the college experience for many students, therefore student debt is as well. The US News college rankings, however, leave this measure out completely. They publish a “Short List" where you can find which schools will leave you with the most debt, but this is ultimately unhelpful when not factored into the overall rankings.

10. When colleges refuse to participate, US News penalizes them.

When schools don't report their numbers, USNWR simply plugs in their own measures and assigns a lower rank… despite nothing actually changing on campus. Colin Dover, who has served as president at Reed College and Dean of the University of Pennsylvania Law School, details his experience with the rankings, as a participant and a bystander.[Also, see what follows about Hampshire College.]

12. Schools know how to work the system… and they do.

In his article, Robert Woodbury describes 10 ways to climb up the rankings (and he proves how skewed they are in the process) and he says, “The ranking of colleges and universities by neat formulae and dubious statistical measures is distorting, illusory and, ultimately, harmful to democratic values we all share." This witty and brutally honest article speaks for itself and is worth the read.

14. When students choose colleges based on US News rankings, they're in trouble.

… Purportedly objective rankings can cause students to choose or feel pushed toward “best" colleges that will not necessarily be the best learning environment for them and, in turn, lead them to either change their major or, even worse, drop out of school.

***

Now, to Hampshire College …

John Courtmanche, Media Relations and Editorial Director at Hampshire College, sent me a press release entitled Hampshire Reports Results of No-SAT No-ACT Admissions Strategy; US News Disqualification Continues. I think it speaks well for a college that has found their independence from the U.S. News bondage rewarding, much like Reed College has. I post it here in its entirety.

John opens his message to me with a prelude to his release:

“… If you're covering changes in standardized testing in education this fall, you might find our results relevant to your reporting; this morning we released the year-two results of our No-SAT No-ACT admissions policy."

In Year Two, Hampshire's No-SAT No-ACT Strategy Continues to Align Admissions with Mission

The College is still disqualified from the U.S. News rankings for rejection of standardized testing

FOR IMMEDIATE RELEASE (Amherst, Mass.) – Hampshire College is reporting its second year of results since it stopped accepting SATS and ACTs, and they reveal sustained progress in aligning admissions with the college's mission. Again, the practice disqualifies Hampshire from the U.S. News annual college rankings this week, as the publisher doesn't have a process for measuring institutions of higher education without standardized test scores.

The College stopped accepting these scores in June 2014 after an internal study of its most successful students revealed that neither the SATs nor the ACTs predicted their success. In making this radical change, the College also cited the bias of these tests against low-income students and the negative influence of standardized testing on education.

This week, Dean of Enrollment Meredith Twombly reported Hampshire's second-year results under this strategy:

• The incoming class is again more racially diverse and includes more students who represent the first generation in their family to attend college than in any year before this policy went into effect.

• Retention of first-year students is again higher than it was before the policy change, 81 percent as opposed to 78 percent two years ago.

• The “yield" percentage of students who accepted the College's offer of admission is again higher than in the years before the policy change.

Regarding the disqualification from U.S. News rankings, Twombly points out those rankings are not based on whether colleges produce successful alums; rather, they measure how well colleges attract high-scoring, high-GPA students, which is influenced in part by the wealth of the schools.

“Standardized test scores have all the value of a broken clock, plus they are biased and cause widespread anxiety for students," Twombly wrote recently in the essay “College Admissions Moneyball." “Many remarkable students whose test scores and GPA fall among the lower 90–95 percent of their class are being shortchanged in the arms race that is college admissions."

Twombly says many colleges are consumed by the competition for rankings and bragging rights and are too quick to bolster entering-class SAT/ACT/GPA data through admissions practices that are harmful to many promising students: “When you see that your U.S. News ranking will improve if you deny more students, then you're tempted to chase more applications though you have no intention of admitting them," she says. “If you see that your U.S. News ranking will improve if you have a higher mean SAT or ACT score, you'll be tempted to chase high test scores, and increasingly give financial-aid dollars to students who don't necessarily need it but have high test scores."

Twombly explains the College's new direction in a YouTube video, “Why We Don't Accept SAT/ACT Scores."

Hampshire remains in the top 1.4 percent of American colleges as measured by the number of alums who go on to earn the highest degree in their field, according to National Science Foundation data released earlier this year. Hampshire also placed in the top 10 of Forbes's list of most entrepreneurial colleges based on LinkedIn data of alums who run their own ventures, organizations, and businesses or are self-employed.

A recent example of Hampshire's success as a consequence of admitting promising students independent of test scores is detailed in President Jonathan Lash's latest Huffington Post essay, “A Study in Project-Based Learning," about a group of students who this summer, after only their first year at Hampshire, outperformed teams of graduate students from much larger research universities in a national ecological-design competition. Lash points out that professors at Hampshire never give grades and instead evaluate student work using narratives, which are exponentially more formative and informative for learning, giving students constructive feedback they can learn from and act on. He writes, “Students learn best — not when they are taught or lectured at — but when they are given independence to direct their own learning under the guidance of teachers; when education is not imposed, but is active."

Hampshire's outcomes also reveal that 89% of our alumni report receiving a job offer within one year of graduation, and two-thirds of our alums earn advanced degrees within ten years of graduating. Hampshire publishes other student outcomes here.

When high school students in their applications were asked “Is there anything you'd like to share about your Hampshire admissions experience?" says Twombly, their responses helped confirm that the College's practices are having a positive effect. Here's some of what they had to say:

• “Hampshire made me feel optimistic about the admissions process because I knew that no matter what, I wasn't going to be judged by a vague SAT or ACT score. I hope that Hampshire's approach to the admissions process is adopted by other colleges."

• “Hampshire was the most individualized and personal experience."

• “It was such a humane and compassionate process. I applaud the admission team at Hampshire."

• “Thank you for treating me as a whole person instead of a stat."

• “You guys did an amazing job personalizing my experience. Out of all the colleges I applied to, you were the only ones who made me feel as if you wanted me as a person instead of just money."

Twombly acknowledges that risk is inherent when making a major policy change because it makes the environment less predictable. This explains why many colleges are averse to risk and innovation, she said, in any area that has an impact on enrollment. Hampshire projected a decline in enrollment as it becomes more selective and admits only students who are a good fit for the College, as well as a short-term budget deficit for the same reason. Indeed, a decline in enrollment this year resulted in the College's decision to leave 15 employee positions unfilled and the board of trustees contributed $1.3 million to balance the budget.

“We entered into this with our eyes wide open and we're very willing to ride out a couple of bumpy years as we get a better handle on yield and retention projections," Twombly said. “We're thrilled with the outcomes of the policy thus far, and will stay the course."

Last September, President Jonathan Lash announced the results of the College's first year of no-SAT, no-ACT.

***

Here's the money quote from the release, with my emphasis of its most important part:

>>The College stopped accepting these scores in June 2014 after an internal study of its most successful students revealed that neither the SATs nor the ACTs predicted their success. In making this radical change, the College also cited the bias of these tests against low-income students and the negative influence of standardized testing on education.<<

This underscores my core contention regarding both standardized testing and the U.S. News rankings. So, high school students and parents … take note!

**********

Check College Confidential for all of my college-related articles.