Taking me to task

The Family Social Scientist takes me to task in the comments on my post on Decision Fatigue:

while this is an interesting quirk of the data and warrants further exploration, its hardly conclusive . To say that the marshmallow test is "known to predict future success in life" is a little misleading and perhaps a misinterpretation of the results.

Touché. I really cannot complain that the FSS is calling me out in the same way that I do to others all the time. It is indeed perilous to try to glean science out of popular articles. I am a rank amateur in the field, and I know how that seems to an expert because I have not always had the grace to deal lightly with those who have trespassed into my own technical specialty.

Yet, nevertheless, I will persist on this topic, despite the many landmines, because it is absolutely fascinating to me. The FSS brings up some really good points in his comment that need to be considered:

grades are hardly a proper measurement of academic progress and intelligence

Quite true. This was arguably less true in the past, but grades, both high school and undergraduate, have a pretty loose connection with both academic progress and intelligence. This is actually one of the things that first attracted me to psychometrics, because it gave me the mental tools to understand why grades aren't a measure of intelligence. 

By way of example, consider this work by Steve Hsu and Jim Schombert on college GPA and SAT scores. Hsu and Schombert explain some of the complexities their work demonstrated in an interview:

“Freshman GPA is not a satisfactory metric of academic success,” Hsu explains. “There is simply too much variation in the difficulty of courses taken by freshmen.” More able freshmen typically take more difficult courses, whereas less able freshmen take introductory courses “not very different from high school classes,” he says. Under these circumstances, academic success—an “A” in an introductory course versus a “B” in an advanced course—becomes too relative to accurately measure. Course variation decreases in later years, as students settle into their respective majors, working hard in required classes.

The new approach bore fruit: SAT and ACT scores, their analysis showed, predict upper-level much better than lower-level college grades, “a significant and entirely new result,” Schombert says. 

Hsu and Schombert are now working on including personality inventories in this assessment to see whether they can improve their model. As a guess, conscientiousness will probably be a big hitter. But, there is a difficulty here. How do you measure conscientiousness? The short answer is: we don't know how. The longer answer is we try various techniques to quantify a quality, such as personality inventories or the marshmallow test. Personality inventories are easy, but they are also easy to game. If you know what the questions are getting at, you can manufacture any result you want. The marshmallow test, and the ice bath test, are a little better in this respect because they push up against a hard limit that we hope is correlated with the thing we are interested in. Thus, even if you knew that holding your hand in the water was going to be used to judge your mental toughness, this would be a good thing because your ability to endure unpleasantness for a positive social judgement is exactly what the test is after.

This is also related to why grades aren't the best predictor either: the system is easy to game. In college admissions, this is part of the reason grades have become de-emphasized. Good grades in high school aren't by themselves a good predictor of doing well in college, but if you factor in participation in sports and other extracurriculars, you can get a rough estimate of a student's ability to stick something through and their ability to manage competing priorities. This can be gamed too, as Amy Chua demonstrates, but if you can successfully game this system, it means you are probably smart and likely to be wealthy, which is something colleges want anyway. 

An American Professor Weighs in on the Academic Process

Via John D Cook, Matt Welsh explains why he left Harvard for Google:

There is one simple reason that I'm leaving academia: I simply love work I'm doing at Google. I get to hack all day, working on problems that are orders of magnitude larger and more interesting than I can work on at any university. That is really hard to beat, and is worth more to me than having "Prof." in front of my name, or a big office, or even permanent employment. In many ways, working at Google is realizing the dream I've had of building big systems my entire career.

A big reason for this is the amount of time academics spend doing things that aren't teaching or research:

The biggest surprise is how much time I have to spend getting funding for my research. Although it varies a lot, I guess that I spent about 40% of my time chasing after funding, either directly (writing grant proposals) or indirectly (visiting companies, giving talks, building relationships). It is a huge investment of time that does not always contribute directly to your research agenda -- just something you have to do to keep the wheels turning.

There are good reasons to be an academic, but it is not for everyone. 

Academic Ponzi Scheme

Here is another take on the pyramid scheme that is higher education today, this time from England.

Why I am not a Professor

Teaching was not the only criterion of assessment.  Research was another and, from the point of view of getting promotion, more important.  Teaching being increasingly dreadful, research was both an escape ladder away from the coal face and a means of securing a raise. The mandarins in charge of education decreed that research was to be assessed, and that meant counting things. Quite what things and how wasn't too clear, but the general answer was that the more you wrote, the better you were. So lecturers began scribbling with the frenetic intensity of battery hens on overtime, producing paper after paper, challenging increasingly harassed librarians to find the space for them.  New journals and conferences blossomed and conference hopping became a means to self-promotion. Little matter if your effort was read only by you and your mates. It was there and it counted.  

Today this ideology is totally dominant all over the world, including North America.  You can routinely find lecturers with more than a hundred published papers and you marvel at these paradigms of human creativity.  These are people, you think, who are fit to challenge Mozart who wrote a hundred pieces or more of music.  And then you get puzzled that, in this modern world, there should be so many Mozarts - almost one for every department.  

When everyone is Mozart, no one is.

Critical Thinking

Browsing through the comments on the last post I linked to, I was struck by the first one:

Based on my grad and undergrad at elite schools (and classes taken at an excellent community college and excellent regional university while I was in high school), my belief was that college taught you how to think instead of just how to memorize. You would think like a mathematician or an economist or an english major or an artist etc. You would approach problems differently, but systematically and in a similar way to other folks in your major.

Now that I teach graduate students at a solid but not elite school, I understand degree creep and why so many employers favor the masters degree over the BA these days. A lot of what I do is teaching students to think rather than to just memorize facts. The first semester is major cognitive dissonance for most (but not all) of them, but they come out critical thinkers.

So prior to teaching I would have said a college degree teaches and demonstrates thinking skills, but as a teacher I realize that many 4 year schools are just 4 more years of high school.

This is bullshit. Or bollocks if you prefer.

How do I know this is bullshit? You would be hard pressed to find a school in the United States at any level from primary to tertiary that requires much in the way of memorization. All educational theory is against memorization, and all educators at all levels are constantly trying to find ways to encourage their students to understand concepts instead of memorizing facts. There is always some memorization to rail against, because memorization is a key part of education and cannot be expunged, but it is deemphasized at all times and in all places.

Don't believe me? If you are under the age of forty, then name all fifty state capitals. Can you do math in your head? Can you recite the entirety of the Gettysburg address from memory? Can you recite any poem from memory? What are the principal exports of Malayasia? Some people of about my age or a little older can do these things, probably the nerdy ones. Elite education is still good. However, this ability used to be more widespread. 

It is notable that this fantasy is contrasted with the sine qua non of modern American education: critical thinking. Critical thinking cannot be taught. Well, let me correct myself. The ideal of critical thinking cannot be taught. The simulacrum that most people actually refer to is taught quite well. 

Almost everyone that I have ever heard use the phrase does not actually mean that they expect students to be able to form independent judgments by carefully weighing evidence, and to reflect upon their own biases before making a decision. If you asked, most would claim this is what they are after, but if you look and see what criterion is used to determine success you will find that it is "the student forms the same opinions I have". 

This is precisely wrong. If you succeed at teaching critical thinking, then you should expect your students to disagree with you and with each other.  Reality is underdetermined by the facts we have available to us. To think that everyone would agree with you if only they could think straight is the sin of Rawls.

Most students do not learn critical thinking, but rather learn the opinions of their teachers. This is normal. This is why everyone cares about the character of teachers. As C. S. Lewis noted in The Abolition of Man, education is fundamentally like the process by which grown birds teach young birds how to fly. It is a kind of propagation. Your mind is passed down to your students. 

What a true university education is capable of teaching is analytical thinking. The ability to construct (or deconstruct) an argument. The ability to reason abstractly, using words or numbers. Enough detailed knowledge of the world to use as premises for further reasoning. This is not the same as the rarer critical thinking. One can be capable of forming good judgments without formal education, and one can also be a great fool despite extensive education. I would be willing to believe that a good education can foster good judgment, but it is far from a sure thing.

Generic Degrees

From Confessions of a Community College Dean:

Ask the Administrator: The Generic Degree

An occasional correspondent writes:

Some jobs out there are advertised as requiring a college degree, but
the employers don't care what was actually studied. So these employers
are in effect using college as a four-year hundred-thousand-dollar
screening test, with perhaps a bit of intellectual calisthenics for
good measure.

I had a chance to discuss this with a supervisor at one of the
management consulting companies, and he confirmed this is in fact
their policy. I suggested that since they don't care about any
specific knowledge -- only smarts and the willingness to work hard --
they should be open to hiring people right out of high school. Some
high-school students can point to significant intellectual
accomplishments, after all. But no, this is Just Not Done.

A four-year degree seems like a very expensive way of doing
intellectual quality control. Could we do better?

I hate to admit it, but there’s some truth to this.

I saw this quite a bit at PU, where some older students were already well into their careers and doing well there, but they needed their hands stamped in order to move up to the next level. They didn’t care much about the actual content of it; the point was to become eligible for management ranks. I took it as a personal victory when one of those students actually found value in a class I taught.

At an individual level, this can be kind of silly. Certainly it’s possible to be brilliant (or better, wise) without a degree, and to be bovine with one. And it’s also true that some jobs that stipulate college degrees don’t really draw on the skills that a degree is supposed to confer, whatever the major. Degree factories exist for that very reason.

That said, though, I like to think that a bachelor’s degree from a real college -- as opposed to a degree factory -- carries some meaning.

At one level, it shows the ability and willingness to stick to a program. Given the prevalence of college dropouts, those who actually finish have at least shown the ability to get their stuff together sufficiently to fulfill a multiyear commitment. (Along similar lines, students who transfer from cc’s with associate’s degrees tend to finish bachelor’s degrees at far higher rates than those who transfer with scattered credits. The graduates are those who finish what they start.) It shows the ability to navigate a bureaucracy, which is an essential workplace skill for most of the higher-paying jobs.

If the college is at least halfway serious, a degree should indicate some ability to handle complexity, to communicate at least functionally, and keep one’s balance when dealing with numbers. One of my personal indices for wisdom is the ability to handle ambiguity. Clueless people can be trained to do almost anything routine; the real test comes when the routine has to change. Some of that is temperamental, but some has to do with the ability to discern the bigger picture.

The actual content of the degree is another issue. I don’t often draw on my study of Restoration England, but I do draw on some of the skills developed through it. My social science training enabled me to stay awake and attempt to wring meaning out of long, boring, poorly-written texts; on the job, I use that skill every single day.

This is the kind of thing Charles Murray was talking about in his book Real Education. The higher education system has changed over time to meet the needs of the marketplace, but for reasons of educational romanticism has retained the dress and language of an earlier dispensation. This change isn't necessarily bad, but we should understand it for what it truly is.

The most persistent misunderstanding, here shown by the questioner's comment "a very expensive way of doing intellectual quality control", is that sending masses of people to college has anything whatsoever to do with the life of the intellect. As Dean Dad correctly notes, today college serves as a filter for high conscientiousness. Whether college imparts more C or simply sorts people by it is a fair question, but it is clear that those who successfully complete a four-year degree have better work ethic, ability to finish what they start, get organized, etc. than those who have not completed such a course. Thus it is entirely rational for businesses to sort applicants in this fashion.

Whether it is best for everyone is another matter entirely.

Physics for Future Presidents

Physics for Future Presidents is a course offered by Richard Muller at Berkeley. The course is intended primarily for liberal arts types, but Muller opened the course up to physics majors because the subject matter is not typically covered in the required courses.

This course really is amazing: it covers the essential science to understand modern technology, especially the technology that is likely to make a big impact on our lives, and is interesting besides. Nuclear power. Spy satellites. Low earth orbits. Moore's Law.

The objective of the course is understanding, not computation. An entirely reasonable goal. As Muller says, if you really need to calculate something physical, you hire a physicist (or an engineer). However, you need to be able to understand the science well enough to make a good judgment about what you are given, and what to do with it. Something along these lines would be much more effective than forcing all college students to take lab science, because this class effectively focuses on the big picture. Learning how to titrate in University Chemistry I does not necessarily make for scientific insight or good judgment. Learning how to compute a surface integral for Gauss' Law is really no better for that purpose (albeit fun).

That being said, I don't know how well this course would work on the general student population at say, Northern Arizona University (my alma mater). Jokes about liberal arts majors aside, anyone who gets into Berkeley is pretty damn smart. Students at a state college are much closer to the population mean in intellectual ability, so this course may be too hard for them as is. However, I think there is something to this. The science courses offered at most universities really are more like vocational training for scientists than the building blocks of a liberal education. The unelected elites that Charles Murray talks about really do need to understand science, but good judgment is more important than the calculus.

If you ever wanted to know more about science, but hated the classes, or couldn't hack the math, buy the popular version of the book, or watch the lectures on YouTube. It would be better if more people understood science in this fashion.