Don't Bite: The Defenestration of Cookie

III. Whither the Cookie Task?

WARNING: What you are about to read may contain graphic statistical content.  Side effects may include: contagious yawning, inappropriate arousal, and / or spontaneous combustion, depending on how you like your math cooked... darling.

Psychologists often think about the cookie task as a test of cognitive control, and in keeping with this, tend to assume that it is a measure both of prefrontal maturity, and of executive brain function, more generally.  The idea, then, is that non-delayers are unable to delay gratification – in the same way that three-year olds are unable to switch – because their frontal lobes simply aren’t as serviceable.  Indeed, in the 2006 Psych Science paper that brought this idea to the big screen, the authors suggested that “performance in the delay-of-gratification task may serve as an early marker of individual differences in the functional integrity of [fronto-striatal] circuitry.”

If this is right, a case could be made that self-control is largely biologically determined.  Indeed, such evidence would suggest that children who exhibit poor “self control” in the delay task have relatively weak executive functioning, which subsequently manifests in long-term attentional and self-regulatory issues.  This is certainly an intriguing, if contentious possibility.  The curious thing about it, however, is that there’s almost no evidence for it.

Take the surprising findings of the Psych Science paper I mentioned: when kids in one of the original delay of gratification tasks were brought back into the lab fourteen years later and their cognitive control was measured (via a go / no-go task), researchers found *no* correlation between the kids’ original delay time and their later control abilities. This is particularly striking given that the whole reason that delay of gratification is interesting, is because delay time does correlate so well with many measures of later attainment.

Even odder, the only correlation that the researchers did find was the *opposite* of what you’d expect if delay of gratification were simply the same thing as adult cognitive control. When the researchers kept delay time constant, they found that how much time the kid had spent oggling the cookie during the task was actually negatively correlated with later cognitive function.

What’s that mean?  Well – what the researchers did, was split the kids into two groups: they wanted to compare the kids in the task who spent proportionally more time looking at the reward – and therefore had more temptation to overcome – with the kids who spent less time looking at the reward – and thus had less temptation to overcome.  Just to be thoroughly offensive, let’s call these the “Catholics” (more temptation) and the “Protestants” (less).

The researchers then controlled for delay times, meaning they looked at Catholics and Protestants who ended up waiting the same amount of time.  You might expect that the Catholics – who spent way more time tempting themselves, but the same amount of time patiently waiting – would have higher marks for cognitive control.  After all, they spent all that time staring at the thing, but didn’t bite!  But actually, the researchers found just the opposite: in fact, the “see-no-evil” Protestants were better on adolescent measures of cognitive control than the woefully-tempted Catholics. (It might be tempting to see this a case of tortoise-vs.-hare syndrome – with the Protestants catching up in the end – but in truth, I'm not sure anyone knows quite what to make of this.)

So, the findings in the Psych Science paper are actually quite astonishing: The results suggest that later neural functioning isn’t predicted either by actual performance in the cookie task, or by how well children overcome temptation.  Which – going back to the question I posed earlier – indicates that the workings of our prefrontal cortex may not be the most important determinate of how we control our behavior.

Given that many papers treat “delay of gratification” tasks as tests of cognitive control, this could have important implications for how behavior in the task is characterized and understood.  However, I don’t think that the Psych Science provides us with a conclusive basis on which to decide yea or nay what the “cookie task” is testing.  Even if the authors had found a correlation between delay times and later executive function, this would not have thereby established that the task was a test of executive function, only that it was predictive of later function.  And similarly, the failure to find such a connection does not thereby rule out whether the cookie task measures cognitive control.

That said, directly testing whether it does should be simple enough.  There are an array of classic cognitive control tasks, such as the DCCS, that are demonstrably related to cortical development.  Why not simply run kids through a battery of such tasks, and then see how their performance on those measures links up with their delay times?

This year, two scientists in my lab, Michael Ramscar and Christine Tran, decided to do just that.  They hypothesized that if the cookie task were indeed a test of cognitive control, then delay times should be tightly coupled with scores on the DCCS, and similar tasks.  But, if the task were better seen as a test of how good children are at strategically distracting themselves, then delay times should be more strongly linked with, say, vocabulary development – which is a good proxy for strategic competence and practice.

To tease apart these factors, they ran 3 – 4 ½ year olds through a battery of tests, which assessed cognitive function, measured delay of gratification, and calculated vocabulary size.  What they found is that vocab is a much, much better predictor of delay-success than cognitive function: in particular, while delay-times strongly correlated with verbal inventory scores, they only weakly correlated with measures of cognitive control; a finding that is in keeping with the Psych Science results.

Taken together, these findings indicate that delay of gratification isn’t merely down to cognitive functioning.  Rather, how long a child delays seems to have more to do with how good they are at strategizing and distracting themselves, an ability which may largely reflect their learning environment.  This is not to say that this rules out the contribution of “cognitive control,” or neural functioning, more generally.  It simply shifts the focus.

Encouragingly, I don’t think this is a conclusion Mischel would disagree with.  Indeed, much of his research in this domain has focused on how strategies for delay of gratification may be learned and taught, and how these strategies may play out effectively in the long run.  As he notes, children who learn early on how to use effective strategies to help regulate their behavior – and thus how to exercise self control – will begin life with a distinct advantage, which may bear more and more fruit over time.

[This is the third in a series of posts on "delay of gratification" that is to be continued. Access articles #1 and #2.]

The Daily Fact Check

If you made it this far, congratulations!  You managed not to succumb to stereotype threat. And really, the stats bit wasn't so bad, was it?

Eigsti, I., Zayas, V., Mischel, W., Shoda, Y., Ayduk, O., Dadlani, M., Davidson, M., Aber, J., & Casey, B. (2006). Predicting Cognitive Control From Preschool to Late Adolescence and Young Adulthood Psychological Science, 17 (6), 478-484 DOI: 10.1111/j.1467-9280.2006.01732.x

Peake PK, Mischel W, & Hebl M (2002). Strategic attention deployment for delay of gratification in working and waiting situations. Developmental Psychology, 38 (2), 313-26 PMID: 11881765

Mischel, W., Shoda, Y., & Rodriguez, M. (1989). Delay of gratification in children Science, 244 (4907), 933-938 DOI: 10.1126/science.2658056

12 responses so far

  • Jason G. Goldman says:

    Which...indicates that the workings of our prefrontal cortex may not be the most important determinate of how we control our behavior.

    ...or that the cookie test isn't as strong a predictor of future behavior as it appears. Or the function of PFC may indeed be a strong determine of how we control our behavior, but delay of gratification is not. Right?

  • Coturnix says:

    'Defenestration' is one of my favorite words in English. Just after "elusive". If my anatomy students ever memorized a foreign-sounding term, it is defenestration, I bet.

  • Tybo says:

    I suppose I'm misreading the statement that they "were brought back into the lab and their cognitive control was measured" as that the same test was done, when it was really a different measure used entirely? Because the former was my first understanding, then I had to go back and realize that couldn't be right.

  • Jason Kerwin says:

    The moment I read this

    >You might expect that the Catholics – who spent way more time tempting
    >themselves, but the same amount of time patiently waiting – would have higher
    >marks for cognitive control

    I immediately thought "no, they're playing an inferior strategy!" But of course you hit that nail right on the head. The whole Catholic-Protestant theory that the Psych Science paper tests seems obviously suspect. To flip the setup on its head, suppose we hold the time spent looking at the cookie constant. One explanation is that conditional on time looking the kids who take the cookie have more self control. But another is that they are literally less tempted by it; in economics we'd say their discount factor is higher. Perhaps you'd consider that to be self-control as well, but it's distinct from devising strategies to control yourself. It's hard to see how you could possibly identify an effect without trying to separate the two.

    Also, I'm a little confused as to how higher vocab scores indicate a better ability to strategize and plan for the future. I think I'm missing a step in the reasoning there.

  • Very interesting. Thanks, I'm really enjoying your blog so far. I can't quite wrap my mind around the vocab as proxy for strategic competence though. Care to expand on that?

  • I didn't understand half the words in this post.

    Mmmm... hungry. I need to eat now!

  • melodye says:

    Nothing more helpful than readers to point out where you missed a step...

    @Tybo -- Thanks so much for the comment! My mistake for not making that explicit. In the experiment, the kids from one of the original cookie experiments came back fourteen years later to take a classic cognitive control task (the go / no-go task). I updated the post to reflect that. And you're right -- while it's often taken for granted that the "delay of gratification" task simply is a cognitive control task, part of the point of the post was to examine whether that assumption is supported by the evidence.

    @Jason & Livia -- I'm going into far more detail about vocabulary and how this may (or may not) relate to behavioral strategy tomorrow. I've very much simplified the idea here in the interest of space constraints, but definitely follow-up if tomorrow's post doesn't answer your questions.

  • katie d says:

    According to the cookie test, I should be absolutely stellar at all aspects of my life, since I would have waited for ages to get that second cookie (everyone knows 2 cookies are way freaking better than 1, especially when they're chocolate chip). Yet I am a classic underachiever and misfit. So frankly, I think the cookie testers are idiots. (BTW, not that I demonstrate it here, but my vocabulary was way advanced when I was a kid.)

  • stripey_cat says:

    katie - I think that if it is down to strategic ability to withstand distraction (or to distract yourself), that doesn't necessarily say how you'll apply that ability in later life. I'm another one with precocious vocabulary and self-control as a kid, excellent academic transcript, and life down the tubes as an adult.

    • melodye says:

      @katie d and @stripey_cat : If you read the pepper story, you'll realize I'm the exact opposite -- a child with raging curiosity and rather poor 'self control.' Yet I had a large vocabulary (highly verbal parents) and did well in high school. So I'm another exception -- but it's important to note that that doesn't refute their findings. The predictive power of the cookie task lies over a population of children; it's not necessarily going to tell you much about the specific outcome for any one individual.

      Of course, one of the things that's appealing about the cookie task -- particularly in the glare of the media spotlight -- is that it seems like we can take this one, tiny little task, and use it as a crystal ball into our children's futures. But that's just not true: if you read the papers, you quickly realize that there's lots of unexplained variance in the task. Which is another way of saying that "yes, it's generally the case that if X then Y" but not always. There are lots of fantastic delayers who end up high school drop-outs and some non-delayers who end up superstars.

  • AnlamK says:

    "This year, two scientists in my lab, Michael Ramscar and Christine Tran, decided to do just that. They hypothesized that if the cookie task were indeed a test of cognitive control, then delay times should be tightly coupled with scores on the DCCS, and similar tasks."

    I'm at a bit loss about what is meant by 'cognitive control'. The ability to move rocks with your mind? Or self-regulation, executive functioning/will power? We'll just assume the latter for now.

    I'm inclined to say that 'DCCS and similar taks' are not measuring self-regulation as opposed to conclude that the marshmallow experiment is not measuring self-regulation. More than anything, this is a conceptual, dare I say philosophical, disagreement on what we mean by terms like 'cognitive control' and 'self-regulation'

    In some sense, I think marshmallow experiment is definitive of what self-regulation is. It is the gold standard.

    • melodye says:

      @AnlamK : It may be helpful to read the whole five-part series -- any individual post may be a bit confusing out of context.

      In post 2, I discuss what is meant by cognitive control, and explain how the DCCS and other similar tasks measure executive function. Then in post 5, I conclude that the marshmallow / cookie task isn't a good measure of 'cognitive control' which it is often taken to be. This isn't to say it doesn't test 'self control' -- though I elaborate some reasons why we, as scientists, might need to think harder about what self control amounts to in the first place.

      Hope that helps!