published on May 14, 2024 in today

Inside the S2 Cognition test thats transforming how NFL teams evaluate draft prospects

The Athletic has around-the-clock coverage of the NFL Draft. Follow our NFL Draft Round 4-7 live blog and round 2-3 winners and losers, round 2 grades and best available players.

FRANKLIN, Tenn. — It’s natural, even forgivable, to get lost in the vortex of the bone-breaking hits, the signature quarterbacking moments or the dazzling speed and mind-bending catches during the most athletically dominant era in NFL history.

Even now, on the doorstep of the NFL Draft, narratives frequently swirl around a prospect’s 40-yard dash or whether he can hit the ceiling with a football. Of course, all of that has its integral place in the game, but it overlooks a cognitive element that has drawn increased focus in recent years.

With players processing an incalculable number of split-second decisions during each play, NFL teams have put more effort into trying to learn, almost quite literally, how their players’ brains are wired.

Within the past seven years, many of them have turned specifically to the S2 Cognition test to gain that understanding.

“Things are happening at speeds that are very different than what you do in everyday interactions,” S2 Cognition co-founder Scott Wylie told The Athletic. “These guys, it’s incredible. Their brains are wired to do things most human beings don’t appreciate and can’t do. They are really at the upper extreme with some of these brains. You can imagine how that gives them huge advantages on the football field if they can see things a tick quicker, when you can anticipate with higher accuracy better than most, when you can process, recognize and track things with better precision and effectiveness.

“That’s what we’re getting at, those intangible qualities that lead to terms like, ‘Man, he’s got a nose for the ball.’ That’s what we’re quantifying.”

The S2 test gained mainstream attention this offseason when The Athletic uncovered quarterback Brock Purdy’s results prior to the 2022 draft and how they could have been a predictor for his success with the 49ers. Last week it gained even more attention after a report surfaced that top 2023 quarterback C.J. Stroud had performed poorly on the test, hurting his draft stock.

This week, Wylie and S2 co-founder Brandon Ally gave The Athletic exclusive access, including the ability to take the test to gain a better understanding for the science behind the software.

“This is challenging, and you will get frustrated,” Ally said before revealing the computerized test. “View it as you’re going in the weight room to get your one-rep max. You know two things. One, it’s going to hurt a little. And two, you’re going to fail at some point. That’s our job to try to find where your point of failure is.”

Ally and Wylie are cognitive neuroscientists who were collegiate athletes and decided in 2014 to blend their passions. They generated a testing system — using tests with decades of scientific backing — to formulate the S2 method, and they began logging data for NFL Draft prospects (and players in other sports) in 2016. On a broad scope, the S2 test measures how quickly the brain can process information in real time.

S2 Cognition has grown each year. Fifteen NFL teams currently employ their services (the Bears, Bills, Broncos, Cardinals, Chiefs, Colts, Commanders, Cowboys, Falcons, Jets, Panthers, Rams, Saints, Steelers and Titans) — testifying to the growing belief around the NFL in their test’s capabilities. Before the start of the all-star game and bowl circuit, those teams prioritize a list of prospects whom they’d like to analyze, and S2 gets to work. S2 tested roughly 850 players this offseason from in-person visits to all-star games, pro days and any other means to get it done.

For so long, the Wonderlic was viewed as the premium cognitive measurement on the pre-draft circuit, but the 50-question, multiple-choice exam is essentially a standardized test. Applied to football, it’s somewhat similar to a player who can learn the playbook and draw up the concepts on the whiteboard, but it essentially ends at that point.

But how do they process their assignments after the snap amid the chaos? That’s where the S2 aims to show its value.

“Understanding the playbook and executing the playbook in real time are vastly different things,” Wylie said. “Every coach knows that. Every player knows that. From a cognitive standpoint, what they’re doing on the field is very different.

“By definition, we’re pushing them (in the S2 test) to the point of failure, to the brink of human performance.”

The S2 Cognition test is broken into eight parts, with each focusing on a particular set of skills thanks to the scientific research behind it.

GO DEEPER

The Panthers, the S2 test and Bryce Young: What does it mean when the No. 1 pick is made?

Part One: Search efficiency

The screen shows a cluttered assortment of colored shapes. The test-taker must determine whether the designated shape is among them and make a quick and accurate choice.

“Some guys have a preference for being accurate,” Ally said. “Some guys have a preference for being quick. It can kind of tell you a little bit how they make decisions. If you have a low score for impulse control and you’re really fast but inaccurate making decisions, that’s going to lead to a lot of impulsive mistakes on the field.”

Low impulse scores correlate to pass rushers who could be more inclined to jump offside on a hard count or fall prey to misdirections or play-action passes.

The test can also serve a predictor for receiver drop rates, especially if the player has to turn quickly and find the ball. Receivers can use this information to compensate by turning for the ball a half step quicker.

“Think about how fast they can search through chaos,” Wylie said. “Our fastest athletes can search in about 600 milliseconds with a high degree of accuracy, so just a little over a half second, all the way up to pro guys searching around 1,200-1,300 milliseconds, so more than twice as slow.

“Think about being a half second slower and finding your target, finding the ball, finding the guy you’re supposed to block, a defensive end making a spin move then having to pick up where people are. If you’re a tick slower, it’s one way you can experience a disadvantage, or a huge advantage if you can scan with lightning speed. That’s a function of the brain system.”

Part Two: Distraction control

Amid a series of arrows, the test-taker must quickly determine which direction the target is facing. Though the premise may sound easy, it feels impossible to overstate how easily the brain can become distracted by the screen’s surrounding characteristics.

Even if the brain recognizes the correct answer, the body doesn’t always follow along.

“You can see on this task, it’s simple, right?” Wylie said. “Everybody thinks so. You’re just responding to an arrow in the middle, left or right, but you see how your brain is taking in all kinds of information? It’s taking in those arrows on the sides. You’re not even supposed to be paying attention to those, but how many times did they get you?”

Remember, on these tests, pace and precision are paramount. Sacrifice one for the other, and the test-taker will fail.

“We don’t have unlimited time for these athletes to respond (on the field),” Wylie said. “We get them in this really tight athletic range where your reaction should be between 240-380 milliseconds. It seems like a tight window, but that’s where 98 percent of athletes should be.”

This particular assessment dates back to the 1970s and has been one of the most studied measures of attention in the history of cognitive psychology. It intends to reveal which athletes can operate with extreme levels of focus.

Think of a cornerback who can lock in on a receiver — or perhaps get beat for losing focus for any minuscule matter around them. With quarterbacks, consider the way Tom Brady kept his cool through so many game-winning drives on the biggest stages.

Future Hall of Fame quarterback Tom Brady always seemed to stay cool under pressure. (Focus on Sport / Getty Images)

Or in baseball, a batter’s concentration could stray with a runner leading off second base and the shortstop hovering behind. And similar to knowingly hitting the wrong buttons on the test, a batter can often recognize immediately that he swung at a pitch out of the strike zone, even before the ball hits the catcher’s mitt.

“That instant thought, you have an error-monitoring system that has detected, ‘Oh, that’s not right,’” Wylie said. “You know it almost instantly when you hit it. It’s an incredibly important (cognitive) system with monitoring and focus.”

GO DEEPER

Introducing the S2 Cognition test that helps predict NFL quarterback success

S2 conducted an on-field study with a high school football team. The quarterbacks initially threw passes to a receiver without any defenders on the field. Then they instructed a linebacker to run in a random direction but not to interfere with the play, and they measured a 20 percent biomechanical efficiency hit for the quarterback. That’s what just a little distraction can do.

Now, intensify that tenfold, and a quarterback with poor distraction control wouldn’t be as accurate of a passer in a collapsing pocket or with hands in their face, and it would have very little to do with their physical traits.

“Some people’s brains can just lock in their attention so well and really minimize the impact of distractions,” Wylie said. “Some brains can block out those distractions. Other brains can get hammered, and they’re double tapping through the whole test.

“You know those quarterbacks who are steely focused in the pocket even as it’s collapsing and executing with incredible accuracy. A lot of it has to do with the way they are shielding all that competing information that’s trying to bombard your system.”

GO DEEPER

NFL Mock Draft 2023: Our beat writers pick 5 QBs in first round with Bryce Young No. 1

Part Three: Perception speed

A diamond with a missing corner quickly appears on the screen. It’s gone in a flash. The test-taker must identify the missing corner.

Again, the rate of speed is incredibly difficult to comprehend for the non-athlete.

“Your brain is better at this than your conscious awareness,” Ally said. “So you can say, ‘Man, I don’t think I saw any of them,’ and, really, your brain saw enough of them. What we’re getting down to is literally 16 one-thousandths of a second is the fastest one in which that corner is missing. It was once thought the brain couldn’t see that, but it’s the marker for where the delineation is for subliminal.”

Wylie said about five to six percent of the pro baseball players tested have declared they saw every single one. And they were correct.

This can be the difference for a solid hitter who can visually determine the spin on the stitches of a baseball or even a pitcher’s arm angle that can tip the pitch.

“It’s unbelievable what an advantage that is,” Wylie said.

If a cornerback has slow perception speed, a coach might back him off the line by an extra step or two in order to give him that much more time to diagnose the play.

“It’s not about the simple arrows,” Wylie said. “It’s about the system we’re measuring underneath that is helping you accomplish something at an incredible speed.”

GO DEEPER

Feldman's NFL Draft confidential: Coaches on 'Draft Dinging' Stroud, weak WRs, sleeper QBs

Part Four: Spatial memory

This is a visual memory test. Can you stare at a number of objects for a few seconds and put them exactly where they belong after they all disappear?

“This is classic visual representation, and that forms the fundamental basis of our memory,” Ally said. “For here, it’s not like you’re learning something with the intent of learning it for a long period of time.”

This is important for a quarterback who is reading the field after the snap. A high-scoring passer can take a quick snapshot of one side of the field, scan the other way and visually map out everything else happening out of sight, giving life to the phrase, “eyes in the back of his head.”

Conversely, those who struggle with this test might be poor two-dimensional learners. Say, for instance, they have trouble applying the playbook or sorting through a film review and then applying that knowledge when they get to the field. That particular athlete would do better learning new plays in a walkthrough setting, which would be important information for a positional coach who can take that extra time.

“Going onto the field, now they’re in 3D space, so those guys may have a little bit of difficulty with alignment where they’re supposed to be and how pieces operate in the spatial awareness piece on the field,” Ally said. “So they’re going to learn better with the walkthrough style, and not only the classic walkthrough style but speeding it up to game speed so they know how all of the spatial pieces are going to fit together.”

Part Five: Impulse control

This task, which dates back to the 1960s, triggers a strong reflexive impulse to react that is hard to control based on following the visual cues and the rules on the screen.

Some athletes have a strong ability to control their reactive impulses by taking an extra moment to diagnose what’s unfolding rather than falling into a particular trap such as play-action or misdirection or missing a tackle because the ball carrier juked. One pass rusher who tested with poor impulse control was said to be deep into the backfield before ever recognizing the play-action.

“The first action that is activated in their mind, it is right next to that point of no return,” Wylie said. “They have a hard time constraining and containing those fast impulses.”

It’s the same cognitive wiring as sitting at a red light and jamming on the gas when the next lane gets the green turn arrow. This measurement has also been a strong predictor of chase rates for hitters who stray too often out of the strike zone, and there are methods to train them to improve in that area.

For a quarterback, it’s about assessing a coverage when a defender might try to sell a certain zone look before falling into another assignment. A QB with poor impulse control might perceive to see an open receiver and force an early throw without ensuring another defender isn’t hovering in an underneath zone. On the other side of the scale, a quarterback who is too cautious with the test might hold the ball too long and take unnecessary sacks.

“(The test) has been one of the best predictors of interception rates,” Wylie said. “Guys with low impulse control, especially under pressure, they’re going to pull the trigger prematurely.”

GO DEEPER

2023 NFL Draft odds: Will Levis moving up the charts

Part Six: Tracking capacity

This test measures a person’s ability to track several specific targets at once while there are distracting elements around. Compare it to driving through a neighborhood with stop signs, kids at play, dogs running around and a ball rolling into the street.

Each trial lasts three to five seconds, or about the length of an NFL play, and the objects move at the same rate as players on the field.

Though the average human can track three to three and a half objects at a time, the S2 pushes the limit to six. Safeties have proven to be tremendous at this test because they’re equipped to monitor the quarterback, a running back coming out of the backfield, slot receivers and tight ends, among other responsibilities.

“Some safeties in the NFL can get all of them right all of the time, which is quite amazing compared to what we see in scientific literature,” Ally said.

Sticking with the safety example, this test also shows how a player can view the field through a wide lens, narrow the scope and broaden it again without losing focus on their responsibilities. If a safety or linebacker scores poorly on this test, a coach might opt to restrict their assignments, such as turning a free safety into a strong safety or a three-down linebacker into a run-down linebacker.

Or for a quarterback, if he can’t make full-field reads, an interested team would need to tailor the offense to his strengths — or simply determine to draft someone else. This test, according to S2, has been a strong predictor of completion percentage at the NFL level.

“Guys with lower scores have a little bit of tunnel vision, so putting them in a position where they have to really broaden their attention can be a struggle for them,” Ally said.

GO DEEPER

As top QBs speak before NFL Draft, Bryce Young feels like the Panthers' pick at No. 1

Part Seven: Instinctive learning

Retired Saints quarterback Drew Brees posted one of the highest scores ever measured on S2 Cognition’s “Instinctive Learning” test. (Butch Dill / Associated Press)

This one is fascinating because the test-taker is essentially provided no rules for this trial and must figure them out on the fly.

It measures a person’s gut instinct. Former Saints quarterback Drew Brees posted one of the greatest scores they’ve ever measured. The test shows a player’s ability to act and react in situations where the results are changing.

“It’s that intuitive sense of knowing something and identifying it even though you can’t put your finger on it,” Wylie said. “Some people are really good at noticing the nuance, the less obvious.”

This is important, particularly for quarterbacks, because defenses try not to give the same look in consecutive weeks. So a team can prepare for a game for days, but they’ve also got to recognize how to adjust once the game starts.

If Patriots coach Bill Belichick presents the same pre-snap formation but changes the coverages after the snap, how quickly can an offense adapt? One series, two series, longer? That can often be the difference between a win and a loss.

“The instinctive ability to notice things, to pick up and adapt on the fly is a really powerful way to capture the (cognitive) system that we all appreciate but don’t talk about,” Wylie said.

Part Eight: Decision complexity

This is about mirroring movements and opposite movements. Take the rules of the test, apply them to a color on the screen and react accordingly as quickly as possible. Move too casually, and the test-taker will fail this portion.

“It’s probably in our database in football one of the most important skills across all positions and probably, if I had to guess, across all sports,” Ally said. “It’s the separator from all of the other (tests) out there, the Wonderlic and AIQ that really get at the pre-snap, how much can they absorb, how much complexity can they understand.

“But S2 is on the other side. Once the ball is snapped, how are they going to execute? It’s a very different approach in how you understand an athlete and how you work with an athlete. I think the default (recourse) is if a kid shows any hesitation or is inefficient in his play, let’s go back to the classroom and learn the rules again. The kid may know the rules very well but just has difficulty executing them.”

This test can apply to “if-then” rules within the playbook. For example, if the tight end stays in to block, the strong-side linebacker should rush. If a cornerback blitzes, a running back should stay in the backfield in pass protection. If a quarterback recognizes a deep zone, throw it short. If a slot receiver diagnoses man coverage, extend the route.

These are easy concepts in the meeting room, but can a player identify and execute under stress in real time?

The necessary context behind the S2 test

Players can study and prepare for the Wonderlic because it’s a standardized test. Ally and Wylie are confident that’s not the case with the S2 Cognition test because it’s more of an evaluation into the way a person thinks and processes information in real time. They said there’s typically no significant changes in an athlete’s score if they’ve taken the test multiple times.

Of course, there are important factors of preparation, such as being well-rested at the time of the assessment. If they aren’t, that context is important, too.

Ally and Wylie also are adamantly against leaking an athlete’s scores. Because they are under contract with 15 NFL teams, those teams legally own the access to the prospects’ results. Why give away something for free — and without context — when their clients are paying for exclusive access?

The context is also paramount. Ally and Wylie have spent hours, if not days, with teams to discuss players’ S2 performances. The evaluation isn’t as much about the number as it is the reasoning behind it.

"The list of @S2Cognition scores that I have seen aren't accurate at all..

At least two of them aren't accurate and those scores don't have context" ~ @brandonally3 #PMSLive pic.twitter.com/N7VGCrmpvc

— Pat McAfee (@PatMcAfeeShow) April 24, 2023

Plus, of the eight segments of the test, each position carries increased importance in about four of them. So if a player does disproportionately well in the four less important segments and poorly in the heightened categories, a leaked overall score will carry absolutely no value. For example, if a quarterback scores in the 90th percentile overall but fails in tracking capacity, he probably wouldn’t be a good fit for certain offenses.

After all, how many times have good players been drafted into bad situations or poor fits and lost a chunk of their career as a result?

“We don’t just drop data on our teams,” Ally said. “We work with them to understand the player, and we try to help the player match. It’s not just reducing the bust rate for the team. It’s reducing the bust rate for the athlete. As an athlete, don’t you want to go to a situation where you’re going to have a higher rate for success? That’s why these scores that are being leaked have zero (context).”

The S2 Cognition test, in their own words, is meant to serve as one piece of the puzzle. At some point down the line, Ally and Wylie believe they’ll have enough data to predict a player’s success rate in the NFL, as they’ve already uncovered in professional baseball.

They’ve heard a few NFL general managers say they’ll consistently match up the S2 results with the player’s performance on film. Teams naturally want to understand how to apply these testing results in an appropriate way.

Still, it’s complicated — because the brain is complicated. And the S2 Cognition test is beginning to help teams learn how these players are wired to react on the field.

“These (tests) are engineered and designed to capture those brain systems that they use in those split-second, rapid timeframes,” Wylie said. “We’re not messing around here. We’re pushing them to the brink of what humans can do and process.”

(Top photo of C.J. Stroud: Michael Reaves / Getty Images)

ncG1vNJzZmismJqutbTLnquim16YvK57k21sbmxkaXxzfJFsZmlsX2eEcLrFpWSdqpGbwW6%2FkWaaqJ%2BensGqu81mmqNlo6m%2FsMHDZpmrsZOaerq71KeeaA%3D%3D