“It was that bad, huh?”

“Oh yeah. I mean a guy who’s probably not even going to be drafted just kicked his ass all up and down the floor.”

“Sheesh.”

“And the interview was just as bad. There were multiple times where we asked him questions we already knew the answers to, and he lied to us.”

“Wait — seriously?”

“Yep.”

I was at a loss for words. “Wow. Just…wow.”

I hadn’t been able to attend the previous day’s draft workout, which had featured a high-level prospect, so my coworker was filling me in. This prospect had talent but also had numerous flaws — all of which, it sounded like, the workout and interview confirmed. It would have been easy to write the player off and not think about him again, but my coworker wasn’t ready to give up so easily.

“You know,” he said to me at one point in the discussion. “I wonder if it’s actually better that you missed the workout. I mean, none of us are going to be able to get this out of our heads now no matter how hard we try. But for you it’s like it never happened. Maybe that gives you a better perspective?”

That prospect has since become a very good player. He overcame the negatives in his game and personality that were exposed in the harsh light of his visit to our practice facility and has easily outperformed his draft slot.

Welcome to the world of pre-draft workouts, a season about to hit full swing: where a team finally gets control of the draft process, where they can learn about a player in ways they couldn’t otherwise — and where attending them can sometimes be worse than not seeing them at all.

The Power of Control

Team draft workouts have become a fixture of the pre-draft process. Scouts have watched players in a variety of settings up to this point: in high school all-star events, in college or international practices and games, and in group workouts like the Draft Combine. But the only way they can get a truly customized look is when the prospects come into their building in June for a workout of the team’s own design.

There are still some restrictions based on a combination of league rules and negotiations with the agent. The league mandates that prospects can, at most, go 3-on-3 in these workouts. No 5-on-5 play is allowed. And agents often have specific demands. They might not want their clients in a workout against a specific other prospect as a matchup. They might not want them in a workout where they play against anyone at all: one-on-none workouts are becoming more common. They might not want them to even work out, and only agree to let them come in for a visit.¹

But despite these constraints, draft workouts remain the most control teams have over their scouting of players. All of the other scouting opportunities are run by someone else. In a draft workout the players come into the team’s building and follow an itinerary of the team’s creation.

A typical workout for a relatively high-level prospect might go like this: the player flies into the city, maybe gets checked out by team doctors and then goes out to dinner with the GM or coach or other members of the front office. The next morning they’ll go through any athletic testing or measurements the team wants them to (e.g. standing reach, vertical leap, or bench press) and then get warmed up and ready for the workout. They’ll typically be set up with other players the team wants to see, often players picked specifically to see how this main prospect plays with or against them. The players will work their way through the drills that the front office and coaching staff choose for them, some of which might be more typical drill work (e.g. shooting coming off a down screen, or crossover combination at the three point line into a drive to the basket) and some of which might be loosely structured live play (e.g. playing 3-on-3 out of a specific action to start the possession). They’ll shower, grab some lunch, and sit down with other members of the staff or anyone else the team wants them to talk to (e.g. the owner or the team psychologist). After that they head to the airport, ready to do it all over again in a different city.

This is the blueprint, but the power of the workout is that teams can design them however they want.² Maybe there are certain measurements a team doesn’t have on a player — they can now get them. Maybe the team is looking at a big man who’s rumored to have a good jump shot but whose college coach didn’t let him shoot — they can put him through a series of different shooting drills. Maybe they’re looking at a player who seems like they struggle finishing against length — they can match them up against a bigger player.

This is the allure of the draft workout: the chance to get at information that would otherwise be inaccessible. It is about the power of control. A power, teams hope, that will help them avoid costly missteps and uncover hidden gems.

Uniquely Dangerous

Anyone who’s worked in the NBA for any significant length of time has heard dramatic stories of workouts that completely threw an organization off. If they weren’t part of a story like that themselves, they’ve at the very least seen first-hand a workout that was not at all representative of the player that prospect would become. The cautionary tales abound. There is no executive, then, that believes a workout is the be-all-and-end-all of a decision. Different decision makers place a different emphasis on how much the workout matters to them and on how much they believe they can learn from the process, but I haven’t met anyone who thinks that it should be anywhere close to the sole basis for their opinions.

Yet, even with this awareness, workouts remain the riskiest part of draft evaluations. Their very nature makes them incredibly difficult to properly weight. Every piece of information in the draft process can be misinterpreted, of course, but workouts are uniquely dangerous for a few key reasons: samples are small, context is hidden, results are subjective, and, most importantly, it’s the last evaluation a team makes before the draft.³

The small sample size of the evidence in a workout is the clearest of these reasons. A workout is only an hour or so of play, an hour in which a player can easily have an off day or a poor shooter can catch fire. How can a team distinguish between these expected fluctuations in performance and a real change in underlying ability?

More pernicious than the small sample size problem is the issue of hidden context. There are many outside factors, having nothing to do with a prospect’s ability, that can impact their workouts. And these factors are hard (or even impossible) for evaluators to know and take into account. Draft prospects are traveling all around the country, following a demanding schedule that no doubt impacts their play.⁴ They might have differing levels of motivation in different workouts — perhaps they don’t like the city, or the roster has someone who plays their position.

Or maybe something else is wrong that the team doesn’t know about. I once watched a player, known for his high-energy style, look slow and listless. It was puzzling, until I overheard one of our interns mention later that they had spent the previous night trying and failing to find the right asthma medication for the player.

This problem of missing context is exacerbated because it’s very easy to fall victim to something called the Fundamental Attribution Error. As Malcolm Gladwell explains in The Tipping Point:

The mistake we make in thinking of character as something unified and all encompassing is very similar to a kind of blind spot in the way we process information. Psychologists call this tendency the Fundamental Attribution Error (FAE) which is a fancy way of saying that when it comes to interpreting other people’s behavior, human beings invariably make the mistake of overestimating the importance of fundamental character traits and underestimating the importance of the situation and context. We will always reach for a “dispositional” explanation for events, as opposed to a contextual explanation. In one experiment, for instance, a group of people are told to watch two sets of similarly talented basketball players, the first of whom are shooting baskets in a well lighted gym and the second of whom are shooting baskets in a badly lighted gym (and obviously missing a lot of shots). Then they are asked to judge how good the players were. The players in the well lighted gym were considered superior.

Performance is often much more about context than we give it credit for. If, as Gladwell relates, we make this mistake even when that context is something we can see (like lighting), imagine what we do when it’s something we can’t (like missing asthma medication).

This is similar to another concept from behavioral psychology, which Nobel Prize-winning psychologist Daniel Kahneman terms: “what you see is all there is”. When presented with evidence we tend to think only about what’s in front of us and construct a story from that information only. We are blind to what we might be missing, and blind to our blindness.

“An essential design feature” of the intuitive brain, Kahneman writes in Thinking, Fast and Slow, “is that it represents only activated ideas. Information that is not retrieved from memory might as well not exist.” He continues:

The measure of success for [the intuitive brain] is the coherence of the story it manages to create. The amount and quality of the data on which the story is based are largely irrelevant. When information is scarce, which is a common occurrence, [the intuitive brain] operates as a machine for jumping to conclusions.

Because of the restrictions placed on draft workouts, they can only show a subset of a player’s skills. Players who are savvy team players or great transition passers won’t be able to show off these attributes. Players who have high skill levels but don’t play defense or who are great athletes without good feel for the game will have their strengths highlighted and weaknesses minimized. The danger is that observers miss the fact that what they see is not all there is in the player’s game.

Maybe these complications could be adjusted for if there was some empirical way to measure workout performance, but in fact the opposite is true: workouts are a highly subjective exercise. It’s very common for multiple people to observe the same actions and come away with different conclusions. If experienced executives can come away with varying impressions from watching the same performance, it casts doubt on the predictive value of that performance.

The biggest problem with workouts, though, is that they are the last evaluation a team makes before the draft. Due to the recency effect, the psychological tendency to overemphasize events that happened lately, the result of the workout ends up sticking in our minds more than it should. Imagine if the NBA suddenly made a rule change and all draft workouts now took place before the college season, in October. The players performed exactly the same in the workout and in the college season, just the order of events was reversed. The workout would feel like a much less important part of the process even though the information it contained would be exactly the same.

Instead, though, it’s the last thing teams see. For all of the reasons just discussed, workouts are a breeding ground for mistakes in evaluation — the fact that they’re also, subconsciously, given more significance due to their spot on the calendar inflates the chance of error further.⁵

Put it all together and draft workouts are the chainsaws of the front office toolkit: when wielded properly they are useful — but if you’re not careful they can do lasting damage and you could end up hacking off a limb, or drafting a Darko Milicic.

Thinking About Thinking

By the time Damian Lillard left the facility after his workout, our entire staff was head over heels. We sat in the conference room, giddy and gushing over what we had just seen. One of our executives summarized our thoughts perfectly: “now that is an impressive player — and person.” The athleticism, skill-level, maturity, poise, and composure under pressure that Lillard has exuded throughout his entire career was on display that day in Portland.

Workouts are not useless. Lillard’s performance in June of 2012 shows that, despite the problems, these workouts can be an accurate reflection of the player. But getting the right information out of workouts, using them to make more accurate decisions, and avoiding the mental landmines strewn along the path is extraordinarily difficult.

Draft workouts are just one piece of the evaluation process, but they illustrate a larger point about personnel decisions: it is easy to take for granted how decisions are made, to simply rely on intuition and gut and experience — but that is also a great way to make a mistake. It’s how you end up writing off a good player because of a bad workout, or overrating a bad player because of a good one. When making these decisions you can’t just think about the basketball. You need to think about thinking.