This is Part 1 of a two-part question-and-answer series. Coming Thursday, for subscribers: Part 2, where I answer a slew of questions on draft strategy. (Not a subscriber? Sign up for a one week free trial.)
Behind the Scenes
What is draft day itself like for the front office?
Draft day can be divided into two parts: the calm before the storm and the storm itself.
On the day of the draft there are still trade talks between GMs and possibly some last-minute internal debates to be resolved, but for the most part front offices are done with their big-picture work. The draft board is set and the general strategic questions are put to bed. Often owners will come to town for the draft, and GMs will meet with them to walk through their thoughts and goals for the night, including the prospects they are targeting.
For the rest of the front office then, it’s just kind of … waiting. There’s a nervous energy providing an undercurrent, but it’s still mostly anticipation. There isn’t much actual work left to do to feel productive, but you know what’s about to happen in a few hours.
During that calm period there is usually time for members of the front office to grab some of the catered food and fuel up to prepare for a time when they have to be as sharp as possible. GMs, though, are often tied up with ownership most of the day and may not even get a chance to eat. I saw multiple GMs basically fast through the draft because they just didn’t have the time to grab a bite.
(As you could imagine, this is probably not great for decision-making performance. One of the weird aspects of the draft is that it is essentially performance for the GM and front office, like an athlete performs. And yet many of the highest-ranking officials don’t sleep for a while leading up to the draft and don’t take care of themselves on draft day. They’re working, of course, but what would we think of a player who stayed up late the night before Game 7 to work out and put up shots? We’d think he was foolish for doing that work instead of getting himself as prepared as possible to perform. It’s not an exact analogy, but I often wonder why we don’t think more in those terms.)
The day was slightly different for me when I worked for a West Coast team (Portland) compared to an East Coast team (Philly). In Portland, we’d start our draft day very early, but the draft is at 4 p.m. local time, so the night would be done by around 10 or 11 p.m. (including press conference, recap with colleagues, etc.). Add three hours to that timeline, though, in the Eastern time zone — after my first draft in Philly we left the office around 2 a.m.
Regardless of the time zone, I’d be exhausted by the end of the draft, because once things start, there’s not much of a break. Even during picks that have nothing to do with your team, you’re locked in, anticipating, thinking about how the draft could unfold and if it is creating opportunities for trades. Multiple members of the front office might be working the phones, taking the temperature of other teams on deal concepts. You might have conversations as a group as things unfold. The GM might ask the room questions like: “If this guy falls to us unexpectedly, how excited are we about him? Would we want to try to trade back or out?”
As it gets nearer to the pick, the anticipation heightens. Everyone in the room has an idea of who they want, and they are constantly eyeing the draft board, praying their guy falls to us. The moment the team ahead of you picks, there’s a release of tension. Whether good or bad, at least you know who’s left.
When your team is on the clock, a representative from the league office calls your draft room on a dedicated line and keeps you apprised of how much time is remaining on the clock. You tell the representative who you’re selecting once you’re ready to lock in the pick. But even if teams know who they will pick, they rarely submit the name early. They’ll wait to see if they get any calls, or maybe make some outbound calls just to see if anyone they had talked with in the past about the pick is still interested. So once again, there’s waiting.
It’s interesting to think that during what seems like the most important time of the draft, when your team is on the clock in the first round, there’s often the least activity. If you like who’s on the board, you’re not making calls to others. You sit and wait in silence, silence sometimes broken by a phone ringing, and a terse conversation with a GM on the other line. (Usually the offer isn’t any good, and from my perspective would sound like: “How are you doing? Good. No, we wouldn’t have any interest in that. OK, good luck.”)
The second round, though, is much more frenzied because of the compressed time frame. Teams are selling and trading picks left and right, selections are coming in so quickly that sometimes it’s hard to keep your internal draft board completely in sync with what’s coming across on the league’s software. It can be very chaotic.
Sometimes teams have traded away their second-rounders before draft night, though. In that case, it can be relatively calm. Owners might leave after the first round, the interesting part of the night (from their perspective) concluded.
After the draft the GM will hold a press conference and answer questions for the media. My first few drafts I wasn’t invited into the draft room during the draft — I watched on TV with other employees from a room down the hall. But when the GM left the room to do media (sometimes even before the last few picks of the draft) I was allowed to go in. Papers would be scattered everywhere, whiteboards marked up, empty soda cans and water bottles dotting the table — one could only imagine what it was like inside during the action.
Front office members are still pretty wired from the adrenaline rush of the night, so this also was a good time to informally gather and rehash what happened — both what occurred in the room itself and around the league. And then it’s time to head home, get some rest and relax a little before preparing for free agency, which is right around the corner.
What influence does a coach have on the draft and how does he work alongside the GM in terms of scouting and drafting? I’m sure this varies by organization, but do higher-respected and more-tenured coaches (Stevens, Pop, Spoelstra) have strong influence on drafting certain prospects?
As you mentioned, this is undoubtedly different based on the organization and the personalities involved.
In general, the healthy organizations have coaches that understand what Warren Buffet calls their circle of competence. They know what they know and what they don’t. They have healthy dialogue with the GM and front office about what they feel the roster needs, the current players’ strengths and weaknesses, and their impressions of the players who came through for workouts and interviews. But they also know that they haven’t spent the year studying these players, and they defer judgment to those who have.
Depending on the power dynamics, coaches may be able to more strongly influence how the front office weighs various aspects of a player. Perhaps the coach favors players who grade out better on certain personality traits, even if they don’t have the most physical ability or skill. He might push hard for the front office to value those traits similarly, and thus shape their evaluation process.
But there are reasons why coaches who have had control over personnel have struggled, historically, as I wrote recently.
How much involvement does the strength & training/biomechanics and the player development staff have in discussions of player evaluations? Do they have some input on how the team evaluates players when big boards are being made? Is their input limited to just those players brought in for draft workouts? I always thought that those departments should be included in the player-evaluation discussion given how, for example, they might be able to suggest a very likely path to increased power in Miles Bridges’ left leg that in theory could drastically reduce the low [free throw] rate issue with him.
As with all of these questions, it will vary by organization. Some organizations will really listen to what this staff has to say, while others won’t even seek out their opinions. But, as you note, I think there’s valuable information to be gained by consulting with the strength staff to understand the biomechanics of players and what is or is not easily alterable.
That said, you also have to be careful with this information. There is an optimism bias here, in the same way that a shooting coach is probably not going to tell you he won’t be able to help a player’s shot — this is what they do, so they are likely to overrate their ability to fix the given problem.
How much film do GMs watch on each player? Have they gone through every single game of 60-to-70 prospects? Or do they mostly rely on others to do that for them?
This depends on the GM, but many GMs do a lot of their own watching as well as listen to their scouts, and try to determine how to blend the two. The GMs will naturally focus on the higher-priority prospects, and do less of their own due diligence on second-rounders or potential undrafted signees. This is natural — if they’re going to stake their reputation to a high pick, they want to make sure they are comfortable with him from a scouting perspective.
(I also want to note that nobody goes through every game of 60 prospects. Let’s do some simple math: let’s say you wanted to watch 25 games of 60 prospects. That’s 1,500 games to watch. It takes me about 2 hours to seriously study one game of tape. So let’s say that’s roughly 2-3 thousand hours of time. Many of those games will have overlap with multiple prospects, but that makes watching a game take longer. So to do that from the start of the college season until the draft would require roughly 300 hours per month or about 10 hours a day of straight tape watching. It’s not feasible for one person.
A full staff might be able to do that, though usually there are strong diminishing returns on additional games beyond a certain number. More likely what happens is an evaluator will watch a handful of games, a mix of live and on film, and then supplement with video from services such as Synergy for more targeted film watching.)
And yet I’ve often argued that, perhaps counterintuitively, GMs actually should do less scouting. In a well functioning organization, a GM needs to be an executive, not a scout. They should be hired and put into that role for the ability to synthesize the information they are provided and turn it into a decision (as well as to set strategy, hire, create culture, negotiate, and be a face of the organization, among other duties). They should be able to hire good player evaluators that they can lean on, rather than feel responsible to do all of the player evaluation themselves.
I’m curious about how much “non-basketball” factors influence team choices in the draft. For example, it’s been reported that the Kings may draft Marvin Bagley III in part because he’s the rare player who showed interest in working out for them. Also, the storied relationship between Mark Bartelstein and Jerry Reinsdorf has been credited for a number of past Bulls draft picks/FA activity, including this year with Michael Porter Jr. How much does the behind-the-scenes mechanics of agent relationships, player workout/medical info availability manipulation, etc. play a role in player selection? Is it more a tie-breaker in cases where you are weighing players with similar evaluations or can it play a much larger role than that?
It’s hard to talk about this one generally — it’s very different depending on the team and decision maker. Some GMs will value this highly, and some will disregard it entirely.
The biggest way these kinds of relationships can impact draft status is through information sharing, as you mention. Withholding or providing medical information or workout access can influence decisions because some GMs might not be comfortable feeling like they’re flying blind on a very important draft selection.
There’s also the potential for GMs to be affected by this kind of thing subconsciously. I’ve written in the past about all the cognitive biases at play in workouts. This is true in the draft process in general — if evaluators aren’t careful, they can easily be swayed by factors that shouldn’t matter. If an evaluator is predisposed to like a player based on a prior relationship or the player wanting to be drafted by that team, their judgment may be clouded, even without being aware of it.
How does a front office evaluate trade offers from teams in such a short time period between picks? Are trades discussed between teams prior to draft night based on situations and if/then possibilities? If so, how do you not show your cards to interested teams by conducting those conversations?
As I wrote earlier, many deals do not happen on the clock. Teams are calling each other in the days before the draft, taking their temperature on various types of deals. They’re aiming to discover team goals: Are they looking to move up or down, in or out of the draft? If it seems like there could be a match in each team’s goals, they might expand the trade talks and get a little more resolution on what a deal might look like. They’ll put the deal on their trade board, and keep it in mind as the draft unfolds.
Because those conversations have already happened, that makes it easier to make a call during the draft and execute a trade. If it seems like the deal is of interest to one side based on how the draft has gone, they’ll reach out to the other a few picks ahead of when a deal needs to be made and re-engage and make sure the interest is still there. If it is, they can finalize the deal contingent on a certain player being available (or not available) and only consummate the deal when the pick gets there.
There are times, then, that a trade happens when a team is on the clock, but these trades are rare because they have to happen in such a short time period. A second-round pick might be sold this way. Usually teams have registered interest earlier in the draft or even in the preceding days, saying they’re looking to buy/sell picks. But because the second round moves so quickly and teams buy or sell depending on who’s left on the board, the actual transaction—including fielding calls from multiple bidders and deciding on a selling price—can happen while the team is on the clock. Which, as you can imagine, makes for a frenzied minute.
As far as not wanting to reveal information, as with any trade deal, you try to walk a fine line. You don’t want to reveal too much, but you have to reveal something to determine whether a viable deal exists. In practice this mostly means teams saying that they’ll make a deal if a certain player is/isn’t available at their pick, which allows them to make the deal contingent on a player without telling the name of that player to another team until the deal is done.
I find that a lot of draft analysis focuses too much on what a player can do right now, when the real question is what that player will be able to do after X number of years in the league. Players improve after getting drafted, some more than others, and this improvement is key to their success in the league. How do teams evaluate a player’s ability or likelihood to improve?
This is in many ways one of the most important questions in draft evaluation, and there are no good answers.
I wrote in my midseason review of Deandre Ayton:
Imagine arranging all a player’s qualities on a spectrum: on one side are the ones that are least likely to change, like height. On the other end are those that are easiest to change, like experience (it’s just a matter of playing time). We can take everything else about a player — shooting, conditioning, shot blocking, weak side defensive recognition, handle, lateral quickness, et al. — and try to put them somewhere in between. Where we put each characteristic tells us how much to value it when we scout.
For example, if we think one of the easiest things to change about a player is adding muscle and strength to a young, skinny player, we might look the other way when we see a skilled beanpole get bullied. But we might be more concerned if that prospect consistently misses open teammates.
Creating this kind of scouting spectrum isn’t easy, and often causes significant disagreements when discussed. It’s also hard to verify statistically, since, while some of these characteristics can be measured, many are much harder to track. But this spectrum tells us a lot about how we project players…
That is a lot of what you’re talking about. How do we determine what can change for a player?
A lot of this is mental, of course. To try to get a sense of that, teams do a lot of background work. They’ll have psychologists evaluate prospects or put them through other tests. They’ll ask questions of them or test them (clearly or subtly) during workouts to try to gain information about how they think about the game and how quickly they learn.
But all of these measures are imperfect. I believe this will be an area that teams will continuously find ways to improve upon, but it will still likely always remain a challenge. It’s just not easy to forecast the future of teenagers, especially as their environments and circumstances undergo enormous changes.
Each NBA team invests millions of dollars into background checks of potential draftees. At the end of the day what are the questions they are trying to answer? Obviously they’re looking into criminal history, medical information, but is there a common question they’re trying to get at that sits at the core of their searches? Do any teams’ searches differ radically than their peers?
I obviously don’t know what all teams do and if anyone is doing something really different, they’re probably keeping it closely guarded.
But most teams are really trying to dig into who a player is as a person. They want as few surprises as possible. So they try to find out about everything they can regarding that player’s personality and past.
There is certainly differentiation in teams’ ability to collect information. But as I wrote about Michael Porter Jr. last week, the gaps there are narrowing across the league. What really matters is how teams factor it into their decisions, and that part is much more difficult.
It’s one thing to discover a player had academic issues, or that his brother was arrested, or that he was a trouble-maker in high school, or that he was hard to motivate to come into the gym while in college. But how much does that matter relative to all of the other information you have on the player? How predictive is it of future success? These are still really big unknowns.
Looking at the draft records of GMs sometimes reveals a consistent pattern of how they value this information. Some teams will take players off their draft board if they feel like those players don’t have the character they are looking for in their franchise; others will roll the dice on talent even if the player has a checkered past.
Which viewpoint is right is something that can only really be answered over a large sample. It requires studying the history of the draft and seeing what has tended to matter over time. Unfortunately, these variables are hard to capture properly historically, which makes it difficult to run any kind of analysis. That doesn’t mean it’s impossible, it just means it’s a real challenge to attack with the tools we use in other areas.
Analytics
Do teams put any weight on predictive analytics on a player’s demographics/upbringing? For instance is a player raised by a single mom more likely to be an All-Star versus a player raised by a single dad or a player raised by both parents? Or does this get bundled up into the broader upbringing/“work ethic” evaluation?
Talent evaluators certainly have their theories as to what types of players perform better or worse, which includes demographic components that you mention. But this data is not easy to collect properly, and therefore is hard to feed into a model. That doesn’t mean teams haven’t tried, just that it’s likely not widespread.
There’s also the question of whether teams should pay attention to these aspects. Michael Lewis’ latest book, The Undoing Project, starts with a chapter about the Houston Rockets’ draft modeling efforts, which includes the tidbit that the Rockets have tried to collect this data and include it in their predictive models. The chapter suggests that it mostly has not been helpful.
If that’s true, it might be because these variables are already captured in other aspects of player performance that we have data on. For example, perhaps guards from Seattle have a penchant for scoring much better than would be expected (Isaiah Thomas, Nate Robinson, Aaron Brooks, and Jamal Crawford are all from Seattle). It’s likely that ability is already clear by the time they enter the draft. The real question is whether being from Seattle is a separator once they get to the NBA above and beyond what we already know about them. That’s a much higher bar for demographic data to clear.
How do analytics and scouting departments collaborate during the draft? Does each department create their own rankings of prospects, then a middle ground is found?
Between scouting and analytics departments preparing for the draft, I am curious in your experience how these departments work together? Do they work in concert together or separately? Is there a person that works between both departments to combine their work into useful insights for the GM to use or does each department submit them separately?
How do you merge subjective and objective scouting/information to come to the best decision?
This differs by team, as you’d expect. It’s also a significant area of differentiation amongst teams, and therefore a potential edge for the teams that do it well. As I wrote in my recent Michael Porter article, the draft is about how you integrate information into the decision-making process, and statistical information is one such type of information.
Ideally, this happens by having people in key roles (whether the communicators of the analysis or the decision-makers themselves) who understand the statistical analysis process, what it might miss and what it might see that traditional evaluation would miss. As I wrote last year in A Roll of the Dice, Part 2:
In his opening chapter [of The Undoing Project] Lewis describes how Rockets’ GM Daryl Morey came to [a] realization after a mistake made in the draft, one where Morey felt he relied too much on the results of his draft model. “And thus began a process of Morey trying as hard as he’d ever tried at anything in his life to blend subjective human judgment with his model. The trick wasn’t to just build a better model. It was to listen both to it and to the scouts at the same time.”
“At the same time” is the key here. It’s not about just trading off between listening to one or the other. The two work hand in hand: experts understanding the strengths and weaknesses of the data, the data employed to mitigate the weaknesses of the experts.
In practice, though, this is often not how it works. Some teams might have their analytics department present their findings and then that’s the extent of it. They’ll consider it if it backs up their opinions, discard it if it conflicts, and therefore it has essentially no impact on the final decision. Others might do something like you suggest, adjusting the draft board based off predictive models. (It is very unlikely that the statistical data would carry equal weighting with the scouting rankings, though.) Still others might just view it as another input to the process, in the same way they look at medical information or background research.
There are a variety of approaches. For the most part this is still something the league is figuring out.
What, if any, are the most common data points that analytics teams provide (and find compelling) that are commonly distrusted/dismissed by GMs for whatever reason during the draft process?
I think the more divorced analysis is from traditional basketball evaluation, the more it’s distrusted. If an analyst tells a GM that they fed all of this data into their model and it spit out a list of the best players, it’s much more likely to be disregarded. GMs are making enormously consequential decisions — as I wrote above, it’s hard for them to even fully trust other scouts speaking their same language. So a statistical model presented without any other justification beyond “you have to trust it” is rarely going to be given much weight. (This is certainly a strike against so-called “black box models” that are becoming more popular with machine learning, where it is hard to get any kind of explanation as to why the model gives the results it does.)
On the other side of the spectrum, I often found that more simple information that could help inform evaluators was often greeted with enthusiasm rather than skepticism. Data on rebounding percentages, finishing ability, passing ability, etc. all helped inform a scout’s evaluation and could be contextualized by that scout. “Ah so his assist rate is high which is usually an important marker but I watched all his passes and he played in a system where he got a lot of very simple assists because they made him a distributor in their offense.” This information might be harder to use in a purely predictive sense, but could actually be more impactful because it is more easily integrated into the traditional evaluation process.
Check out Part 2 (for subscribers), where I dive into questions regarding the draft strategy of teams. How do they decide on selecting undervalued players? How do they think about mock drafts? Should teams draft the best player available or care more about fit? And much more.