2017-18 Answers: Plus-Minus Machines

Four players who excelled in plus-minus stats switched teams before last season. What did we learn from their performance?

I debuted Cleaning the Glass Insider last year with three articles that detailed the big-picture questions I had about the coming season. As I wrote:

There’s a lot to be excited about when the NBA starts again, but for me no part of it is more exciting than that feeling of suspense as we get ready to find out how all of the new teams, new players, and new schemes play out. We get to learn the answers to our questions.

Today, the third and final question I asked: Will the plus-minus machines keep humming on new teams?

When I asked this question before last season, I started off by introducing the pros and cons of relying on plus-minus in its various forms for player evaluation, concluding by writing:

One helpful rule of thumb: the more data, across more teams and in more roles, the more confidence we have that these metrics are picking up on something real and repeatable…But a front office, of course, can’t wait until a player’s career is done to judge whether these metrics were picking up on something real.

That led right into what I wanted to watch in the 2017-18 season:

Four players changed teams this summer that have looked much better by plus-minus metrics than their conventional reputations would suggest: Ricky Rubio, Jae Crowder, Amir Johnson, and Patrick Patterson. For each of them there are reasons to think the plus-minus is picking up on something real, and for each there are reasons to be skeptical.

That all four players changed teams in the same offseason provided a nice test. Decision-makers who might be swayed to acquire a player because of his strongly positive plus-minus (or high rating in one of its adjusted flavors, like ESPN’s Real Plus-Minus) could watch these cases and see how much they could count on that performance continuing. All of them had a good-sized track record, across multiple teams and roles, to suggest that their plus-minus was picking up on something real. Now we could see how robust it was.

It didn’t take long for the alarm bells to start going off. A few months into the season, all four players’ plus-minus metrics had gone in the opposite direction. It was only four players, all of whom had reasons that might explain the drops, but it was notable that not one of those plus-minus machines had continued apace.

Does this mean plus-minus is a junk stat? That we can’t trust it enough to make decisions? Not at all. As we’ll see when we dig into each player more deeply, there are reasons behind the changes, some of which actually work in the metric’s favor. But these results do show us how cautious we have to be with plus-minus, particularly as contexts change around players.

Kevin Pelton touched on this recently when he mentioned in his 2018-19 season projections that he tweaked his methodology to weight plus-minus less when players change teams, citing a study by Andrew Johnson at Nylon Calculus.

It’s not just changing teams that we should pay attention to, however. Any change in context can make an impact: a different coach; large roster turnover; a new role. As I wrote in my original article:

Depending too much on plus-minus can lead to mistaken evaluations, for two main reasons: basketball is random, and basketball is contextual….Who a player shares the court with, who they go up against, and how the coach uses the player — all of these make a big difference in the team’s results when the player plays, potentially drowning out anything the stat can capture about the player’s true impact.

The performance of these four players captured exactly this weakness in plus-minus metrics. Their context shifted, as did their play, and their plus-minus reflected that.

Ricky Rubio

Andrew Johnson’s study found that Real Plus-Minus was less likely to change when stars switched teams than when other players did. This could be because stars’ performances are based less on context: LeBron is LeBron no matter where he plays, the theory goes, while Tristan Thompson looks a lot better when he’s running alongside LeBron. It could also be because stars, when they switch teams, are not asked to change their games to adapt. The team adapts to them.

Rubio may have been a plus-minus star in Minnesota, but he didn’t have the reputation that would suggest that a team should adapt to him. So his game changed in Utah: He became more of a scorer, taking the most shot attempts per minute in his career, and posting his lowest assist-to-usage ratio by far.

He also played often with two non-shooting big men, with Derrick Favors at power forward and Rudy Gobert at center. That required a large adjustment on Rubio’s part and a more congested paint is perhaps the reason he drew shooting fouls at one of the lowest rates of his career and went to floaters and runners more often.

How could Rubio, whose shooting has always been a question mark, produce effective offense with two other non-shooters on the court? It took time for the Jazz to figure it out. But they did. As I wrote in March, before Gobert went down with an injury, the Jazz just could not score with all three of those players on the court. But by the time Gobert returned from his injury, something was different. After that point the Jazz were remarkably effective with the Rubio/Favors/Gobert trio, scoring much more efficiently while continuing to post a very solid defensive mark.

So, as the season went on, as Donovan Mitchell shouldered more of the scoring burden, as Rubio’s three-point shot started falling at a higher rate (he made 39% of his attempts after a poor first month), as he developed more familiarity with his teammates, those plus-minus numbers changed. If this theory is right, if it really was just a matter of fit and familiarity, then this coming year we should see the strong plus-minus differentials that he posted in past years.

Much of the criticism of Rubio, though, has not just been about whether he helps his team’s point differential, but whether he helps his team win. As I wrote last October:

Critics would say that Rubio’s impact reverses in key moments late in games, that the Wolves’ streak of winning fewer games than their point differential would suggest is due to Rubio’s flaws being much more exposed in the harsh light of crunch time.

We don’t have a definitive answer for that concern, but we do have one more data point: The Jazz underperformed their point differential by more than four wins last season, while the Wolves’ difference between their expected and actual win totals was as small as it has been since 2014-15—when Rubio missed most of the season with an injury. That’s a question that’s beyond the scope of this article, but is something to keep an eye on. Even if Rubio’s plus-minus impact returns in full-force next season, it may not have the expected impact on wins.

Jae Crowder

When the Celtics traded Crowder to Cleveland, those who believed in his plus-minus track record thought the Cavs were getting a key piece for a championship run and that the Celtics would suffer without him. It didn’t take long for that to seem laughable. Crowder struggled throughout his half-season in Cleveland, and was sent at the deadline to Utah (who, on purpose or not, seems to be collecting these plus-minus machines).

Things went better with the Jazz, but Utah was also better with Crowder on the bench than when he was in the game. Is Crowder’s poor performance in 2018-19 another strike against plus-minus as a good metric for evaluating players?

I’d argue the opposite, actually. Crowder’s performance as measured by conventional statistics was also quite poor last year. He had one of the most inefficient seasons of his career: His jumper seemed to desert him, as he made just 32% of his threes and 30% of his long midrange jumpers, a year after making 40% and 37%, respectively. He also finished at the rim worse (63% compared to 73% the prior year), drew the lowest rate of shooting fouls of his career and converted his fouls into and-ones at by far his lowest rate.

It’s not hard to find a potential reason for this dip in performance. As detailed by Dave McMenamin on ESPN last November:

[Crowder is] only a few months removed from the death of his mother, Helen, who died of cancer the same day he was traded to Cleveland. Not only did her death take an emotional toll, it also disrupted Crowder’s offseason training regimen, causing him to come into training camp not up to his normal tip-top standards after spending the final part of the summer making flights around the country to take care of his family’s needs once his mother died.

It’s no surprise, then, that Crowder’s plus-minus went in the tank as well. As a metric that purports to measure his impact on winning, it should have declined. He played worse than usual. We would worry more if a metric somehow showed that Crowder’s impact on winning was unaffected when he was out of shape and making fewer shots.

Crowder’s numbers weren’t much better in Utah. The positive news for the Jazz, though, is that even with Crowder not on top of his game, he was part of a very effective unit for them: Their starting lineup but with Crowder at PF instead of Favors was a +34.8 per 100 possessions in 704 possessions. That performance is so good—on a reasonable sample of possessions and without any crazy fluky shooting on either end—that we should be fairly confident there’s something real there. It’s a lineup to watch out for this coming season, because if it can remain even close to as effective, the Jazz will have a grouping that is powered by two conventional stars (Donovan Mitchell and Rudy Gobert) and two plus-minus machines.

Amir Johnson

If Crowder’s dip in on court/off court differential is explainable by his difficult offseason, Johnson’s can similarly be explained by a change in underlying play. Johnson is 31 years old, but as a high-school draftee, he just finished his 13th season—there are a lot of miles on his tires. His conventional numbers also showed some signs of aging: Johnson had the least efficient season of his career.

His plus-minus is difficult to interpret, though, because of the Joel Embiid effect: Johnson never shared the court with Embiid last season, so his on/off differentials are heavily influenced by the fact that he’s being compared to one of the most dominant defensive players in the league. For example, the Sixers defended the rim much worse with Johnson in the game. This is usually Johnson’s biggest statistical strength: possessing elite defensive awareness and a master of verticality, Johnson had never before played a season where opponents had a better FG% at the rim when he was in the game compared to when he was on the bench. In fact, of his 10 seasons where he played more than 500 minutes, in seven of them he ranked above the 90th percentile in FG% differential at the rim. Johnson has been one of the league’s best rim defenders for a decade.

Yet this past year, not only were the Sixers better defending the rim with Johnson on the bench, they were massively better. Part of that is surely the Embiid effect, but part is something else. It’s not just that the Sixers were great defending the rim with Embiid playing and so-so with Johnson—they were downright awful defending the rim with Johnson in the game. Philadelphia allowed opponents to make 68% of their shots at the basket with Johnson playing, a mind-boggling number for a player who had only once in his career seen his team give up higher than a 60% rate at the rim while he was in the game.

It’s hard to tell if that number is fluky, a result of Johnson’s slowing down, or something else. But despite the problems in rim protection, and despite the Embiid effect, Johnson actually came out pretty well on defense by plus-minus. The Sixers forced many more turnovers when he was in the game and rebounded better on the defensive end. Ultimately their defense didn’t slip a ton with Johnson on the floor in place of a Defensive Player of the Year candidate. ESPN’s Real Plus-Minus placed Johnson’s defense as the 9th-best for centers in the league last year.

So perhaps Johnson’s decline on the defensive end was overstated by the raw plus-minus numbers. He may be slowing down, particularly on offense, but this plus-minus machine may not have run out of battery power quite yet.

Patrick Patterson

Patterson was, ostensibly, signed to be the starting power forward for the Thunder, a perfect complement to the foursome of Russell Westbrook, Paul George, Steven Adams and Andre Roberson. The trade for Carmelo Anthony before training camp changed that, as did Patterson’s offseason knee surgery. Patterson ended up having to play his way into a bench role, usually in lineups featuring mostly bench players.

Once again, the context changed and so did the plus-minus. It’s not clear what that says about what Patterson’s performance will be like if healthy and slotted into a lineup that better fits his strengths, where he can be a floor-spacer and ball-mover. He played only about 150 possessions with the four non-Melo starters, and Oklahoma City was +14.1 per 100 possessions in that time with a very strong defense. But he also played almost 700 possessions with a Paul George + bench unit, a grouping that was absolutely awful, with a -19.9 per 100 possession differential.

Patterson’s great plus-minus numbers were the most brittle of the four players in this group: they had only been apparent in his last three-plus seasons in Toronto, playing in a bench unit that had been remarkably effective. It’s not particularly surprising that he struggled while coming off surgery and playing  in a rotation that exposed his weaknesses.

As Patterson, Johnson, Crowder and Rubio show, plus-minus metrics, as with all statistics, are not immune to contextual changes. Injuries, age, not being in shape and being used in a different role all will impact the player’s performance, and therefore all will impact a player’s plus-minus.

That’s as it should be. Basketball is a contextual sport. Health matters. Role matters. Fit matters. That’s a challenge in relying on any measure of a player’s past performance to predict his future production, whether that measure is points per game, plus-minus or a scouting report.

But plus-minus may be even more dependent on context than other methods of evaluation for a few reasons: It isn’t easy to see why plus-minus changes; we need a large sample of data to make strong conclusions; and plus-minus can be impacted heavily by the other players on the roster. All of those factors were present with these four players last season. Those results helped reinforce the principle: While plus-minus can be a tool in identifying undervalued players, it is still a very blunt instrument.