What About Schaub?

Author's note:  I began this analysis prior to Matt Schaub getting injured as a way to look at both his perceived and statistical value.  I've since altered it slightly to look at what we're missing from Schaub and what we need from Matt Leinart.  I also decided to keep the caption and title because they made me laugh, and that's really the most important thing.

When not writing about the Texans or getting/being blindingly drunk, I work in the field of Six Sigma and Continuous Improvement.  What that means is that I'm kind of an internal consultant that goes from department to department within our company, and helps them become more efficient (remember the efficiency experts from Office Space?  I'm that guy). 

Part of this job includes taking metrics and statistics from other departments, studying them, and making a determination on what's going right and what's going wrong.  One of the key things that I look for is variation within the data.  A metric may have a good average, but if it has a lot of variation, people will feel that something is wrong.  That "feeling" is what causes people to complain, and thus causes me to be called in.  You'd be amazed at the arguments I get when someone complains about an area only to have the departmental manager say something along the lines of, "But our average shows we're doing good" (sound familiar?).  In this business, variation equals pain. 

Furthermore, it's important for me to be as much of an outsider as possible because it allows me to be as objective as possible.

With both of these things in mind, I'd like to discuss Matt Schaub.  There are some out there who have the opinion that Matt Schaub is overrated and an unreliable quarterback.  These opinions on Schaub stem from his extreme and emotional moments in close games, such as the game-ending interceptions against Oakland or Baltimore (last year).  On the other hand, his statistics show him to be a quality quarterback.  So I decided to start looking at Schaub's performances in close games this year.

I originally planned to release this during the bye week since it would be the time when we could be the most objective.  With the recent news of Schaub's injury, however, and Matt Leinart's ascension to the starting lineup, we should look at this not from the perspective of "how has Schaub played' but rather from the perspective of "how much do we need to rely on Leinart?"

Let's start with the bland, unemotional statistical analysis.  It's the vanilla pudding of analysis.  The underseasoned grilled chicken breast.  The preseason Wade Phillips defense.  The Kubiak press conference.  The Matt Schaub.  Ok, that's enough.  

I'm just going to state the facts, here, so it would help if you read this in as monotone a voice as possible.  Imagine you're Ben Stein.

Schaub currently ranks sixth in the league in passer rating (96.8), 13th in completion percentage (61.0%), 10th in yards (2,479), 2nd in yards per attempt (8.49), 9th in touchdowns (15), and tied for 4th in fewest interceptions (6; minimum 200 attempts).  This pretty much covers it for the standard stats.

Let's shift to Football Outsiders.

Schaub ranks 5th among quarterbacks in DYAR (799), 6th in DVOA (29.4%), and 8th in effective yards (2,675).

In Advanced NFL Stats, Schaub ranks 8th in Win Probability Added (1.74), 5th in Expected Points Added (69.9), and 5th in Success Rate (50.6%).

Alright, that's enough numbers.  Based on that alone, it's hard to argue that Schaub is not a top 10 quarterback.  Each metric looks at quarterback play from a slightly different perspective, and outside of completion percentage, he ranks in the top 10 in every single one.  Generally speaking, if you have something that looks good from just about any angle, it's probably pretty good.

What this tells me is that the frustration and negative attitude towards Schaub is primarily driven by a handful of negative highlights (i.e., the variation).  It doesn't help that Schaub doesn't have equally memorable positive plays, and this gives life to the notion that Schaub is not "clutch." 

Even when Schaub does have those moments -- the win against Washington last year and victories against Green Bay and Miami in 2008 -- it's easy to brush them off by arguing that Andre Johnson bailed him out or because they occured so long ago.  The fourth quarter failures are more vivid memories that are more fresh in our minds.

A few weeks back, Rivers had an excellent analysis of Schaub where he discusses the idea that Schaub does a lot of things well, but doesn't excel at any one thing (other than play action).  Rivers was pretty much on point in the sense that Schaub doesn't have that signature that makes you say, "That's my quarterback!" 

Stir that in with the mediocre records of seasons past, and it's easy to argue that you simply have a quarterback that cannot elevate his team in key moments.  The numerical arguments are a difficult retort in a discussion where a single play can be submitted as "Exhibit A."

We're not going to be able to bring the "clutch" argument to a definitive conclusion (and so the debate will rage on), but we can look at data to get a better understanding of what's going on.  

In order to do so, we should start by looking at the data set of close games (let's define this as games that have been within 7 points at some point in the 4th quarter) and evaluating Schaub's performance late in those games to see how he performed. 

Through Week 10 in the 2011 NFL season, we have six such games in our data set:  Miami, New Orleans, Pittsburgh, Oakland, Baltimore, and Jacksonville.  Statistically speaking, that's probably not a huge data set, but for the sake of brevity, let's keep it at this for now (perhaps a more detailed analysis would be a good offseason project... hmmm..../writes it down).

• @ Miami:  This was a highly contested game when the fourth quarter began, as the Texans led 16-10.  The Dolphins started out with possession, and converted a field goal to pull within 3.  The Texans got the ball with 12:53 on the clock and Schaub went 3 for 3 on the drive, capped off with a beautiful touchdown to Andre, to put the team ahead by 10.  The defense held the Dolphins for the rest of the game, and Schaub wouldn't throw another pass. 
  
• @ New Orleans:  Well, we all remember this pretty vividly, huh?  The Texans led 19-17 when the quarter began, and the first play was a touchdown pass to James Casey.  He followed this up with a terrible interception later in the quarter, and had what could definitely be considered a lucky touchdown to Kevin Walter later in the game that gave the Texans a 1 point lead.  These are the types of games that give the anti-Schaub crowd ammunition.  Even though he did have two touchdowns, one was pretty lucky and he had a brutal interception.  Meanwhile, Drew Brees led his team to 23 fourth quarter points and the victory.
• vs Pittsburgh:  The fourth quarter began with the Texans leading 10-7, and the Steelers opened fourth quarter play with a game tying field goal.  On the ensuing drive, Schaub went 2-2, including a huge 30 yard gain by Owen Daniels before Arian Foster ripped off his 42 yard touchdown run.  Schaub would go 1-2 for the rest of the quarter as the Texans primarily led and used Foster to kill time.
• vs Oakland:  The Texans started the fourth quarter with a 17-15 lead, but two plays into it, they found themselves trailing 22-17.  The Texans went 3 and out on their next possession (0-1 by Schaub) before the Raiders drove down for a field goal.  Then the Texans went 3 and out again (0-3 by Schaub).  After a 3 and out by the Raiders, the Texans drove down for a field goal, but Schaub was hardly clicking on all cylinders, going 6-12 on the drive.  After another Raider 3 and out, Schaub went 3-5 on the ensuing drive (excluding an intentional spike) before the final fateful play.  We all know what happened on that play, and it resulted in a 9-21 fourth quarter for Schaub.  Yeach. 
  
• @ Baltimore:  Baltimore led Houston 16-14 at the start of this fourth quarter, and extended the lead by 3 only three plays in.  In general, Schaub was unable to spur the team to victory and was an uninspiring 5 for 12 in the fourth quarter (including the "got screwed by the replay" Jacoby Jones non-catch).
• vs Jacksonville:  The Texans led by 7 to start the fourth quarter and scored another touchdown only a few minutes in to go up by 14.  Though the Jaguars would score again, the Texans were pretty much able to control the game by running the ball and Schaub was not asked to do a whole lot.  He ended up going 3-3 in the fourth quarter with zero interceptions and zero touchdowns.

Here are a few numbers on the same analysis. 

Game	Att	Comp	Comp %	Int	Int/Att	TD	TD/Att
Miami	3	3	100%	0	0.0%	1	3.3%
New Orleans	17	8	47%	1	5.8%	2	1.2%
Pittsburgh	4	3	75%	0	0.0%	0	0.0%
Oakland	21	9	43%	1	4.8%	0	0.0%
Baltimore	12	5	42%	0	0.0%	0	0.0%
Jacksonville	3	3	100%	0	0.0%	0	0.0%
4th Quarter Total 2011	60	31	52%	2	3.3%	3	5.0%
Non-4th Quarter Stats	232	147	63%	4	1.7%	12	5.2%

 

As an admitted member of the pro-Schaub camp, I was a bit surprised by these numbers.  It would appear that in games where the Texans have asked Schaub to carry the fourth quarter load, he has struggled.  In the three games where he has attempted more than 10 fourth quarter passes, he has had a completion percentage less than 50% and the Texans have lost.  Meanwhile, the interception rate has more than doubled, while the touchdown rate decreased by 2%. 

At first glance, we would look at that and say that Schaub has not been "clutch".  I wouldn't be surprised to see completion percentage drop and interception rate increase for most quarterbacks in the fourth quarter -- they are likely, after all, to be taking more risks -- but the lack of a correlating increase in TD percentage suggests that Schaub's risks have not paid off.

Here's where it gets interesting, though.  I did some hypothesis testing on these numbers and found that most of these figures are statistically insignificant.  The closest one was completion percentage which had a p-value of 0.087.  In non-geek terms, that means that if I were to pick a random sample of 60 passes from Schaub's other pass attempts, there's about a 9% probability that Schaub could have a 52% completion percentage.  The rule of thumb in hypothesis testing is that anything with a p-value over 0.05 cannot be considered statistically significant, therefore we cannot say with any level of confidence that the perceived difference is anything other than normal variation.  Interception rate and touchdown rate both had p-values that were nowhere close, so any conclusion drawn from that should be immediately squashed. 

On the other hand, if we look at the data set of only those games where the team was trailing and tried to throw their way back into it, Schaub's drop in completion percentage does become statistically significant.  So we can confidently say that in 2011, Matt Schaub has definitely experienced a drop in accuracy when trying to rally the team from behind.  The interception and touchdown rates are still insignificant.

Here's what we can draw from this:  While we can't absolutely say that Schaub is the primary reason for the team's failures in those games -- the other players also deserve some portion of the blame -- he definitely hasn't "put the team on his shoulders," "raised the level of play of his teammates," or any other cliche. 

On the other hand, what does it mean if you have a player who hasn't excelled late in games, but still ranks as one of the best in his position?  It means that that player has been really damn good during the first three quarters.

Let's take another quick look at the Texans' ten games this season.  In the seven victories this season, the Texans have trailed for a grand total of -- wait for it -- zero minutes and zero seconds.  Yes, you read that right.

Schaub has been absolutely phenomenal early in games -- specifically on opening drives.  The Texans have scored touchdowns on six of their ten opening drives this season, and five of those have been courtesy of Schaub (five TD passes, one TD run). 

Note:  Oddly, the Texans have received the opening kickoff in nine out of ten games this season, with the Oakland game being the lone exception.  In that game, the defense forced a three and out before the offense drove down the field and Schaub hit Kevin Walter for a touchdown.

When you add this all together, what we find is that while Schaub may not be the fourth quarter magician that say, John Elway or Joe Montana were (though he's not quite the Heimlich man some paint him as), but this season he has absolutely excelled at getting and keeping his team in front. 

This allows Barian Fostate to exert their dominance on the ground and allows the defense to pin their ears back and come after the quarterback.  In other words, it allows the team to play to their strengths.

So when looking at what the team may have lost from Schaub and what they need from Leinart, the question should not be "can you win a Super Bowl with Leinart" ('tis a silly question as quarterback play is one of about a million factors that play into winning a Super Bowl -- though admittedly a big one) or "can Leinart carry the team", but rather "can Leinart get this team off to a quick start?". 

If the Texans lose their ability to come out strong and are forced to play from behind, then things may get rough.  They weren't able to do that very successfully with Schaub, and it's unlikely that Leinart will suddenly provide this skill. 

On the other hand, if Leinart can maintain the team's ability to put first half points on the board, then he will not be relied upon to do all those cliche-ish things that make one "clutch," and the team will be able to maintain its success.