What To Make of the Generic Ballot Results?
Posted by wpcomimportuser1 | Email This | Permalink | Email Author
I have been a harsh critic of the generic ballot for most of the campaign season. Through all of my critiques, however, I have maintained that -- when we get near Election Day -- the generic ballot becomes a useful measure.
I still think that is true; however, before we use it, we need to engage in some deconstruction. Specifically -- which generic ballot should we use? There are very conflicting generic ballot results that have emerged in the last 24 hours. Time, Newsweek, CNN and Fox News have found little-to-no shrinkage in the margin that separates Democrats from Republicans. Meanwhile, Gallup, Pew and ABC News/Washington Post (not to mention Democracy Corps) have found a fairly dramatic shrinkage.
What do we do with these divergent results? The first thing we do is verify that they are not reducible to random statistical variation. The most efficient way to do this is to take the two results from the different camps that are closest to one another: Gallup and Fox News. It is 94.74% likely that there is a real difference between the two polls. While this is not enough to fully guarantee that the results are irreconcilable, it is strong evidence that they indeed are. It seems, then, that we are faced with a choice. We cannot simply average these results out, as we might for candidate ballots. They are very clearly divergent because of different methodologies.
What to do?
The smart move, obviously, would be to go with the poll that has the best track record. And, fortunately for us, on that front there is no contest. That is Gallup -- (for better or for worse) by default. Only Gallup has been conducting the generic ballot long enough to evaluate its accuracy, and as it turns out, their generic ballot is incredibly accurate. The final Gallup generic ballot actually explains 89% of all variation in final vote outcomes in midterms since 1950. In fact, for Fox to be correct would require the Gallup generic to underestimate Democratic strength by nearly 3.6%, which is more than twice the size of the largest Democratic underestimation in the history of the Gallup statistic.
Gallup has the historical track record. Practically speaking, it is the poll to follow. In 50 years, we can evaluate the Fox, CNN, Newsweek and Time polls to see how they fare. Until then, Gallup is the indicator.
So, what does Gallup tell us? If we run an ordinary least squares regression analysis that uses the generic ballot to predict the Democratic share of the vote, we get 54% D to 46% R.
That gets to the next big question: how many seats does that imply?
Unfortunately, I do not have more than a rough answer to this question. We can run a straight-up regression analysis that uses final popular vote to predict seat swings. And, with any such regression, we get an estimate of the final result that produces an error or residual. However, two problems with the residual present themselves. First, the variance of the residual is not constant across all observations. In other words, the accuracy of each point estimate varies systematically, depending upon whether or not the observation was taken before or after 1994. Second, the error term seems to be correlated with whether the observation was taken after 1994. We can fix the second problem by simply inserting a dummy variable into the regression to control for 1994. The intuition behind this is that 1994 simply increased the GOP's minimal seat share. So, a dummy variable to control for post-1994 would be a way to increase the floor. However, the first problem persists even when we insert this dummy variable (which is statistically significant). Even with our 1994 dummy variable, the accuracy of our point estimates still varies depending upon when the observation was taken.
What does this mean?
It probably means that the effect of 1994 was more than a simple increase of the GOP floor, though that is certainly the case. It seems that whatever happened in 1994 has systematically diminished the predictive power of vote share. House seat share is now less responsive to changes in House vote totals. This makes sense in light of what I have been arguing all year -- the post-1994 environment produced many districts where one's vote for Congress and one's vote for President came to align. Accordingly, districts became less responsive to aggregate swings -- as they were sufficiently filled with sympathetic partisans to help the in-party withstand such divergences.
N.B. Charles Franklin provides a great graphical illustration of this discussion here. The gray line represents a regression of popular vote to seat share for observations between 1946 and 2004. The green line represents a regression line for observations between 1946 and 1992. The red line represents a regression line for observations between 1994 and 2004. Notice the drastically divergent slopes on the green, 1946-92, line and the red, 1994-2004, line. This implies that an increase in the popular vote is now having a smaller effect on seat share than it used to. Because the gray line is closer to the green line than the red line -- the observed error for each point will be much grater for the 1994-2004 points than the 1946-1992 points. This invalidates the use of the gray line to draw inferences about seat changes. Its estimates are inefficient.
With this being the case, controlling for 1994 is simply not enough. We could do what Franklin does, which is to run two separate regression equations, one for 1994/pre-1994 and one for post-1994.
The two equations yield very different results. In the pre-1994 equation, a Democratic victory of 54/46 in the popular vote yields an expected caucus of 261 seats. That would be a 58 seat pickup -- the quintessential wave of which many have spoken. This equation, furthermore, explains an impressive 86% of all variation in House seats. In the post-1994 equation, a 54/46 Democratic victory in the pouplar vote yields an expected caucus of 222 seats. That would be a 19 seat pickup for the Democrats. The standard error of this estimate is about 4 seats, so 68% of the time we would get a result between 15 seats and 23 seats, 95% of the time we would get a result between 11 and 27 seats.
Here is the major problem: the latter equation only has 5 observations. While I am confident that (a) there has been some kind of change in the political landscape that (b) cannot be captured by a simple post-1994 dummy variable -- I am not confident of the post-1994 equation. The reasons are several and technical -- but they all boil down to the fact that there are just too few observations. I would like more than 5 observations to draw an inference from votes to seats. What I do know is that it is a problematic inference to use the last 15 or so midterm elections all at once. Something has happened in the post-1994 era -- the House is now less responsive than it was prior to this date. Of this I am positive. This equation implies that its relative unresponsiveness will give the Democrats only a nominal majority. However, I do not think we have had enough observations of the post-1994 era to really draw a confident inference.
We can see, then, in the final day of the campaign the question that I think has been with us for more than a year: just how stable is the current House structure? Just how invulnerable is it to change?
I honestly do not have an answer to the question. I have a hypothesis -- and tomorrow is the day that I will be testing that hypothesis. So...I'll tell you on Wednesday! Honestly, my sense is that 19 is a tad high. Just a tad, which is to say that seat changes are less responsive to vote changes than this equation predicts. This, however, might be my own personal bias creeping in. No, Chuck, not my partisan bias. Given that my day-to-day interests are like 90% process and 10% policy/substance, I really do not have one. I find, rather, that I have an "institutional bias" when I study politics. I am fascinated by the political institutions that surround and condition behavior -- and, so, I often tend to overestimate their power. Fortunately, political science is not metaphysics, so my biases are not decisive. They just make for good hypotheses! Like I said...Wednesday!
I will say that I feel pretty good about that 11 to 27 range. On a good night, I see the Democrats picking up 23 to 27. On a bad night, I see them pick up only 11 to 15. On an average night, well...19 is not terribly far from the estimate I have had in my mind for a while. It is also what Bob Novak currently estimates.
--------------------------------------------
Follow the RCP Blog on Twitter.
Become a fan of RCP on Facebook.
--------------------------------------------

