States with new poll data added since the last update: None, this isn’t that kind of update. Instead of new data, there is a change to how averages are calculated.
TL;DR
I changed the details on the way the poll averages are calculated to prevent pollsters who release multiple results on the same poll from having an outsized influence on the average. This changed the overall national summary as follows:
- Trump’s best case improved from winning by 4 electoral votes to winning by 24 electoral votes, since Wisconsin now looks like a close state that Trump has a shot in.
- The tipping point moved from Clinton by 4.0% in NH to Clinton by 4.4% in OH as the average moved a little toward Clinton in Ohio.
- Clinton’s best case improved from winning by 226 electoral votes to winning by 248 electoral votes since Arizona once again looks like a close state that Clinton has a shot in.
The changes also modified the shape of the historical graphs a bit. Details follow for those who are interested.
The Problem
The issue is that sometimes pollsters release multiple results from a single underlying poll sample. For instance they will give results when only the two major party candidates are mentioned, but also results if respondents are also asked about third party candidates. Or they will release both “registered voter” and “likely voter” results. Or other variants.
In the interest of simplicity and a bias toward including as much data as possible, when this happened, I included all of the different results a pollster released, and just threw them into the average as if they were completely independent results.
But of course they are not.
Since the same underlying sample was used, the results are often much more similar to each other than separate polls tend to be. So I was essentially giving extra weight to pollsters who tended to release multiple results. This has the effect of making the averages more volatile than they would be otherwise, since effectively I’d be really using fewer data sources to determine the average rather than the normal minimum of five.
In some cases pollsters were even starting to release four separate results, which since I generally use five poll averages could mean that four of the five polls in the average came from the same sample. In other cases almost every poll in a state included multiple results. It was clear this was having an impact and I would probably have to change something.
The Solution
I considered a number of possible ways to address this over the last couple months. Over the last few weeks I have also been engaging with a number of Election Graphs readers about this issue and the possible solutions via the comments on this site, email, and even by phone in one case. You can see some of this discussion in the post here.
In the end I selected a solution where I weight these types of results so that if a poll includes two different variants, each of those results count as half a poll, both for the averaging itself and for determining which polls to include in the average. If the poll released three variants, each would count as a third of a poll, if they release four, each would count as a fourth, etc.
Using Iowa as an example:
Quinnipiac and Suffolk released two results each, one with just the two major candidates, one with Johnson and Stein also added into the question. The polls in question get a grey background to highlight that they are being weighted. The more variants a poll has, the darker the grey. The number in [] indicates how many variants were in that poll. The summary still says this is an average of five polls, as each of the Quinnipiac and Suffolk variants count as half a poll in the averaging.
The graphs also change the color of data points that are weighted in this way. The lighter the color of blue, the less weight to the data point. This makes it as clear as possible when viewing these charts that some of the data points count more than others.
The Impact
Obviously any change to the way in which averages are constructed will cause those averages to be different. There are any number of choices made on this site that would have similar impact. As an example, the choice of five poll averages rather than using six polls or four polls. The general picture wouldn’t be expected to change much, but some details will.
In this case there were three states that changed in ways that caused an adjustment to the national summary. For each of these I’ll show an animation of the state chart before and after the change. The frame of the animation that includes lighter blue dots is the new version of the chart.
Ohio [18 EV]
Ohio moved from a 2.8% Clinton lead to a 4.4% Clinton lead, a 1.6% move toward Clinton. Ohio is classified as “Weak Clinton” either way, but this changed the tipping point. This also eliminated a brief period earlier in the year when Ohio had been “Strong Clinton”.
Arizona [11 EV]
Arizona moved from a 5.5% Trump lead to a 3.1% Trump lead, a 2.4% move toward Clinton. This undid the recent Arizona move from “Weak Trump” to “Strong Trump” and improved Clinton’s best case from winning by 226 electoral votes to winning by 248 electoral votes since Arizona now once again looks like a close state that Clinton has a shot in.
Wisconsin [10 EV]
Wisconsin moves from a 7.5% Clinton lead to a 4.8% Clinton lead, a 2.7% move toward Trump. This moves Wisconsin from “Strong Clinton” to “Weak Clinton”, putting the state in play for Trump and improving his best case from winning by 4 electoral votes to winning by 24 electoral votes.
Other states
A number of other states were impacted by the change in how averages are constructed, but did not do so in ways that in turn changed the national summary. These were:
- California [55 EV]: Clinton 28.6% -> Clinton 28.3% (0.3% toward Trump)
- Texas [38 EV]: Trump 10.0% -> Trump 10.3% (0.3% toward Trump)
- Florida [29 EV]: Clinton 1.3% -> Clinton 2.0% (0.7% toward Clinton)
- New York [29 EV]: Clinton 21.7% -> Clinton 19.8% (1.9% toward Trump)
- Pennsylvania [20 EV]: Clinton 7.3% -> Clinton 7.1% (0.2% toward Trump)
- Georgia [16 EV]: Trump 0.8% -> Trump 0.8% (Historical graph change only)
- Michigan [16 EV]: Clinton 5.4% -> Clinton 5.5% (0.1% toward Clinton)
- North Carolina [15 EV]: Clinton 1.2% -> Clinton 1.8% (0.6% toward Clinton)
- New Jersey [14 EV]: Clinton 15.0% -> Clinton 12.7% (2.3% toward Trump)
- Virginia [13 EV]: Clinton 10.6% -> Clinton 11.2% (0.6% toward Clinton)
- Washington [12 EV]: Clinton 16.0% -> Clinton 14.4% (1.6% toward Trump)
- Massachusetts [11 EV]: Clinton 17.7% -> Clinton 18.8% (1.1% toward Clinton)
- Colorado [9 EV]: Clinton 8.3% -> Clinton 8.4% (0.1% toward Clinton)
- South Carolina [9 EV]: Trump 2.0% -> Trump 3.2% (1.2% toward Trump)
- Connecticut [7 EV]: Clinton 7.6% -> Clinton 7.4% (0.2% toward Trump)
- Iowa [6 EV]: Clinton 0.9% -> Clinton 0.8% (0.1% toward Trump)
- Mississippi [6 EV]: Trump 15.2% -> Trump 13.0% (2.2% toward Clinton)
- Nevada [6 EV]: Clinton 1.8% -> Clinton 1.8% (Historical graph change only)
- Utah [6 EV: Trump 12.6% -> Trump 10.9% (1.7% toward Clinton)
- New Hampshire [4 EV]: Clinton 4.0% -> Clinton +4.0% (Historical graph change only)
- Alaska [3 EV]: Trump 10.8% -> Trump 12.3% (1.5% toward Trump)
- Maine-All [2 EV]: Clinton 4.4% -> Clinton 2.0% (2.4% toward Trump)
If anybody is interested in the before/after animations for any of these states, ask me. It would have been too much to include them all here in this post.
National View
The changes in all these states of course changed the two national graphs as well. The versions with data a little further to the right are the “after” frames.
The overall shape of the electoral college trend chart is similar before and after, although Trump’s best case is now a bit better off in the time between Clinton clinching and the start of the conventions. The brief July spike toward Trump in the expected case also disappears. And of course both candidate’s best cases improve a bit at the current time.
The main impact on the tipping point chart seems to have been to mute the extremes. The high in July wasn’t quite as high, the low in August wasn’t quite as low.
The map view would have changed too of course, but unfortunately I did not save off a “before” version of the map, so couldn’t produce a nice animation. The graphs above tell the story nicely though I think.
In the end, while I think this correction was important to make, it doesn’t significantly change much in the national view, or the narrative of this campaign. Trump has been behind the entire race. At times he has been able to make it a bit closer, but then seems to fall back again. Electoral college wise Clinton seems to be bouncing between winning by 144 electoral votes and winning by 188 electoral votes in the “expected case”. The range between the best cases is wider and a bit more volatile, and usually includes a narrow Trump win as a possibility, but often does not. On the tipping point, Trump’s ceiling seems to be about a 2.5% Clinton lead, and his floor seems to be about an 8% Clinton lead.
All in all, with these bookkeeping changes in place to stop giving extra weight to some pollsters, the race looks even more stable than it did before. Yes, things bounce around a bit, but fundamentally we’re in basically the same place we have been since mid-May with only minor changes.
We can and will continue to talk about the ups and downs day to day and week to week, but absent something really big happening, this race looks like it doesn’t want to budge much.
69.1 days for that to change.
Note: This post is an update based on the data on ElectionGraphs.com. Election Graphs tracks both a poll based estimate of the Electoral College and a numbers based look at the Delegate Races. All of the charts and graphs seen in this post are from that site. Additional graphs, charts and raw data can be found there. Follow @ElectionGraphs on Twitter or like Election Graphs on Facebook to see announcements of updates or to join the conversation. For those interested in individual general election poll updates, follow @ElecCollPolls on Twitter for all the polls as they are added. If you find the information in these posts interesting or useful, please consider visiting the tip jar.
[Blog Post] Change to handling of polls with multiple results https://t.co/SPl6EokJAH
RT @ElectionGraphs: [Blog Post] Change to handling of polls with multiple results https://t.co/SPl6EokJAH
These changes look good. Thanks for all the hard work you did revamping the website. I know it may not look like the graph changed by much, but that’s actually a good thing. It shows how stable the metric you picked is. And, frankly, I would have been skeptical if there had been a drastic shift.
The new graph is acting more like you’d expect a moving average to do, flowing smoothly. That weird sudden spike in July in the old graph had always made my eyebrows pop-up and go “Huh?”. The fact that the spike is gone is proof enough for me that this change, despite the complexity it added, was worth it. (But please don’t rush to add any more complexity just because this was so successful. Simple and understandable is beautiful.)
BTW, the graying out of the background cells in the polls-used-in-average chart is a tad confusing to read. Could I suggest trying something else: Maybe make each *poll* always take the same amount of space, but multiple variants would be shrunk to fit. Ideally, variants would be separated by lighter (or dashed) lines so that one can easily see the separation between polls. (Notice how Quinnipac and Suffolk sort of run into each other in the above chart).
By the way, you mentioned you didn’t save a “before” version of the map. I found a cached copy in Yandex here: http://hghltd.yandex.net/yandbtm?fmode=inject&url=http%3A%2F%2Felectiongraphs.com%2F2016ec%2F&tld=ru&lang=en&la=1472047616&tm=1472818275&text=http%3A%2F%2Felectiongraphs.com%2F2016ec%2F&l10n=ru&mime=html&sign=fcdc4e0bcf36f73ff901906706249e01&keyno=0
Or, e-mail me and I’ll send you the SVG/PNG/GIF.
A simpler solution to the colour or shrinking…
Just add a note at the end of the poll name (or extra field) that gives the weight. Either as just a % number, or a fraction, having W=1, W=1/2 or W=33% would be simple enough to understand.
Richard, you mean like “Quinnipiac [2]”? It looks like that’s already been implemented. Your idea of making more explicit what the number represents is a good idea, though. I particularly like “Quinnipiac [W=1/2]” as it conveys that this is a fraction.
I still like the idea that visual size correlates with how much weight a poll is given, but I can imagine it might take some tricky CSS or SVG coding to make it look good. (OTOH, From looking at the great information visualisation on this site, my hunch is it wouldn’t be too tricky for whoever does the web coding here.)
Ben,
I think the [2] you mean refers to variations of the poll at the moment, not weighting. That is kind of the same thing I was meaning though. As I said, I don’t know if a weight column would be better either.
Not suggesting going back and changing historical either, this could happily just add in as new from now as its only providing information.
Not knowing much about coding, I have a question.
If you reduce the size, how will that affect looking at the site/poll on a device like a tablet or phone? Will it render it too small to read properly without a lot of zooming and messing.
Good point. I guess there would have to be a minimum text size to keep it from getting too small. There’s nothing stopping pollsters from tossing in more than four variations. (Did you see the actual poll that compared Clinton vs Trump vs Giant Meteor Crashing Into The Earth?)
I think having the background get darker makes those rows look more important, even though they actually have less weight. Perhaps, if font size is too hard, the foreground text could be lightened?
Giant meteor…? Really?
*boggle*
Given the way some people believe everything they are told..
Nah… They couldn’t.. Could they?
Yeah, it was a real poll by PPP. Giant Meteor was doing surprisingly well in June, don’t know if there’s been more updates…
http://fusion.net/story/320621/vote-giant-meteor-in-2016/
Hah…
It’s just bizarre. I wonder how many of those responding were actually serious?
At least it didn’t get the 15% needed for a debate invite. I’d love to see how that was handled…
Sorry for not answering this one for so long. Glad you generally liked these updates. On specific UI things… I made the lines darker grey they more they were split on the notion of “greying them out”. You are kinda right that that makes those polls look more important vs less though. Having multiple font sizes ends up looking really odd though and makes things more difficult to read. I’d experiment with other options, but too busy entering polls at this point.
In terms of [2] rather than [1/2], the [1/2] would of course make it more immediately obvious what was going on, but basically I was running out of space, every character counted. On polls with longer names, I already was having issues with things “fitting” in the space allocated. Since then I’ve had to shrink the font on that whole data table for that reason. Even having [2] instead of [1/2] made a difference for such things. Since the weighting is just the inverse of the number of results, I explain this in a footnote. If I was actually using some complex weighting algorithm with every poll having a different weight, I’d have to have a weight column.
Also, thank you very much for the snapshot before picture. It wasn’t from exactly the right time I would have needed for a before and after on this particular change though. (It was before other changes as well.) But really appreciate you finding it!
Oh, and PPP always has great curveball questions in their polls. In their last one in Florida they polled a three way race with Clinton, Trump, and Ken Bone. Ones like that actually have some purpose (showing how much 3P support actually goes to ANY 3P as a protest, rather than to the actual specific candidates). But they poll all kinds of other fun things as well. Almost every one of their public polls has something or other that is odd and funny thrown in. The one about the giant meteor was just one of them.
I still would love to have seen the meteor with a debate invite…
Perhaps it should have just..ah.. crashed the debate instead??
:)