Tremors But No Youthquake: Measuring Changes in the Age and Turnout Gradients at the 2015 and 2017 British General Elections

In the aftermath of the 2017 UK General Election, some claimed that Labour performed unexpectedly well because of a surge in youth turnout. Polling estimates for the size of this ‘youthquake’ ranged from 12 to 21 points amongst 18–24 year olds. Using conventional and Bayesian statistical methods, we analyse British Election Study and British Social Attitudes random probability surveys and find no evidence of a shift in the relationship between age and turnout of this scale. Using the pooled BES and BSA reported turnout data with an informative prior that there was a modest increase in 18–24 turnout (N{6, 3}), our 95% credible interval for that change is between 1.2 and 6.3 points. Even with a strong youthquake prior (N{15.5, 3.5}), this data suggest that there is only a 4% probability that the change in turnout amongst 18–24 years olds was 12 points or higher.

In the wake of this surprise outcome, political analysts and commentators began to look for an explanation. One proposition quickly gained prominence: Jeremy Corbyn had mobilised previously disengaged young voters who turned out in droves. Some claimed that turnout amongst 18-24 year olds was as high as 72%. 2 To use the term that soon gained traction: there had been a 'youthquake'.
Initial evidence seemed to suggest that there was something to the youthquake claim. At the aggregate constituency level, turnout change between 2015 and 2017 was correlated with the number of young people in a constituency (Heath and Goodwin 2017). Two polls released after the election suggested that youth turnout had risen dramatically. One suggested that 18-24 year old turnout went up by 12 points . Another suggested a larger increase of 16 percentage points (Ipsos MORI 2017). Amongst 2 academics, Whiteley and Clarke (2017) reported a 19 point increase in 18-29 turnout, whilst Sloam and Ehsan (2017) reported an increase of 21 points amongst 18-24 year olds. 3 In this paper we assess these claims using British Election Study (BES) and British Social Attitudes (BSA) survey data. Whilst it is clear that Labour increased its support amongst young people (see appendix 1), we find no good evidence to support the claims made about a large increase in youth turnout. The lessons from our analysis are not just important for understanding the 2017 UK General Election, they have wider implications for the measurement of and interpretation of political behaviour.

Assessing the case for youthquake
It is important to emphasise that we are examining the question of turnout specifically.
Youthquake may be powerful rhetorical device, but as a social scientific concept, it is slippery. Originally, youthquake described the 'shock [2017 general election] result founded on an unexpected surge in youth turnout' (Sloam and Ehsan 2017, 5). Later formulations included changes in party support amongst young people (Whiteley and Clarke 2017), and some proponents argued that a youthquake also encompassed different 3 Neither Britton, nor Whiteley and Clarke, say whether they are measuring turnout amongst registered voters or the voting eligible, or age, population. Since they make no mention of registration, it seems reasonable to assume they are talking about either the voting eligible or voting age populations. Sloam and Ehsan's 21 point figure comes from Ipsos MORI, who report two numbers-a 16 point rise amongst all resident 18-24 year olds, and a 21 point rise amongst registered 18-24 year olds. Sloam and Ehsan do not make clear why they prefer the more dramatic number and do not report the lower number, nor do they report that it is measured as a proportion of registered voters. Ipsos MORI themselves say 'we believe the first figure [turnout amongst all resident adults] is both more reliable and more meaningful'.
3 campaigning styles, the role of social media, the Labour party's policy proposals, and the subsequent reaction of the Conservatives to the election result (Ehsan, Sloam, and Henn 2017). It is impossible to determine whether a youthquake so flexibly defined has taken place. 4 We deal with 'youthquake' as it was originally formulated, and examine the question of whether there was a substantial rise in turnout amongst young people.

Explanations for the relationship between age and turnout
It is long established that voter turnout varies with age (Milbrath 1965), a pattern has been attributed to life-cycle effects, in particular the accumulation of resources (Verba, Schlozman, and Brady 1995) and acquisition of adult roles and experience (Strate et al. 1989).
Other research suggests that voting is habit forming (Cutts, Fieldhouse, and John 2009).
Because young voters are yet to acquire these habits, they are more sensitive to contextual factors, such as how close an election is likely to be (Fieldhouse, Tranmer, and Russell 2007). There are also more mechanical reasons to question a disproportionate rise in turnout amongst young people. Overall turnout increased by 1.5 million people in 2017. Those aged 18-24 make up 11% of the electorate-about 5.2 million people. If there was a 16 point increase in 18-24 year old turnout, this would mean that more than half of the total increase in turnout would have come from this group. A 19 point increase in turnout amongst 18-29 year olds (18% of the electorate) would be 1.6 million people, more than the total increase in turnout at the election. 5 Such dramatic changes are not impossible, but claims of this scale should obviously be accompanied by compelling evidence.

Pre and post-election polling evidence
As far as we are aware, there were three claims of a surge in youth turnout based on pre and post-election polling: 1) an NME 'exit poll' that claimed a 12 point rise in 18-24 year old turnout, 2) an Ipsos MORI estimate of a 16 point rise in 18-24 year old turnout, and 3) an estimate of a 19 point rise in 18-29 turnout from the Essex Continuous Monitoring 5 The wider the age band used, the less plausible large turnout changes seem. A 16 point rise amongst those aged 18-39, for example, would be a turnout increase in this age range of about one million more voters than the total turnout increase. Without a (substantial) decrease in turnout amongst those 40 and older, the ceiling turnout change amongst 18-39 year olds would be about nine points. 5 Survey (CMS). We examine these polls in more detail in appendix 2, but summarise our main concerns here.
The NME 'exit' poll was conducted using a research panel launched in 2016, which raises the obvious question of how it can compare 2017 turnout with what happened in 2015. It also cannot be an 'exit' poll as described, because such polls cannot be used to measure turnout. Exit polls necessarily exclude non-voters because they sample people as they exit polling stations after having voted.
The Ipsos MORI (2017) data is reweighted pre-election polling. Assigning turnout on the basis of pre-election turnout likelihood is far from straightforward. As we show in appendix 2, the relationship between reported turnout likelihood during an election campaign and reporting voting afterwards is not stable between elections, over the course of different campaigns.
The CMS uses non-probability internet survey data which, in publicly available and comparable data, has reported turnout levels well above actual turnout levels (90%+).
These concerns do not mean that the claims made about turnout using these data are necessarily wrong, but given the magnitude of those claims, it does suggest that other data is needed. 6

Aggregate evidence
Aggregate constituency data shows that the change in constituency-level turnout between 2015 and 2017 is correlated with the number of young people in a constituency in the 2011 census. However, when we examine the aggregate data more closely (see appendix 3) we find that the relationship between turnout change and the 18-24 bracket is actually the weakest of the five voter eligible age brackets and is strongest with an age bracket that cannot vote: children aged zero to four. Few people, it is safe to say, would argue that the increase in turnout in 2017 was due to a sudden surge in the number of British toddlers voting in elections. 2017 was not the 'toddlerquake' election. What this analysis shows is the dangers of making inferences about individual behaviour from aggregate data because of the 'ecological fallacy' problem (Robinson 1950).  reported that 'an internal Labour Party estimate, based on analysis of several million records from marked electoral registers… [found] an increase in the involvement of those aged 18-24, of about ten percentage points'. As we detail in appendix 4, analysis of the electoral registers in Britain is far from straightforward because they contain no demographic information. Even ignoring these issues however, an analysis of the registers necessarily excludes those who are not registered. Taking registration levels into account, the Labour Party's analysis suggests a 6.6-point rise in 7 turnout amongst eligible 18-24 year olds, just over half of the lowest claim made about the change in youth turnout on the basis of post-election polling data.

Evidence from the UK Household Longitudinal Study
The final piece of evidence was presented by Sturgis and Jennings (2019), who analysed data from Understanding Society (the UK Household Longitudinal Study). Sturgis and Jennings report a nine-point rise in turnout for 18-24 year olds between 2015 and 2017.
We need to take this evidence particularly seriously. Understanding Society (University of Essex et al. 2017) is a rigorously conducted scientific survey that uses a gold-standard methodology to randomly sample and retain its respondents. However, one aspect of the Understanding Society data makes us cautious about taking this data as the final word about turnout change between 2015 and 2017, namely that it is panel data.
Nonresponse bias with regards to turnout is a well-known problem with survey research and surveys are well known to overestimate turnout (e.g. Dahlgaard et al. 2019). In the first wave of Understanding Society, we would expect turnout nonresponse bias to be comparable to other random probability surveys. However, further nonresponse bias can occur in subsequent waves of panel surveys (attrition bias). 6 6 An additional concern with panel data is that the simple fact of being in a panel survey might change how people behave, a problem known as panel conditioning (Sturgis, Allum, and Brunton-Smith 2009). There is also some evidence that being asked about electoral participation specifically might increase the likelihood of voting (e.g. Greenwald et al. 1987). Because we do not observe the voting behaviour of nonrespondents, we cannot assess the extent that these problems affect the Understanding Society data.

8
If we examine whether a respondent who appeared in wave one of Understanding Society (conducted between 2009 and 2011) also appears in the 2015 turnout data, we see that a respondent who said they were very interested in politics in wave 1 is 12 percentage points more likely to appear in the 2015 data than a respondent who said they were not at all interested in politics. Understanding Society mitigates the effects of attrition by weighting, but even with weights, the gap between official turnout and that recorded in Understanding Society has grown over time: it was 9.1 points in 2010, 11.6 in 2015, and 12.4 in 2017. 7 Although attrition bias might be a problem for Understanding Society in general, it is an additional question whether this affects estimates of change in turnout amongst young people in particular. There are no population data about the demographics of turnout in the UK, so we cannot answer this question with absolute certainty. However, we can show that even with minimal assumptions, it would be reasonable to expect that attrition bias will particularly affect youth turnout estimates. In appendix 5 we report simulation results that show that if turnout tends to increase with age and attrition affected people of all ages equally, this would lead us to overestimate the level of turnout change amongst the youngest age groups. These are conservative assumptions given that we know that 7 For sake of comparison, the equivalent numbers for the BES were 7.3 in 2015 and 8.5 in 2017, and for the BSA they were 5.2 in 2015 and 4.6 in 2017. The fact that the overall level of turnout bias is lower in the BES and BSA data should also give us confidence that this data is our best chance of accurately assessing turnout in 2015 and 2017. 9 turnout does increase with age and young people are more likely to drop out of Understanding Society These concerns notwithstanding, it is important to note that Sturgis and Jennings's estimate is ten points lower than Whiteley and Clarke's, half the size of Ipsos MORI's, and two-thirds the size of NME's.

Using survey data to measure turnout
How, then, can we better assess claims made about changes to turnout in 2017? In the absence of official population register data-which we do not have in Britain-our only option is survey data.
Online and phone surveys are prone to representativeness problems, and tend to include too few non-voters in samples (Sturgis et al. 2017;Mellon and Prosser 2017). The best way to minimise these problems is to use a random probability sample (Sturgis et al. 2017). As we discussed above, we are also likely to be on safer ground if we use cross-sectional surveys rather than panel surveys. We use the 2015 and 2017 editions of two such surveys: the BES face-to-face survey (Fieldhouse et al. 2015(Fieldhouse et al. , 2017 and the BSA face-toface survey (NatCen Social Research 2017, 2019. Both are address-based random probability samples, such that respondents have a known chance of selection-in contrast to telephone and online opt-in samples. Measuring turnout is still not straightforward. We need to overcome three challenges: 1) gathering accurate targets to adjust surveys for demographic imbalances due to differential response; 2) further adjusting surveys to account for the fact that, even with probability sampling and after accounting for demographic imbalances, people who turn out to vote are more likely to take part in surveys; and 3) dealing with the fact that people over-report having voted in elections.
The methodology behind survey weighting is well established, but the BES data raise an additional challenge. The sampling frame for most surveys (including the BSA) is the resident adult population. The sampling frame for the BES is the resident voting eligible population (VEP). Weighting targets are generally available for the resident adult population, but in order to create VEP weights, we must estimate them. We describe our approach to doing this in appendix 6.
Even after weighting, however, each survey has too many people who claim to have voted (relative to the actual level of turnout at the election). As we discussed above, this is likely due to two factors: one, turnout related nonresponse bias, and two, misreporting.
We mitigate the impact of the first by weighting to the result of the relevant election in terms of turnout and party vote shares. As we show in appendix 7, although we only weight to overall levels of turnout, this weighting should also make our estimates of turnout in demographic subgroups more accurate.

11
To reduce the impact of misreporting, we make use of validated turnout in the BES. In order to validate whether or not a respondent had voted, their details were checked against the marked electoral register (see appendix 8 for more details). The vote validation process comes at a slight cost of sample size (and was not conducted for the BSA), and the process itself cannot be perfect.
We present our turnout analysis using self-reported and validated vote measures. The likely errors in these approaches work in the opposite direction: self-reported vote might overestimate turnout and validated vote might underestimate it. Where these two measures agree on the age/turnout relationship-and they generally do-we can be more confident in our findings.

Age and turnout in 2015 and 2017
We now turn to our analysis of turnout and age at the 2015 and 2017 elections. We conduct each set of analyses on four (overlapping) sets of data: on 1) BES reported vote, 2) BSA reported vote, 3) pooled BES and BSA reported vote data, treating the BSA data 12 as if it were a Voter Eligible Population (VEP) sample, 8 and 4) the BES validated vote data. 9

Nonparametric analysis
First we analyse our data by plotting the nonparametric smoothed local mean (Fan and Gijbels 1996) analysis of turnout by age, shown in Figure 1. This method allows us to examine turnout without having to impose arbitrary cut-offs between age groups, or to impose a particular functional form on the relationship between age and turnout (i.e. it does not have to be linear or quadratic). Each set of comparisons tells the same storythe relationship between age and turnout was remarkably similar in 2015 and 2017. There is no evidence here of a substantial rise in turnout amongst young people. The validated turnout data suggests that turnout rose more substantially amongst 30 to 40 year olds, but given the relatively small validated sample we should be cautious about this result, since we are more likely to observe large differences by chance in smaller samples (Gelman and Carlin 2014), and it is not corroborated by our other analysis. 8 Because VEP turnout change is necessarily greater than VAP turnout change, this slightly biases the BSA data towards a larger overall change in turnout. Given that our results are very similar between the separate BES and BES data when weighted appropriately and when they are pooled together, we do not think this bias has unduly influenced our conclusions. 9 We additionally conducted a series of logistic regression models examining changes to the overall age/turnout gradient between elections. This analysis corroborates our other analysis, but for reasons of space we report these in appendix 9.

Pairwise comparison of age groups
Next we analyse turnout changes between 2015 and 2017 by categorising respondents into age groups. We use two sets of age groups, the first with the youngest age band of 18-24, and the second with a wider band of 18-29. We report the sample sizes for each comparison in appendix 10. Given concerns about the statistical power of relatively small 14 samples, we also conduct a series of power analyses. We report the full results of these comparisons in appendix 10, but highlight some key comparisons here.
If we take the lowest youthquake claim of a 12 point increase as our benchmark (we will necessarily have more statistical power to detect larger changes), using the 18-24 age band, the BES reported turnout data has 0.62 power, the BSA data 0.68, the pooled BES We compare turnout in 2015 and 2017 using two approaches. First, we use a conventional statistical approach to calculate confidence intervals around our observed differences in turnout. Second, we use a hierarchical Bayesian approach (with a logistic link function) to partially pool our results within age groups across elections, and across age groups within elections, and within age groups at each election, using the BRMS package (Bürkner 2017), which compiles the model using Stan (Stan Development Team 2018).
The Bayesian approach allows us to incorporate information we can be more certain about-namely the overall level of turnout at each election-by specifying informative priors for the overall intercept and turnout increase in 2017. 10 We report these comparisons in Figure 2.
First, we examine our conventional estimates. For the 18-24 and 18-29 age groupings, across all four sets of data, the confidence intervals overlap zero. In other words, we cannot reject the null hypothesis that there was no change in the levels of turnout amongst young voters between 2015 and 2017. The absence of sufficient evidence to reject the null hypothesis is not the same thing as evidence in favour of the null. The confidence intervals in all our comparisons are fairly wide and compatible with a wide range of changes in turnout-from a moderate increase to a small decrease (or even a large decrease in the case of the validated vote, but the other data suggest that this is unlikely to be the case).
10 More specifically, we specify priors for the VEP turnout model as normal distributions (on the log odds scale) with means of .622 for the intercept (2015 turnout) and .142 for the increase in turnout in 2017. For VAP turnout the equivalent numbers are .424 and .097. When transformed into probabilities, these coefficients give our estimates of overall VEP and VEP turnout for each election. Because our measures of VEP and VAP turnout are still estimates, we allow for a small amount of uncertainty, giving the distribution of each prior a standard deviation of 0.02. The standard deviations of the distributions of subgroup means are also given priors (hyperparameters). We use weakly informative (regularizing) priors using folded tdistributions (df=3) with a mean of zero and scale parameter of 10. As well as the null hypothesis, we can also examine whether our data are compatible with the youthquake hypothesis. If we test the smallest (and therefore, most difficult to reject) youthquake claim about turnout change-a 12 point increase-we can reject the youthquake hypothesis using both the combined BES and BSA reported turnout data (p= 0.0466) and the BES validated vote data (p= 0.0124). In other words, whilst we cannot be exactly sure about the level of turnout change amongst young voters between 2015 and 2017, by conventional statistical standards, we can say that it did not increase by the amount claimed by those who say there was a youthquake.
If we turn to the Bayesian analysis, the most striking feature is the relatively small credible intervals compared to the conventional confidence intervals. Pooling across age groups and elections increases the precision of our estimates, as does correctly accounting for the fact that we know the change in overall turnout between elections. Using the pooled BES and BSA reported turnout data, the 95% credible interval for the change in turnout between 2015 and 2017 for 18-24 year olds is between 1.2 and 6.3 points (with a median estimate of 3.6). The 95% credible interval for the validated vote data is considerably wider-between -8.5 and 6.6 points, with a median of 1.3. 11 Using the Bayesian approach gives us more confidence that turnout amongst young people did increase in 2017, but it also suggests that it increased by levels similar to the overall increase in turnout as a whole-in other words, it gives us even more confidence that there was not a turnout 'youthquake'.

The impact of subjective priors on estimates of turnout change
Some advocates of the 2017 youthquake may still be sceptical of our results. How much weight should people with strong prior beliefs about a youthquake put in on our data? 11 The equivalent numbers for the 18-29 age band using the pooled BES and BSA data are a 95% credible interval of between 1.1 and 5.7 points, with a median of 3.5. Using the BES validated vote data, the 95% credible is -3.3 to 9.2, with a median of 3.3.

18
The Bayesian approach to statistics allows for a principled and transparent approach to incorporating subjective beliefs into statistical estimates. In order to do so, however, we need to transform those beliefs into statistical distributions.
There have been three specific claims about the size of a youthquake level increase in turnout amongst (eligible) young people-12 points, 16 points, and 19 points. Three data points is too small to draw a full distribution, but if we assume the shape of the distribution is normal, we can make reasonable assumptions about what it might look like. If there were a normal distribution of estimates, we would expect random draws from this distribution to be one standard deviation away from the mean of the distribution on average. The highest (19) and lowest (12) claims are similarly far apart from the third claim (16). A straightforward approach, then, is to specify a prior where the two extremes are each one standard deviation away from a mean. We adopt this approach and specify a youthquake prior as a normal distribution with a mean of 15.5 and a standard deviation of 3.5.
As is hopefully clear from our earlier examination of these claims, we do not actually believe this is a reasonable prior. What would a more reasonable prior look like? Without any further information about the relationship between age and changes in turnout, our first guess would probably be that the overall level of turnout change applied equally across different age groups. The law of dispersion (Tingsten 1937), however, implies that when turnout rises we might expect a slightly larger increase in turnout among lower turnout groups because those groups contain a larger number of potential new voters to be mobilized (Persson, Solevid, and Öhrvall 2013). We can, then, take an alternative 'modest increase' prior, which is consistent with the Labour party's analysis of a 6.6 point change, and Sturgis and Jennings's 9 point change. If we follow the same logic as above and treat the overall (VEP) change in turnout (3 points) as being one standard deviation below the mean of our distribution, and the higher of the two plausible claims as being one standard deviation above the mean, we get a normal distribution with a mean of 6 points and a standard deviation of 3.
We examine the effects of these priors using linear probability models on the 18-24 year olds in each dataset. 12 Furthermore, in order to make clear what the effect of these prior distributions is, we specify a weakly informative prior distribution with a mean of 0 and a standard deviation of 30 (which yields a normal distribution in which 99.99% of the distribution lies between -100 and 100, the strict mathematical limits by which turnout could change between elections).  As we would expect, the analysis using the weakly informative prior is very similar to the conventional estimates of turnout change we showed in Figure 2. If we examine the posterior distributions using our modest increase prior using the pooled BES and BSA data, the 95% credible interval for the posterior is 0.9 to 8.8 points, with a median of 4.9.
These numbers are slightly higher but broadly similar to our partially pooled analysis reported in Figure 2. Using the BES validated vote data, the 95% credible interval is -1.6 to 8.9, with a median of 3.6.
We are primarily interested in how the youthquake prior affects our estimates. The results are clear. Even with a strong prior that there was a large increase in turnout amongst 18-24 years olds in 2017, the BES and BSA data should lead to a substantial downward revision of beliefs about the scale of that change. Although the midpoint of the youthquake prior distribution is ten points higher than the midpoint of our modest increase prior, the median of the posterior distribution using the pooled BES and BSA data with a youthquake prior is 8.5, just 3.7 higher than the posterior with our modest increase prior. The Bayesian approach allows us to make probabilistic statements about our results. Even with a strong youthquake prior-which we emphasise we do not think is a reasonable prior-the pooled BES and BSA data suggest that there is a 4% probability that the change in turnout amongst 18-24 years olds was 12 points or higher-the lowest claim made in favour of youthquake. With our modest increase prior, the same data suggest that the probability of a 12 point or higher increase in 18-24 year old turnout is less than 0.00025%.

Conclusions
It is perhaps unsurprising that following the dramatic and unexpected result of the 2017 election, a similarly dramatic explanation was put forward to explain what happened. As we have shown in this paper, however, we do not have good evidence that the relationship between age and turnout changed dramatically between the 2015 and 2017 elections.
Given this, we need to look elsewhere to account for the rise in Labour's support in the 2017 general election. One such explanation points to the importance of Brexit over the two year period between 2015 and 2017, and the importance of leaders to the short campaign in 2017 , and also the degree to which the Brexit referendum caused a potential realignment of electoral choice on the basis of Brexit and liberalauthoritarian values, which are strongly related to age and education (Fieldhouse et al. 2019).
The idea of a 'youthquake' may not tell us much about turnout at the 2017 election. It does reveal, however, the perils of making inferences about electoral turnout and political behaviour more generally. Analysing political behaviour is hard. We have shown here that aggregate level analysis of turnout can easily be misleading and that individual data is necessary. Analysing turnout in surveys is fraught with difficulties. We have detailed our approach to dealing with the problems due to non-response bias and turnout misreporting. We also set out a number of ways in which we can analyse the data,

Measuring changes in the age and turnout gradients at the 2015 and 2017 British General Elections
Electronic copy available at: https://ssrn.com/abstract=3111839

Appendix 1: The relationship between age and vote choice in 2015 and 2017
It is important to be clear that although we find no evidence of a large change in the relationship between age an turnout, young voters were distinctly Labour supporting. Figure A1.1 shows the relationship between age and Labour and Conservative support at the 2015 and 2017 general elections. At both elections, Labour were more popular among young voters, and the Conservatives were more popular with older voters. These relationships increased between 2015 and 2017. It would be a mistake, however, to assume that because young voters are typically Labour supporting, Labour voters are typically young. According to the combined BES and BSA data, voters under the age of 25 made up only 11% of all Labour voters in 2015. In 2017, they made up 13%. The median age of Labour voters was 49. In 2017 it was 48. The increase in Labour support was highest amongst young voters, but on their own, young voters cannot account for the change in Labour's electoral fortunes. Instead, the explanation for the large increase in Labour's vote share in 2017 is more prosaic-Labour gained support across a wide spectrum of the electorate.

Appendix 2: More detailed examination of polling evidence relating to turnout change
Here we examine three claims of a surge in youth turnout based on pre and postelection polling: 1) An NME 'exit poll' that claimed a 12 point rise in 18-24 year old turnout.
2) An Ipsos MORI estimate of a 16 point rise in 18-24 year old turnout.
3) An estimate of a 19 point rise in 18-29 turnout from the Essex Continuous Monitoring Survey (CMS).
We should also note that the election polling evidence is not uniformly in favour of a youthquake. Kantar's post-election polling gives a turnout gradient broadly comparable to the one we present in this paper. YouGov's multilevel regression and poststratification (MRP) model-the only successful 2017 pre-election forecastestimated turnout for particular demographic groups based on 2010 and 2015 BES face-to-face data and assumed that they would not change much in 2017 ).

A2.1 NME exit poll
The NME 'exit' poll reported by Britton (2017)  A key factor in the 2015 polling miss was the failure to sample representative levels of turnout (Mellon and Prosser 2017a)-our key variable of interest, and the magnitude of this problem was highest in the youngest age groups (Sturgis et al. 2017). It is unclear whether these problems were fixed in 2017.
Ipsos MORI were cautious about measuring turnout with pre-election polling, noting that: '…estimating turnout is one of the hardest challenges when relying solely on survey data… polls may still be more likely to interview politically engaged people than those who are disengaged, people may over-estimate their likelihood of voting, and they may think they are registered when in fact they are not.' Assigning turnout on the basis of pre-election turnout likelihood is far from straightforward. The relationship between reported turnout likelihood during an election campaign and reporting actually voting after the election is not stable between elections, over the course of a campaign, or indeed, over the course of the campaign between elections. We can see this using the BES Internet panel data  we can compare the likelihood of post-election reported turnout by pre-election reported likelihood of voting. Figure A2.1 reports this comparison for the 2015 (wave 5) and 2017 (wave 12) data and shows that across all pre-election turnout likelihoods, 2017 respondents were less likely to report having actually voted after the election. We can illustrate the impact of these differences by applying the 2015 expected turnout proportions to the 2017 data. If we did this we would overestimate 2017 turnout by 2.4 points-an error almost as large as the actual increase in turnout and so a substantively large error. Electronic copy available at: https://ssrn.com/abstract=3111839 We can also examine how the relationship between turnout likelihood and actual turnout changes over the course of the campaign, which we do in figure A2.2. This plot shows two important things. One, that the relationship between reported turnout likelihood and turnout is unstable across the election campaign, and two the size and direction of that instability is not the same at the two elections. These issues do not mean that we should dismiss Ipsos MORI's findings completelythey are suggestive of a large rise in turnout amongst young people, but they are not decisive.

A2.2 CMS
Whiteley and Clarke (2017) reported a 19 point rise in turnout amongst those aged 18-29 using the Essex Continuous Monitoring Survey (CMS). The relevant data from the Essex CMS are not publicly available, nor accompanied with methodological detail, so it is difficult to assess the basis for this reported turnout rise. We do know that the CMS uses non-probability internet survey data collected by YouGov. Non-probability internet survey data have many invaluable uses (c.f. Fieldhouse and Prosser 2018) but are not well suited for measuring turnout given the over-sampling of voters (Mellon and Prosser 2017a). Earlier BES CMS data are available online, and so we can use these available data to examine the validity of CMS data for studying turnout. In the June 2010 wave of the CMS (the last available dataset conducted after an election), 91% of the sample claim to have voted. Likewise, similar proportions of respondents claim to have voted in 2015 and 2017 in the BES Internet Panels (also conducted by YouGov). Given they share a common supplier, it is likely that these problems also affected the 2015 and 2017 post-election waves of the Essex CMS. Again, these concerns do not mean that Whiteley and Clarke's claims about changes in 18-29 year old turnout are wrong, but they suggest that other data are needed before we accept their conclusion.

Appendix 3: Aggregate level analysis
Much of the on-the-night and immediate post-election analysis rested on constituency patterns, as is understandable given that the only data are in the form of aggregate level results. Heath andGoodwin (2017), andFetzer (2017) showed that the change in constituency level turnout between 2015 and 2017 is correlated with the number of young people in a constituency-supporting the idea that there was an increase in turnout amongst young people. However, aggregate analysis is not without risk of spurious inference because of the 'ecological fallacy' problem (Robinson 1950), where aggregate level correlations do not reflect individual level relationships but arise as a result of omitted variables and aggregation problems. There are many cases where the ecological fallacy has been shown to have occurred (see, for example, Tam Cho and Gaines 2004;Matsusaka and Palda 1993). An illustrative example is Gelman's (2008) demonstration that the negative correlation between US state level income and Republican support (poorer states have higher levels of Republican support) is the reverse of the individual level correlation between income and voting (poorer individuals are more likely to vote Democrat).
We need to be very cautious before taking constituency-level evidence at face value. Table A3.1 shows the results from a series of regression models that analyse the relationship between the proportion of the population falling into particular age brackets, and the change in constituency turnout levels between the 2015 and 2017 elections. At first glance the results appear to support the argument that there was an increase in turnout amongst young people: there is a positive relationship between the proportion of the population aged 18-24 and negative relationships between change in turnout levels and residents aged 45 and older. However, the relationship between turnout change and the 18-24 bracket is actually the weakest of the five voter eligible age brackets. Increases in turnout are much more strongly related to adults aged 25-29 and 30-44. More importantly, the first column raises alarm bells about inferring individual behaviour from aggregate data. The relationship between turnout change and the proportion of the population that are children aged zero to four is far stronger than the relationship between turnout change and any of the adult age groups. The relationship is not trivial-for every additional percentage point, children aged zero to four in a constituency turnout increased by 0.9 percentage points between 2015 and 2017, 11 times greater than the coefficient for adults aged 18-24, with an R 2 nearly 10 times as large.
Few people, it is safe to say, would argue that the increase in turnout in 2017 was due to a sudden surge in the number of British toddlers voting in elections. 2017 was not the 'toddlerquake' election. Yet, at the aggregate level, the statistical evidence for a surge in toddler voting is seemingly more compelling than the evidence of a youthquake. Of course, what this relationship is showing is not that turnout went up amongst toddlers but that turnout went up in the sorts of places with lots of toddlers. The same is true of the relationship between the number of young adults and turnout. Turnout went up slightly in the sorts of places with lots of young adults. That does not necessarily mean it was those young adults doing the extra voting. The sorts of places with lots of young adults are cities. Table A3.2 shows that once we include population density as a control variable, the relationship between the number of young adults living in a constituency and change in turnout disappears. For all three of the youngest adult age brackets the coefficient (previously positive and statistically significant) is now negative and not statistically significant. The same is also true for the oldest age bracket, which flips sign and is no longer statistically significant. Only the relationships between turnout change and 45 to 64 year and zero to four year olds are still significant and in the same direction (albeit substantially reduced in size). This is a strong indication that the apparent relationship between the number of young adults in a constituency and turnout change is spurious. The results in this section are robust to an alternative specification with an age 18-29 bracket. Likewise if we use the proportion of full time students in a constituency instead of age we see the same pattern. The results of these models are show in Table  A3.3.   reported that 'an internal Labour Party estimate, based on analysis of several million records from marked electoral registers… [found] an increase in the involvement of those aged 18-24, of about ten percentage points'. We have to assume that the ten-point rise is a proportion of 18-24 year olds who are registered to vote (since it is based on analysis of the electoral register). Only one other claim was specifically amongst registered 18-24 year olds; Ipsos MORI's figure of 21 points. This suggests that the size of any 'youthquake' might have been half of Ipsos MORI's estimate. Even here, however, the case is not completely clear-cut.

Appendix 4: Labour party analysis of the electoral registers
Labour's analysis has not been released publicly, so we cannot be sure about its methodology, but we know that any analysis of demographic differences in turnout based on the electoral registers must overcome a sizeable barrier: the electoral registers contain no demographic information, and there is no official population register in Britain that contains such information. Instead, it must be approximated from other sources, such as credit agencies. No assessment of the accuracy of these sorts of data in the UK has ever been conducted, although studies of similar data in other contexts report substantial errors (Igielnik et al. 2018).
Analysing the electoral registers faces other problems, particularly when comparing turnout between 2015 and 2017. In 2014, Great Britain shifted from a household system for electoral registration to Individual Electoral Registration (IER). Although IER came into force in 2014, register entries from the previous system were not removed until after the 2015 election. The shift to IER affects the calculation of turnout because it changes the denominator. Turnout as a proportion of register entries is not the same thing as turnout as a proportion of registered people. In other words, if exactly the same proportion of people had voted in both elections but the registers were more accurate after the introduction of IER (which research by the Electoral Commission (2016) suggests is the case), this would look like a proportional increase in turnout, even though there was no change in behaviour.
Not everyone who is eligible to vote is on the electoral register. As a simple matter of mathematics, registered voter turnout is higher than turnout amongst all those who are eligible to vote. In December 2015, the Electoral Commission (2016) estimated that 65% of 18-19 year olds and 67% of 20-24 year olds were registered to vote. If we assume a stable level of registration between 2015 and 2017 of 66% for 18-24 year olds, a ten-point rise in turnout amongst those registered to vote would mean a 6.6-point rise in turnout amongst all those eligible to vote. This figure is more than 12 points lower than Whiteley and Clarke's 19-point rise, nine points lower than Ipsos MORI's 16-point rise, and just over half of NME's 12 point rise. If we take the Labour party figure at face value, it shows evidence of an increase in turnout amongst 18-24 year olds, but it also suggests that previous claims about the scale of that change were vastly over-inflated.

Appendix 5: Attrition bias simulations
In order to demonstrate the potential problem of attrition bias, we specify a simplified turnout model where turnout probability is a function of two things: 1. General underlying propensity to take part in things (i.e. both turnout and surveys), which we specify as random uniformly distributed variable (i.e. not correlated with age) between 0 and 1, which we call turnScore. 2. Age (because of life-cycle stuff, e.g. home ownership, stability of residency etc -things we know tends to be true about turnout) which for convenience is uniformly distributed between 18 and 90.
Taking the age-turnout relationship (approximately) from the 2017 BES face-to-face, and adjusting the turnScore and constant parameters by trial and error to give an overall turnout level that approximates what we see in British elections (~64%) we specify the data generating model for turnout at election 1 as a logistic model where: 1 * = .07 − .0004 2 + .2 − 1.8 Then for the second election, we increase the coefficient for turnScore to 0.6 to give a turnout at the second election that is ~4 points higher overall: 2 * = .07 − .0004 2 + .6 − 1.8 The logic of changing the turnScore parameter and leaving the other parameters constant is that the age parameters are capturing stable life-cycle effects (so shouldn't change) whereas the turnScore coefficient is capturing election specific 'motivation' type effects, i.e. people are more motivated in election 2, but the extent to which that motivation translates into vote choice is still going to be a function of how likely people are to ever be motivated to turnout.
There is no age*turnScore interaction, but because this is a logistic model, there is a compression effect, i.e. as people get older, turnScore has a smaller effect because there's an age/turnout ceiling effect. Given what we know about turnout being habit forming and life cycle effects, we think this is a reasonable assumed effect in the data generating model, and not just an artefact of the functional form. Figure A5.1 illustrates the change in turnout between elections over the age range. The highest change is in the young-to-middle age range (which is where there is the least compression) but overall the change is similar across the age range -the increase is between about 3 and 5 points for all ages.

Figure A5.1 Simulated turnout change between two elections.
We then simulate a sampling process where the propensity to be included in the sample is correlated with turnScore at different rates between 0 and 1, and record the error in the turnout increase for four age bands from 1000 simulations of each level of correlation. Figure A5.2 reports the results of these simulations and show the more correlated turnout propensity and response propensity, the more change is overestimated for everyone. Importantly though, despite no relationship between age and survey response propensity, turnout is the most overestimated for the two youngest age groups.
Of course reality is not going to be as simple as our simulation but we can be confident we know two things: 1. Turnout is correlated with age 2. Propensity to take part in surveys is correlated with turnout (because turnout is too high in surveys and not just because people over report voting) in a way that is not just a function of age (because otherwise we would expect age weighting to give us the correct turnout) Figure A5.

Simulated turnout change between two elections.
Our simulation shows that you get age related turnout change biases from a very simple data generating model that works of these two ideas. While reality will be more complicated, it would have to be more complicated in a very specific way to counteract this problem -propensity to take part in things is would have to be less correlated with either turnout or non-response for younger people. Our expectation us that the likely relationship runs in the opposite direction (i.e. turnout/response propensity will be correlated with age). If this is the case, it would make the problem of attrition bias worse.

Appendix 6: Voting Eligible Population weighting targets
Even with the best sampling methodology and good response rates, demographic imbalances due to differential response rates are an inevitable part of survey research. Weighting survey samples to correct for these imbalances is required and when performed correctly, reduces the error in estimates of population values. The two main challenges for survey weighting are identifying the relevant weighting variables and gathering accurate targets to weight to.
Gathering weighting targets for election surveys is more complex than is generally acknowledged. Surveys generally use population level weighting targets derived from high quality official statistics such as the census and the Annual Population Survey (APS). These targets are appropriate when the target population and sampling frame are the whole population. For election surveys, however, the target population is not the whole population, but the sub-population who are eligible to vote in elections. The necessity to distinguish between these targets has expanded substantially since the rise in EU migration from 2004 onwards. EU citizens are not eligible to vote in UK General Elections and have substantially increased the gap between the Voting Age Population (VAP) and the Voting Eligible Population (VEP) . The demographic characteristics of migrants to the UK are not uniformly distributed across demographic characteristics-they are concentrated in younger age brackets. Nor are they uniformly distributed across the UK-they are particularly concentrated in London. Failure to account for these differences introduces systematic error into weighting targets, which will reduce the accuracy of population estimates.
Despite these problems, because demographic data is more readily available for the population as a whole political surveys in Britain are weighted to VAP targets. In contrast we develop at set of VEP weighting targets-the first time that this has been done, to our knowledge. To gather the targets we use APS microdata on nationality to distinguish between voting eligible and ineligible respondents, including respondents only if they state either a nationality or country of birth that is eligible to vote in UK parliamentary elections. 1 We then derive interlocked targets for age and education (degree level/below), and separate targets for gender and region.
An additional complication is that the APS does not ask respondents older than 69 who are not in the labour force their qualification levels. To provide this data we supplement the APS with an education target for this oldest age group derived from Understanding Society data (University of Essex et al. 2017). Understanding Society does not have a nationality variable, so we derive the proportion of 65-74 and 75+ year olds who have degrees from Understanding Society and multiply this proportion by the number of eligible people in each age group to get the relative size of the degree and non-degree groups. This percentage will still be relatively accurate, because the eligible proportion of the oldest age groups is very high.
To illustrate the differences between VAP and VEP, we show differences between each of these targets (both gathered using the same data sources) for 2017 in Table A6.1. Compared to the VAP targets, the VEP targets have lower numbers of younger and educated respondents than the VAP targets and have a lower share of respondents in London compared to other regions. response bias, such as political engagement). The correlation between and is a uniformly distributed random parameter that is restricted to plausible values of the correlation between demographics and turnout/response bias: 0.1 and 0.6. 3 As with continuous demographic variables such as age, is divided into quintiles for weighting purposes.
The 'turnout' variable is determined by a logistic model of the form: * = + And whether an observation is included the final 'survey' sample is determined by a logistic model of the form: * = The parameters and are set to approximate a plausible scenario where turnout is between 58% and 78% (depending on the correlation between and ) and the 'response rate' is 50%: = 0.05 and = 0.
Each round of the simulation begins with 5000 cases (the approximate size of the original sampling frame in the 2017 BES face-to-face) and the , , , and variables are generated as described above. Two sets of weights are then generated for the sample: 1) 'demographic' weights that weight the population quintiles of so they are evenly distributed in the sample. 2) + 'turnout' weights that add an additional weighting factor that weights ('turnout') to the correct population proportion.
We are primarily interested in how these weights affect the estimation of the levels of within quintiles of (i.e. turnout within age groups). For each simulation we calculate the error between the population level of in each quintile of and the estimated level of level of in each quintile of in the sample calculated with each set of weights. Figure A7.1 displays the distribution of these errors from 10,000 simulations. It clearly shows that the 'demographic' weights do not recover the accurate relative levels of turnout between quintiles, and that because and are correlated, the level of error is correlated with : it is highest in the lower quintiles (i.e. the younger age groups). In contrast, the + 'turnout' weights perform a much better job of recovering the correct level of turnout within quintiles of : the peak of the error distribution is close to zero for all quintiles.
The advantages to weighting to the correct level of turnout are hopefully clear. However, implementing turnout weighting in reality is more complicated than it would first appear. The turnout figures reported after each election are calculated as the proportion of total votes to total entries on the electoral register. This is not the same thing as the proportion of total votes to the total number of registered voters.
The total number of entries on the electoral register is greater than the total number of registered voters due to a combination of legitimate multiple registrations (i.e. students registering at home and at university, and home owners registering in two locations) and inaccuracies in the register. The actual turnout amongst registered voters is substantially higher than the official turnout statistics would suggest . Even accounting for these problems with the turnout denominator, registered voter turnout ignores those who are eligible to vote but who are not registered.
To properly weight a VEP sample, we need a VEP turnout estimate. Mellon et al. (2018) estimate VEP turnout for the UK. This VEP turnout target is also calculated using the APS and estimating the number of over 18 year olds resident in the UK who are eligible to vote in UK elections based on their nationality. The VEP turnout targets for 2015 and 2017 are 65% and 68%, respectively. 4 For similar reasons to those outlined above, in addition to weighting to the correct turnout figures, we also weight to the correct party vote shares amongst those that cast a vote in the election. Inflated turnout figures are indicative of turnout related response bias, and weighting to turnout reduces the impact of this response bias. Similarly, incorrect party shares are suggestive of other forms of response bias, and weighting to the result reduces the impact of these biases.

Appendix 9: Logistic regression model of age and turnout
Finally we test whether the relationship between age and turnout changed between 2015 and 2017 with a series of logistic regression models of turnout on the pooled data from both years. Specifically, for each of the self-reported and validated samples we estimate two logistic models of the form: (1) * = + + .
Where is an interval variable recording respondent ages, 2 is its squared term and is a binary variable that indicates whether the data comes from the 2015 or 2017 elections (2017 = 1). We are primarily interested in whether the age/turnout gradient has changed between elections, which is measured by the and ² terms. The results from these models are shown in table A9.1. Again, these results show no evidence of a change in the relationship between age and turnout between 2015 and 2017: the interaction terms have different signs in the self-reported and validated turnout data and in none of the models are the interactions statistically significant.