Executive summary
This study tested the impact of multiple different variants of an early voter vote tripling (EVVT) program, in which voters were messaged after casting early ballots and asked to remind three friends to vote. The experiment was partially conducted in partnership with Voto Latino, AAPI Victory Fund, and Woke Vote, in the Florida August 2020 State Primary.1
We find that:
The EVVT program increases the turnout of copartisan housemates by 0.95pp (p<.05). The effect is significantly smaller among non-copartisan housemates (p<.01), suggesting that targets are discerning in the partisanship of the housemates they choose to remind.
Having EVVT messages sent by an organization demographically aligned with the voter (based on race data on the voter file) significantly increases response rates for all groups and appears to significantly increase turnout effect, although the turnout effect increase is only significant for the AAPI Victory Fund group.
The best performing outbound script (out of eight possible variants) is: “Hi {{VoterName}}! It’s {{TexterName}}, a volunteer with {{OrganizationName}}. Public records show you voted in the Florida Primary — thanks for being a voter! We have a quick favor to ask: can we count on you to remind 3 friends to vote?”
The best performing follow-up script to pledgers (out of four possible variants) is: “Hi {{VoterName}}, it’s {{TexterName}}, with {{OrganizationName}}. Remember when you told us last week you’d get {{Names}} to vote? The time has come! Tomorrow is Primary Day in Florida, and polls are open 7am-7pm. If they didn’t vote yet, will you remind them to get out and vote tomorrow?”
There is such a thing as too many reminders to pledgers; those assigned to the four-reminder group have lower turnout effects among housemates, and appeared based on SMS data to simply start ignoring the messages after the first round of follow-up.
1: Vote Rev thanks the partner organizations without which this experiment would not have been possible — AAPI Victory Fund (Varun Nikore), Voto Latino (Ameer Patel), Woke Vote (Tristan Wilkerson). We also thank our implementing partner TextOut (Isaac Vazquez).
1. Research questions and experimental design
This experiment was designed to investigate three areas of inquiry. First, what is the expected impact of “early vote vote tripling” (EVVT), and what are the best practices in implementing it; second, what is the ideal follow-up schedule and script for vote tripling pledges; third, what is the best practice in the opening sentences of vote tripling SMS messages?
1.1 EVVT
EVVT is the vote tripling variant in which voters are messaged after they have already cast early ballots via in-person early vote (IPEV) or vote by mail (VBM), specifically thanking them for having voted and asking them to remind a few friends as one last action. EVVT had not previously been the subject of direct inquiry, although the research team had begun to suspect that the large results seen in the initial Texas 2018 study were because of the heightened impact of this specific VT variant. This experiment was the first one to explicitly and intentionally test EVVT. The goal was to validate the model’s effectiveness, as well as test A/B variants for a couple of components of the initial outgoing message — and to work out any kinks in the experimental design for an EVVT test, since the organization would be running a much larger one in the 2020 General Election months later.
The EVVT A/B variations are shown in the table below. The research team opted not to test a script that did not explicitly refer to the fact that the target had voted; the team felt that there was enough behavioral evidence for the benefits of a more specific approach as to make such a message not worth testing.
Chart 1: A/B variants for EVVT messaging
Note one important wrinkle in interpreting the results of EVVT studies using housemate spillovers. When we target early voters with vote tripling messaging, seeking turnout spillovers onto housemates, it is rational to restrict the experiment to households where the early voter has another housemates who has not yet voted. In so doing, we are selecting for high-enthusiasm prospective triplers — people who care enough about the election to have voted early — who are, furthermore, very likely to have a lower-propensity voter in their life to remind — the voter who lives with them, and did not yet vote, despite the prospective tripler having voted. These are both potentially higher-value triplers than in a typical vote tripling implementation (where there is no guarantee that the target is especially engaged in this election, and no guarantee that the potential tripler has a non-voter readily at hand to remind) — as well as, at the very least, cases where the tripling spillover effect is more likely to be visible within the household. As such, it is hard to compare EVVT estimates to other, classic vote tripling estimates. The former are not only a slightly different design, but are also estimated only in households that are especially likely to yield effects. This is not an issue for the internal validity of this study, but rather for its interpretation and generalizability.
1.2 Reminders
Prior to this experiment, VoteTripling.org (now Vote Rev) had done extremely limited experimentation on the number, nature, and schedule of reminder text messages, a question that was of growing import to partners over the summer of 2020. While the EVVT setting contains marked differences from the usual vote tripling pledge collection (VTPC) setting (with pledges occurring quite near the election, and pledgers likely to message friends immediately upon pledge solicitation), this experiment was the best opportunity to test reminder variants before the November elections. First, we experimentally varied the follow-up schedule in three groups. Each group was eligible to receive follow-up messaging for a different set of dates, and would receive follow-up messages on those dates if they were first messaged at least one day prior. For example, a voter first messaged on August 12, who pledged to triple, and was assigned to the Most follow up schedule, would receive August 14 and August 17 messages. A voter first messaged on August 9 who did not pledge and was assigned to the Least follow-up schedule would receive no reminders at all.
Chart 2: Schedules for follow-up messaging
Second, we experimentally varied three components of the follow-up message to pledged triplers. (Non-triplers did not receive different messaging variants, as this was of less interest to partners.)
Chart 3: Variants of follow-up messaging
The overall follow-up message also varied the middle sentence according to the date being sent. Depending on the subtreatment and the date, follow-up messages might read:
Hi [ContactName], it’s [TexterName], with Voto Latino. Remember when you told us last week you’d get Larry, Moe, and Curly to vote? The time has come! Tomorrow’s the last day to request a mail ballot in the Florida Primary. Will you remind them to request their ballots? Reply YES for details.
Hi [ContactName], it’s [TexterName], with VoteTripling .org. Thanks for agreeing earlier this month to get 3 friends to vote! Folks are encouraged to mail their ballots now, to ensure they arrive by 8/18. Will you remind them to submit their mail ballots today?
Hi [ContactName], it’s [TexterName], with Woke Vote. It’s your last chance to remind Peter, Paul, and Mary to vote! Tomorrow is Primary Day in Florida, and polls are open 7am-7pm. If they didn’t vote yet, will you remind them to get out and vote tomorrow?
1.3 Opening sentences
Finally, we varied two elements of the initial outbound text message, of general interest. First, we varied whether or not the sender explicitly identified as a volunteer. Second, we varied the organizational messenger, and in some cases the description thereof. Previous research had shown lower pledge rates among communities of color, and we hypothesized that organizations with more credibility in those communities would solicit higher pledge rates. To test the hypothesis, we partnered with AAPI Victory Fund, Voto Latino, and Woke Vote to serve — respectively — as the messengers to AAPI, Latin, and Black voters. Control messages — and messages to voters not identified as AAPI, Latin, or Black — came from VoteTripling.org. Finally, within VoteTripling.org messages, we varied whether the organization was described as a “voter turnout group” or a “progressive voter turnout group.”
1.4 Summary of subtreatments
In total, then, the initial outbound message independently varied four parameters, with up to 36 possible treatment permutations and 60 possible outbound messages:
Chart 4: Initial outbound scripts
Overall, the initial outbound script might read like any of the following:
Hi [ContactName]! It’s [TexterName], a volunteer with AAPI Victory Fund. Thanks for voting in the Florida Primary and making your voice heard! Could we ask you to reach out to 3 friends and urge them to join you in voting?
Hi [ContactName]! It’s [TexterName], with VoteTripling .org, a voter turnout group. Thanks for voting in the Florida Primary and making your voice heard! Now that you’ve voted, will you remind 3 friends to do so as well?
Hi [ContactName]! It’s [TexterName], a volunteer with VoteTripling .org, a progressive voter turnout group. Public records show you voted in the Florida Primary — thanks for being a voter! We have a quick favor to ask: can we count on you to remind 3 friends to vote?
And, voters were assigned to one of three follow-up schedules, and one of up to 8 possible messaging variants, yielding a total of 864 unique treatment assignments.2
The response messages sent to pledgers to request names in the project were not experimentally varied; previous research had settled on a best method to request names, and had concluded that providing some forwardable information to triplers was valuable. Voters who said yes were sent the following message:
Great! Let them know they can order a mail ballot online: openprogress.com/fl-vbm-request Just because it helps to think it through, what are the first names or nicknames of the 3 friends you'll remind?
The URL in the message linked to a page created by VoteTripling.org in partnership with Textout/OpenProgress, containing a dedicated link to the VBM request application for each county in Florida. No such page, to our knowledge, previously existed.
2: Complete script blocks are available here: https://docs.google.com/spreadsheets/d/1qOD9LTjCzLzS6_rWrziVIJ-aj1jPCgjQoEffV2OZj_k/
2. Sample universe, randomization, and implementation
The experiment was implemented during the Florida state primary in August 2020, beginning on August 4 and continuing through August 17.
The experiment was conducted in partnership with TextOut, who oversaw all texting for the project, and with AAPI Victory Fund, Voto Latino, and Woke Vote, who provided consultation on scripts, and served as the messengers for certain subsets of the messages. The primary project data was sourced from Alloy, and the sample universe contained any likely Democratic voter with a usable cell number, regardless of whether or not she had plausible spillover targets in the household. To simplify experimental implementation and ensure integrity, only one phone number was used per voter, and phone numbers that cross multiple voters were dropped, such that phone numbers are unique. 1,182,777 voters in the state were viable messaging targets by this definition.
Because the program was implemented on a rolling basis, randomizations had to be performed daily, as new data came in. We used daily TargetSmart updates to identify voters who were viable messaging targets and had newly cast ballots in the primary. If the new voter’s household had not yet been assigned a treatment status, they were included in that morning’s randomization, stratified by the race of the target and balanced on turnout score and age. Subtreatments were assigned sequentially, stratified on all previous subtreatments. The overall counts assigned to various treatments by day are shown in Table 1. Table 2 shows balance on the main treatment axis.
These new voters who trigger a household’s randomization are known as triggers. If there were two new voters in the same household, they were randomized together, with race ties broken according to a specified algorithm. The randomization included all subtreatments in the initial outbound message, and the overall follow-up schedule. If the household was already assigned to randomization, the new voter inherited the household’s randomization status. Note that the number of messages sent to any given household is endogenous; if there is a positive treatment effect, the treatment may produce additional early voters, which is to say additional valid messaging targets and thus additional tripling messages. As a result, the estimates do not have a straight-forward interpretation in terms of impact per message sent.
Note that the daily randomization did not confirm the presence of a spillover target before randomizing the household. Consider for example a two-person household, where both voted on August 4. When the randomization occurred on August 5, there were no valid spillover targets, but the randomization code did not detect this and did not throw out the household. As a result, Table 1 shows that there are relatively few valid spillovers (that is, spillovers who had not voted by the time the randomization occurred) compared to triggers. In total, there are 212,658 triggers, but only 97,591 spillovers, and only 61,515 copartisan spillovers.
Recorded outcomes from each round of texting were based both on the coding of TextOut volunteers and manual review by VoteTripling.org. Rounds of follow-up were sent on August 7, 11, 14, and 17, with scripts tailored to the situation and messages sent according to the amount of follow-up assigned in randomization. Note that voters could be dropped from follow-up for various reasons. For pledger follow-ups, voters were removed from further follow-up if, during an earlier follow-up, they reported that their pledge was already complete, or otherwise indicated — in the data team’s view — that they did not want further messaging, whether or not they explicitly opted out. For non-pledger follow-ups, voters were only included if they had originally entirely ignored the outgoing message; if they responded to the text in any way, they were not eligible for the follow-up. Outcomes coded from pledger follow-ups were not in all cases reviewed by VoteTripling.org and may not be consistent.
Chart 5: Follow-up schedule
The randomization for race-coded messaging was done as follows. Each voter was defined as Black, AAPI, Latin, or other. If at least one trigger was Black, AAPI, or Latin, the household was eligible for race-coded messaging; otherwise the household was not eligible. Notably, however, the actual messaging to the voter was based on their own race data, not on the household’s. So, in a household with two triggers, one white and one Black, the household would be randomized along the race-coded-messaging axis; if they were assigned to race coded messaging, the Black voter would get Woke Vote messaging, but the white voter would get VoteTripling.org messaging. This means that an individual’s receipt of race-coded messaging can differ from the household’s. The assignment of “progressive” messaging applied only to cases where race-coded messaging was not implemented — so, conversely, individual and household levels do not line up.3 Additionally, Woke Vote did not finalize approval for scripts until several days after the experiment began. As a result, Black voters assigned on or before August 6 received the generic VoteTripling.org messaging in all cases. This policy was implemented at the household level, so even a Black voter who only received messaging on August 12 but had a housemate who had triggered the randomization on August 5 would still get VoteTripling.org messaging.
Note that Florida collects race on Secretary of State forms, so race data is more likely to be accurate here than in other states.
Parameters at the tripler level are ambiguous, since multiple voters may be messaged in a single household, and some of these may be endogenous. For simplicity, tripler characteristics are the mean of all triggers’ characteristics for continuous variables, and one level chosen at random for categorical variables. This definition obscures some trigger characteristics via averaging, but only 4.5% of spillover targets are in households with more than one trigger.4 This definition ignores any messaged individuals who are not triggers, since they may be endogenous and are not evenly defined for treatment and control. Messaged non-triggers account for only 2.2% of the overall messaged population.
There was one meaningful mistake in implementation. Due to a coding error, 510 triplers who had pledged and provided names were not actually sent those names in follow-up messages, instead receiving the generic messaging provided to triplers who did not provide names. These 510 triplers comprise 28% of the overall sample who provided names. The error affected specifically those triplers who pledged after a certain date, meaning that the two groups are not necessarily comparable and the error can’t be treated as an additional accidental treatment axis.
3: In the data, individual_progressive and individual_race_coded_msg are properly specified randomizations for implementation of those messages at the individual level. Hh_progressive and hh_race_coded_msg are the household-level assignments, and are coded as missing in interracial households where the actual treatments delivered may have varied across voters.
4: 4.3% of spillovers with any partisanship, and 4.7% for Democratic spillovers.
3. Process results
Table 3 shows the overall results of the texting campaigns. Of note are the astronomically high pledge rates, with 9.4% of targets pledging on the first round, and nearly 12% pledging overall. 18% of pledgers provided names, a relatively low rate compared to other implementations, although this could be a function of the higher number of pledges solicited, with more marginal voters being induced to pledge but not willing to go the extra step. The high engagement did not translate into especially high opt-outs, with opt-out rates of 3.5% in the first round.
Anecdotally, it is also worth noting the qualitative impression of the research team and the texters that EVVT messages elicited a positive and engaged response, from voters who appreciated the notice that their VBM ballot had indeed arrived, and seemed happy to be recognized for having voted.
Table 4 shows the results of the initial outbound result by the main messaging variants. Note that the analysis only includes triggers and only includes the first initial outbound round, to ensure comparability across all subtreatments. The findings are relatively unambiguous: (1) “volunteer with” language increases pledges, pledges with names, and engagement; (2) “public records show” language produces more pledges and engagement, although the marginally generated pledges tend not to provide names, so that the pledges with names generated are similar to “thanks for voting” language; and (3) “we have a quick favor” language outperforms the other two phrasings of the ask. There are no meaningful differences in opt-outs across the scripts.
Table 5 shows the result of varying the messenger. Columns (1)-(3) show the results of race-coded messaging in general, showing that it significantly increases pledges and engagement, as well as lowering opt-outs. Columns (4)-(6) disaggregate these results by race/organization. The increased pledges are especially pronounced among Latin households, while the decreased opt-outs appear to be driven by differences among Black households. AAPI households show no significant differences, although the sample sizes are far smaller, with only 889 messaging targets included in this comparison. Columns (7)-(9) show the impact of describing VoteTripling.org as a “progressive” group. The results are inconclusive but suggest that if anything the “progressive” moniker depresses pledges, and perhaps increases opt-outs.
Table 6 examines the impact of sending multiple rounds of follow-up to non-pledgers. Note that targets were eligible for different numbers of follow-up rounds depending on their initial assignment date, and targets may systematically vary across assignment dates. To account for this heterogeneity, Panel A examines the marginal impact of the follow-up round for non-pledgers eligible to receive at least one round of follow-up, while Panel B examines the marginal impact of two follow-up rounds for non-pledgers eligible to receive both. The analysis assumes that the number of follow-ups — and not their timing — is of import. In line with previous studies, there are significantly diminishing marginal returns from additional follow-up, in both pledges but also (thankfully) in opt-outs. Note that the third round does not fall as far as the second round does, a replication of this perhaps surprising result from the CA-25 study.
Table 7 shows the impact on SMS outcomes of different follow-up scripts. The informal phrasing and the reference to the pledge timing both significantly increase response rates, and the informal phrasing also leads more targets to respond that their pledge is complete. (This success may have been a double-edged sword, as it caused us to remove the targets from further rounds of follow-up.) Optouts — either true optouts (“STOP”) or soft optouts (indications that the target should not be messaged again — are not impacted by either subtreatment. Table 8 shows the impact of the “reply YES” info treatment on August 7, showing no difference between the variants — or, if anything, the “reply YES” discouraged responses. Presumably the apparently-automated nature of “reply YES” outweighed any positive inducement to replying that it created.
Finally, Table 9 shows the marginal impact of each round of follow-up to pledgers. As in Table 6, the cohorts are separated by how many rounds they were eligible for. Note the high rate of fall-off in response rates: while about a third of pledgers engage with the first follow-up message, they basically stop engaging entirely with follow-up messages. While the further messages do not seem to meaningfully drive up opt-outs, the evidence is consistent with the idea that after the first follow-up the additional messages fade into the background and pledgers simply start ignoring them. This story suggests a world in which campaigns have one shot to remind a pledger, and further reminders (at least via SMS) will tend to be disregarded, implying a potential downside for intensive reminder schedules that blow their impact too early in the cycle.
4. Turnout results
Table 10 shows headline turnout results. While the turnout effect on all spillovers is not quite statistically significant (p=.184 without controls, p=.139 with controls), the turnout effect on copartisan spillovers is substantively large and highly significant (p=.044 without controls, p=.014 with controls). The point estimates of .8pp and .95pp are consistent with the 1pp estimate from the Texas 2018 Vote Tripling EVVT study.
Table 11 shows these headline results for a variety of other voting-related dependent variables. There is some evidence that most of the effect operates through increasing VBM requests — although, in a sample where 81% of voters who voted used VBM, this is perhaps not surprising. The results show no effect on Election Day voting, but given that only 2.5% of the sample voted on Election Day this result is again not surprising.
Tables 12 and 13 show results by follow-up schedule. Table 12 pools pledges and non-pledgers, while Table 13 disaggregates them; a pledger household is defined as a household with at least one pledger, even if other targets do not pledge. Analysis for pledger households is restricted to households who were randomized before August 7, in time to be eligible for all rounds of follow-up. In Table 12, results are relative to the control; in Table 13, they are relative to the no follow-up group, which, in the non-pledger case, may be thought to serve as a pseudo-control group. Overall, the medium follow-up group appears to be the optimal strategy, although we cannot reject the hypothesis that the low follow-up group performs just as well. The high follow-up strategy, on the other hand, appears to be actively counter-productive; the difference appears to be driven by significant differences between the high- and medium-follow-up strategies in pledger households. This evidence is consistent with the idea that high follow-up schedules used up their impact too early in the cycle, and pledgers were tuning them out by the time Election Day rolled around. Overall, if the high follow-up group were excluded from the analysis, the estimated impact of the program would rise to .106pp without controls (p=.025) or .120pp with controls (p=.007).
Table 14 shows turnout effect by the main script subtreatments, relative to a reference script. The only significant result here is in the third phrasing of the ask, which shows significantly lower turnout effect than the other two phrasings, consistent with its poor performance in the SMS outcomes.
Table 15 shows the impact of the messenger subtreatments relative to a reference script. In Columns (1) and (4), the overall race-coded messaging estimate is positive, but not significant (p=.249 without controls , p=.241 with controls). Columns (2) and (5) disaggregate the race-coded messaging by race/organization. The significant and astronomical positive effect among AAPI households is notable (p=.053 without controls, p=.070 with controls), although given the tiny sample size of AAPI households the point estimate should be taken with caution. The positive impact on Black households is marginally significant (p=.117 without controls, p=.119 with controls) while Latin households have no effect and negative point estimates (p=.393 without controls, p=.465 with controls). The story here confirms the value of race-coded messaging, but provides yet another example of an instance where pledge rates are not a reliable guide to turnout effects. Columns (3) and (6) show the effect of referring to VoteTripling.org as a “progressive” group, which has no detectable effect on turnout.
Table 16 shows the effect of different follow-up messaging variants on turnout. The only marginally significant result here is the impact of including a reference to pledge timing, which increases turnout among copartisan spillovers (p=.089 without controls, p=.129 with controls). The ‘reply YES’ messaging may depress turnout slightly, but this effect is not significant (p=.209 without controls, p=.198 with controls).
Finally, Table 17 investigates heterogeneity along several axes. Columns (1) and (2) directly test the theory implied by the overall results, that triplers are more likely to mobilize their Democratic housemates. The theory is true: the interaction between partisanship and treatment status is highly significant (p=.058 without controls, p=.008 with controls). (These columns test all spillovers; further columns so as to run analysis where effects are largest, restrict the universe to copartisan spillovers.) Columns (3) and (4) test a theory that became clear from manual inspection of the results: that much of the effect stemmed from households assigned early in the experiment. Indeed, the interaction between date assigned and treatment is negative and highly significant (p=.003 with and without controls).
Columns (5) and (6) test heterogeneity by salience of election, taking advantage of the fact that some congressional districts had contested primaries and some did not. (Congressional primaries were the highest-salience elections on the ballot.) There is no detectable difference in effect by contestedness of the election. Columns (7)-(10) test heterogeneity based on the turnout scores of the spillover targets — first in absolute terms, and second relative to the trigger. Neither test has significant effects, although the relative test shows weak suggestive evidence that the treatment effect is larger when the spillover target’s turnout score is lower than the trigger’s. (Turnout gap is defined as spillover turnout minus trigger turnout.)
Finally, Columns (7) and (8) test treatment effects by racial group. (Note this is different from Table 15, which tests the impact of the race-coded-message subtreatment by racial group.) None of the differences reported across racial groups are significant.
5. Discussion
This study provides evidence for large and significant effects on household spillovers from EVVT programs. While the effect size is perhaps biased upwards compared to a true implementation by selecting only for housemates where a promising spillover target exists, it is also biased downwards by — as with other vote tripling studies — only capturing effect that remains within the household, and by only reporting the effect on a single householder. The effect appears to come primarily from those who voted early in the early vote period, perhaps because these early voters are the most enthusiastic voters and therefore the most effective triplers, at least in the context of this low-salience election.
The evidence on scripts is fairly clear. The best performing outbound script (with the variable portions underlined) is: “Hi {{VoterName}}! It’s {{TexterName}}, a volunteer with {{OrganizationName}}. Public records show you voted in the Florida Primary — thanks for being a voter! We have a quick favor to ask: can we count on you to remind 3 friends to vote?” The best organization messenger is one known and trusted in the community. If using VoteTripling.org as a generic messenger, there is no need to identify as “progressive.”
The best performing follow-up script is: “Hi {{VoterName}}, it’s {{TexterName}}, with {{OrganizationName}}. Remember when you told us last week you’d get {{Names}} to vote? The time has come! Tomorrow is Primary Day in Florida, and polls are open 7am-7pm. If they didn’t vote yet, will you remind them to get out and vote tomorrow?” That is, use the informal phrasing, and if possible, refer to when the pledge was made.
The evidence suggests that multiple rounds of follow-up to pledgers may backfire, causing them to start ignoring the messages and then neglect to mobilize housemates when it really counts, although it is possible that the medium-follow-up schedule (up to two reminders for pledgers and one reminder for non-pedgers) outperforms the low-follow-up schedule (one for pledgers, none for non-pledgers).
Table 1: Counts
Table 2: Balance check
Notes: P-values are from OLS regressions. Copartisan spillovers are housemates who are modeled to be likely Democrats. Valid spillovers are those who have not voted as of the randomization date.
Table 3: Process results — headline outcomes
Notes: P-values are from OLS regressions with stratum fixed effects. “Any response” may be a more reliable indicator of engagement, as whether a given conversation should be coded as a pledge or optout can be subjective, and could be biased by sender fixed effects if individual texters were assigned to individual subtreatments.
Table 4: Process outcomes — by subtreatment
Notes: OLS regressions, standard errors in parentheses. * = p<.1, ** = p<.05, *** = p<.01. SE clustered by household. All regressions contain stratum fixed effects for the first randomization. Target controls are gender, race, 2020 turnout score, partisanship score, age, and high-quality phone indicator. Outcomes relative to baseline version: (1) no “volunteer” language, (2) “thanks for voting,” and, (3) “now that you’ve voted…” Only triggers are included.
Table 5: Process outcomes — messenger
Notes: OLS regressions, standard errors in parentheses. * = p<.1, ** = p<.05, *** = p<.01. SE clustered by household. All regressions contain stratum fixed effects for the first randomization. Target controls are gender, race, 2020 turnout score, partisanship score, age, and high-quality phone indicator. Outcomes relative to baseline text messages. Only triggers are included.
Table 6: Process outcomes — marginal impact of non-pledger followups
Table 7: Process outcomes — follow-up subtreats
Notes: OLS regressions, standard errors in parentheses. * = p<.1, ** = p<.05, *** = p<.01. SE clustered by household. All regressions contain stratum fixed effects for the first randomization. Target controls are gender, race, 2020 turnout score, partisanship score, age, and high-quality phone indicator. Outcomes relative to baseline text messages. Only triggers are included.
Table 8: Process outcomes — follow-up subtreats 2
Notes: OLS regressions, standard errors in parentheses. * = p<.1, ** = p<.05, *** = p<.01. SE clustered by household. All regressions contain stratum fixed effects for the first randomization. Target controls are gender, race, 2020 turnout score, partisanship score, age, and high-quality phone indicator. Outcomes relative to baseline text messages. Only triggers are included.
Table 9: Process outcomes — marginal impact of pledger follow-ups
Table 10: Headline results
Notes: OLS regressions, standard errors in parentheses. * = p<.1, ** = p<.05, *** = p<.01. SE clustered by household. All regressions contain stratum fixed effects for the first randomization. Controls are gender, race, 2020 turnout score, partisanship score, age, and — for the associated trigger — gender, race, age, 2020 turnout score, partisanship, and high-quality phone indicator. Only includes spillovers who had not voted at the time of randomization. Copartisan spillovers are any housemates modeled to be likely Democrats.
Table 11: Effect on other voting-related outcomes — copartisan spillovers only
Notes: OLS regressions, standard errors in parentheses. * = p<.1, ** = p<.05, *** = p<.01. SE clustered by household. All regressions contain stratum fixed effects for the first randomization. Controls are gender, race, 2020 turnout score, partisanship score, age, and — for the associated trigger — gender, race, age, 2020 turnout score, partisanship, and high-quality phone indicator. Only includes spillovers who had not voted at the time of randomization. Copartisan spillovers are any housemates modeled to be likely Democrats.
Table 12: Turnout effects by follow-up schedule — initial randomization before August 7
Notes: OLS regressions, standard errors in parentheses. * = p<.1, ** = p<.05, *** = p<.01. SE clustered by household. All regressions contain stratum fixed effects for the first randomization. Controls are gender, race, 2020 turnout score, partisanship score, age, and — for the associated trigger — gender, race, age, 2020 turnout score, partisanship, and high-quality phone indicator. Only includes spillovers who had not voted at the time of randomization. Copartisan spillovers are any housemates modeled to be likely Democrats. Households randomized on or before August 6.
Table 13: Turnout effects by follow-up schedule by pledger status
Notes: OLS regressions, standard errors in parentheses. * = p<.1, ** = p<.05, *** = p<.01. SE clustered by household. All regressions contain stratum fixed effects for the first randomization. Controls are gender, race, 2020 turnout score, partisanship score, age, and — for the associated trigger — gender, race, age, 2020 turnout score, partisanship, and high-quality phone indicator. Only includes spillovers who had not voted at the time of randomization. Copartisan spillovers are any housemates modeled to be likely Democrats. Effects relative to low-follow-up subtreatment. Columns (1) and (2) are households where at least one trigger pledged, with assignment before August 7. Columns (3) and (4) are other households, assigned before August 14.
Table 14: Turnout effects by script subtreatment
Notes: OLS regressions, standard errors in parentheses. * = p<.1, ** = p<.05, *** = p<.01. SE clustered by household. All regressions contain stratum fixed effects for the first randomization. Controls are gender, race, 2020 turnout score, partisanship score, age, and — for the associated trigger — gender, race, age, 2020 turnout score, partisanship, and high-quality phone indicator. Only includes spillovers who had not voted at the time of randomization. Copartisan spillovers are any housemates modeled to be likely Democrats. Effect relative to reference script: (1) no “volunteer” language, (2) “thanks for voting,” and, (3) “now that you’ve voted…”
Table 15: Turnout effect by messenger subtreatments
Notes: OLS regressions, standard errors in parentheses. * = p<.1, ** = p<.05, *** = p<.01. SE clustered by household. All regressions contain stratum fixed effects for the first randomization. Controls are gender, race, 2020 turnout score, partisanship score, age, and — for the associated trigger — gender, race, age, 2020 turnout score, partisanship, and high-quality phone indicator. Only includes spillovers who had not voted at the time of randomization. Copartisan spillovers are any housemates modeled to be likely Democrats. Effect relative to reference script.
Table 16: Turnout effect by follow-up messaging
Notes: OLS regressions, standard errors in parentheses. * = p<.1, ** = p<.05, *** = p<.01. SE clustered by household. All regressions contain stratum fixed effects for the first randomization. Controls are gender, race, 2020 turnout score, partisanship score, age, and — for the associated trigger — gender, race, age, 2020 turnout score, partisanship, and high-quality phone indicator. Only includes spillovers who had not voted at the time of randomization. Copartisan spillovers are any housemates modeled to be likely Democrats. Effect relative to reference script. Includes only households with one pledger, assigned on or before August 6.
Table 17: Heterogeneity — 3g-heterogeneity
Notes: OLS regressions, standard errors in parentheses. * = p<.1, ** = p<.05, *** = p<.01. SE clustered by household. All regressions contain stratum fixed effects for the first randomization. Controls are gender, race, 2020 turnout score, partisanship score, age, and — for the associated trigger — gender, race, age, 2020 turnout score, partisanship, and high-quality phone indicator. Only includes spillovers who had not voted at the time of randomization. Copartisan spillovers are any housemates modeled to be likely Democrats.
