Policy

# 74

Note: A version of this document was reviewed by Hauke Hillebrandt, David Manheim, Samuel Hilton, Aaron Gertler, and Tara Law, who I thank profusely for their useful and encouraging feedback.

At the end of this document, I make a proposal for high-leverage lobbying for altruistic ends, conduct a preliminary cost-effectiveness analysis, and offer some rough suggestions for policies that might be worth advocating in Congress. Several reviewers said this was their favorite part of this piece, so you may want to read it first.

### Conclusions

My confidence in each conclusion leads each bullet point, in parentheses.

• (85%) Informational lobbying can shift policy relative to the counterfactual.
• (80%) Well-resourced interest groups are no more or less likely to achieve policy success, in general, than their less well-resourced opponents. I've eliminated this conclusion in light of some thoughtful questions from @jackva and @Jamie_Harris below. In general, I think that there's no evidence that resource-rich groups are effective irrespective of their deployment of those resources. But that conclusion is (1) itself not well-supported enough to include here and (2) not practically useful. The most useful takeaway here is probably: resources are not a guarantor of success, because success is contingent upon many other factors. Basically, less well-resourced groups can win and often do.
• (70%) Dollar for dollar, lobbying against a proposal is more effective than lobbying for a proposal.
• (65%) Lobbying is more effective when issues are not salient to the public, or (more particularly) to the lobbied member's constituency. After more review, I've eliminated this conclusion as well. I'm now somewhere around 50/50 on this, with some expectation that the direction of the effect is actually reversed. I think this was in fact a serious error on my part: I (1) heavily overweighted the theoretical work (2) included apples-to-oranges work that focused on PAC expenditure and (3) did not take into account explicit evidence to the contrary. I also overweighted my own deductions, which are worth spelling out here: There is good evidence that lobbying expenditure increases with issue salience. If the below conclusion about net direction lobbying is true, then lower salience translates into less overall spending, which translates into less total expenditure for the bigger spender, and a higher probability of success per dollar spent.
• (60%) The net direction of lobbying, e.g. the difference between the number of groups lobbying for or against a proposal, or the difference between total expenditure for or against a proposal, influences policy outcomes. Organizations that lobby more on an issue relative to their adversaries are slightly more likely to win.

The quality of the evidence in this domain is considerably worse than in other areas of inquiry in the social sciences. In general, the question I tried to ask myself when assessing these potential conclusions was this: "How likely would I have been to see this evidence, in aggregate, if this statement was false?"

### Introduction

A few months ago, I started thinking about the Tullock paradox: "Why is there so little money in politics?" That is, given the extraordinarily large rewards available to an interest that captures the cooperation of government, why is outright corruption so rare, and why is industry's expenditure on political influence relatively small? Tullock's observation was made mostly with regard to campaign contributions, but it applies just as well to lobbying: the market for new vehicles in the U.S. was worth nearly half a trillion dollars in 2019, but the entire automotive industry spent only $70 million on lobbying -- 0.01% of the value of the market. Tullock's observation suggests that, for groups that want something from government, lobbying is potentially an extraordinarily long lever with which to move policy. The question motivating this review is whether that lever is also available for altruistic ends. This review has three purposes: 1) To serve as a reference regarding the effectiveness of informational lobbying for groups or individuals considering funding advocacy or influence campaigns. 2) To provide a starting point for others who may be interested in conducting further research themselves. 3) To try to answer to my satisfaction the question of whether lobbying works before we try to uncover the most effective lobbying tactics and techniques. #### A caveat It's clear that lobbying has a negative connotation. Stories about the pernicious impacts of lobbying by cigarette companies, oil companies, and professional organizations are common, and it seems clear that the influence of moneyed interests can hurt the public. But lobbying is just a tool: informational lobbying, as I'll discuss further down, is no more or less than talking to government. Although the research I review here is focused on professional lobbyists, lobbying in general can be done by anyone, from K Street professionals to high school students. I don't take a moral stance on whether lobbying in general is good or bad, or whether it should be restricted in some way. ### My process #### Focus The empirical section of this review is intentionally narrow. I focus on lobbying directed at legislative bodies at the national level (e.g. the Senate and House) in the United States. Aside from my being American and therefore more familiar with the American system, there are a few reasons for this narrowness: • The structure and effectiveness of lobbying at the federal level is likely different than at the state or local level, or at the national level in other countries. It's not clear to me that we can draw meaningful conclusions if we mix these contexts, so I choose not to. My review indicates this is probably a good choice: Bennedsen and Feldmann (2002), for example, write that "observers of the policy process are often struck by the intensity of lobbying—or lack thereof—in the system on the other side of the Atlantic." • Since the U.S. Congress wields considerably more power than other bodies that are plausibly susceptible to lobbying, determining whether Congress can be influenced seems (by comparison to others) to be a top priority. • Lobbying of federal regulatory agencies, which also seems like a very promising avenue for investigation, is likely substantively different than lobbying legislators. For the sake of clarity, I try not to mix these different settings. Crucially, I focus on informational lobbying: when lobbyists communicate directly with policymakers to shift policy. Policymakers include members of Congress and their staffs, members of the executive branch, bureaucrats, and regulators. I expressly do not investigate the empirical impact of political campaign contributions or grassroots advocacy here. However, the topics are closely tied together, as I describe at the beginning of the "Theoretical Background" section. #### Methodology This is a narrative review. I have tried to synthesize what I have learned about the current state of academic knowledge on informational lobbying at the level of the U.S. Congress, communicate my read on the evidence, and highlight the main points of uncertainty. I took a snowball approach and started with two origin points: (1) a handful of relevant Google Scholar queries (e.g. "empirical lobbying effectiveness", "interest group influence in politics"), and (2) the empirical studies described in the literature reviews I summarize below. Starting from these lists, I followed both backward- and forward-citation chains until I was satisfied I had encountered most of the main themes in the literature and had read most of the empirical work relevant to the review. Not all major studies are included, largely as a function of my own resource constraints. John Mark Hansen's 280-page Gaining Access: Congress and the Farm Lobby, 1919-1981 (1991), for example, does not get the attention it deserves here. Readers may note that the theoretical section is heavy on the political science literature and comparatively light on relevant subfields in economics, such as public choice theory. The primary reason for this is that the literature in public economics, while theoretically instructive, is somewhat orthogonal to my main concern here, which is this: How effective, as a means of influencing policy, is the provision of information directly to legislators, and what circumstances influence a lobbyist's chances of realizing the policy that s/he advocates? ### Background and Terminology #### Why informational lobbying? I use the definition provided by De Figueiredo and Richter (2013): Lobbying is "the transfer of information in private meetings and venues between interest groups and politicians, their staffs, and agents." Lobbying can be done by the interested parties themselves, but the term "lobbyist" generally refers to individuals and firms that attempt to influence government on behalf of private clients, such as corporations or nonprofits. It is also conventional to distinguish between "inside" lobbying, which consists of direct communication with policymakers, and "outside" lobbying, in which organize interests attempt to shift public opinion in hopes of affecting legislative decisions. This review concerns only inside lobbying. Lobbying and political campaign contributions, as to PACs, are both types of corporate political activity (CPA), along with in-kind gifts, participation in Chambers of Commerce, legal action, grassroots influence, and other similar activities. It's necessary for me to specify my focus on informational lobbying because lobbying is often conflated with other kinds of CPA, particularly PAC contributions, in both the academic literature and the mainstream press, as in this FT article by the economist Tim Harford. However, not only are these avenues of influence structurally and legally very different, but spending on lobbying in a given year is typically an order of magnitude larger than spending on campaign contributions. Some researchers (e.g. Lux, Crook and Woehr 2010) view the spending differential between lobbying and PACs as an indication of the greater relative importance of lobbying. My focus on informational lobbying as opposed to other avenues of influence is pragmatic. Evidence about the efficacy of informational activities is valuable and portable. I see two possible takeaways from this research: • If lobbying is effective under certain conditions, well-resourced organizations might be justified in taking a portfolio ("hits-based") approach, paying professional lobbyists to advocate for a menu of highly impactful policies meeting those conditions, or conducting such advocacy on their own initiative. For a proposal on this front, please see the section entitled "Effective Lobbying." • Information about the circumstances under which political influence campaigns are successful can improve resource allocation efficiency for groups that already fund political advocacy. #### How does lobbying work? In 2019, reported lobbying expenditures totaled around$3.5 billion. This money, spent largely by industry, pays the salaries of registered lobbyists; it doesn't go directly to members of Congress or their staff. Lobbyists conduct research on relevant topics, including the political constituencies that may support or oppose a given policy, and set meetings with bureaucrats and politicians, which they use to provide information that they hope will sway votes or policy positions. Many firms keep lobbyists on retainer in order to protect the status quo, and change their political influence strategies only when some new threat arises. This means that the effect of lobbying can be hidden: the absence of new legislation can also be a goal of lobbying. Unlike campaign contributions, lobbying expenditures are not limited by law; nevertheless, most firms do not lobby—although they are often members of trade associations and other groups that do engage in political advocacy. There is some evidence that lobbyists sometimes help legislators write legislation (Hall and Deardorff 2006).

In the U.S., lobbyists are required by The Lobbying Disclosure Act of 1995 to register with the government and to file regular reports on their activities and on their spending. A lobbyist is anyone who makes more than one communication to a covered official while on contract to a particular client. Covered officials include not just members of the Senate and House but their employees, employees of Congressional committees, etc. - regardless of their job level or function.

The Executive branch is also covered by the Lobbying Disclosure Act, but is less extensive: only the President, the Vice President, Schedule C political appointees (such as a special counsel), and a handful of other top-level positions are covered by the Act.

#### How can lobbyists influence policy?

• Lobbyists can help to set the agenda. In other words, they can help to define the things that people in government are paying attention to: making legislators aware of issues, creating public awareness and pressure, or publicizing causes and specific items of legislation.
• They can provide evidence and arguments to legislators or staffers that either (a) persuade them to change their opinion or (b) supply them with rhetorical ammunition that they can use in committee, during floor debate, or in media appearances.
• They can provide outside research that legislative offices or bureaucracies may not have the internal capacity to develop on their own (see the "legislative subsidy" section below).
• They can demonstrate alignment between constituents and the lobbyists' preferred policy outcomes. For example, lobbyists can conduct polling showing that a policy is preferred by a majority of a Senator's constituents.
• They can participate in "issue networks" or subgovernments, attempting to shift policy over a longer time scale by publicizing and propagating their preferred policies and stances within a given policy domain.

### Theoretical background

I don't claim to include all relevant theories in what follows, but I aim to make some useful distinctions and synthesize some of the main themes in the literature. I do not include the sizable literature on the drivers of policy change in general; for that I refer the reader to compendia such as Theories of the Policy Process (Sabatier and Weible 2018).

#### Exchange and persuasion

Theoretical work about political influence has tended to fall into two broad (though not exclusive) categories. In general, there are "exchange theories" and "theories of persuasion" (Hall & Deardorff 2006).

In exchange models, campaign contributions or in-kind gifts are exchanged for Congressional votes. This is, to a first approximation, the model assumed in most lay conversations about "money in politics." There is mixed evidence regarding this phenomenon, also known as vote buying. For a good (if now somewhat dated) lit review on the topic see Ansolabehere, de Figueiredo, and Snyder (2003).

In persuasion models, information is at the center of the relationship between legislators and lobbyists. Lobbyists, in this view, can use information to either (1) genuinely persuade legislators or (2) convince self-interested legislators that their chances of electoral success would be enhanced by taking the advocated position. Lobbyists have a comparative advantage in their ability to expend significant resources collecting and presenting such information, e.g. polls of relevant constituencies.

#### Money, access, and information

As Wright (1990) notes, many studies on the impact of contributions on Congressional voting fail to make a distinction between contributions and informational lobbying, which makes it impossible to separate the impact of lobbying from the impact of the contributions themselves. This puts a finer point on an important way that the exchange and persuasion models come together: money may not buy votes, but it's possible that money buys access.

Legislators have limited time, and if they prioritize contributors when arranging meetings and phone calls, then these big spenders will have more opportunities to influence policy. This is the view advanced by Wright, as well as by Austen-Smith (1995), and supported by some empirical evidence (Tripathi, Ansolabehere and Snyder 2000 and Langbein 1986). Contributions may also serve a signaling function to legislators with reference to possible future lobbying, demonstrating that interest groups have both the resources and the intent to provide necessary information.

#### Theoretical models of lobbyist behavior

##### Pluralism and the communications approach

In the mid-20th century, a prominent strain in political theory was the pluralist model, which conceived of policy as being primarily a result of competition between diverse interest groups (Denzau and Munger 1986). To pluralist thinkers like Truman (1951) and Latham (1952), competition between interest groups was a source of stability and a fundamental component of democracy. Archetypal pluralist works (e.g. Dahl 1956) held that political outcomes are the result of struggle between interest groups, who compete within specific issues or policy areas and tend to be influential primarily within those domains, rather than dominant in society in general. In the pluralist view, power is dispersed across society; policy is a result of mediation and healthy discussion between a large number of competing interests. The influence of interest groups is viewed as a benign representation of the will of the people, weighted by its salience to a specific subset of citizens (Lowery and Gray 2004). The pluralism hypothesis implies that interest groups primarily serve the purpose of raising legislators’ consciousness of the preferences of existing constituencies; they cannot make legislators do things that are against their own electoral interests.

Pluralism fell from favor under the weight of empirical research finding (a) that American voters were largely ignorant and apathetic about politics and (b) that organized interests seemed to work less like "pressure groups" and more like "service offices" for members of Congress already agreeing with them. At the time, lobbyists spent considerable resources communicating with already-friendly legislators (Bauer, Poole and Dexter 1963). Several other works of the same period and much later came to similar conclusions, holding broadly that lobbyists are at the beck and call of legislators, who have a great deal of agency in how they allocate their time (Ellis 1992).

In his book The Washington Lobbyist (1963), Lester Milbrath presented interest group politics as a game of communication, rather than of influence. The communications approach provided a compelling explanation for the tendency of groups to lobby already-agreeing legislators: targeting opposing legislators constituted an attempt to pass a message through a closed channel. Milbrath, along with Bauer et al., is largely responsible for the modern conception of lobbying as being largely about information. (McGrath 2017).

##### The supply price for policy

Denzau and Munger (1986), building partly on earlier ideas developed by Stigler (1971), offered a formal model predicting that legislators are most likely to respond to "organized group pressures" when constituents are uninformed or indifferent. In their model, the amount that interest groups must offer a legislator for his services (e.g. his vote) depends on the legislator's productivity and the preferences of voters in his district. The price of policy depends on a legislator's comparative advantage in providing votes. Comparative advantage is determined by the hostility of voters at home to a given policy and the productive capacity of the individual legislator (e.g. seniority, committee assignment, competence). The primary conclusion of this influential paper is that "contributions can have some influence on policies about which voters are divided, ignorant, or indifferent."

##### Counteractive lobbying

Austin-Smith & Wright (1994) proposed a theory of counteractive lobbying in response to the by-then mainstream awareness that interest groups often lobbied legislators predisposed to agree with them. Counteractive lobbying occurs when groups lobby in order to offset the pressure applied by opposing interests. These authors propose that lobbying does, in fact, influence the positions taken by legislators, but that lobbying by friendly interests takes place in order to prevent allied legislators from changing their positions.

##### Legislative subsidy

Hall & Deardorff (2006) have proposed a legislative subsidy model, in which lobbying operates with respect to budget, not to preferences. Lobbyists offer a "gift" to like-minded legislators, providing assistance in the form of legislative intelligence, information, connections to other groups and supporters, structured arguments, and other types of free intellectual labor. This more recent model, like the counteractive lobbying theory, addresses the demonstrated tendency of lobbyists to spend time and effort lobbying those who already agree with them. The subsidy model and other models are not mutually exclusive; lobbyists are free to select from a "menu" of available strategies. This model is (for example) consistent with the Denzau & Munger model described above: lobbying under the subsidy model can be seen as an in-kind contribution to legislators, so the supply price of policy is coterminous with lobbying labor purchased. Hall & Deardorff nonetheless argue that "preference-centered" lobbying (in which lobbyists seek to change legislators' preferences) is relatively rare.

##### Interpretation

Smith (1984) observed the lobbying activities of the National Education Association in the spring and summer of 1977 and proposed a model in which advocates—including not just lobbyists but also fellow members of Congress, staffers and others— influence members' interpretations of the consequences of a policy. Smith proposes a two-stage model of policy formulation. In the first stage, members interpret the policy; in the second, they attempt to discern how their interpreted policy consequences impact their own policy goals. Advocates can shift policy by demonstrating to legislators how a given policy could advance their own goals. However, since basic human constraints (e.g. bounded rationality) cause interpretations to be unstable, advocates' influence on policy is also itself unstable.

##### Political accountability

Smith (1995) singles out Arnold (1990) as having a rare comprehensive theory in which policy outcomes are driven by (1) whether legislators perceive constituents as attentive or inattentive to the issue at hand, (2) how salient the issue is to constituents, and (3) how capable constituents are of holding legislators accountable. Arnold's theory holds that pressure succeeds only when an issue is viewed as important to attentive citizens and not to inattentive citizens, and when it has champions in Congress who will do everything "procedurally necessary" to ensure that legislators are held accountable by constituents.

### How lobbying is studied empirically

Research about political influence has tended to focus overwhelmingly on campaign contributions, to the neglect of informational lobbying. Several of the papers I reviewed (e.g. Evans 1996, Bombardini and Trebbi 2019, Caldeira and Wright 1998) suggest that the disproportionate attention afforded to campaign contributions is mostly a function of greater data availability; at least one (Milyo, Primo and Groseclose 2000) suggests that this tendency in the literature is a case of "looking under the lamppost": data about campaign contributions has historically been easier to come by.

#### Data sources

While information about federal political contributions has been available since 1974, data about federal lobbying expenditures has only been available since 1999 as a condition of the Lobbying Disclosure Act. The Honest Leadership and Open Government Act of 2007 made electronic filing mandatory, and data since 2015 is now available from the Senate. Senate data is formatted as XML; more helpfully structured data is available from the Center for Responsive Politics (CRP). A considerable portion of recent research on this topic uses data from CRP.

A significant chunk of the research I consider below focuses on the relationship between lobbying expenditure and firms' financial outcomes; these studies often merge CRP data with business information from Compustat. Some sources also use lobbyists.info (now washingtonrepresentatives.com) for biographical information about lobbyists and about their relationships.

Some enterprising researchers (e.g. Evans 1996) have pursued much more labor-intensive data collection operations, with significant amounts of hand-coding and manual review. Baumgartner et al 2009, one of the most impressive recent works on this topic, followed a random sample of issues over a four-year period, collecting information via large numbers of interviews with lobbyists and policymakers and hand-coding their responses.

#### Research designs

To my knowledge, there has never been a controlled experiment about informational lobbying— this kind of thing would be tricky to pull off, though it's probably not out of the question. In general, work attempting to assess political influence through communications channels takes one or more of the following approaches:

• Cross-sectional studies
• e.g. Alexander, Scholz and Mazza (2009), who model tax savings due to repatriated earnings from the American Jobs Creation Act of 2004 (AJCA) as a function of lobbying expenditures by firms.
• Panel studies
• e.g. Meade and Li (2015), who follow a group of firms over a 13-year period and model effective tax rate as a function of lobbying expenditure with year and firm fixed effects.
• Instrumental variables (IV) / 2-stage least squares (2SLS)
• e.g. Gawande, Maloney and Montes-Rojas (2009), who find a source of outside variation in lobbying expenditure (an instrumental variable, in econometrics-speak), and use it to estimate the effect of pro-tourism lobbying by foreign countries on tourism from the U.S. to the lobbying nations. (See here for a good explanation of the IV design)
• Interviews and Surveys
• Baumgartner et al (2009) contains a considerable statistical element, but the research is heavily informed by qualitative interviews that provide a lot of background context (and were also put into structured form and analyzed)
• Case studies
• In addition to interviews, Kingdon (1984) reports and codes policy case studies in order to identify where interest groups seem to be important.

### Previous Literature Reviews

#### Smith (1995)

Smith identifies a body of work that concludes, from formal models of the lobbyist-legislator relationship, that a primary persuasive virtue of lobbyists is their ability to affect decision-making by reducing legislators' uncertainty about bill consequences. Smith finds little empirical support for these models, with the exception of Austin-Smith and Wright's (1992) theory of counteractive lobbying, for which the authors find empirical support in Austin-Smith and Wright (1994).

Smith identifies eight studies from the preceding decade that assess the impact of lobbying on roll call votes, and finds that the research presents a "mixed picture" of the possible effects of lobbying. Smith finds that most empirical studies suffer from measurement and specificity errors, and draws no firm conclusions about the effectiveness of lobbying in general.

#### Potters and Sloof (1996)

This review explicitly focuses on quantitative data around "how and when" interest groups influence public policy, but does not distinguish between different types of interest group influence (informational lobbying is among those types studied). The authors make a useful distinction between two types of variables in empirical interest group research: (1) the behavior of individual policymakers and (2) policy outcomes. Potters and Sloof find that empirical research on interest group influence outside of campaign contributions is rare. A key observation of this paper is that single-equation models are often used to estimate interest group influence, but that in reality causality flows in both directions: politicians' positions influence group behavior, as well as the reverse.

Potters and Sloof conclude that there is evidence for the effectiveness of lobbying, particularly when bills have "a narrow focus and low public visibility." In addition, they find that interest groups have greater success influencing policy when politicians are under low democratic pressure (e.g. low intra-party competition) and less success when there is a well-informed electorate.

#### Burstein and Linton (2002)

Burstein and Linton analyze every article on policy change in the top three sociology and top three political science journals. Their unit of analysis is the effect of a predictor on a dependent variable, e.g. between a measure of either the resources or activities of an organization and a policy outcome. They find 53 articles containing a total of 230 effects. They find that statistical significance is a more common standard in this literature than substantive significance, and reviewed findings using a scale that takes both standards into account. Burstein and Linton find that the impact of interest groups is statistically significant roughly half the time, and substantively important (according to the authors' own assessment) in about a fifth of cases.

Burstein and Linton conclude that the hypothesis that interest groups influence policy significantly is "not as well supported by the data as we might expect." However, they offer three potential sources of bias in their estimate.

First, they suggest that their estimate of the frequency with which political organizations influence policy may be too high relative to its actual effect—they note that few studies include variables for public opinion, and suggest that the inclusion of such a variable might cause other apparently significant relationships to disappear.

Second, they suggest that their estimate may be too low: they note that theory predicts that it is the way in which resources are deployed, not the size of the resources themselves, that influences policy; nonetheless, most studies rely on data about resources, regardless of deployment. This thesis is supported by their data: measures of resources alone impacted policy 45% of the time, but resources and activities together impacted policy 65% of the time.

Finally, they suggest that the range of extant lobbying research is very narrow— it covers only a small number of issues and countries, and may simply not be representative enough to draw meaningful general conclusions.

#### Lowery (2013)

Lowery addresses himself to the findings of three previous literature reviews—Smith (1995), Baumgartner and Leech (1998) and Burstein and Linton (2002)—which held that evidence about the effectiveness of lobbying is inconclusive or inconsistent. Lowery starts from the premise that null findings are common—assuming that, due to the file drawer effect, significant findings are even rarer than they at first appear—and offers some explanations for why that might be so.

Lowery suggests several possible explanations for null findings:

• The most important instances of influence may not be observable
• Issues compete for space on the agenda, and exogenous forces, not interest groups, ultimately determine what ends up on the agenda.
• Lobbying is two-sided, and a failure on one side is a success on another. Null findings may simply leave out the influence of major indirect determinants of policy, such as public opinion.
• Selection bias: The most common type of research is an analysis of a small number of groups working on one issue over a specific amount of time, and most influence studies focus on "substantive large and controversial issues on which many organizations are engaged." However, "these are precisely the issues on which influence might be least expected because public opinion is likely especially constraining." In point of fact, most policy proposals see little or no lobbying.
• A quote that gave me a lot to think about: "It could well be that lobbying is far more successful on these kinds of proposals. But we will not know this for sure unless we stop confusing studying lobbying influence with studying lobbying influence on controversial issues."

#### De Figueiredo and Richter (2013)

De Figueiredo and Richter list "empirical regularities" in the study of lobbying. The uncontroversial topline summary is that lobbying is pervasive, that lobbying increases when issues are more salient or when stakes are high for an organized interest, and that both expertise and connections seem to matter for lobbyist effectiveness, but the magnitude of these effects is unclear.

They also list three main challenges in the empirical study of lobbying:

• Time series persistence: very little within-interest-group variation
• Omitted variables: datasets typically used to study lobbying may not contain important data necessary to adjust for these variables
• Endogeneity: groups may be more likely to lobby when they believe they are more likely to succeed

In general, the authors find that lobbying "can be valuable in some contexts" but that its success is heavily dependent on timing, targets, and tactics.

### Studies that were instrumental in my conclusions

You can read summaries and key findings for 25 of the studies I reviewed in a Google sheet here. What follows are brief summaries of the studies that I found most useful and convincing.

#### Caldeira and Wright (1998)

Caldeira and Wright model individual Senate votes in the Bork, Souter, and Thomas Supreme Court nominations as a function of lobbying. They use an instrumental variables approach in which votes are modeled (in the second stage) as a function of partisan affiliation, ideological orientation, constituency support for the nominee, and a lobbying index constructed (in the first stage) as a function of party, ideology, constituency, campaign contribution, and organizational strength (e.g. the number of interest groups lobbying for and against the nominee) that contribute independently to the value of the lobbying index. They find a significant relationship between lobbying and Bork vote; the authors find counterfactually that if the opposition lobbying effort had been 10% smaller, the opposition to Bork would have decreased by two votes. In the case of the Thomas nomination, the authors find that "a 25% reduction in opposition lobbying effort would have garnered an additional 33 votes for Thomas."

#### Goldstein and You (2017)

In a pretty novel approach, these authors examine the impact of lobbying expenditure by U.S. cities with populations over 25,000 on disbursements via earmarks and the American Recovery and Reinvestment Act. They also use a novel instrument: the existence of a direct flight between a given city and Washington, D.C, which the authors argue is a source of exogenous variation in lobbying expenditure. They present several convincing robustness checks on this instrument in their Appendix, but an unaddressed area of uncertainty is whether the existence of a direct flight can cause legislators to be more familiar with a city and therefore more likely to disburse funds to it. If so, this could compromise the instrument.

Goldstein and You find that a 10% increase in lobbying expenditures is associated with increases in earmarks and ARRA grants of 10% and 4%, respectively. As in the case of Montes-Rojas (2013), this framing can obscure the magnitude of the impact. Cities spent about $20k on lobbying on average, and received an average of around$1m and $130m in earmarks and ARRA funds respectively, so the coefficients suggest marginal returns of 50x and 2600x, respectively. #### Ludema, Mayda and Mishra (2018) Ludema, Mayda, and Mishra examine tariff suspension bills, of which 1400 were introduced in Congress between 1999 and 2006. They model passage of these bills as a function of lobbying expenditure for the Congresses spanning that time period and use a set of instruments that they argue (convincingly) are not readily available to legislators and are therefore unlikely to influence tariff suspensions. They find that increases in effective lobbying expenditure (expenditures relative to a minimum baseline) in opposition to suspensions are significantly associated with suspension, expenditure in support is likewise associated with no suspension (with a smaller effect size); likewise number of opponents. ### Findings #### Does lobbying work? A very heavily qualified yes. I think that there are enough good-quality quantitative studies, with enough convincing theoretical backing, to conclude that informational lobbying can effectively shift policy. There are a number of reasonable concerns with the state of the research: much of the existing work is not causal in nature, and I found almost no published papers on this topic with null results, suggesting that a bit of the file drawer effect may be at play here. For this reason, it's not advisable to attempt to extract any quantitative estimate of the effectiveness of lobbying from the published work. Two separate factors give me some confidence in this conclusion. The first is the relatively high quality of many of the quasi-experimental studies and the evidence these higher quality studies represent in aggregate. The second, perhaps more debatable, factor is my sense that some of the repeating patterns in correlational and observational work on this topic are best explained by a causal story rather than omitted variables. These patterns include the observed association between the net gap in lobbying (supporting minus opposing) and policy success, consistently higher coefficients on negative lobbying relative to positive lobbying, and the broad agreement on conditions for lobbying success that I discuss below. #### When does lobbying work? Previous literature reviews have identified a number of stylized facts about the determinants of lobbying success that seem to have a strong empirical basis. Please note that the scope of my review centered on legislative lobbying at the national level, and that the following areas of agreement should be considered to be restricted to the legislative domain. The first area of broad agreement is that lobbying success seems to be more likely on issues of low salience to the electorate, issues with low public visibility, or issues about which the public is largely ignorant. This is a theoretical prediction made by Arnold (1990) and Denzau and Munger (1986) and observed empirically by e.g. Evans (1986). Literature reviews conducted by de Figueiredo and Richter (2013) and Potters and Sloof (1996) consider it well-established. EDIT: I've included the above paragraph because it accurately reflects the research referenced, but after digging in more detail into to the studies referenced in the literature reviews, I don't have confidence that it's actually the case. The second area of broad agreement is that, all other things being equal, lobbying against policies seems to be more effective than lobbying for policies (see e.g. McKay 2011, Caldeira and Wright 1998, Ludema, Mayda and Mishra 2018); in other words, lobbyists for the status quo are likelier to succeed. The third area of agreement is that having high-level allies, either in committee leadership, the executive branch, or among Congressional staffers, is consistently associated with policy success (see e.g. Baumgartner et al 2009, de Figueiredo and Silverman 2006, Evans 1996). Some other areas for which I found some empirical support but no broad consensus are as follows: • Interest groups with a larger membership are more influential (Potters and Sloof 1996) • Intergroup conflict damages a group's chances of regulatory success. In addition, conflict is more likely to occur on issues where there are winners and losers, e.g. regulatory and redistributive issues (Evans 1996). • Policies that fail in a given Congress have a poor shot in subsequent Congresses (Kang 2016) ### Case Studies I think it's worth listing a few instances when minor interventions have significantly impacted policy on a large scale. I'm not picky about whether these examples would have been quantifiable by any of the methods reviewed here; I just want to give an idea of the settings in which lobbying (external or internal) can influence policy (in the bureaucracy or the legislature) and some sense of the magnitude of its potential effects. These cases demonstrate instances in which policy was meaningfully impacted simply by communication, even though they do not all resemble a typical picture of lobbying. • Terry Dawson, an advisor to the House Science Committee, was apparently responsible for influencing committee chair George Brown to require, in a piece of 1991 legislation, that NASA investigate asteroid detection and deflection strategies (Morton 2017 and Morrison 2005). The vast majority of such asteroids are now known and tracked as part of the resulting program. • The famous Einstein-Szilárd letter, though perhaps not commonly thought of as lobbying, directly influenced FDR's decision to initiate the Manhattan Project. • OPP's History of Philanthropy includes several salient examples including the following: • Nuclear arms control research funded by the Carnegie and MacArthur Foundations resulted in a report co-authored by Ashton Carter, the Harvard professor (and future Secretary of Defense). Carter briefed Senators Nunn and Lugar on the report and apparently helped to write the text of the resulting bill. • The Robert Wood Johnson Foundation funded the SmokeLess States initiative, which shifted in 2000 to focus entirely on policy change. Ben Soskis's OPP-funded research reports that various evaluations have found that SmokeLess States successfully lobbied for increased excise taxes on tobacco and other state-level policies intended to combat smoking. • Corporate lobbyists have had considerable success engaging in the type of legislative subsidy known as legislative drafting: they literally write sections of bills that become law. Ellis and Groll (2019) report that lobbyists from Genentech and Citigroup have provided talking points or proposed language to lawmakers that found their way, verbatim, into legislation. • Stokes & Breetz (2018) report that during the early 2000s, the solar industry targeted Republican members of Congress "whose districts benefitted [sic] from solar employment." Success came in 2005 with a Business Solar Investment Tax Credit, which was meant to expire at the end of 2007, but which was extended and expanded for eight years as a part of the 2008 economic stimulus. In 2016, solar leasing companies again lobbied successfully for an extension of the tax credit. Some sources see the ITC as an important factor behind the recent growth of solar power. ### "Effective lobbying" As observed by many of the papers referenced here, most issues face no lobbying. If the counteractive lobbying theory is true, then the simple act of introducing lobbying on one of these issues can stimulate opposition; if the opposition is from an opponent willing to "outbid" on policy, then they are likely to win, and lobbying can fail, or fail to be cost-effective. Yet in some cases there simply is no organized opposition, and a relatively small investment can meaningfully alter wording, put a policy on the agenda, or cause a bill to become law. I want to propose something along the lines of "effective lobbying": a rigorous approach to institutional-level change, starting with the legislature, that would take a portfolio approach to policy advocacy. Although my review has focused on lobbying Congress, any approach to effective lobbying would have to operate with an eye to all possible venues: state and local governments, regulatory agencies, and the executive branch. #### How would this work? 1) Identify a set of high-impact policies that meet the following criteria: • Very high expected value (e.g. in DALYs) • Mass of distribution of potential outcomes is mostly above zero 2) Identify the policies least likely to face organized opposition (e.g. from entrenched and well-resourced interests inside or outside the government, or from the public) 3) Identify what avenues of lobbying are most appropriate for enabling the policy goal • Determine the level (e.g. regulatory, congressional, state, municipal) at which a policy would be implemented • Identify the appropriate lever by which this policy could be brought about (e.g. via legislation, rules change, budget processes, or executive order) 4) Identify the proximate course of action that would be required to increase the likelihood of each such policy change in the near term: • Re-wording of an existing bill • Bringing the attention of influential legislators to the policy (e.g. putting it on the agenda) • Narrow lobbying to shift votes for the passage of a bill already on the floor or in committee, or to allow a bill to come to a vote in either setting • Broad advocacy of bills already on the floor • Writing model legislation in cooperation with legislative staffers 5) Fund research necessary to maximize the effectiveness of lobbying for each policy • Low-cost public opinion polls in relevant constituencies to identify public support or opposition • Identification of key legislators or legislative staff who can serve as champions • Independent analysis, as by think tanks or external research groups • Indirect lobbying to encourage political allies to expend resources on an issue. 6) For each policy in the portfolio, fund lobbying and advocacy informed by the research produced in (4). 7) Measure impact • Any such program needs to clearly define policy success, systematically track resource expenditure, and —ideally—conduct experiments in order to draw causal conclusions about strategy choices. The bulk of the work here needs to happen in (1) and in (4). What would make this approach "effective" is the rigorous restriction of the policy portfolio to only potentially very high-value policies, and the deployment of a systematic approach to tracking and facilitating policy success. #### Does effective lobbying already exist? Traditional lobbyists are hired by clients, either for- or non-profit, to advocate for specific policies. Once hired, their success or failure within the specified domain is only partially under their control. As the research detailed here has found, policy success or failure is determined mostly exogenously, and lobbying success is heavily context-dependent. The approach I've suggested is focused on finding the overlap between policies that have high social value and those that face favorable contexts for lobbying success. So the first difference is in issue selection: traditional lobbyists advocate as they're hired to do, whether or not they're likely to be successful, but favorable context is a necessary condition (though not a sufficient one) for effective lobbying. There's still the question of the degree to which lobbyists working today actively seek to maximize their efficiency, and the degree to which they make use of the mountain of data that is presently available to assist in this end. To conclusively answer these questions would require a whole separate literature review, but I did review some existing studies that paint a broad picture. Hojnacki and Kimball (1998) find evidence that interest groups take into account the degree of support they believe they have in a legislator's district when deciding whether to lobby her and that interest groups do target influential committee members. Victor (2007) reviews the literature and finds that (a) interest group perceptions of legislator knowledge about a topic impact their choice of whether to lobby directly or indirectly, that (b) interest groups consider how the public views an issue before starting to lobby, that (c) lobbyists expend more resources when there is less consensus on a topic, and that (d) groups lobby more when they perceive that there is a procedurally high likelihood of success. So there is evidence that lobbyists are aware of the factors that seem to enhance success. The degree to which they take a systematic approach is unclear; several of the papers I read for this review hinted at a belief in considerable heterogeneity in effectiveness across lobbyists. If this heterogeneity is significant, then there is probably a lot of value to be gained by identifying, engaging, and facilitating the work of only the most skilled lobbyists. Foundations and social change organizations already engage in this type of activity, funding advocacy across a wide range of topics. This is very close to what I am proposing; the only difference is that I am suggesting a simultaneously more diversified and more targeted approach. Rather than plowing millions into advocacy groups that fight for a broad roster of changes on a small number of hotly contested issue areas (e.g. climate change), my proposal is to invest smaller amounts on targeted advocacy for specific, high-expectation policies across a wide range of possible focus areas. #### Some ideas Just to give a flavor of some of the potentially high-impact (in expectation) policies that one might want to lobby for. Please note that some of these are likely to face significant opposition, so they're not ideal candidates for effective lobbying, but they're not ipso facto bad candidates (the expected value of avoiding nuclear war, for example, is very high). #### Cost-effectiveness analysis I conducted a limited cost-effectiveness analysis via Monte Carlo simulation by "backing out" distribution parameters from (in my view, conservative) 95% confidence intervals. Here are my assumptions: • The per-policy lobbying cost lies with 95% confidence between$20,000 and $10,000,000 dollars, and is distributed log-normally. • The change in probability of policy implementation lies with 95% confidence between 0 and 5%, and is distributed normally. Note that (1) the baseline probability of policy implementation is very low to start with and that (2) this setup also includes a 2.5% chance that lobbying could actually decrease the likelihood of policy implementation. • The expected impact of policy implementation lies with 95% confidence between 1 and 100,000,000 DALYs, and is distributed log-normally. I think I'm using a pretty conservative upper end here, given the potential value to be gained by genuinely effective longtermist policies. • I also assume that in 10% of cases, effective lobbyists get the sign of impact wrong, so the sign of impact is distributed binomially, with a 10% chance of a negative sign. This results in the following setup: The punchline here is that the cost per DALY of this approach lies with 95% confidence between$-157,632 and $27,974,720, with a median value of$879, and with 70% confidence between $0 and$247,117. There's also a 20% chance that the cost per DALY lies in the "sweet spot" of between $0 and$50. So the median value is semi-promising, but there's a lot of uncertainty and there's a 10% chance effective lobbyists will pay as much as $100,000 to make people's lives one year shorter. However, it's worth noting again that I tried to be both very conservative and very uncertain here, and that further rationalization of this setup could result in more precise (and perhaps more optimistic) results. ##### Sensitivity In addition to the basic cost-effectiveness calculation outlined above, I present in this graph four variations with different parametrizations. The initial calculation is the red distribution (labeled "Original"). In Variations 1 through 4, I adjust only one parameter relative to the original. The key takeaway of this analysis is that there's an important tradeoff in the way I've set this up: lower median values (and lower, narrow ranges) for positive costs per DALY come at the price of a higher probability of a negative cost per DALY. The interpretation of a negative cost per DALY would be, as I mentioned above, that substantial expenditures are made lobbying for a policy that ends up having a significant negative impact. This is obviously a function of the binomial term in the setup described above. I think this term is defensible: for any policy that could plausibly affect large numbers of people in a significant way, there's a small but nontrivial possibility of severe, negative, unintended consequences. One of the counterintuitive consequences of this setup is that reducing the likelihood of harm increases the price per DALY: that's because reducing harm doesn't stop us from spending: we get positive value that otherwise would be missed, but we still end up spending a lot of money on policies at the low end of the impact distribution. So (with this setup, at least) we can't get a better cost distribution just by reducing harm. So there's two main preconditions to getting a good cost/DALY in lobbying-based policy change. The first is to advocate policies, as I indicated above, that have some probability of producing very large-scale benefits; the second is (obviously) to spend less (see Variation 3). I think this suggests a couple things. First, the bundle of lobbied policies should include as many genuinely longtermist policies, or policies addressing global catastrophic risks, in order to shift as much mass as possible onto the right side of the impact distribution with as little risk of catastrophic error as possible. Second, if you accept the premise that accidental error in the form of a "sign reversal" is a plausible model, then reducing the error rate becomes proportionally more important as policies that affect larger numbers of people are advocated. One possible avenue here is to preference policies that enhance decision-making in the long term, that improve the functioning of institutions, over policies that encourage near-term changes to government decisions on specific topics. # Appendix ### How does federal legislation work in the U.S.? Lobbyists interested in advancing a policy proposal must first find a bill sponsor. After a bill is introduced, it is considered by the standing committee that has jurisdiction over the bill, which depends on its content. Most bills fail at this stage. Committee chairmen have enormous power, and (as things currently stand) they can decide unilaterally whether or not to bring a bill to a subcommittee, which would hold a public hearing on the bill's content. On some occasions, these public hearings are held in front of the entire committee. After a public hearing, the bill moves up to the committee level, where it is subject to "markup": amendments, rewrites, and votes that can make or break the bill. The Congressional leadership then decides whether or not to schedule a floor vote on the bill. At this point, the processes in the Senate and House diverge. In the Senate, the Senate Majority Leader can immediately schedule a vote—or refuse to schedule one. Once on the floor, amendments can be offered indefinitely until debate is cut off by a cloture vote (sixty senators). In the House, if a majority of the committee approves the bill, it is sent to the House Rules Committee, which receives amendments from members and determines which amendments will be considered on the floor. For any particular bill, the Rules committee can institute an "open rule," which allows consideration of all amendments, a "closed rule," which allows none, or a rule defining specifically which amendments can be considered. On the floor, House members must abide by the rules set by the Rules Committee. Differences between House and Senate versions of the same bill are reconciled, and then final, identical versions of the bill are approved or rejected by floor votes in each chamber. Lobbyists can attempt to intervene at any stage during this process: they can attempt to modify the wording of a measure, to encourage votes at the subcommittee, committee, or floor level, to persuade committee or Congressional leadership to bring a bill to a vote, or to influence the way in which House and Senate versions of the same measure are reconciled. One of the primary ways in which corporate lobbyists can impact their clients' earnings is via earmarks—specific appropriations within a bill that define precisely where and how some segment of the appropriated funds are to be spent: this is what many empirical researchers are considering when they attempt to determine the effects of lobbying expenditure on firm financial performance. For some classes of issue, significant policy changes are possible via lobbying at other levels and branches of government. Policies of federal agencies, for example, can have a significant impact: think here of rulemaking by the FDA, EPA, and other regulatory agencies. Considerable lobbying takes place at the state level as well, and can indirectly influence federal policy: in major states such as California and New York, state regulations can be a blueprint for federal ones. ### Code for future in-depth analyses As part of my research, and to support the derivation of parameters for the above cost-effectiveness model, I went through the lobbying disclosure data in detail and explored a bit. I discovered that it's hard to get your hands on it in a useful format, so I wrote some code to convert it from XML to tabular, as well as to produce some hopefully useful summary analysis like those below. That code is available publicly on my GitHub. It is clean enough, but is mostly meant to serve as a starting place for others who want to pick up where I have left off here. The three summary graphics below are extracted from lobbying disclosures for 2018. Each filing contains a number of issue-specific reports, some of which contain references to specific bills and some of which do not. In order to get thes plots below plots, I took reported amounts from quarterly lobbying reports and allocated them across the bills and issues extracted from the text of each report. • For the red and green plots, I extracted mentions of all House Resolutions from the text of the quarterly statements and divided the reported amounts evenly between them and reported lobbying issues, then plotted the density. If a quarterly filing in the amount of$10,000 contained 6 issues, 2 of which referenced 3 specific bills each, then each bill was allocated $1,000, and each issue without a bill was allocated$1000.
• The green plot includes all the bills that were references in disclosure filings. This is several thousand bills, 99% of which never came to the floor for a vote.
• The red plot includes only the ~100 bills that came up for a floor vote in 2018.
• For the blue plot, I divvied the amounts up only at the issue level, without reference to specific bills. The unit of observation here is an individual issue report, of which there are four annually, so the distribution pictured is the distribution of (allocated) expenditure per issue, per lobbyist, per quarter. So the median amount that a lobbyist spent on any given issue in any given quarter of 2018 was about $200. ### Bibliography, etc. My bibliography is on Google Sheets, along with a tab providing short summaries, research designs, and key results of about 25 of the empirical studies that I reviewed. # 74 New Comment 20 comments, sorted by Click to highlight new comments since: Wow, this is really fantastic work! Thank you for the effort you put into this. Overall I think this paints a more optimistic picture of lobbying than I would have expected, which I find encouraging. To follow up on a couple specific points: (1) Just in terms of my own project planning, do you have an estimate of how long you spent on this? If you had another 40 hours, what uncertainties would you seek to reduce? (2) Your discussion of Bumgartner et al. (2009) is super interesting. You write "Policy change happens over a long time frame." I wonder if you could expand on this briefly. Do you mean that it takes a lot of lobbying over years before a policy change happens, or do you mean that meaningful policy change happens through incremental policy changes over time? (3) Your finding that lobbying which protects the status quo is much more likely to be effective seems particularly actionable. I mean, once put into words it seems obvious, but it's a point I hadn't thought about before. I notice, though, that your list of ideas seems to consist of positive changes rather than status quo protection. I wonder if it would be worth brainstorming a list of good status quo issues that might be under threat. Protecting these would be less exciting than big changes, but for exactly the reasons you outline here more likely to work! (4) I'm interested in thinking a bit more about uncertainty about policy implementation. This is something that we're currently grappling with in our models of policy change where I work (Founders Pledge). On the one hand, the Tullock Paradox suggests that we should expect lobbying to be extremely difficult (otherwise everyone would do a lot more of it). On the other hand, we've noticed that very good policy advocates seem to quite regularly affect meaningful policy changes (for example, it seems like the Clean Air Task Force regularly succeeds in their work). In your model you write that "the change in probability of policy implementation lies with 95% confidence between 0 and 5%, and is distributed normally." I'm not sure about this, but I imagine the distribution of "chance of affecting policy success" over all the possible policies we could work on is much flatter than this. Or perhaps it's bimodal: there are some issues on which it is near impossible to make progress and some issues where we could definitely get policies implemented if we spent a certain amount of money in the right way. Perhaps we want to start with a low prior chance of policy success, and then update way up or down based on which policy we're working on. Do you think we'd be able to identify highly-likely policies in practice? (5) I found this post super helpful, but overall I think I'm still quite puzzled by the Tullock Paradox. If anything I'm more confused now, given that this post made me update in favour of policy advocacy. I think perhaps something that's missing here is a discussion of incentives within the civil service or bureaucracy. A policy proposal like taking ICBMs off hair-trigger alert just seems so obvious, so good, and so easy that I think there must be some illegible institutional factors within the decision-making structure stopping it from happening. I don't blame you for excluding this issue considering the size of this post and the amount of research you've already done, but it seems worth flagging! Thanks again for a great post! I'm really excited about more work in this vein. Thanks for your response! (1) I spent something like 100 hours on this over the course of several months. I think I could have cut this by something like 30-40% if I'd been a little bit more attentive to the scope of the research. I decided on the scope (assessing the effectiveness of national-level legislative lobbying in the U.S.) at the beginning of the project, but I repeatedly wound up off track, pursuing lines of research outside of what I'd decided to focus on. I also spent a good chunk of time on the GitHub repo with the setup for analyzing lobbying data, which wasn't directly related to the lit review but which I felt served the goal of presenting this as a foundation for further research. If I had 40 more hours, I'd intentionally pursue an expanded scope. In particular, I'd want to fully review the research on lobbying of (a) regulatory agencies and (b) state and local governments. I explicitly excluded studies along those lines, some of which were very interesting. (2) Thanks for asking for clarification on this. Baumgartner et al mean that it takes a long time for policy change to be observed on any given issue. After starting to pursue a policy goal, lobbyists are more likely to see success after four years than after two. Baumgartner et al include a chapter that is mostly critical of the incrementalist idea of policy change, which they trace to Charles Lindblom's 1959 article The Science of "Muddling Through". Incrementalism is tied to Herbert Simon's idea of "bounded rationality." Broadly, the incrementalist idea is that policymakers face a broad universe of possible policy options, and in order to reduce the landscape to a manageable set, they choose from only the most available options, e.g. those closest to the status quo: "incremental" changes. Frank Baumgartner, with Bryan Jones, is now well-known for their theory of "punctuated equilibrium." This is a partial alternative to incrementalism which uses the analogy of friction to understand policy change. Basically: the pressure builds on an issue over a period of time, during which no change occurs. After the pressure is overwhelming, policy shifts in a major way. I say that punctuated equilibrium is a "partial" alternative because Baumgartner and Jones actually collected data that seems to demonstrate that policy change follows a steeply peeked, fat-tailed distribution. Their overall takeaway is that very small changes are overwhelmingly common, but moderate changes are relatively uncommon, and very large changes are surprisingly common. To come back to your question, Baumgartner et al might say that although most policy change is incremental—like year-to-year changes in agency budgets—meaningful policy change happens in a big way, all of a sudden. (3) I agree with you. I think some of my suggested policies are not likely to be those most effectively advocated for, and I included them just to give a flavor of the types of things we might care about lobbying for. Coming up with more practicable ideas is, I think, a much bigger, much longer-term project. I also think that although lobbying for the status quo is more effective all other things being equal, it may not be the best use of EA resources to focus exclusively on that side of things. That's because (per the counteractive lobbying theory) on many issues there is are latent interests that will arise to lobby against harmful proposals. It's hard to identify beforehand which proposals will stimulate this opposition, so there's a lot of prior uncertainty as to whether funding opposition to policy change is marginally useful in expectation. (4) There are a lot of takes on the Tullock paradox, but I'll present two broad possible explanations. • Explanation A: Lobbying is basically ineffective, and the reason we don't see more lobbying is that most organizations recognize its ineffectiveness. • Explanation B: Lobbying is highly effective, and the reason we don't see more lobbying is that relatively small expenditures can exert enormous amounts of leverage. Given the evidence here, I'm starting to be a lot more inclined toward Explanation B. I think it's demonstrably not the case, as you have noted with respect to the Clean Air Task Force, that organizations that lobby are wasting their money. For both altruistic and self-interested interest groups, the rewards to be captured are very large, and they make it worth the risk of wasting money. Alexander, Scholz, and Mazza (2009), for example, find a 22,000% return on investment. If Explanation B holds, then the question is really just why the market for policy isn't efficient. Why hasn't the price of lobbying been bid up to the value of the rewards to be captured? I think it seems likely that this is down to multiple layers of information asymmetry (between legislators and their staffs, between these staffers and lobbyists, between lobbyists and their clients, etc.), which create multiple layers of uncertainty and drive the expected value of lobbying down from the standpoint of those in a position to purchase it. I agree with you that a normal distribution is probably not the best choice to model the expected incremental change in probability. I felt like, given my CI for this figure and my sense that values closer to 0% and values closer to 5% were each less likely than values in the middle of that range, this served my purposes here - but please take my code and modify as you see fit! Perhaps we want to start with a low prior chance of policy success, and then update way up or down based on which policy we're working on. Do you think we'd be able to identify highly-likely policies in practice? I don't know. I think it's worth investigating. It seems like, given an already-existing basket of policies we'd be interested in advocating for, we can make lobbying more cost-effective just by allocating more resources to (e.g.) issues that are less salient to the public. I have a sense that lobbyists, do, in fact, do something like what you're describing, and that this is part of the resolution to the Tullock paradox. Money spent on lobbying is not spent all at once: lobbyists can make an effort, check their results, report to their clients, and identify whether or not they're likely to meet with success in continued expenditure. If lobbying expenditure on a given topic seems unlikely to make a difference, then it can just stop. I wasn't able to find anything on how this process actually works, so the next step in this research is to actually talk to some lobbyists. (5) I think perhaps something that's missing here is a discussion of incentives within the civil service or bureaucracy I agree with this too. I'd love for an EA with a public choice background to tackle this topic. I didn't consider it as part of my scope, but I do want to note something: A policy proposal like taking ICBMs off hair-trigger alert just seems so obvious, so good, and so easy that I think there must be some illegible institutional factors within the decision-making structure stopping it from happening. I think this is probably true in many if not most cases of yet-to-be-implemented policy changes that are obvious, good, and easy. It is probably true in this case. But I want to warn against concluding that, because some obvious, good, and easy policy change has not been implemented, that means that there is some illegible institutional factor that is stopping it from happening. It could just be that no one has been pushing for it. In EA terms, it's an important and tractable policy change that's neglected by the policy community. Given what I know about the policy community, it's not at all difficult for me to imagine that such policies exist. Thanks for this! Something that came to my mind as I was reading this was that it might be time for an update of CEA's list of good policy ideas that won't happen (yet). You wrote that "It seems like, given an already-existing basket of policies we'd be interested in advocating for, we can make lobbying more cost-effective just by allocating more resources to (e.g.) issues that are less salient to the public." This made me think it might be useful be to make a list of EA-relevant policy ideas and start organizing them into a Charity Entrepreneurship-style spreadsheet. Something I'll keep musing on! I'm also curious about what motivated you to take on this project, and what you're planning to work on next? I'm replying again here to note that I've struck the salience point from my conclusions. I've noted why up top. I now have a lot of uncertainty about whether this is the case or not, and don't stand by my suggestion that salience is a good guide to resource allocation. I like this spreadsheet idea and think I may kick it off (if you haven't already done so!) I took the project on because I got interested in this topic, went looking for this, couldn't find it, and decided to make it so that it might be useful to others. I wasn't feeling very useful in my day job, so it was easy to stay motivated to spend time on this for a while. I tend to be most interested in generalizable or flexible approaches to improving welfare across different domains, and this seemed like it might be one of those. Some areas I'm thinking about exploring. These are pretty rough thoughts: • Some more exploration of strategies for ameliorating child abuse in light of the well-known ACES Study. GiveWell and RandomEA have both explored Nurse-Family Partnerships. This problem is just so huge in terms of people affected (and in terms of second-order effects) that I think it's worth exploring a lot more. I'm particularly interested in focusing on child sexual abuse in particular. • Aggregating potentially cost-effective avenues to improve institutional performance. I'm curious about thinking at a higher level of abstraction than institutional decision-making. It seems worthwhile to put together the existing cross-disciplinary evidence on the question: what steps outside of those explicitly focusing on rationality and decision-making can companies/nonprofits/government agencies take to increase the probability that they make good decisions? A good example of one such step is in the apparent evidence that intellectually diverse teams make better decisions. • Long-term cost-effectiveness of stress reduction for pregnant women (with potential effects of infant mortality, maternal health, and long-term outcomes like brain development and violence). • Review of recent innovations that seem to like they might have potential for expediting scientific progress (like grant lotteries) Very impressive and interesting piece, thanks for this! I am a colleague of smclare at Founders Pledge and work a lot on modeling policy advocacy across charities. Would be great to have a chat. It's a great and very useful summary of the literature, hugely valuable. That being said, I am less convinced by the approach taken in the cost-effectiveness model which seems to somewhat contradict the prior analysis which stresses (a) contextuality and (b) strategic choice. As far as I can tell, the modeling tries to be entirely general (contradicting a)) and assumes (b) independence between variables that are very likely related (such as spending and success probability). Please let me know if I completely misunderstand what you are doing! My sense from doing this kind of work in charity evaluation is that we would want to move towards a "suite" of models for different situations -- e.g. (i) pushing genuinely new ideas, (ii) opportunistically exploiting policy windows (such as stimulus season), (iii) pushing high-impact low probability policies, (iv) averting policy rollbacks etc. and that the cost effectiveness for these kinds of things will be very different and essentially unrelated to a generic estimate such as the one featured at the end of your piece. To me it seems that this kind of classifying into the sub-models will be the only way to realistically bound parameters of interest and also that the dynamics underlying are sufficiently different that they should probably have their own models. Some minor points/comments: Something I found missing is a bit are standard explanations for why there is so little money in politics, namely (1)because dominant firms in districts have lots of indirect power via employees voting so that they do not need to spend money. (2) It also seems that the report somewhat under-emphasizes the idea of lobbying equilibria where marginal increases by one side would be quickly countered, which would make it look like additional money could be effective when in effect it is not. I also think that the conclusion which, I believe, mostly draws from Baumgaertner " (80%) Well-resourced interest groups are no more or less likely to achieve policy success, in general, than their less well-resourced opponents." is quite surprising and I would be curious to find out why you think that / in how far you trust that conclusion. Hello and thank you for your response! Your criticism of the cost-effectiveness model is fair. Thematically, I guess it does contradict the spirit of my prior analysis in that it avoids the concerns of strategic choice. I was actively trying to be as general as possible, and actively trying to err on the side of greater uncertainty by not including any assumptions about correlatedness, though it occurs to me now that making such an assumption (e.g. a correlation between expenditure and likelihood of success) would actually have increased the variance of the final estimate, which would have been more in line with my goals. When I have time, I may comment here with an updated CEA. I also agree that the only useful way to do this analysis is, as you've described, with a suite of models for different scenarios. I don't have a defense for not having done this beyond my own capacity constraints, though I hope it's more useful to have included the flawed model than not to have one at all (what do you think?). I also think that the conclusion which, I believe, mostly draws from Baumgaertner " (80%) Well-resourced interest groups are no more or less likely to achieve policy success, in general, than their less well-resourced opponents." is quite surprising and I would be curious to find out why you think that / in how far you trust that conclusion. Thanks for this, in particular. I think your surprise stems from a lack of clarity on my part. The reason I have high confidence in this conclusion is that it's a much weaker claim than it might seem. It does stem primarily from Baumgartner et al and from Burstein and Linton (2002). The claim here is that resource-rich groups are no more or less likely to get what they want--holding all else equal, including absolute expenditure and the spending differential between groups and their opponents. There are three types of claim that are closely related: 1) Groups that spend more relative to their opposition on a given policy are likelier to win 2) Groups that spend more in absolute terms are likelier to win 3) Groups that have more money to spend are likelier to win So I found fairly consistent evidence for (1), some evidence for (2), and no real evidence for (3). It's not obvious to me that (3) should be the case irrespective of (1): why would resource-rich groups succeed in lobbying if they deploy those resources poorly? It seems like the success of resource-rich groups is dependent upon (1), and that (3) should not be true when in isolation, unmediated by (1). Although Baumgartner et al conduct an observational study, the size of their (to me, convincingly representative) sample to me suggests that if such an effect exists, it should be observable as a correlation in their analysis. The association they observe is pretty small. I have to say, though, that in writing this comment, my confidence in this conclusion has eased up a bit, so I'm curious to hear your response. I also think that since Baumgartner et al do find a small effect, I probably overstate the case here. Baumgartner et al offer a theoretical take on this: "...organizations rarely lobby alone. Citizen groups, like others, typically participate in policy debates alongside other actors of many types who share the same goals. For every citizen group opposing an action by a given industrial group, for example, there may also be an ally coming from a competing industry with which the group can join forces" (p.12). So it's important to recognize that the finding here is about individual parties, not "sides" or coalitions advocating a given policy. Finally, I'm curious to hear your take on the two potential money-in-politics explanations you mentioned. I've never found (1) particularly convincing—it's not clear to me that firms and their employees have the same interests, or that (if they do) the marginal value of regulatory capture isn't still high. But I agree that I underemphasized (2) and think it would be useful to have in this thread the "inside view" on lobbying equilibria from someone who works in the field. Groups that spend more relative to their opposition on a given policy are likelier to win... Although Baumgartner et al conduct an observational study, the size of their (to me, convincingly representative) sample to me suggests that if such an effect exists, it should be observable as a correlation in their analysis. The association they observe is pretty small. I just skimmed this thread, so apologies if I missed a comment on this. But Baumgartner et al. don't argue that money doesn't matter. They believe that the reason there's little evidence in their study that money affects outcomes is because “the status quo already reflects the distribution of power in previous rounds of the policy process." I.e. they don't see large changes during their 4 year study period, because the situation at the start of their study period tended to favour the resource-rich groups. After your comments and @jackva's, I actually struck this conclusion. I was trying to make a more modest statement that upon reflection (thanks to you) is (1) not such a valuable claim and (2) not well-supported enough to have >50% confidence in. It's true Baumgartner don't find that money doesn't matter; my initial (now disavowed) read was that if resources mattered independent of deployment strategy, then we'd expect to see a much stronger correlation even in the observational context. I sort of think that this observation holds true even given the passage you've cited, but it's definitely not a top-level extract from the lit review and definitely needs a considerably more robust defense than I am prepared to muster. Hi Matt, Thanks for your reply and sorry for the slight delay! Would you like to present your work in a session at Founders Pledge (we have a Journal Club where we discuss relevant research and having you as a guest speaker there sounds like a good idea)? On the cost effectiveness model On the model generality, I think my view is that even a general and simple model should not be unrealistic / systematically biased; because we trust the models more than our intuitions and generally fail -- I think -- to do intuitive adjustments on the model (e.g. we probably underestimate intuitively how much independence/dependence assumptions matter). On the substance of that question, I am not sure I understand your reasoning (but see point above :)). To me it seems that when expenditure is positively correlated with success probability -- what seems to be implied by a view where actors are strategic and at least mildly successful at being so -- would that not (a) increase the cost effectiveness and (b) reduce the overall uncertainty? Because we often trust models more than we should, I weakly lean towards having less models -- personally, for me the conclusion “this is a really fascinating piece and now we need to think about building models for these different situations and considerations” would be fine whereas with the very rough model I see the risk of incorrect updates, e.g. people not looking into it more because they think that the model fairly represents the uncertainty and under the given range it does not look attractive for some interventions (where the benefit is less large). But these are just personal philosophical views on modeling, weakly held. On Baumgartner Thanks for this, this is really clarifying. The rephrasing makes this a lot clearer, I had originally read this as “giving more money might be useless” which would undermine the whole case for investigating advocacy charities from an EA perspective, so I am glad I misunderstood that and that this is now clearer. I would agree that one should not update too much from this correlational evidence. Mostly, because what seems to be the dependent variable here -- success probability -- is itself confounded by a strategic choice to engage with issues so it is not clear that the same success probability across different levels of resources expresses no difference in strength, rather than being confounded by smaller groups not trying harder things. On money in politics (1) On employees as explanation of influence, I do think this is a pretty strong explanation for a couple of reasons. (i) While employees and employers might not see eye to eye with regards to such as issues as labor policy, they have essentially the same interests with regards to the companies they own/work for -- and this is where the lack of money paradox is focused, the lack of money in pork barrel political settings. (ii) The argument does not rest on explicit voting intentions of employees, but can simply work with references to employment levels in districts, etc., it hands a very powerful argument to local business leaders vis-a-vis their political representatives. (iii) Campaign finance as well as general influence of business leaders can lead to a situation where the political representatives are already “captured”, where there is no need for additional spending. (iii) Empirically, it seems well-supported (or so I remember from my political science days, but I cannot find the paper so I might recall that wrongly). (2) On lobbying equilibria, I am unsure who the relevant experts would be -- but I would trust the political science / political economy literature there more than people with very local “inside view” expertise (as it is a dynamic system-level feature, something that seems more accurately to observe with data than based on individual experience). And just to conclude on this, I think there are many cases where lobbying is probably very good, I just think that the introduction overstates this in not fully considering explanations that make this less surprising and give reasons to think that there can also be many situations where additional money will not lead to additional influence, just higher spending. Points all well-taken. I'd love to share with FP's journal club, though I hasten to add that I'm still making edits and modifications based on your feedback, @smclare's, and others. With respect to uncertainty in the CE calculation, my thinking was (am I making a dumb mistake here?) that because and , then . So if covariance is nonzero, then (I think?) the variance of the product of two correlated random variables should be bigger than in the uncorrelated counterfactual. To me, the main value of the CE model was in the sensitivity analysis - working through it really helped me think about what "effective lobbying" would have to be able to do, and where the utility would lie in doing so. I think if it doesn't serve this purpose for the reader, then I agree this document would have been better off without the model altogether. Thanks for your thoughts on money in politics. Vis (1) I have to think more about this, but I do definitely view the topic a little differently. For instance, it's not obvious to me that economic arguments and political representation do the necessary work of regulatory capture. Boeing is in Washington and Northrop Grumman is in Virginia. It seems clear that the representatives of the relevant districts are prepared to argue for earmarks that will benefit their constituents... but these companies are still in direct competition, and it seems like there's still strategic benefit to each in getting the rest of Congress on their side. I might misunderstand- maybe we're reaching the limits of asynchronous discussion on this topic. Vis (2), the "inside view" I was talking about was actually yours, as someone who thinks about this professionally- so thank you for your thoughts! Hi Matt, 1) I don't see incompleteness as an issue -- what is good for Journal Club is bringing in lots of interesting ideas which your post certainly does, updates you made and are working on are fine. So if that would work for you, I would suggest you as a speaker for Journal Club and we could see when it would fit over the next month or so? 2) My reading of your model -- which might be wrong -- was that you assumed independence between variables related to cost/effort and variables of success probability. It seems to me that when they are positively correlated rather than independent, cost efficiency would increase and become more narrow, because what this says is that worlds of high spending and success will be more likely to co-occur and worlds of high spending and no success less likely to occur than under independence. Does this make sense? 3) I think on money in politics my understanding is that a couple of intensely motivated politicians -- e.g. the representatives where headquarters of companies are -- can be quite sufficient for pork barrel style politics because they tend to fill committee positions important for their respective economic interests and they can easily bargain with other legislators. 1) Sounds good to me! We can connect about it over DM. 2) Your reading is right. A priori, a positive correlation means lower cost-effectiveness in expectation. However, I'm not sure if it means anything generally for the median cost-effectiveness (which I tried to work with in my existing CEA), irrespective of the other model parameters. And in my existing setup, if worlds of high spending and high success are more likely co-occur, and worlds with low spending and low success are more likely to co-occur, then I believe the distribution of their product would have been more dispersed, since there would be more values at the extremes (high/high and low/low) then there would be if they were independent. But I'm pretty convinced now that a better approach would have been, as you've suggested, to do separate CEAs conditional on various assumed interventions. Rather than change the parameters of independent distributions as I did in the posted analysis, the true next step is probably to re-model under varying assumptions about the covariance of the different variables. 3) I have a different sense of this, but not an overwhelmingly different sense, and I'm going to think about it some more. It also seems that the report somewhat under-emphasizes the idea of lobbying equilibria where marginal increases by one side would be quickly countered, which would make it look like additional money could be effective when in effect it is not. Thanks for sharing this. I'd be interested if you're aware of empirical evidence of this effect/reaction happening? Or is this largely a theoretical concern? Thanks for this! FWIW, I'd love to see a follow-up review on lobbying Executive Branch agencies. They're less powerful than Congress, but often more influenceable as well, and can sometimes be the most relevant target of lobbying if you're aiming for a very specific goal (that is too "in the weeds" to be addressed directly in legislation). I found Godwin et al. (2012) helpful here, but I haven't read much else. Interestingly, Godwin et al. find that some of the conclusions from Baumgartner et al. (2009) about Congressional lobbying don't hold for agency lobbying. Though I didn't read Godwin (now on my to-do list), I encountered some useful research that seemed to point toward the idea that regulatory lobbying could be a lot more efficient than legislative lobbying. By the end of my review, I had started to think that it would have more productive to do that instead. Since I finished, though, I've been thinking about one of the main concerns I have about regulatory lobbying. The fact that it's probably (comparatively) easy to influence regulatory agencies means that it's pretty easy to walk back any positive rule changes. This seems to happen fairly frequently, e.g. with EPA regulations. From that standpoint, the stickiness of the status quo in the legislative context is also an advantage: when policy change succeeds legislatively, the new policy becomes part of the difficult-to-change status quo. For longermist-oriented policies, it seems like this is a major advantage over regulatory changes. Curious to hear your thoughts. I worked on influencing healthcare policy during both the Obama and Trump presidencies, which I think is about as big of a swing as you can get on the executive side. My experience was that there was moderate leeway on the executive side. For example, legislation would require a certain amount of money to be distributed amongst healthcare providers who had "high quality" care, but "high-quality" shifted from "scores better than their peers" to "reports any amount of quality data to the government." (The latter standard effectively meaning that everyone was "high-quality", so the program was approximately useless.) However, the government has a ton of inertia and executives have limited resources, so things often continued on as they were before, even if executives really wanted things to change. I can think of a couple of ways in which executive branch lobbying can be "sticky": 1. Something which I didn't appreciate until working on this is that it's often quite hard for the government to actually do the thing it is trying to do. Officials often simply haven't thought through how some policy would affect a stakeholder, or what would happen in some unusual circumstance, simply because there are so many different things to consider. Many of my suggestions were things like "this section contradicts this other section if some circumstance occurs so you should fix that" and I expect to those to stick relatively well because it's pretty uncontroversial. 2. As I alluded to above, most government employees are nonpolitical staffers who mostly just do their job the way their predecessor trained them. I'm sure you've heard stories about government departments using computer systems from the 1970s or whatever, and a similar thing can happen at the process level. Even if the executive branch has the ability to change the interpretation of some term, they often won't, just because changing is hard. This is just from my personal experience, and I'm not sure how it would compare to working with other branches of government (or even other executive-branch agencies). Thank you for this review! Ever since I did a literature review of mostly political science literature on "Is the US Supreme Court a Driver of Social Change or Driven by it?" I've been interested in comparable reviews of legislative tactics. I especially appreciate the summary of the empirical research. Some of those findings seem really impressive to me and that has increased my optimism about lobbying. I look forward to reading through your spreadsheet with summaries of other papers. (80%) Well-resourced interest groups are no more or less likely to achieve policy success, in general, than their less well-resourced opponents. I was very surprised by this claim... it was part of what spurred me to read the piece immediately rather than save it to my reading list. But I don't recall seeing evidence for the claim in the post. What has encouraged this high confidence in this claim (which seems very counterintuitive to me)? Apologies if I just missed it. Yet in some cases there simply is no organized opposition, and a relatively small investment can meaningfully alter wording, put a policy on the agenda, or cause a bill to become law. The suggestion for the spending strategy in the "Effective lobbying" section seems to rest on this claim. But again, I don't recall seeing much empirical support in the review; does this rest on the theoretical discussions that you summarise? Actually, Baumgartner et al. provide some contrary evidence to this hypothesis: "The table [table 3.1, page 58] shows that a surprisingly large number of issues (seventeen cases) consist of a single side attempting to achieve a goal to which no one objects or in response to which no one bothers to mobilize. Ironically, the lack of countermobilization is a good predictor of failure. Many of these reflect efforts to put an issue on the agenda, but these efforts are either too early in the process for anyone yet to have reacted or they are clearly not moving so others have not gotten involved in the issue." They discuss this a little more on the following pages. They note that 17 cases had only one side -- 15 of these were 1 opposing the status quo, none defending it. Unfortunately, from a quick look back, I can't see the proportion of these 15 cases that did result in a policy change. Thanks for your response! With respect to your first point, I'm considering striking this conclusion upon reflection - see my discussion with @jackva elsewhere in this thread. In any case, my confidence level here is certainly too high given the evidence, and I really appreciate your close attention to this. With respect to your second point, I don't mean to imply that the lack of organized opposition is the only thing that justifies lobbying expenditure, and think my wording is sloppy here as well. I used "lack of an organized opposition" to refer broadly to oppositions that are simply doing less of the (ostensibly) effective things — lower "organizational strength" as in Caldeira and Wright (1998), number of groups, as in Wright (1990), or simply lower relative expenditure, as in Ludema, Mayda, and Mishra (2018). The evidence in Baumgartner et al that you reference about the apparent association between lack of countermobilization and success is also related to @jackva's concern about my underemphasis on potential lobbying equilibria here. On the one hand, I think this is clearly evidence in favor of the hypothesis that there is some efficiency in the market for lobbying- perhaps most lobbyists have a good idea of which efforts succeed, and don't bother to countermobilize against less sophisticated opposition. On the other hand, lobbying is a sequential game, and, since the base rate for policy enactment is so low to start with, it makes sense that opposition wouldn't appear until there's a more significant threat. EDIT: I've actually struck the first bit, with a note. I wanted to add one more thing, which is that I don't know how much you've adjusted your prior on lobbying, but I wouldn't say this has made me "optimistic" about lobbying. The core thing I've come away with is that lobbying for policy change is extraordinarily unlikely to succeed, but that marginal changes to increase the probability of success are (1) plausible, based on the research and (2) potentially cost-effective, based on the high value of some policies. Hello. I studied lobbying in DC during my MA in International Trade and Economic Diplomacy. I also tried to estimate the cost-effectiveness of lobbying per QALY - and came up with$280 per quality life (or about \$4.7/QALY) (p. 3) or the first draft on the EA Forum.

I think that EA Brazil and some others are working on starting an EA lobbying group. I dm'd you regarding this.