Note: A version of this document was reviewed by Hauke Hillebrandt, David Manheim, Samuel Hilton, Aaron Gertler, and Tara Law, who I thank profusely for their useful and encouraging feedback.

At the end of this document, I make a proposal for high-leverage lobbying for altruistic ends, conduct a preliminary cost-effectiveness analysis, and offer some rough suggestions for policies that might be worth advocating in Congress. Several reviewers said this was their favorite part of this piece, so you may want to read it first.

Conclusions

My confidence in each conclusion leads each bullet point, in parentheses.

• (85%) Informational lobbying can shift policy relative to the counterfactual.
• (80%) Well-resourced interest groups are no more or less likely to achieve policy success, in general, than their less well-resourced opponents. I've eliminated this conclusion in light of some thoughtful questions from @jackva and @Jamie_Harris below. In general, I think that there's no evidence that resource-rich groups are effective irrespective of their deployment of those resources. But that conclusion is (1) itself not well-supported enough to include here and (2) not practically useful. The most useful takeaway here is probably: resources are not a guarantor of success, because success is contingent upon many other factors. Basically, less well-resourced groups can win and often do.
• (70%) Dollar for dollar, lobbying against a proposal is more effective than lobbying for a proposal.
• (65%) Lobbying is more effective when issues are not salient to the public, or (more particularly) to the lobbied member's constituency. After more review, I've eliminated this conclusion as well. I'm now somewhere around 50/50 on this, with some expectation that the direction of the effect is actually reversed. I think this was in fact a serious error on my part: I (1) heavily overweighted the theoretical work (2) included apples-to-oranges work that focused on PAC expenditure and (3) did not take into account explicit evidence to the contrary. I also overweighted my own deductions, which are worth spelling out here: There is good evidence that lobbying expenditure increases with issue salience. If the below conclusion about net direction lobbying is true, then lower salience translates into less overall spending, which translates into less total expenditure for the bigger spender, and a higher probability of success per dollar spent.
• (60%) The net direction of lobbying, e.g. the difference between the number of groups lobbying for or against a proposal, or the difference between total expenditure for or against a proposal, influences policy outcomes. Organizations that lobby more on an issue relative to their adversaries are slightly more likely to win.

The quality of the evidence in this domain is considerably worse than in other areas of inquiry in the social sciences. In general, the question I tried to ask myself when assessing these potential conclusions was this: "How likely would I have been to see this evidence, in aggregate, if this statement was false?"

Introduction

A few months ago, I started thinking about the Tullock paradox: "Why is there so little money in politics?" That is, given the extraordinarily large rewards available to an interest that captures the cooperation of government, why is outright corruption so rare, and why is industry's expenditure on political influence relatively small? Tullock's observation was made mostly with regard to campaign contributions, but it applies just as well to lobbying: the market for new vehicles in the U.S. was worth nearly half a trillion dollars in 2019, but the entire automotive industry spent only $70 million on lobbying -- 0.01% of the value of the market.

Tullock's observation suggests that, for groups that want something from government, lobbying is potentially an extraordinarily long lever with which to move policy. The question motivating this review is whether that lever is also available for altruistic ends.

This review has three purposes:

1) To serve as a reference regarding the effectiveness of informational lobbying for groups or individuals considering funding advocacy or influence campaigns.

2) To provide a starting point for others who may be interested in conducting further research themselves.

3) To try to answer to my satisfaction the question of whether lobbying works before we try to uncover the most effective lobbying tactics and techniques.

A caveat

It's clear that lobbying has a negative connotation. Stories about the pernicious impacts of lobbying by cigarette companies, oil companies, and professional organizations are common, and it seems clear that the influence of moneyed interests can hurt the public. But lobbying is just a tool: informational lobbying, as I'll discuss further down, is no more or less than talking to government. Although the research I review here is focused on professional lobbyists, lobbying in general can be done by anyone, from K Street professionals to high school students. I don't take a moral stance on whether lobbying in general is good or bad, or whether it should be restricted in some way.

My process

Focus

The empirical section of this review is intentionally narrow. I focus on lobbying directed at legislative bodies at the national level (e.g. the Senate and House) in the United States. Aside from my being American and therefore more familiar with the American system, there are a few reasons for this narrowness:

• The structure and effectiveness of lobbying at the federal level is likely different than at the state or local level, or at the national level in other countries. It's not clear to me that we can draw meaningful conclusions if we mix these contexts, so I choose not to. My review indicates this is probably a good choice: Bennedsen and Feldmann (2002), for example, write that "observers of the policy process are often struck by the intensity of lobbying—or lack thereof—in the system on the other side of the Atlantic."
• Since the U.S. Congress wields considerably more power than other bodies that are plausibly susceptible to lobbying, determining whether Congress can be influenced seems (by comparison to others) to be a top priority.
• Lobbying of federal regulatory agencies, which also seems like a very promising avenue for investigation, is likely substantively different than lobbying legislators. For the sake of clarity, I try not to mix these different settings.

Crucially, I focus on informational lobbying: when lobbyists communicate directly with policymakers to shift policy. Policymakers include members of Congress and their staffs, members of the executive branch, bureaucrats, and regulators. I expressly do not investigate the empirical impact of political campaign contributions or grassroots advocacy here. However, the topics are closely tied together, as I describe at the beginning of the "Theoretical Background" section.

Methodology

This is a narrative review. I have tried to synthesize what I have learned about the current state of academic knowledge on informational lobbying at the level of the U.S. Congress, communicate my read on the evidence, and highlight the main points of uncertainty.

I took a snowball approach and started with two origin points: (1) a handful of relevant Google Scholar queries (e.g. "empirical lobbying effectiveness", "interest group influence in politics"), and (2) the empirical studies described in the literature reviews I summarize below. Starting from these lists, I followed both backward- and forward-citation chains until I was satisfied I had encountered most of the main themes in the literature and had read most of the empirical work relevant to the review. Not all major studies are included, largely as a function of my own resource constraints. John Mark Hansen's 280-page Gaining Access: Congress and the Farm Lobby, 1919-1981 (1991), for example, does not get the attention it deserves here.

Readers may note that the theoretical section is heavy on the political science literature and comparatively light on relevant subfields in economics, such as public choice theory. The primary reason for this is that the literature in public economics, while theoretically instructive, is somewhat orthogonal to my main concern here, which is this: How effective, as a means of influencing policy, is the provision of information directly to legislators, and what circumstances influence a lobbyist's chances of realizing the policy that s/he advocates?

Background and Terminology

Why informational lobbying?

I use the definition provided by De Figueiredo and Richter (2013): Lobbying is "the transfer of information in private meetings and venues between interest groups and politicians, their staffs, and agents."

Lobbying can be done by the interested parties themselves, but the term "lobbyist" generally refers to individuals and firms that attempt to influence government on behalf of private clients, such as corporations or nonprofits. It is also conventional to distinguish between "inside" lobbying, which consists of direct communication with policymakers, and "outside" lobbying, in which organize interests attempt to shift public opinion in hopes of affecting legislative decisions. This review concerns only inside lobbying.

Lobbying and political campaign contributions, as to PACs, are both types of corporate political activity (CPA), along with in-kind gifts, participation in Chambers of Commerce, legal action, grassroots influence, and other similar activities. It's necessary for me to specify my focus on informational lobbying because lobbying is often conflated with other kinds of CPA, particularly PAC contributions, in both the academic literature and the mainstream press, as in this FT article by the economist Tim Harford.

However, not only are these avenues of influence structurally and legally very different, but spending on lobbying in a given year is typically an order of magnitude larger than spending on campaign contributions. Some researchers (e.g. Lux, Crook and Woehr 2010) view the spending differential between lobbying and PACs as an indication of the greater relative importance of lobbying.

My focus on informational lobbying as opposed to other avenues of influence is pragmatic. Evidence about the efficacy of informational activities is valuable and portable. I see two possible takeaways from this research:

• If lobbying is effective under certain conditions, well-resourced organizations might be justified in taking a portfolio ("hits-based") approach, paying professional lobbyists to advocate for a menu of highly impactful policies meeting those conditions, or conducting such advocacy on their own initiative. For a proposal on this front, please see the section entitled "Effective Lobbying."
• Information about the circumstances under which political influence campaigns are successful can improve resource allocation efficiency for groups that already fund political advocacy.

How does lobbying work?

In 2019, reported lobbying expenditures totaled around $3.5 billion. This money, spent largely by industry, pays the salaries of registered lobbyists; it doesn't go directly to members of Congress or their staff. Lobbyists conduct research on relevant topics, including the political constituencies that may support or oppose a given policy, and set meetings with bureaucrats and politicians, which they use to provide information that they hope will sway votes or policy positions. Many firms keep lobbyists on retainer in order to protect the status quo, and change their political influence strategies only when some new threat arises. This means that the effect of lobbying can be hidden: the absence of new legislation can also be a goal of lobbying. Unlike campaign contributions, lobbying expenditures are not limited by law; nevertheless, most firms do not lobby—although they are often members of trade associations and other groups that do engage in political advocacy. There is some evidence that lobbyists sometimes help legislators write legislation (Hall and Deardorff 2006).

In the U.S., lobbyists are required by The Lobbying Disclosure Act of 1995 to register with the government and to file regular reports on their activities and on their spending. A lobbyist is anyone who makes more than one communication to a covered official while on contract to a particular client. Covered officials include not just members of the Senate and House but their employees, employees of Congressional committees, etc. - regardless of their job level or function.

The Executive branch is also covered by the Lobbying Disclosure Act, but is less extensive: only the President, the Vice President, Schedule C political appointees (such as a special counsel), and a handful of other top-level positions are covered by the Act.

How can lobbyists influence policy?

• Lobbyists can help to set the agenda. In other words, they can help to define the things that people in government are paying attention to: making legislators aware of issues, creating public awareness and pressure, or publicizing causes and specific items of legislation.
• They can provide evidence and arguments to legislators or staffers that either (a) persuade them to change their opinion or (b) supply them with rhetorical ammunition that they can use in committee, during floor debate, or in media appearances.
• They can provide outside research that legislative offices or bureaucracies may not have the internal capacity to develop on their own (see the "legislative subsidy" section below).
• They can demonstrate alignment between constituents and the lobbyists' preferred policy outcomes. For example, lobbyists can conduct polling showing that a policy is preferred by a majority of a Senator's constituents.
• They can participate in "issue networks" or subgovernments, attempting to shift policy over a longer time scale by publicizing and propagating their preferred policies and stances within a given policy domain.

Theoretical background

I don't claim to include all relevant theories in what follows, but I aim to make some useful distinctions and synthesize some of the main themes in the literature. I do not include the sizable literature on the drivers of policy change in general; for that I refer the reader to compendia such as Theories of the Policy Process (Sabatier and Weible 2018).

Exchange and persuasion

Theoretical work about political influence has tended to fall into two broad (though not exclusive) categories. In general, there are "exchange theories" and "theories of persuasion" (Hall & Deardorff 2006).

In exchange models, campaign contributions or in-kind gifts are exchanged for Congressional votes. This is, to a first approximation, the model assumed in most lay conversations about "money in politics." There is mixed evidence regarding this phenomenon, also known as vote buying. For a good (if now somewhat dated) lit review on the topic see Ansolabehere, de Figueiredo, and Snyder (2003).

In persuasion models, information is at the center of the relationship between legislators and lobbyists. Lobbyists, in this view, can use information to either (1) genuinely persuade legislators or (2) convince self-interested legislators that their chances of electoral success would be enhanced by taking the advocated position. Lobbyists have a comparative advantage in their ability to expend significant resources collecting and presenting such information, e.g. polls of relevant constituencies.

Money, access, and information

As Wright (1990) notes, many studies on the impact of contributions on Congressional voting fail to make a distinction between contributions and informational lobbying, which makes it impossible to separate the impact of lobbying from the impact of the contributions themselves. This puts a finer point on an important way that the exchange and persuasion models come together: money may not buy votes, but it's possible that money buys access.

Legislators have limited time, and if they prioritize contributors when arranging meetings and phone calls, then these big spenders will have more opportunities to influence policy. This is the view advanced by Wright, as well as by Austen-Smith (1995), and supported by some empirical evidence (Tripathi, Ansolabehere and Snyder 2000 and Langbein 1986). Contributions may also serve a signaling function to legislators with reference to possible future lobbying, demonstrating that interest groups have both the resources and the intent to provide necessary information.

Theoretical models of lobbyist behavior

Pluralism and the communications approach

In the mid-20th century, a prominent strain in political theory was the pluralist model, which conceived of policy as being primarily a result of competition between diverse interest groups (Denzau and Munger 1986). To pluralist thinkers like Truman (1951) and Latham (1952), competition between interest groups was a source of stability and a fundamental component of democracy. Archetypal pluralist works (e.g. Dahl 1956) held that political outcomes are the result of struggle between interest groups, who compete within specific issues or policy areas and tend to be influential primarily within those domains, rather than dominant in society in general. In the pluralist view, power is dispersed across society; policy is a result of mediation and healthy discussion between a large number of competing interests. The influence of interest groups is viewed as a benign representation of the will of the people, weighted by its salience to a specific subset of citizens (Lowery and Gray 2004). The pluralism hypothesis implies that interest groups primarily serve the purpose of raising legislators’ consciousness of the preferences of existing constituencies; they cannot make legislators do things that are against their own electoral interests.

Pluralism fell from favor under the weight of empirical research finding (a) that American voters were largely ignorant and apathetic about politics and (b) that organized interests seemed to work less like "pressure groups" and more like "service offices" for members of Congress already agreeing with them. At the time, lobbyists spent considerable resources communicating with already-friendly legislators (Bauer, Poole and Dexter 1963). Several other works of the same period and much later came to similar conclusions, holding broadly that lobbyists are at the beck and call of legislators, who have a great deal of agency in how they allocate their time (Ellis 1992).

In his book The Washington Lobbyist (1963), Lester Milbrath presented interest group politics as a game of communication, rather than of influence. The communications approach provided a compelling explanation for the tendency of groups to lobby already-agreeing legislators: targeting opposing legislators constituted an attempt to pass a message through a closed channel. Milbrath, along with Bauer et al., is largely responsible for the modern conception of lobbying as being largely about information. (McGrath 2017).

The supply price for policy

Denzau and Munger (1986), building partly on earlier ideas developed by Stigler (1971), offered a formal model predicting that legislators are most likely to respond to "organized group pressures" when constituents are uninformed or indifferent. In their model, the amount that interest groups must offer a legislator for his services (e.g. his vote) depends on the legislator's productivity and the preferences of voters in his district. The price of policy depends on a legislator's comparative advantage in providing votes. Comparative advantage is determined by the hostility of voters at home to a given policy and the productive capacity of the individual legislator (e.g. seniority, committee assignment, competence). The primary conclusion of this influential paper is that "contributions can have some influence on policies about which voters are divided, ignorant, or indifferent."

Counteractive lobbying

Austin-Smith & Wright (1994) proposed a theory of counteractive lobbying in response to the by-then mainstream awareness that interest groups often lobbied legislators predisposed to agree with them. Counteractive lobbying occurs when groups lobby in order to offset the pressure applied by opposing interests. These authors propose that lobbying does, in fact, influence the positions taken by legislators, but that lobbying by friendly interests takes place in order to prevent allied legislators from changing their positions.

Legislative subsidy

Hall & Deardorff (2006) have proposed a legislative subsidy model, in which lobbying operates with respect to budget, not to preferences. Lobbyists offer a "gift" to like-minded legislators, providing assistance in the form of legislative intelligence, information, connections to other groups and supporters, structured arguments, and other types of free intellectual labor. This more recent model, like the counteractive lobbying theory, addresses the demonstrated tendency of lobbyists to spend time and effort lobbying those who already agree with them. The subsidy model and other models are not mutually exclusive; lobbyists are free to select from a "menu" of available strategies. This model is (for example) consistent with the Denzau & Munger model described above: lobbying under the subsidy model can be seen as an in-kind contribution to legislators, so the supply price of policy is coterminous with lobbying labor purchased. Hall & Deardorff nonetheless argue that "preference-centered" lobbying (in which lobbyists seek to change legislators' preferences) is relatively rare.

Interpretation

Smith (1984) observed the lobbying activities of the National Education Association in the spring and summer of 1977 and proposed a model in which advocates—including not just lobbyists but also fellow members of Congress, staffers and others— influence members' interpretations of the consequences of a policy. Smith proposes a two-stage model of policy formulation. In the first stage, members interpret the policy; in the second, they attempt to discern how their interpreted policy consequences impact their own policy goals. Advocates can shift policy by demonstrating to legislators how a given policy could advance their own goals. However, since basic human constraints (e.g. bounded rationality) cause interpretations to be unstable, advocates' influence on policy is also itself unstable.

Political accountability

Smith (1995) singles out Arnold (1990) as having a rare comprehensive theory in which policy outcomes are driven by (1) whether legislators perceive constituents as attentive or inattentive to the issue at hand, (2) how salient the issue is to constituents, and (3) how capable constituents are of holding legislators accountable. Arnold's theory holds that pressure succeeds only when an issue is viewed as important to attentive citizens and not to inattentive citizens, and when it has champions in Congress who will do everything "procedurally necessary" to ensure that legislators are held accountable by constituents.

How lobbying is studied empirically

Research about political influence has tended to focus overwhelmingly on campaign contributions, to the neglect of informational lobbying. Several of the papers I reviewed (e.g. Evans 1996, Bombardini and Trebbi 2019, Caldeira and Wright 1998) suggest that the disproportionate attention afforded to campaign contributions is mostly a function of greater data availability; at least one (Milyo, Primo and Groseclose 2000) suggests that this tendency in the literature is a case of "looking under the lamppost": data about campaign contributions has historically been easier to come by.

Data sources

While information about federal political contributions has been available since 1974, data about federal lobbying expenditures has only been available since 1999 as a condition of the Lobbying Disclosure Act. The Honest Leadership and Open Government Act of 2007 made electronic filing mandatory, and data since 2015 is now available from the Senate. Senate data is formatted as XML; more helpfully structured data is available from the Center for Responsive Politics (CRP). A considerable portion of recent research on this topic uses data from CRP.

A significant chunk of the research I consider below focuses on the relationship between lobbying expenditure and firms' financial outcomes; these studies often merge CRP data with business information from Compustat. Some sources also use lobbyists.info (now washingtonrepresentatives.com) for biographical information about lobbyists and about their relationships.

Some enterprising researchers (e.g. Evans 1996) have pursued much more labor-intensive data collection operations, with significant amounts of hand-coding and manual review. Baumgartner et al 2009, one of the most impressive recent works on this topic, followed a random sample of issues over a four-year period, collecting information via large numbers of interviews with lobbyists and policymakers and hand-coding their responses.

Research designs

To my knowledge, there has never been a controlled experiment about informational lobbying— this kind of thing would be tricky to pull off, though it's probably not out of the question. In general, work attempting to assess political influence through communications channels takes one or more of the following approaches:

• Cross-sectional studies
• e.g. Alexander, Scholz and Mazza (2009), who model tax savings due to repatriated earnings from the American Jobs Creation Act of 2004 (AJCA) as a function of lobbying expenditures by firms.
• Panel studies
• e.g. Meade and Li (2015), who follow a group of firms over a 13-year period and model effective tax rate as a function of lobbying expenditure with year and firm fixed effects.
• Instrumental variables (IV) / 2-stage least squares (2SLS)
• e.g. Gawande, Maloney and Montes-Rojas (2009), who find a source of outside variation in lobbying expenditure (an instrumental variable, in econometrics-speak), and use it to estimate the effect of pro-tourism lobbying by foreign countries on tourism from the U.S. to the lobbying nations. (See here for a good explanation of the IV design)
• Interviews and Surveys
• Baumgartner et al (2009) contains a considerable statistical element, but the research is heavily informed by qualitative interviews that provide a lot of background context (and were also put into structured form and analyzed)
• Case studies
• In addition to interviews, Kingdon (1984) reports and codes policy case studies in order to identify where interest groups seem to be important.

Previous Literature Reviews

Smith (1995)

Smith identifies a body of work that concludes, from formal models of the lobbyist-legislator relationship, that a primary persuasive virtue of lobbyists is their ability to affect decision-making by reducing legislators' uncertainty about bill consequences. Smith finds little empirical support for these models, with the exception of Austin-Smith and Wright's (1992) theory of counteractive lobbying, for which the authors find empirical support in Austin-Smith and Wright (1994).

Smith identifies eight studies from the preceding decade that assess the impact of lobbying on roll call votes, and finds that the research presents a "mixed picture" of the possible effects of lobbying. Smith finds that most empirical studies suffer from measurement and specificity errors, and draws no firm conclusions about the effectiveness of lobbying in general.

Potters and Sloof (1996)

This review explicitly focuses on quantitative data around "how and when" interest groups influence public policy, but does not distinguish between different types of interest group influence (informational lobbying is among those types studied). The authors make a useful distinction between two types of variables in empirical interest group research: (1) the behavior of individual policymakers and (2) policy outcomes. Potters and Sloof find that empirical research on interest group influence outside of campaign contributions is rare. A key observation of this paper is that single-equation models are often used to estimate interest group influence, but that in reality causality flows in both directions: politicians' positions influence group behavior, as well as the reverse.

Potters and Sloof conclude that there is evidence for the effectiveness of lobbying, particularly when bills have "a narrow focus and low public visibility." In addition, they find that interest groups have greater success influencing policy when politicians are under low democratic pressure (e.g. low intra-party competition) and less success when there is a well-informed electorate.

Burstein and Linton (2002)

Burstein and Linton analyze every article on policy change in the top three sociology and top three political science journals. Their unit of analysis is the effect of a predictor on a dependent variable, e.g. between a measure of either the resources or activities of an organization and a policy outcome. They find 53 articles containing a total of 230 effects. They find that statistical significance is a more common standard in this literature than substantive significance, and reviewed findings using a scale that takes both standards into account. Burstein and Linton find that the impact of interest groups is statistically significant roughly half the time, and substantively important (according to the authors' own assessment) in about a fifth of cases.

Burstein and Linton conclude that the hypothesis that interest groups influence policy significantly is "not as well supported by the data as we might expect." However, they offer three potential sources of bias in their estimate.

First, they suggest that their estimate of the frequency with which political organizations influence policy may be too high relative to its actual effect—they note that few studies include variables for public opinion, and suggest that the inclusion of such a variable might cause other apparently significant relationships to disappear.

Second, they suggest that their estimate may be too low: they note that theory predicts that it is the way in which resources are deployed, not the size of the resources themselves, that influences policy; nonetheless, most studies rely on data about resources, regardless of deployment. This thesis is supported by their data: measures of resources alone impacted policy 45% of the time, but resources and activities together impacted policy 65% of the time.

Finally, they suggest that the range of extant lobbying research is very narrow— it covers only a small number of issues and countries, and may simply not be representative enough to draw meaningful general conclusions.

Lowery (2013)

Lowery addresses himself to the findings of three previous literature reviews—Smith (1995), Baumgartner and Leech (1998) and Burstein and Linton (2002)—which held that evidence about the effectiveness of lobbying is inconclusive or inconsistent. Lowery starts from the premise that null findings are common—assuming that, due to the file drawer effect, significant findings are even rarer than they at first appear—and offers some explanations for why that might be so.

Lowery suggests several possible explanations for null findings:

• The most important instances of influence may not be observable
• Issues compete for space on the agenda, and exogenous forces, not interest groups, ultimately determine what ends up on the agenda.
• Lobbying is two-sided, and a failure on one side is a success on another. Null findings may simply leave out the influence of major indirect determinants of policy, such as public opinion.
• Selection bias: The most common type of research is an analysis of a small number of groups working on one issue over a specific amount of time, and most influence studies focus on "substantive large and controversial issues on which many organizations are engaged." However, "these are precisely the issues on which influence might be least expected because public opinion is likely especially constraining." In point of fact, most policy proposals see little or no lobbying.
• A quote that gave me a lot to think about: "It could well be that lobbying is far more successful on these kinds of proposals. But we will not know this for sure unless we stop confusing studying lobbying influence with studying lobbying influence on controversial issues."

De Figueiredo and Richter (2013)

De Figueiredo and Richter list "empirical regularities" in the study of lobbying. The uncontroversial topline summary is that lobbying is pervasive, that lobbying increases when issues are more salient or when stakes are high for an organized interest, and that both expertise and connections seem to matter for lobbyist effectiveness, but the magnitude of these effects is unclear.

They also list three main challenges in the empirical study of lobbying:

• Time series persistence: very little within-interest-group variation
• Omitted variables: datasets typically used to study lobbying may not contain important data necessary to adjust for these variables
• Endogeneity: groups may be more likely to lobby when they believe they are more likely to succeed

In general, the authors find that lobbying "can be valuable in some contexts" but that its success is heavily dependent on timing, targets, and tactics.

Studies that were instrumental in my conclusions

You can read summaries and key findings for 25 of the studies I reviewed in a Google sheet here. What follows are brief summaries of the studies that I found most useful and convincing.

Caldeira and Wright (1998)

Caldeira and Wright model individual Senate votes in the Bork, Souter, and Thomas Supreme Court nominations as a function of lobbying. They use an instrumental variables approach in which votes are modeled (in the second stage) as a function of partisan affiliation, ideological orientation, constituency support for the nominee, and a lobbying index constructed (in the first stage) as a function of party, ideology, constituency, campaign contribution, and organizational strength (e.g. the number of interest groups lobbying for and against the nominee) that contribute independently to the value of the lobbying index. They find a significant relationship between lobbying and Bork vote; the authors find counterfactually that if the opposition lobbying effort had been 10% smaller, the opposition to Bork would have decreased by two votes. In the case of the Thomas nomination, the authors find that "a 25% reduction in opposition lobbying effort would have garnered an additional 33 votes for Thomas."

De Figueiredo and Silverman (2006)

De Figueiredo and Silverman model academic earmarks for universities against lobbying expenditure by those institutions in an instrumental variables setting. They find a marginal $5 return to an additional dollar of lobbying expenditure. However, this relationship only holds for universities with representation on Congressional appropriation committees.

Baumgartner et al (2009)

This book, Lobbying and Policy Change: Who Wins, Who Loses, and WHY is a contemporary classic. Baumgartner et al take a random sample of issues and conduct a correlational analysis, following the issues over several years and collecting data primarily via interviews with lobbyists. Though this process has its weaknesses, the authors present a number of conclusions that I view as broadly convincing correlational evidence when viewed in light of the full scope of related research.

Key findings:

• Resources in general have no significant correlation with a positive policy outcome
• "When the mobilization of resources is unbalanced, the wealthy side tends to win." Increased comparative resource advantage is associated with success maintaining the status quo and greater success achieving a policy change. However, in most cases there is no such imbalance.
• Policy change happens over a long time frame.
• "The most consistent finding throughout our book is that defenders of the status quo usually get what they want."
• Lobbying follows a power law distribution: most lobbying occurs in a small number of cases.
• High-level government allies are more important for success than financial resources: particularly at the level of Congressional leadership and the presidency—when they get involved, they will likely win.
• Baumgartner et al find support for the counteractive lobbying thesis

Montes-Rojas (2013)

Montes-Rojas models foreign aid disbursed by the U.S. as a function of lobbying and finds that a 1% increase in lobbying expenditures in a given year is associated with a 0.04% increase in foreign aid the following year, and up to 0.075% in the long run. I want to note that this effect is huge, given the differences in magnitude between lobbying expenditure and foreign aid. Foreign lobbying expenditures total about half a billion dollars, but the foreign aid budget is around $40 billion. Montes-Rojas uses an IV strategy in which lagged values of the parameter of interest are used as instrumental variables; this is not as crazy as it sounds, and I recommend you read the paper, which leans heavily on the work of David Roodman.

Goldstein and You (2017)

In a pretty novel approach, these authors examine the impact of lobbying expenditure by U.S. cities with populations over 25,000 on disbursements via earmarks and the American Recovery and Reinvestment Act. They also use a novel instrument: the existence of a direct flight between a given city and Washington, D.C, which the authors argue is a source of exogenous variation in lobbying expenditure. They present several convincing robustness checks on this instrument in their Appendix, but an unaddressed area of uncertainty is whether the existence of a direct flight can cause legislators to be more familiar with a city and therefore more likely to disburse funds to it. If so, this could compromise the instrument.

Goldstein and You find that a 10% increase in lobbying expenditures is associated with increases in earmarks and ARRA grants of 10% and 4%, respectively. As in the case of Montes-Rojas (2013), this framing can obscure the magnitude of the impact. Cities spent about $20k on lobbying on average, and received an average of around $1m and $130m in earmarks and ARRA funds respectively, so the coefficients suggest marginal returns of 50x and 2600x, respectively.

Ludema, Mayda and Mishra (2018)

Ludema, Mayda, and Mishra examine tariff suspension bills, of which 1400 were introduced in Congress between 1999 and 2006. They model passage of these bills as a function of lobbying expenditure for the Congresses spanning that time period and use a set of instruments that they argue (convincingly) are not readily available to legislators and are therefore unlikely to influence tariff suspensions. They find that increases in effective lobbying expenditure (expenditures relative to a minimum baseline) in opposition to suspensions are significantly associated with suspension, expenditure in support is likewise associated with no suspension (with a smaller effect size); likewise number of opponents.

Findings

Does lobbying work?

A very heavily qualified yes. I think that there are enough good-quality quantitative studies, with enough convincing theoretical backing, to conclude that informational lobbying can effectively shift policy. There are a number of reasonable concerns with the state of the research: much of the existing work is not causal in nature, and I found almost no published papers on this topic with null results, suggesting that a bit of the file drawer effect may be at play here. For this reason, it's not advisable to attempt to extract any quantitative estimate of the effectiveness of lobbying from the published work.

Two separate factors give me some confidence in this conclusion. The first is the relatively high quality of many of the quasi-experimental studies and the evidence these higher quality studies represent in aggregate. The second, perhaps more debatable, factor is my sense that some of the repeating patterns in correlational and observational work on this topic are best explained by a causal story rather than omitted variables. These patterns include the observed association between the net gap in lobbying (supporting minus opposing) and policy success, consistently higher coefficients on negative lobbying relative to positive lobbying, and the broad agreement on conditions for lobbying success that I discuss below.

When does lobbying work?

Previous literature reviews have identified a number of stylized facts about the determinants of lobbying success that seem to have a strong empirical basis. Please note that the scope of my review centered on legislative lobbying at the national level, and that the following areas of agreement should be considered to be restricted to the legislative domain.

The first area of broad agreement is that lobbying success seems to be more likely on issues of low salience to the electorate, issues with low public visibility, or issues about which the public is largely ignorant. This is a theoretical prediction made by Arnold (1990) and Denzau and Munger (1986) and observed empirically by e.g. Evans (1986). Literature reviews conducted by de Figueiredo and Richter (2013) and Potters and Sloof (1996) consider it well-established.

EDIT: I've included the above paragraph because it accurately reflects the research referenced, but after digging in more detail into to the studies referenced in the literature reviews, I don't have confidence that it's actually the case.

The second area of broad agreement is that, all other things being equal, lobbying against policies seems to be more effective than lobbying for policies (see e.g. McKay 2011, Caldeira and Wright 1998, Ludema, Mayda and Mishra 2018); in other words, lobbyists for the status quo are likelier to succeed.

The third area of agreement is that having high-level allies, either in committee leadership, the executive branch, or among Congressional staffers, is consistently associated with policy success (see e.g. Baumgartner et al 2009, de Figueiredo and Silverman 2006, Evans 1996).

Some other areas for which I found some empirical support but no broad consensus are as follows:

• Interest groups with a larger membership are more influential (Potters and Sloof 1996)
• Intergroup conflict damages a group's chances of regulatory success. In addition, conflict is more likely to occur on issues where there are winners and losers, e.g. regulatory and redistributive issues (Evans 1996).
• Policies that fail in a given Congress have a poor shot in subsequent Congresses (Kang 2016)

Case Studies

I think it's worth listing a few instances when minor interventions have significantly impacted policy on a large scale. I'm not picky about whether these examples would have been quantifiable by any of the methods reviewed here; I just want to give an idea of the settings in which lobbying (external or internal) can influence policy (in the bureaucracy or the legislature) and some sense of the magnitude of its potential effects. These cases demonstrate instances in which policy was meaningfully impacted simply by communication, even though they do not all resemble a typical picture of lobbying.

• Terry Dawson, an advisor to the House Science Committee, was apparently responsible for influencing committee chair George Brown to require, in a piece of 1991 legislation, that NASA investigate asteroid detection and deflection strategies (Morton 2017 and Morrison 2005). The vast majority of such asteroids are now known and tracked as part of the resulting program.
• The famous Einstein-Szilárd letter, though perhaps not commonly thought of as lobbying, directly influenced FDR's decision to initiate the Manhattan Project.
• OPP's History of Philanthropy includes several salient examples including the following:
• Nuclear arms control research funded by the Carnegie and MacArthur Foundations resulted in a report co-authored by Ashton Carter, the Harvard professor (and future Secretary of Defense). Carter briefed Senators Nunn and Lugar on the report and apparently helped to write the text of the resulting bill.
• The Robert Wood Johnson Foundation funded the SmokeLess States initiative, which shifted in 2000 to focus entirely on policy change. Ben Soskis's OPP-funded research reports that various evaluations have found that SmokeLess States successfully lobbied for increased excise taxes on tobacco and other state-level policies intended to combat smoking.
• Corporate lobbyists have had considerable success engaging in the type of legislative subsidy known as legislative drafting: they literally write sections of bills that become law. Ellis and Groll (2019) report that lobbyists from Genentech and Citigroup have provided talking points or proposed language to lawmakers that found their way, verbatim, into legislation.
• Stokes & Breetz (2018) report that during the early 2000s, the solar industry targeted Republican members of Congress "whose districts benefitted [sic] from solar employment." Success came in 2005 with a Business Solar Investment Tax Credit, which was meant to expire at the end of 2007, but which was extended and expanded for eight years as a part of the 2008 economic stimulus. In 2016, solar leasing companies again lobbied successfully for an extension of the tax credit. Some sources see the ITC as an important factor behind the recent growth of solar power.

"Effective lobbying"

As observed by many of the papers referenced here, most issues face no lobbying. If the counteractive lobbying theory is true, then the simple act of introducing lobbying on one of these issues can stimulate opposition; if the opposition is from an opponent willing to "outbid" on policy, then they are likely to win, and lobbying can fail, or fail to be cost-effective. Yet in some cases there simply is no organized opposition, and a relatively small investment can meaningfully alter wording, put a policy on the agenda, or cause a bill to become law.

I want to propose something along the lines of "effective lobbying": a rigorous approach to institutional-level change, starting with the legislature, that would take a portfolio approach to policy advocacy.

Although my review has focused on lobbying Congress, any approach to effective lobbying would have to operate with an eye to all possible venues: state and local governments, regulatory agencies, and the executive branch.

How would this work?

1) Identify a set of high-impact policies that meet the following criteria:

• Very high expected value (e.g. in DALYs)
• Mass of distribution of potential outcomes is mostly above zero

2) Identify the policies least likely to face organized opposition (e.g. from entrenched and well-resourced interests inside or outside the government, or from the public)

3) Identify what avenues of lobbying are most appropriate for enabling the policy goal

• Determine the level (e.g. regulatory, congressional, state, municipal) at which a policy would be implemented
• Identify the appropriate lever by which this policy could be brought about (e.g. via legislation, rules change, budget processes, or executive order)

4) Identify the proximate course of action that would be required to increase the likelihood of each such policy change in the near term:

• Re-wording of an existing bill
• Bringing the attention of influential legislators to the policy (e.g. putting it on the agenda)
• Narrow lobbying to shift votes for the passage of a bill already on the floor or in committee, or to allow a bill to come to a vote in either setting
• Broad advocacy of bills already on the floor
• Writing model legislation in cooperation with legislative staffers

5) Fund research necessary to maximize the effectiveness of lobbying for each policy

• Low-cost public opinion polls in relevant constituencies to identify public support or opposition
• Identification of key legislators or legislative staff who can serve as champions
• Independent analysis, as by think tanks or external research groups
• Indirect lobbying to encourage political allies to expend resources on an issue.

6) For each policy in the portfolio, fund lobbying and advocacy informed by the research produced in (4).

7) Measure impact

• Any such program needs to clearly define policy success, systematically track resource expenditure, and —ideally—conduct experiments in order to draw causal conclusions about strategy choices.

The bulk of the work here needs to happen in (1) and in (4). What would make this approach "effective" is the rigorous restriction of the policy portfolio to only potentially very high-value policies, and the deployment of a systematic approach to tracking and facilitating policy success.

Does effective lobbying already exist?

Traditional lobbyists are hired by clients, either for- or non-profit, to advocate for specific policies. Once hired, their success or failure within the specified domain is only partially under their control. As the research detailed here has found, policy success or failure is determined mostly exogenously, and lobbying success is heavily context-dependent. The approach I've suggested is focused on finding the overlap between policies that have high social value and those that face favorable contexts for lobbying success. So the first difference is in issue selection: traditional lobbyists advocate as they're hired to do, whether or not they're likely to be successful, but favorable context is a necessary condition (though not a sufficient one) for effective lobbying.

There's still the question of the degree to which lobbyists working today actively seek to maximize their efficiency, and the degree to which they make use of the mountain of data that is presently available to assist in this end. To conclusively answer these questions would require a whole separate literature review, but I did review some existing studies that paint a broad picture. Hojnacki and Kimball (1998) find evidence that interest groups take into account the degree of support they believe they have in a legislator's district when deciding whether to lobby her and that interest groups do target influential committee members. Victor (2007) reviews the literature and finds that (a) interest group perceptions of legislator knowledge about a topic impact their choice of whether to lobby directly or indirectly, that (b) interest groups consider how the public views an issue before starting to lobby, that (c) lobbyists expend more resources when there is less consensus on a topic, and that (d) groups lobby more when they perceive that there is a procedurally high likelihood of success.

So there is evidence that lobbyists are aware of the factors that seem to enhance success. The degree to which they take a systematic approach is unclear; several of the papers I read for this review hinted at a belief in considerable heterogeneity in effectiveness across lobbyists. If this heterogeneity is significant, then there is probably a lot of value to be gained by identifying, engaging, and facilitating the work of only the most skilled lobbyists.

Foundations and social change organizations already engage in this type of activity, funding advocacy across a wide range of topics. This is very close to what I am proposing; the only difference is that I am suggesting a simultaneously more diversified and more targeted approach. Rather than plowing millions into advocacy groups that fight for a broad roster of changes on a small number of hotly contested issue areas (e.g. climate change), my proposal is to invest smaller amounts on targeted advocacy for specific, high-expectation policies across a wide range of possible focus areas.

Some ideas

Just to give a flavor of some of the potentially high-impact (in expectation) policies that one might want to lobby for. Please note that some of these are likely to face significant opposition, so they're not ideal candidates for effective lobbying, but they're not ipso facto bad candidates (the expected value of avoiding nuclear war, for example, is very high).

• National adoption of the Asilomar principles for AI safety
• Set aside money for an outbreak contingency fund at the U.S. level or multilaterally
• Enable cooperative threat reduction (CTR) programs with respect to North Korea
• Reprioritize NIH funding away from expensive orphan drugs and into higher-impact areas such as genetic counseling, as George Church recommends in this Future of Life podcast
• Remove U.S. ICBMs from hair-trigger alert
• Implement more efficient grant allocation systems along the lines of DARPA's Program Manager system to speed up scientific research.
• The passage of the Endless Frontiers Act, which will allocate additional funds toward "natural or anthropogenic disaster prevention"
• Revive Congress' Office of Technology Assessment

Cost-effectiveness analysis

I conducted a limited cost-effectiveness analysis via Monte Carlo simulation by "backing out" distribution parameters from (in my view, conservative) 95% confidence intervals. Here are my assumptions:

• The per-policy lobbying cost lies with 95% confidence between $20,000 and $10,000,000 dollars, and is distributed log-normally.
• The change in probability of policy implementation lies with 95% confidence between 0 and 5%, and is distributed normally. Note that (1) the baseline probability of policy implementation is very low to start with and that (2) this setup also includes a 2.5% chance that lobbying could actually decrease the likelihood of policy implementation.
• The expected impact of policy implementation lies with 95% confidence between 1 and 100,000,000 DALYs, and is distributed log-normally. I think I'm using a pretty conservative upper end here, given the potential value to be gained by genuinely effective longtermist policies.
• I also assume that in 10% of cases, effective lobbyists get the sign of impact wrong, so the sign of impact is distributed binomially, with a 10% chance of a negative sign.

This results in the following setup:

The punchline here is that the cost per DALY of this approach lies with 95% confidence between $-157,632 and $27,974,720, with a median value of $879, and with 70% confidence between $0 and $247,117. There's also a 20% chance that the cost per DALY lies in the "sweet spot" of between $0 and $50. So the median value is semi-promising, but there's a lot of uncertainty and there's a 10% chance effective lobbyists will pay as much as $100,000 to make people's lives one year shorter. However, it's worth noting again that I tried to be both very conservative and very uncertain here, and that further rationalization of this setup could result in more precise (and perhaps more optimistic) results.

Sensitivity

In addition to the basic cost-effectiveness calculation outlined above, I present in this graph four variations with different parametrizations. The initial calculation is the red distribution (labeled "Original"). In Variations 1 through 4, I adjust only one parameter relative to the original.

The key takeaway of this analysis is that there's an important tradeoff in the way I've set this up: lower median values (and lower, narrow ranges) for positive costs per DALY come at the price of a higher probability of a negative cost per DALY. The interpretation of a negative cost per DALY would be, as I mentioned above, that substantial expenditures are made lobbying for a policy that ends up having a significant negative impact.

This is obviously a function of the binomial term in the setup described above. I think this term is defensible: for any policy that could plausibly affect large numbers of people in a significant way, there's a small but nontrivial possibility of severe, negative, unintended consequences.

One of the counterintuitive consequences of this setup is that reducing the likelihood of harm increases the price per DALY: that's because reducing harm doesn't stop us from spending: we get positive value that otherwise would be missed, but we still end up spending a lot of money on policies at the low end of the impact distribution. So (with this setup, at least) we can't get a better cost distribution just by reducing harm.

So there's two main preconditions to getting a good cost/DALY in lobbying-based policy change. The first is to advocate policies, as I indicated above, that have some probability of producing very large-scale benefits; the second is (obviously) to spend less (see Variation 3).

I think this suggests a couple things. First, the bundle of lobbied policies should include as many genuinely longtermist policies, or policies addressing global catastrophic risks, in order to shift as much mass as possible onto the right side of the impact distribution with as little risk of catastrophic error as possible. Second, if you accept the premise that accidental error in the form of a "sign reversal" is a plausible model, then reducing the error rate becomes proportionally more important as policies that affect larger numbers of people are advocated. One possible avenue here is to preference policies that enhance decision-making in the long term, that improve the functioning of institutions, over policies that encourage near-term changes to government decisions on specific topics.

Appendix

How does federal legislation work in the U.S.?

Lobbyists interested in advancing a policy proposal must first find a bill sponsor. After a bill is introduced, it is considered by the standing committee that has jurisdiction over the bill, which depends on its content. Most bills fail at this stage. Committee chairmen have enormous power, and (as things currently stand) they can decide unilaterally whether or not to bring a bill to a subcommittee, which would hold a public hearing on the bill's content. On some occasions, these public hearings are held in front of the entire committee.

After a public hearing, the bill moves up to the committee level, where it is subject to "markup": amendments, rewrites, and votes that can make or break the bill. The Congressional leadership then decides whether or not to schedule a floor vote on the bill. At this point, the processes in the Senate and House diverge.

In the Senate, the Senate Majority Leader can immediately schedule a vote—or refuse to schedule one. Once on the floor, amendments can be offered indefinitely until debate is cut off by a cloture vote (sixty senators).

In the House, if a majority of the committee approves the bill, it is sent to the House Rules Committee, which receives amendments from members and determines which amendments will be considered on the floor. For any particular bill, the Rules committee can institute an "open rule," which allows consideration of all amendments, a "closed rule," which allows none, or a rule defining specifically which amendments can be considered. On the floor, House members must abide by the rules set by the Rules Committee.

Differences between House and Senate versions of the same bill are reconciled, and then final, identical versions of the bill are approved or rejected by floor votes in each chamber.

Lobbyists can attempt to intervene at any stage during this process: they can attempt to modify the wording of a measure, to encourage votes at the subcommittee, committee, or floor level, to persuade committee or Congressional leadership to bring a bill to a vote, or to influence the way in which House and Senate versions of the same measure are reconciled.

One of the primary ways in which corporate lobbyists can impact their clients' earnings is via earmarks—specific appropriations within a bill that define precisely where and how some segment of the appropriated funds are to be spent: this is what many empirical researchers are considering when they attempt to determine the effects of lobbying expenditure on firm financial performance.

For some classes of issue, significant policy changes are possible via lobbying at other levels and branches of government. Policies of federal agencies, for example, can have a significant impact: think here of rulemaking by the FDA, EPA, and other regulatory agencies. Considerable lobbying takes place at the state level as well, and can indirectly influence federal policy: in major states such as California and New York, state regulations can be a blueprint for federal ones.

Code for future in-depth analyses

As part of my research, and to support the derivation of parameters for the above cost-effectiveness model, I went through the lobbying disclosure data in detail and explored a bit. I discovered that it's hard to get your hands on it in a useful format, so I wrote some code to convert it from XML to tabular, as well as to produce some hopefully useful summary analysis like those below. That code is available publicly on my GitHub. It is clean enough, but is mostly meant to serve as a starting place for others who want to pick up where I have left off here.

The three summary graphics below are extracted from lobbying disclosures for 2018. Each filing contains a number of issue-specific reports, some of which contain references to specific bills and some of which do not. In order to get thes plots below plots, I took reported amounts from quarterly lobbying reports and allocated them across the bills and issues extracted from the text of each report.

• For the red and green plots, I extracted mentions of all House Resolutions from the text of the quarterly statements and divided the reported amounts evenly between them and reported lobbying issues, then plotted the density. If a quarterly filing in the amount of $10,000 contained 6 issues, 2 of which referenced 3 specific bills each, then each bill was allocated $1,000, and each issue without a bill was allocated $1000.
• The green plot includes all the bills that were references in disclosure filings. This is several thousand bills, 99% of which never came to the floor for a vote.
• The red plot includes only the ~100 bills that came up for a floor vote in 2018.
• For the blue plot, I divvied the amounts up only at the issue level, without reference to specific bills. The unit of observation here is an individual issue report, of which there are four annually, so the distribution pictured is the distribution of (allocated) expenditure per issue, per lobbyist, per quarter. So the median amount that a lobbyist spent on any given issue in any given quarter of 2018 was about $200.

Bibliography, etc.

My bibliography is on Google Sheets, along with a tab providing short summaries, research designs, and key results of about 25 of the empirical studies that I reviewed.