Zach Stein-Perlman

Looking for new projects
4285 karmaJoined Nov 2020Working (0-5 years)Berkeley, CA, USA

Bio

Participation
1

AI strategy & governance. Blog: Not Optional.

Comments
427

Topic contributions
1

See https://ea-internships.pory.app/board, you can filter for volunteer.

It would be helpful to mention if you have background or interest in particular cause areas.

I'm annoyed at vague "value" questions. If you ask a specific question the puzzle dissolves. What should you do to make the world go better? Maximize world-EV, or equivalently maximize your counterfactual value (not in the maximally-naive way — take into account how "your actions" affect "others' actions"). How should we distribute a fixed amount of credit or a prize between contributors? Something more Shapley-flavored, although this isn't really the question that Shapley answers (and that question is almost never relevant, in my possibly controversial opinion).

Happy to talk about well-specified questions. Annoyed at questions like "should I use counterfactuals here" that don't answer the obvious reply, "use them FOR WHAT?"

I don’t feel 100% bought-in to the Shapley Value approach, and think there’s a value in paying attention to the counterfactuals. My unprincipled compromise approach would be to take some weighted geometric mean and call it a day.

FOR WHAT?

Let’s assume in all of these scenarios that you are only one of the players in the situation, and you can only control your own actions.

If this is your specification (implicitly / further specification: you're an altruist trying to maximize total value, deciding how to trade off between increasing X and doing good in other ways) then there is a correct answer — maximize counterfactual value (this is equivalent to maximizing total value, or argmaxing total value over your possible actions), not your personal Shapley value or anything else. (Just like in all other scenarios. Multiplicative-ness is irrelevant. Maximizing counterfactual value is always the answer to questions about what action to take.)

My current impression is that there is no mechanism and funders will do whatever they feel like and some investors will feel misled...

I now agree funders won't really lose out, at least.

Hmm. I am really trying to fill in holes, not be adversarial, but I mostly just don't think this works.

the funder probably recognizes some value in [] the projects the investors funded that weren't selected for retrofunding

No. If the project produces zero value, then no value for funder. If the project produces positive value, then it's retrofunded. (At least in the simple theoretical case. Maybe in practice small-value projects don't get funded. Then profit-seeking investors raise their bar: they don't just fund everything that's positive-EV, only stuff that's still positive-EV when you treat small positive outcomes as zero. Not sure how that works out.)

the funder probably recognizes some value in . . . aligned investors likely devoting their "profits" on other good projects

Yes.

If those gains aren't enough for the retrofunder, it could promise 100% payment up to investment price, but only partial payment of impact over the investment price -- thus splitting the surplus between itself and the investor in whatever fraction seems advisible.

Surely this isn't optimal, there's deadweight loss. And it's still exploitable and this suggests that something is broken. E.g. Alice can do something like: write a bad proposal for her project to ensure it isn't funded in advance, self-fund at an investment of $10, and thereby extract $10 from the funders.

This is ultimately up to retro funders, and they each might handle cases like this differently.

Oh man, having the central mechanism unclear makes me really uncomfortable for the investors. They might invest reasonably, thinking that the funders would use a particular process, and then the funders use a less generous process...

In my opinion, by that definition of true value which is accounting for other opportunities and limited resources, they should just pay $100 for it. If LTFF is well-calibrated, they do not pay any more (in expectation) in the impact market than they do with regular grantmaking, because 99% of project like this will fail, and LTFF will pay nothing for those. So there is still the same amount of total surplus, but LTFF is only paying for the projects that actually succeeded.

What happened to "operate on a model where they treat retrospective awards the same as prospective awards, multiplied by a probability of success." Can you apply that idea to this case? I think the idea is incoherent and if not I want to know how it works. [This is the most important paragraph in this comment.] [Edit: actually the first paragraph is important too: if funders aren't supposed to make decisions in a particular way, but just assign funding according to no prespecified mechanism, that's a big deal.]

(Also, if the funder just pays $100, there's zero surplus, and if the funder always pays their true value then there's always zero surplus and this is my original concern...)

There's a different type of "true value", which is like how much would the free market pay for AI safety researchers if it could correctly account for existential risk reduction which is an intergenerational public good.

Sure. I claim this is ~never decision-relevant and not a useful concept.

Actually I'm confused again. Suppose:

Bob has a project idea. The project would cost $10. A funder thinks it has a 99% chance of producing $0 value and a 1% chance of producing $100 value, so its EV is $1, and that's less than its cost, so it's not funded in advance. A super savvy investor thinks the project has EV > $10 and funds it. It successfully produces $100 value.

How much is the funder supposed to give retroactively?


I feel like ex-ante-funder-beliefs are irrelevant and the right question has to be "how much would you pay for the project if you knew it would succeed." But this question is necessarily about "true value" rather than covering the actual costs to the project-doer and giving them a reasonable wage. (And funders have to use the actual-costs-and-reasonable-wage stuff to fund projects for less than their "true value" and generate surplus.)

Ah, hooray! This resolves my concerns, I think, if true. It's in tension with other things you say. For example, in the example here, "The Good Foundation values the project at $18,000 of impact" and funds the project for $18K. This uses the true-value method rather than the divide-by-P(success) method.

In this context "project's true value (to a funder) = $X" means "the funder is indifferent between the status quo and spending $X to make the project happen." True value depends on available funding and other available opportunities; it's a marginal analysis question.

I agree this would be better — then the funders would be able to fund Alice's project for $1 rather than $10. But still, for projects that are retroactively funded, there's no surplus-according-to-the-funder's-values, right?

Load more