I think you attribute an overly strong position to the highly skeptical EAs in post 2, as discussed in my longer comment on this. To illustrate this briefly, I'd distinguish between two positions:

(1) We should never fund activities which are [much?] more speculative than GiveWell recommendations.

(2) We should sometimes fund activities which are [much?] more speculative than GiveWell recommendations, but less often than the average EA would accept.

I'd only defend (2), not (1).

[My mini defense of "Charity Science"-style empirical EA]

Earlier I said I was skeptical of uncertain causes. I still think this approach is largely correct (though with some revisions) and I have hoped for quite some time to elaborate again in length.

Even if it is self-defeating to request evidence in some respect (even if you have evidence for evidence, do you have evidence for evidence for evidence?), the opposite position "we need absolutely no evidence" is also flatly ridiculous (though, admittedly, not self-defeating) and we end up chasing after Pascal's Mugging.

So we need some evidence, clearly -- the question is, how much?

This is a matter of epistemology and priors, but I think the "evidence crowd" has done quite well. Remember that EA was founded on the "evidence crowd". The crowd that didn't request evidence gave us Play Pumps, whereas the crowd that did request evidence has saved thousands of lives through AMF.

But what about moonshots? How much evidence went into eradicating smallpox or producing vaccines? Is GiveWell underinvesting in these sorts of things? I don't really know personally and haven't thought about it much, but it's possible.

I still think, as I've argued before, that it really comes down to what the optimal strategy is under uncertainty and I believe that game theory has established the optimal strategy is to do an explore-exploit pattern, where you start out by exploring the space widely and then slowly transition into exploiting the best things found so far.

I think this explore-exploit is done very well by Good Ventures and GiveWell with OpenPhil doing a lot of the exploring but still exploiting the best so far by investing in AMF.

However, it's pointless to explore unless you actually learn -- otherwise you're not really exploring, you're just exploiting randomly, which is even worse. And the way you learn is by collecting evidence -- evidence that is high-quality enough that you can use it to improve your future actions.

Does this need to be 22 RCTs, as which backs the idea of distributing malaria nets? No -- notably, GiveWell still gives millions to orgs that have much less RCTs and there still are several outstanding questions about AMF's impact.

But on the other hand I see EAs acting without thinking statistically about their activities, running surveys, and showing any signs of skepticism in their work.

I figure I fall into the "skeptical EA" camp, so let me try defending it. :) It'd be good to make progress on this issue so I appreciate your engaging with it! Here I'll consider your two key steps in post 2.

To take your first step first:

"Highly skeptical EAs think you need strong evidence to prove that something works / is true / will happen."

This would be self-refuting, as you say. But I don't think most have quite as strong a position as that. After all, 'provenness' is a matter of degree. It's more that we're relatively negative about speculative activities.

It's also worth distinguishing between:

(1) charities which definitely have a 1% chance of doing an enormous of good. For example, a charity which'd definitely do 101 times as much good as AMF if a 100-sided die were rolled and came up 100.

(2) charities which may have a 1% chance of doing an enormous amount of good, but lack robust evidence for this. E.g. they have no track record, no precedent, no broad expert/academic endorsement. But a back of the envelope calculation including some very rough guesses suggests that they have a 1% chance of doing 101 times as much good as AMF.

I'd give to (1) over AMF, but not (2).

To consider your third step:

"But the claim that it's higher expected value to do the best 'proven' thing rather than a speculative/unproven thing (that on its face looks important, neglected and tractable) is itself unproven to a high standard of evidence."

True. This would be a reductio ad absurdum of the claim that we should only ever believe 'proven' propositions (which we could perhaps define as those which an ideally rational agent in our position would have >90% credence in). But 'skeptical EAs' rarely claim anything so implausible.

The best epistemic approach on these issues is clearly far from proven, so we have no choice but to pick our best bet (adjusting for our uncertainty). It could still be the case, without inconsistency, that the best epistemic approach is to rate relatively speculative activities lower than the average EA does.

"Indeed, it's a very hard claim to ever prove and would require a very large project involving lots of people over a long period of time looking at the average return on e.g. basic science research."

This sort of look at the historical track record of different epistemic approaches does indeed seem the best approach. You're right that the correct answer is far from 90% proven.

"(In my view, we don't really know at the moment and should be agnostic on this issue.)"

If by 'agnostic' you mean 'completely neutral' (which you may very well not?) then I disagree. Some approaches seem better than others, and we should take our best bet.

I'm worried there's a motte and bailey argument where people start out by saying "we need to be speculative because impact will happen through taking risks and focusing on growth!" (motte) and then go to "therefore we don't need to bother considering things like counterfactuals or spending that much time on impact reports" (bailey).

On post 1, this is a good observation, and it certainly speaks in favour of riskier donations. But it's not clear that it does so very strongly, and I'd expect it to be a relatively weak consideration once you start directly considering candidate charities (e.g. AMF vs. an outfit that lobbies for a US carbon tax).