Comments
141
BLUF:
* To determine whether AI is ‘improving exponentially’, ‘hitting the wall’, or any other claim which involves a quantity or magnitude (e.g. ‘This model was a big leap/small increment’). We need a good y-axis: an interval scale of AI capability which means +1 unit always represents the same degree of ‘how much better’, in the same way +1 degree Celsius is always the same amount of ‘how much hotter’.
* Yet there is no good y-axis for AI capability. All our...
I think most frequentists would agree that Bayesian inference is more intuitive. Bayesian inference is much more computationally difficult though, and you usually get the same answer anyways. (Bayesian estimators are typically asymptotically equivalent to classical estimators!)
> and you usually get the same answer anyways
I don't agree with this! In reality we don't get asymptotic properties, we get finite sample properties, and these can vary greatly. E.g. MLE often won't even converge for hierarchical models without a fair amount of data. Also, for bespoke models there often isn't a published frequentist estimator available, and attempting to derive one would be a much bigger issue for most people than the computational resources required for MCMC or variational inference.