Hamish Huggard

Comments

Statistics for Lazy People, Part 1

Thank you. I have corrected the mistake.

The relationship between Lindy, Doomsday, and Copernicus is as follows:

  • The "Copernican Principle" is that "we" are not special. This is a generalisation of how the Earth is not special: it's just another planet in the solar system, not the centre of the universe.
  • In John Gott's famous paper on the Doomsday Argument, he appeals to the the Copernican Principle to assert "we are also not special in time", meaning that we should expect ourselves to be in a typical point in the history of humanity.
  • The "most typical" point in history is exactly in the middle. Thus your best guess of the longevity of humanity is twice its current age: Lindy's Law. 
I scraped all public "Effective Altruists" Goodreads reading lists

This is brilliant!

I think we can actually do an explicit expected-utility and value-of-information calculation here:

  • Let one five-star book = one util 
  • Each book's quality can be modelled as a rate  of producing stars. 
  • The star rating you give a book is the sum of 5 Bernoulli trials with rate 
  • The book will produce  utils of value per read in expectation.
  • To estimate , sum up the total stars awarded  and total possible stars .
  • The probability distribution is then  (assuming uniform prior for simplicity).
  • For any pair of books, we can compute the probability that book 1 is more valuable than book 2 as .
  • Let's say there's a prescribed EA reading list. 
  • Let people who encounter the list be probabilistic automata.
  • These automata start at the top of the list, then iteratively either: 1) read the book they are currently looking at, 2) move down to the next item on the list, 3) quit.
  • Intuitively, I think this process will result in books being read geometrically less as you move down the list.
  • For simplicity, let's say the first book is guaranteed to be read, the next book has a 50% chance of being read, then 25%, ..., and then -th book has  chance of being read (with  starting at zero).
  • The expected value of the list is then 
  • To calculate the value of information for reading a given book, you enumerate all the possible outcomes (one-star, two-stars, ...., five-stars), calculate the probability of each one, look at how the rankings would change, and re-calculate the the expected value of the list. Multiply the expected values by the probabilities et voila

Can I get the data please?

BitBets: A Simple Scoring System for Forecaster Training

It just occurred to me that you don't actually need to convert the forecaster's odds to bits. You can just take the ceiling of the odds themselves:
 

Which is more useful for calibrating in the low-confidence range.

BitBets: A Simple Scoring System for Forecaster Training

Additional note: BitBets is a proper scoring rule, but not strictly proper. If you round report odds which are rounded up to the next power of two you will achieve the same scores in expectation.

BitBets: A Simple Scoring System for Forecaster Training

Thanks for the insightful comments.

One other thought I've had in this area is "auctioning" predictions, so that whoever is willing to assign the most bits of confidence to their prediction (the highest "bitter") gets exclusive payoffs. I'm not sure what this would be useful for though. Maybe you award a contract for some really important task to whoever is most confident they will be able to fulfill the contract.