Thanks for raising this issue! I agree with some of your points, but I also think things are somewhat more complicated than you suggest.
The comments here suggest that most people have the intuition that a system should not treat individuals differently based only on the colour of their skin. That is a completely reasonable intuition, especially as doing so would be illegal in most contexts. Unfortunately, such a policy can still lead to undesired outcomes. In particular, a system which does not use race can nevertheless end up having differential impacts at the group level, which may also be illegal. (see here; "Disparate impact refers to policies or practices that are facially neutral but have a disproportionately adverse impact on protected classes.")
The first important thing to note here is that, except for one specific purpose (see below), it typically makes little difference whether a protected attribute such as race is explicitly included in a model such as this or not. (In the ProPublica example, it is not). There are enough other things that correlate with race, that it is likely that you can predict it from the rest of the data (probably better than you can predict recidivism in this case).
As a trivial example, one could imagine choosing to release people based only on postal code (zipcode). Such a model does not use race directly, but the input is obviously strongly correlated with race, and could end up having a similar effect as a model which makes predictions using only race. This example is silly because it is so obviously problematic. Unfortunately, with more complex inputs and models, it is not always obvious what combination of inputs might end up being a proxy for something else.
Part of the problem here is that whether or not someone will be re-arrested is extremely difficult to predict (as we would expect, since it may depend on many other things that will happen in the future). The OP wrote that "any unbiased algorithm will have more false positives for blacks, even if it is equally accurate for both races at any given level of risk", but strictly speaking that is not correct. First, I would say that the use of the term "unbiased" here is somewhat tendentious. Bias is a term that is somewhat overloaded in this space, and I expect you mean something more precise. More to the point, however, if we could predict who will be re-arrested with 100% accuracy, then there would be no difference between groups in rates of false positives or accuracy. That being said, it is true that if we cannot achieve 100% accuracy, and there is a difference in base rates, there is an inherent trade-off between being having equal levels of accuracy and equal rates of false negatives / false positives (which is sometimes referred to as equality of odds).
For example, in the extreme, a system which does not use race, and which supposedly "treats all individuals the same", could end up leading to a decision to always release members of one group, and never release members of another group. Although such a system might be equally accurate for both groups, the mistakes for one group would only be false positives, and mistakes for the other group would only be false negatives, which seems both morally problematic and possibly illegal.
The above example is a caricature, but the example reported on by ProPublica is a less extreme version of it. We can debate the merits of that particular example, but the larger problem is that we can't guarantee we'll obtain an acceptable outcome with a rule like "never let the model use race directly."
Unfortunately, the only way to explicitly balance false negatives and false positives across groups is to include the protected attribute (which is the exception hinted at above). If this information is explicitly included, it is possible to construct a model that will have equal rates of false negatives and false positives across groups. The effect of this, however, (ignoring the 100% accuracy case), will be to have different thresholds for different groups, which can be seen as a form of affirmative action. (As an aside, many people in the US seem strongly opposed to affirmative action on principle, but it seems in general like it might plausibly be given consideration as a possibility, if it can be shown to lead to better outcomes.)
Finally, the details of this example are interesting, but I would say the broader point is also important, namely, that models do in fact pick up on existing biases, and can even exacerbate them. Although there clearly are differences in base rates as highlighted by the OP, there are also clear examples of widespread discrimination in how policing has historically been conducted. Models trained on this data will capture and reproduce these biases. Moreover, given the lack of public understanding in this space, it seems likely that such systems might be deployed and perceived as "objective" or unproblematic, without a full understanding of the trade-offs involved.
While ProPublica's headline definitely suppresses some the complexity involved, the article was highly effective in initiating this debate, and is one of the best known examples of how these problems manifest themselves in practice. I would say that people should know about it, but also be provided with pointers to the subsequent discussion.
There are a number of other issues here, including the difficulty of learning from censored data (wherein we don't know what the outcome would have been for those who were not released), as well as the potential for a single model to prevent us from learning as much as we might from behaviour of many separate judges, but that goes well beyond the scope of the original post.
For more on this, there is a great interactive visualization of the problem here and some more discussion of the underlying technical issues here.
Thanks for raising this issue! I agree with some of your points, but I also think things are somewhat more complicated than you suggest.
The comments here suggest that most people have the intuition that a system should not treat individuals differently based only on the colour of their skin. That is a completely reasonable intuition, especially as doing so would be illegal in most contexts. Unfortunately, such a policy can still lead to undesired outcomes. In particular, a system which does not use race can nevertheless end up having differential impacts at the group level, which may also be illegal. (see here; "Disparate impact refers to policies or practices that are facially neutral but have a disproportionately adverse impact on protected classes.")
The first important thing to note here is that, except for one specific purpose (see below), it typically makes little difference whether a protected attribute such as race is explicitly included in a model such as this or not. (In the ProPublica example, it is not). There are enough other things that correlate with race, that it is likely that you can predict it from the rest of the data (probably better than you can predict recidivism in this case).
As a trivial example, one could imagine choosing to release people based only on postal code (zipcode). Such a model does not use race directly, but the input is obviously strongly correlated with race, and could end up having a similar effect as a model which makes predictions using only race. This example is silly because it is so obviously problematic. Unfortunately, with more complex inputs and models, it is not always obvious what combination of inputs might end up being a proxy for something else.
Part of the problem here is that whether or not someone will be re-arrested is extremely difficult to predict (as we would expect, since it may depend on many other things that will happen in the future). The OP wrote that "any unbiased algorithm will have more false positives for blacks, even if it is equally accurate for both races at any given level of risk", but strictly speaking that is not correct. First, I would say that the use of the term "unbiased" here is somewhat tendentious. Bias is a term that is somewhat overloaded in this space, and I expect you mean something more precise. More to the point, however, if we could predict who will be re-arrested with 100% accuracy, then there would be no difference between groups in rates of false positives or accuracy. That being said, it is true that if we cannot achieve 100% accuracy, and there is a difference in base rates, there is an inherent trade-off between being having equal levels of accuracy and equal rates of false negatives / false positives (which is sometimes referred to as equality of odds).
For example, in the extreme, a system which does not use race, and which supposedly "treats all individuals the same", could end up leading to a decision to always release members of one group, and never release members of another group. Although such a system might be equally accurate for both groups, the mistakes for one group would only be false positives, and mistakes for the other group would only be false negatives, which seems both morally problematic and possibly illegal.
The above example is a caricature, but the example reported on by ProPublica is a less extreme version of it. We can debate the merits of that particular example, but the larger problem is that we can't guarantee we'll obtain an acceptable outcome with a rule like "never let the model use race directly."
Unfortunately, the only way to explicitly balance false negatives and false positives across groups is to include the protected attribute (which is the exception hinted at above). If this information is explicitly included, it is possible to construct a model that will have equal rates of false negatives and false positives across groups. The effect of this, however, (ignoring the 100% accuracy case), will be to have different thresholds for different groups, which can be seen as a form of affirmative action. (As an aside, many people in the US seem strongly opposed to affirmative action on principle, but it seems in general like it might plausibly be given consideration as a possibility, if it can be shown to lead to better outcomes.)
Finally, the details of this example are interesting, but I would say the broader point is also important, namely, that models do in fact pick up on existing biases, and can even exacerbate them. Although there clearly are differences in base rates as highlighted by the OP, there are also clear examples of widespread discrimination in how policing has historically been conducted. Models trained on this data will capture and reproduce these biases. Moreover, given the lack of public understanding in this space, it seems likely that such systems might be deployed and perceived as "objective" or unproblematic, without a full understanding of the trade-offs involved.
While ProPublica's headline definitely suppresses some the complexity involved, the article was highly effective in initiating this debate, and is one of the best known examples of how these problems manifest themselves in practice. I would say that people should know about it, but also be provided with pointers to the subsequent discussion.
There are a number of other issues here, including the difficulty of learning from censored data (wherein we don't know what the outcome would have been for those who were not released), as well as the potential for a single model to prevent us from learning as much as we might from behaviour of many separate judges, but that goes well beyond the scope of the original post.
For more on this, there is a great interactive visualization of the problem here and some more discussion of the underlying technical issues here.