This is a linkpost for https://psyarxiv.com/hduyx/

This article would be worth considering by EAs using animal cognition studies to assess their sentience; it suggests a lot of the literature is biased towards overly positive conclusions about animal cognition.

A prominent vein of comparative cognition research asks which cognitive abilities may be ascribed to different species. Here, we argue that the current structure of comparative cognition makes it near impossible to evaluate the accuracy of many of the claims produced by the field’s empirical research. ... This argument is based on the following observations:
1) Phenomenon-based comparative cognition uses confirmatory research methods that are directionally biased
2) In combination with a publication bias and a likely high rate of false discoveries, this bias suggests our literature contains many false positive findings
3) This directional bias persists even with strong methodological criticism, and when researchers explicitly consider alternative explanations for the phenomena studied
4) No formal method exists for generating and assessing theory-disconfirming evidence that could counter the biased positive evidence
5) Ambiguity in definitions allow us as researchers to flexibly adjust our substantive claims depending on whether we are refuting criticism or selling the results
6) The small size of comparative cognition as a research field perpetuates and reinforces points 1 to 5.

My own research training is in neuroethology, where Krogh's Principle is commonly used as a guide:

for such a large number of problems there will be some animal of choice, or a few such animals, on which it can be most conveniently studied.

While these days the principle is helpful for encouraging people to work on animals other than model organisms, it probably reinforces a lot of the biases above (particularly #1). For instance, desert ants have been studied as a model of visual navigation for many years, but were originally chosen for this because they lived in an environment that was hard to navigate and so initial studies mostly confirmed researchers opinions of their exceptional navigation abilities. If cognition is usually tested in exceptional cases then it probably also reduces the generality of findings between species - most ant species don't live in the Saharan Desert and will have had less selection pressure to evolve exceptional navigation abilities.

27

0
0

Reactions

0
0
Comments3


Sorted by Click to highlight new comments since:

Further work from the authors of the original article:

Claims and statistical inference in animal physical cognition research.

Overall, our analysis provides a cautiously optimistic analysis of reliability and bias in animal physical cognition research, however it is nevertheless likely that a non-negligible proportion of results will be difficult to replicate.

I'm not sure how important Krogh's Principle is in animal cognition research of the kind we're interested in; my impression is that the animals that are studied are primarily animals that are well-studied, like fruit flies, bees, mice, rats, cats, dogs, farmed animals and the stereotypically smart ones (corvids, parrots, elephants, cetaceans, primates), and the animals EAs are interested in fall into these groups. When I want to know about chicken cognition, I just look for studies on chickens. It's worth mentioning that Rethink Priorities stuck to relatively narrow taxons in their report.

I do agree that this research is likely to be biased overall to produce more positive than could be reproduced or generalized. However, I also think that the priors are already very skeptical (e.g. Morgan's canon over Occam's razor and despite common descent) so scientists are also likely to attribute fewer and less complex mental states to animals than I think best explains the evidence, and it's pretty clear that we've systematically underestimated their capacities, so it's likely the current state of research underestimates them overall, too.

Or, rather, researchers aren't using Bayesian reasoning in the first place, so they aren't really using priors at all in interpreting evidence; I think Morgan's canon is more like a p-value threshold than a prior.

Of course, we can just use our own priors in interpreting the evidence, and in doing so, we should take into account biases towards positive results in research.

I think I remember reading a book or paper by Noam Chomsky which said that chimpanzees don't have minds because they beg for food usually directly facing a human, but if that doesn't work they will use the same strategy and beg behind the human. He says this proves chimpanzees dont know what a human is---they view humans as food dispensers. My interpretation was if you are going to beg, maybe display subserviance--ask from the back, nor face to face. (Chomsky if i recall also said something like this in 2 lectures on linguistics i attended (along with many other things like 'language did not evolve for communication'--his view was language first evolved to talk to yourself (he has a 'technical term' for this called 'I(nternal) language', which was then later found to have a slight evolutionary use for communication ('E-language') --eg Amazon books found it valuable. ).

Chomsky's view is not what is found in my limited study of social biology, sociobiology, ethology (R Hinde had a reasonable book on this if i recall), etc. (Chomsky has written a few reasonable things, but also many which seem 'off the wall').

While i tend to think animals of all sorts have various forms of cognition, as noted with the 'ant' example, these often will be quite different. (Chomsky has also said the only animal language even close to human language is 'bee language' --none of the other animals like birds or non-human primates have language in his view. I guess this is how you become the most famous public intellectual--Chomksy said it, i beleive it, and that is all there is to it. Chomsky related language to recursion---which as many have noted, he basically did not define until the 2000's. Animals use recursion all the time in my experience. Squirrels know exactly where their acorns are and can point to them. )

Many if not most studies of animal cognition (and human) are flawed or imperfect--not really well thought out, but there is a bias to publish something. Sometimes they seem to be on the right track, and sometimes not.






Curated and popular this week
trammell
 ·  · 25m read
 · 
Introduction When a system is made safer, its users may be willing to offset at least some of the safety improvement by using it more dangerously. A seminal example is that, according to Peltzman (1975), drivers largely compensated for improvements in car safety at the time by driving more dangerously. The phenomenon in general is therefore sometimes known as the “Peltzman Effect”, though it is more often known as “risk compensation”.[1] One domain in which risk compensation has been studied relatively carefully is NASCAR (Sobel and Nesbit, 2007; Pope and Tollison, 2010), where, apparently, the evidence for a large compensation effect is especially strong.[2] In principle, more dangerous usage can partially, fully, or more than fully offset the extent to which the system has been made safer holding usage fixed. Making a system safer thus has an ambiguous effect on the probability of an accident, after its users change their behavior. There’s no reason why risk compensation shouldn’t apply in the existential risk domain, and we arguably have examples in which it has. For example, reinforcement learning from human feedback (RLHF) makes AI more reliable, all else equal; so it may be making some AI labs comfortable releasing more capable, and so maybe more dangerous, models than they would release otherwise.[3] Yet risk compensation per se appears to have gotten relatively little formal, public attention in the existential risk community so far. There has been informal discussion of the issue: e.g. risk compensation in the AI risk domain is discussed by Guest et al. (2023), who call it “the dangerous valley problem”. There is also a cluster of papers and works in progress by Robert Trager, Allan Dafoe, Nick Emery-Xu, Mckay Jensen, and others, including these two and some not yet public but largely summarized here, exploring the issue formally in models with multiple competing firms. In a sense what they do goes well beyond this post, but as far as I’m aware none of t
LewisBollard
 ·  · 6m read
 · 
> Despite the setbacks, I'm hopeful about the technology's future ---------------------------------------- It wasn’t meant to go like this. Alternative protein startups that were once soaring are now struggling. Impact investors who were once everywhere are now absent. Banks that confidently predicted 31% annual growth (UBS) and a 2030 global market worth $88-263B (Credit Suisse) have quietly taken down their predictions. This sucks. For many founders and staff this wasn’t just a job, but a calling — an opportunity to work toward a world free of factory farming. For many investors, it wasn’t just an investment, but a bet on a better future. It’s easy to feel frustrated, disillusioned, and even hopeless. It’s also wrong. There’s still plenty of hope for alternative proteins — just on a longer timeline than the unrealistic ones that were once touted. Here are three trends I’m particularly excited about. Better products People are eating less plant-based meat for many reasons, but the simplest one may just be that they don’t like how they taste. “Taste/texture” was the top reason chosen by Brits for reducing their plant-based meat consumption in a recent survey by Bryant Research. US consumers most disliked the “consistency and texture” of plant-based foods in a survey of shoppers at retailer Kroger.  They’ve got a point. In 2018-21, every food giant, meat company, and two-person startup rushed new products to market with minimal product testing. Indeed, the meat companies’ plant-based offerings were bad enough to inspire conspiracy theories that this was a case of the car companies buying up the streetcars.  Consumers noticed. The Bryant Research survey found that two thirds of Brits agreed with the statement “some plant based meat products or brands taste much worse than others.” In a 2021 taste test, 100 consumers rated all five brands of plant-based nuggets as much worse than chicken-based nuggets on taste, texture, and “overall liking.” One silver lining
 ·  · 6m read
 · 
Cross-posted from Otherwise. Most people in EA won't find these arguments new. Apologies for leaving out animal welfare entirely for the sake of simplicity. Last month, Emma Goldberg wrote a NYT piece contrasting effective altruism with approaches that refuse to quantify meaningful experiences. The piece indicates that effective altruism is creepily numbers-focused. Goldberg asks “what if charity shouldn’t be optimized?” The egalitarian answer Dylan Matthews gives a try at answering a question in the piece: “How can anyone put a numerical value on a holy space” like Notre Dame cathedral? For the $760 million spent restoring the cathedral, he estimates you could prevent 47,500 deaths from malaria. “47,500 people is about five times the population of the town I grew up in. . . . It’s useful to imagine walking down Main Street, stopping at each table at the diner Lou’s, shaking hands with as many people as you can, and telling them, ‘I think you need to die to make a cathedral pretty.’ And then going to the next town over and doing it again, and again, until you’ve told 47,500 people why they have to die.” Who prefers magnificence? Goldberg’s article draws a lot on author Amy Schiller’s plea to focus charity on “magnificence” rather than effectiveness. Some causes “make people’s lives feel meaningful, radiant, sacred. Think nature conservancies, cultural centers and places of worship. These are institutions that lend life its texture and color, and not just bare bones existence.” But US arts funding goes disproportionately to the most expensive projects, with more than half of the funding going to the most expensive 2% of projects. These are typically museums, classical music groups, and performing arts centers. When donors prioritize giving to communities they already have ties to, the money stays in richer communities. Some areas have way more rich people than others. New York City has 119 billionaires; most African countries have none. Unsurprisingly, Schill