Some thoughts on the effectiveness of the Fraunhofer Society

The Fraunhofer Society is a german research organization that focuses on applying science to develop new technologies. With over 28000 employees and over three billion dollar annual budget the Fraunhofer Society is the biggest research organization in Europe. Therefore, I find it surprising that very little is written about the organization from independent sources. Having worked at Fraunhofer for over one year, I want to share some thoughts and insights on the Fraunhofer organization.

The Fraunhofer Society was founded in 1949 as an attempt to accelerate the growth of West Germany. Today, the Fraunhofer Society says on its website: With its focus on developing key technologies that are vital for the future and enabling the commercial exploitation of this work by business and industry, Fraunhofer plays a central role in the innovation process. As a pioneer and catalyst for groundbreaking developments and scientific excellence, Fraunhofer helps shape society now and in the future [1] . In this sentence no goal of the Society is mentioned. That is because the goal of the society has never been postulated clearly, or even vaguely. The Fraunhofer Society is a non-profit that is uniquely funded receiving a public basic funding of thirty percent of its budget (about one billion dollar annually). Seventy percent of the funding comes from contract research for industry and public sector (more details on that later). The Fraunhofer Society receives only negligible amounts of donations from private donors. Despite that fact, the Fraunhofer Society is a non-profit with a budget that (at least in part) comes
straight from all tax-payers. This budget can be increased or decreased. As a result, the Fraunhofer Society is under pressure to work effectively. If it is no longer a organization that german citizen find worth investing in, it will cease to exist.

This obviously raises the question: How effective is the Fraunhofer Society ? Surprisingly (or maybe not), you will find no answer googling this question. Having worked at Fraunhofer for a bit now I have gained the following insights:

1. The Fraunhofer Society has a very ill-faded incentive structure

This begins with the basic funding of the Society. The Society states, that the basic funding is in place so that research projects can be undertaken whose payoffs lie too far in the future to be of interest for private companies [2]. Although this particular explanation does not make a lot of sense, it is true that the basic financing gives absolute freedom to pursue research that is for whatever reason (market failures etc.) not economical for companies, but could be extremely valuable. However, an unrestricted basic funding also allows for interventions that are not within this space. Furthermore, since the Fraunhofer Society also does contract research for the industry it provides an unfair comparative cost advantage. If you were to provide contract research privately you would have to beat the Fraunhofer Society by far more than 30% in efficiency since you have to pay your competitor. Surprisingly (maybe not), you probably will be able to do that. At Fraunhofer I learned that, despite their advantage, rather rarely teams have the performance and efficiency to challenge private sector companies for contract research. Almost all of the contract research is done for public projects, often in joint-ventures with companies. That way, most of the funding comes from public sources.

Although the funding structure of the Fraunhofer Society is troubling in and of itself, it actually still allows for the Fraunhofer Society kicking ass and being an extremely effective research organization. Now here is why it is not: In order to undertake effective interventions the Society has to be either forced and restricted to effective projects or has to have some built-in form of goal-oriented behavior that incentivizes effectiveness. Since there is not even a goal, the latter is not the case. There is some effort by the public sector to only fund projects that are effective. As stated before, besides the basic funding most of the funding comes from public projects. The public sector in Germany always invites for tenders in a broad range of areas or causes. The institutes (and other companies, NGOs etc.) provide project descriptions of the projects that they offer to carry out to improve on those causes. How the decision which projects get funded and which do not is made is very intransparent. The thing is, the project choice is very, very poor. I see this as the most fundamental system flaw. If there were an intact system that would prohibit the worst projects from being realized, that would be huge.

I will try to quickly give an idea about how bad the situation is. I have worked only on a handful of projects so my sample size is limited to say the least. One project that is supposed to mitigate the "plastic problem" relies on inventing a thermoplastic material that can be mechanically repurposed infinitely often. Not having a particular background in chemistry, I know that this is almost certainly physically impossible, not speaking of whether this intervention would be effective if it were. The project budget for this one is around 3 million dollars. An other project focused on developing a new technology that was supposed to be used to safeguard products. After about 2 million dollars and 3 years of development, there is no single product known that can benefit from this protection. The value of these interventions is downright zero. These were fully publicly funded projects. That is around $5/0.7\approx 7$ million dollars flushed. I did not cherry-pick these projects. I have spent most of my time at Fraunhofer with these two projects. They make up most of my sample size. It gets worse. I have read a couple of project descriptions. The project descriptions are really terrible. A couple of distinct features are:

• There is never any total metric of impact (like this intervention raises y by x). Instead the benefits of the intervention are described very buzzwordy and non-quantitative (like this intervention helps transition to industry 4.0, protect workplaces in the region b and improves the digitalization of x)
• The project descriptions are full of logical errors and unproven statements, most often when describing the benefits of the intervention (like stating that a intervention improves x, but there is no obvious or stated correlation or causation). For example, it was stated that a technology that was developed would basically end counterfeiting of pharmaceuticals, without giving a hint why this assumption has been made. It took me a couple hours of internet research to find out that there is no way of even applying the technology to most pharmaceuticals. This should have come up during the decision making.
• Deception and deliberately lying about the teams capabilities and equipment is normal. I have been in a team that manipulated their measurement results to exhibit a correlation (that they knew was not there) in order to land a project, stating that they were able to reproduce and apply the effect. If there are quantitative estimates in project descriptions they rely on physical boundaries (like this antenna will be able to broadcast 10 meters far). In the end, results from demonstrators often lie orders of magnitude below the estimates made in the project description.

Of course, it is not that the employees at Fraunhofer want to do harmful things. Many are cognitively dissonant, actually thinking that they do tremendous good. But many are aware of the problematic situation they are in. The dilemma is: Not having any goal-oriented incentive system, the Fraunhofer Society is dominated by the personal incentive of its members: Job security. At Fraunhofer people do have a little less job security than the average worker. If your team has not enough funding you will find yourself working elsewhere within months. Therefore many groups and institutes are desperate for funding and just send in a lot of trashy project offers. I have the feeling that given enough project offers, someone is going to decide to fund a project, which is almost the definition of non-scientific.

2. Fraunhofer has the image of being a place where the brightest minds work to achieve innovations that the average person can not conceive of

This is the feeling that I have when I speak about Fraunhofer with friends, family and strangers. You can watch this german image movie to get a sense of what I am referring to. I think the creation of this image is done to help justify the huge public expenses of Fraunhofer. People are supposed to think Fraunhofer is a place where smart scientists in lab coats shoot with quantum-lasers at mutated plants to retrieve a pharmaceutical that heals all cancer. Fraunhofer is not the place where the brightest minds work. To make this a quick one, a couple of citations from three different senior scientists and team or project leaders:

"I have contacted the colleagues from the Institute B. I just want another Institute on the title page of our project description to give the project more weight (importance)"

"If you are not a scientist, than you do not have to work scientifically"

When I told a senior scientist about CoolEarth, she replied:

"When it comes to climate change, we have to stop thinking in numbers"

When I asked her why, she said : "Because you can´t just throw a couple of dollars at the ground and ask mother nature to do it one more year"

(This has not been previously suggested by anyone)

To end on a more positive note: Almost all of my colleagues are nice and pleasant people with whom I would really like to work on something else. Especially as a student or PhD student Fraunhofer is worth considering as a investment in your career capital. You can very easily get into a decently paid position at Fraunhofer where you are learning a lot quickly because the set of tasks is often very diverse. Working at Fraunhofer can be very fun and pleasant if you are actually working on something meaningful or when you are distanced from the economic reality of the Society and unsuspecting. Sorry about that one. I feel as if the Fraunhofer Society was (similar to the catholic church) founded with good intentions but the underengineered structure took on a life of its own and has deviated from the intended path.

-------------------------------------------------------------------

Concluding, the effectivity of the Fraunhofer Society is likely much lower than competing alternatives. The main problem seems to be that public funding is not provided based on evidence.

My guess is that great improvement in this area comes with considerable effort since multiple parts of the system are broken and all have to be fixed to mitigate the problem. However, I can think of some "quick fixes" that might be worth investing in, yielding greater result per unit of effort. One possible measure would be to just demand a quantitative assessment of impact in the project descriptions. This would force a calculation which I think is easier to detect as flawed and allows for easier cross-project comparison by grant-makers. But I am not very sure about that. An additional measure could be restricting the project funding or future funding to actual results (in essence to the track record of the organization, ideally using a gittins index backoff). This would create incentive to estimate realistic project outcomes.

------------------------------EDIT_Nov20---------------------------

After reading "Inadequate Equilibria" by Eliezer Yudkowsky, I come to the conclusion that the Fraunhofer-Public-Funding-System is in a state of inadequacy.

1. Lack of sensible incentive-structures

Decision makers do not benefit from better judgment. Nobody gets fired when the money is allocated to the wrong project OR when the most important project does not get funded.

2. Asymmetrical information

The Institutes hold way more information about the potential project than grant-makers. That way they can easily create a false impression (e.g. estimated likelihood of project success) .

3. Nash-equilibria that is not the best possible outcome

Everybody does what is best for them. Employees at Fraunhofer secure their job by securing funds, lying about their capabilities and the expected impact of the project if necessary. Decision-makers in charge of allocating funds do not look closely for the most efficient projects since this would mean investing a lot of time and thought without extra pay.