Hide table of contents

ALTER is excited to announce our continued progress on a number of fronts, despite some clear challenges and setbacks. Below we summarize our work, then our funding situation.

Before the main update, we are excited to launch our new website, here! (Direct link to English version here.) We’ve redesigned to update from the original version, and better highlight our current focus areas, contributors, and staff.

Substantive Project Areas

AI Policy and Standards

AI Policy and standards has been ALTER’s primary focus recently, and we are pursuing a number of avenues simultaneously. 

We have continued working with the NIST AI Safety Consortium, primarily around CBRN and especially biorisks from foundation models, but also on reporting standards for AI systems. David gave a presentation on understanding a threat model from AIxBio use by sophisticated actors, with encouragement and assistance from Holden Karnofsky and Luca Righetti. This work is primarily impactful in assisting AI firms and informing key actors about risks, and to build norms around testing and reporting for frontier AI, rather than any expectation that NIST will impose standards.

David has also been engaging with the EU code of practice as an individual expert in order to emphasize and prioritize external auditors and evaluations for AI models as part of the safety process. The process has benefited greatly from the excellent work of the chairpersons, and seems likely to focus on key issues much more than we would otherwise have expected. (There remains an unfortunate level of focus on risks that we believe do not pass a cost/benefit assessment for regulation, especially for non-frontier models.)

We have also begun work with the ISO on an in-progress standard on human oversight of AI systems. David is participating as an author, and believes that having clear international standards pointing out the challenge and non-trivial nature of providing such oversight is high leverage, and benefits from having clear vision about large scale risks and challenges. This is true despite the fact that the process is very slow, is plausibly being outpaced by the risk itself, and does not lead directly to any form of binding requirements.

Finally, we have been able to attend a number of Knesset (Israeli Parliament) Science and Technology committee meetings on AI progress and regulation, and met some of the key actors in Israel. We currently see Israeli regulation as unlikely to be directly impactful, but will continue to monitor and engage if opportunities present themselves.

AI Community Building

ALTER has continued to look for ways to support the Israeli AI safety community, including working directly with and mentoring individuals. Work on this has been slow due to both the ongoing war, and insufficient funding for us to support promising safety-focused researchers directly - but there is discussion of restarting the AI safety coworking days in the office to foster the community. We have also been excited to see work supported by Open Philanthropy with EA Israel focused on AIxCyber, and will continue to engage with them, and hope to help support the program and help the participants in finding useful career paths.

Learning-Theoretic AI Safety

As we announced earlier this year, Vanessa Kosoy has joined ALTER as a researcher working on the ARIA Safeguarded AI program. As mentioned in past updates, this work is being separated from ALTER Israel, and the ARIA contract is with AshGro; it has ALTER as a subcontractor which employs Vanessa, and a small amount of David’s time for management. Vanessa is PI for the research, alongside the US-based team which she is managing. Both she and David will be attending the quarterly research update and review meeting in February. There is substantive progress on this work, much of which was outlined here and we expect that additional work will be shared separately on the alignment forum and in publications.

Biorisk / AIxBio

The joint work with RAND started more slowly than hoped, but is now proceeding apace. The details of the planned projects are not publicly disclosable, but we expect at least one publication to be released by mid-2025.

Bioweapons / BWC

David just attended the meeting of state parties in Geneva; the conference itself was unsuccessful due to the inability to find someone to chair the meeting, but (in part because the meeting was suspended while looking for a chairperson most of the day,) David was able to connect with a number of people regarding the potential obstacles and paths forward for an eventual Israeli accession to the BWC, as well as to discuss other work in biosecurity.

Salt Iodization

We have continued our side-project to have Israel iodize table salt, and we are happy to announce that the bill we have been working to introduce now has several co-sponsors, and was introduced to the Knesset December 23rd! We will continue to monitor and engage on this process. Note that this work allows us to engage with lawmakers we otherwise would not have access to, and advances an important object-level high-leverage intervention, albeit outside of our highest-priority focus areas.

Funding Situation

Unfortunately, ALTER remains constrained by funding with minimal runway. We are planning to apply to LTFF and other funders in the coming months for further funding. Our ongoing budget through September is mostly met via the RAND contract and ARIA work; this non-grant income was initially pursued in part because it allows us flexibility - notably, the (inexpensive) work with a political lobbyist for iodization, which was otherwise difficult to fund because of restrictions for grants. However, at this point the contract income is almost all of our 2025 income, and it is unclear whether it will extend past October 2025.

For the remainder of our funding, and a buffer for flexibility, we were previously hopeful that we would be funded by SFF. While we received a small speculation grant, we were given nothing in the main round; the grant round was both oversubscribed, and we received disappointing feedback. (In addition to the general feedback that it was oversubscribed, one grantor said that Israel was not a priority country, which seems to have been a slight misapprehension; as explained above, our policy focus is almost exclusively international, and only our talent development is primarily domestically focused. Another grantor’s feedback said that generalist organizations with multiple priorities were harder to evaluate; this seems unfortunate.)

We would be excited to hear any feedback or suggestions on our work this year, either in the comments or directly!

37

0
0

Reactions

0
0

More posts like this

Comments1


Sorted by Click to highlight new comments since:

Executive summary: ALTER Israel reports progress in AI policy, standards, and safety work in 2024, but faces funding constraints that may impact future operations beyond October 2025.

Key points:

  1. Major progress in AI policy work through engagement with NIST AI Safety Consortium, EU code of practice, and ISO standards development, focusing on biorisks and safety protocols
  2. Continued support of Israeli AI safety community, though activities were slowed by war and funding limitations
  3. Advancement in learning-theoretic AI safety through ARIA Safeguarded AI program, led by Vanessa Kosoy
  4. Successfully introduced salt iodization bill to Knesset with multiple co-sponsors
  5. Critical funding challenges: primarily reliant on contract income from RAND and ARIA work, with uncertain funding beyond October 2025 after disappointing results from grant applications

 

 

This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.

Curated and popular this week
 ·  · 10m read
 · 
I wrote this to try to explain the key thing going on with AI right now to a broader audience. Feedback welcome. Most people think of AI as a pattern-matching chatbot – good at writing emails, terrible at real thinking. They've missed something huge. In 2024, while many declared AI was reaching a plateau, it was actually entering a new paradigm: learning to reason using reinforcement learning. This approach isn’t limited by data, so could deliver beyond-human capabilities in coding and scientific reasoning within two years. Here's a simple introduction to how it works, and why it's the most important development that most people have missed. The new paradigm: reinforcement learning People sometimes say “chatGPT is just next token prediction on the internet”. But that’s never been quite true. Raw next token prediction produces outputs that are regularly crazy. GPT only became useful with the addition of what’s called “reinforcement learning from human feedback” (RLHF): 1. The model produces outputs 2. Humans rate those outputs for helpfulness 3. The model is adjusted in a way expected to get a higher rating A model that’s under RLHF hasn’t been trained only to predict next tokens, it’s been trained to produce whatever output is most helpful to human raters. Think of the initial large language model (LLM) as containing a foundation of knowledge and concepts. Reinforcement learning is what enables that structure to be turned to a specific end. Now AI companies are using reinforcement learning in a powerful new way – training models to reason step-by-step: 1. Show the model a problem like a math puzzle. 2. Ask it to produce a chain of reasoning to solve the problem (“chain of thought”).[1] 3. If the answer is correct, adjust the model to be more like that (“reinforcement”).[2] 4. Repeat thousands of times. Before 2023 this didn’t seem to work. If each step of reasoning is too unreliable, then the chains quickly go wrong. Without getting close to co
 ·  · 11m read
 · 
My name is Keyvan, and I lead Anima International’s work in France. Our organization went through a major transformation in 2024. I want to share that journey with you. Anima International in France used to be known as Assiettes Végétales (‘Plant-Based Plates’). We focused entirely on introducing and promoting vegetarian and plant-based meals in collective catering. Today, as Anima, our mission is to put an end to the use of cages for laying hens. These changes come after a thorough evaluation of our previous campaign, assessing 94 potential new interventions, making several difficult choices, and navigating emotional struggles. We hope that by sharing our experience, we can help others who find themselves in similar situations. So let me walk you through how the past twelve months have unfolded for us.  The French team Act One: What we did as Assiettes Végétales Since 2018, we worked with the local authorities of cities, counties, regions, and universities across France to develop vegetarian meals in their collective catering services. If you don’t know much about France, this intervention may feel odd to you. But here, the collective catering sector feeds a huge number of people and produces an enormous quantity of meals. Two out of three children, more than seven million in total, eat at a school canteen at least once a week. Overall, more than three billion meals are served each year in collective catering. We knew that by influencing practices in this sector, we could reach a massive number of people. However, this work was not easy. France has a strong culinary heritage deeply rooted in animal-based products. Meat and fish-based meals remain the standard in collective catering and school canteens. It is effectively mandatory to serve a dairy product every day in school canteens. To be a certified chef, you have to complete special training and until recently, such training didn’t include a single vegetarian dish among the essential recipes to master. De
 ·  · 1m read
 · 
 The Life You Can Save, a nonprofit organization dedicated to fighting extreme poverty, and Founders Pledge, a global nonprofit empowering entrepreneurs to do the most good possible with their charitable giving, have announced today the formation of their Rapid Response Fund. In the face of imminent federal funding cuts, the Fund will ensure that some of the world's highest-impact charities and programs can continue to function. Affected organizations include those offering critical interventions, particularly in basic health services, maternal and child health, infectious disease control, mental health, domestic violence, and organized crime.