Effective Altruism Forum
EA Forum

All of JoyOptimizer's Comments + Replies

FTX Crisis. What we know and some forecasts on what will happen next
JoyOptimizer1y2
0
2

While they are insolvent, FTX and SBF have not declared bankruptcy. In developing scenarios, information is unclear and from unknown sources. (Alameda's balance sheet may prove incomplete.)

Reply
2
Jason
1y
Bahamian authorities have obtained a court order to begin provisional liquidation proceedings for "FTX Digital Markets and related parties."
FTX Crisis. What we know and some forecasts on what will happen next
JoyOptimizer1y6
2
13

Calm down. It's a complex situation developing rapidly, let's wait and see for what happens as a final outcome.

Reply
8
Greg_Colbourn
1y
I think it's only natural for EAs to be doing a bit of collective soul searching now.
5
Jason
1y
There is wisdom in this comment: for many people, following this story closely as it rapidly develops doesn't add any value to their lives or their work. However, I suspect it will be many months before we have a truly "final outcome." At least as of this morning, people have enough knowledge of the situation (and the range of likely outcomes) that it should be affecting -- or at least causing them to delay -- decisions that need to be made in the next weeks to months. I think it is extremely likely that FTX is done for and that the bulk of Bankman-Fried's assets are gone. It wouldn't be responsible to ignore that likelihood in making any decisions from this time forward, unless and until the probabilities change.  People involved in managing EA organizations, people who were considering sizable donations in the near term, and people who are employed at organizations highly dependent on FTX-affiliated funding probably shouldn't wait for a more final outcome before assessing their situations. But for those who don't have any decisions to make in the next few weeks to months (including decisions not to change funding or operations) that could be affected by the breaking news, I think you're absolutely correct.
7
Charles He
1y
This is very sad and terrible, but I think the relevant companies are bankrupt and rescue seems unlikely. Maybe if it's helpful in some way, the newspapers Reuters and NYT contain information, if you want to read it.    Onlookers: Guys, it's a highschooler, I think there might be special duties  here to people of different ages and backgrounds.
Open Thread: October — December 2022
JoyOptimizer2y1
0
0

I used a model I fine-tuned to generate takes on Effective Altruism. 

was unclear. It should be:

I used a model that I fine-tuned, in order to generate takes on Effective Altruism.

This model was not fine-tuned specifically for Effective Altruism. It was developed to explore the effects of training language models on a twitter account. I became surprised and concerned when I noticed it was able to generate remarkable takes regarding effective altruism, despite not being present in the original dataset. Furthermore, these takes are always criticism.

This p... (read more)

Reply
Open Thread: October — December 2022
JoyOptimizer2y5
1
0

I used a model I fine-tuned to generate takes on Effective Altruism. The prompt is "effective altruism is." Here are its first three:

effective altruism is vampirism, except instead of sucking blood you suck hours and happiness from helping people who would otherwise have spent the time improving their lives.

effective altruism is parasitic. it latches onto the success of actual altruism, which is genuine and humanizing, to justify its cold calculations and make them feel virtuous too.

effective altruism is rich kid hobbyism pretending to be a moral imperativ

... (read more)
Reply
2
Charles He
2y
I have some moderately useful comments if you're interested. Some basic questions: Are you running this on GPT-NeoX-20B? If so, how are you rolling this? Are you getting technical support of some kind for training? Are you hand selecting and cleaning the data yourself?
1
EdoArad
2y
Beautiful
[AMA] Announcing Open Phil’s University Group Organizer and Century Fellowships
JoyOptimizer2y1
0
0

Who is responsible for evaluating the success of the Century Fellowship?

Reply
2
abergal
2y
Interpreting this as “who is responsible for evaluating whether the Century Fellowship is a good use of time and money”, the answer is: someone on our team will probably try and do a review of how the program is going after it’s been running for a while longer; we will probably share that evaluation with Holden, co-CEO of Open Phil, as well as possibly other advisors and relevant stakeholders. Holden approves longtermist Open Phil grants and broadly thinks about which grants are/aren’t the best uses of money.
[AMA] Announcing Open Phil’s University Group Organizer and Century Fellowships
JoyOptimizer2y1
0
0

What role do different people in reviewing applications for the fellowship, and who fills those roles?

Reply
2
abergal
2y
Each application has a primary evaluator who is on our team (current evaluators: me, Bastian Stern, Eli Rose, Kasey Shibayama, and Claire Zabel). We also generally consult / rely heavily on assessments from references or advisors, e.g. other staff at Open Phil or organizations who we work closely with, especially for applicants hoping to do work in domains we have less expertise in.
1
JoyOptimizer
2y
Edited.
Open Thread: June — September 2022
JoyOptimizer2y1
0
0

Can you help write test prompts for GPT-EA? I want testcases and interesting prompts you want to see tried. This helps track and guide the development of GPT-EA versions. The first version, GPT-EA-Forum-v1 has been developed. GPT-EA-Forum-v2 will include more posts and also comments.

Reply
When 2/3rds of the world goes against you
JoyOptimizer2y1
0
0

this is why we're building an AI to make humans kinder to each other

Reply
1
Jeffrey Kursonis
2y
Yes, well if it can solve many of our problems, it will remove the issues that cause us to defend our own against the other...but if we keep being bad, it will make alignment that much harder, so I think we need it before, though whenever it happens is fine. Love your name. 
JoyOptimizer's Quick takes
JoyOptimizer2y1
0
0

This is a call for test prompts for GPT-EA. (announcement post: https://forum.effectivealtruism.org/posts/AqfWhMvfiakEcpwfv/training-a-gpt-model-on-ea-texts-what-data) I want testcases and interesting prompts you want to see tried. This helps track and guide the development of GPT-EA versions. The first version, GPT-EA-Forum-v1 has been developed. GPT-EA-Forum-v2 will include more posts and also comments.

Reply
How to become more agentic, by GPT-EA-Forum-v1
JoyOptimizer2y7
0
0

One goal is to make it easier to understand Effective Altruism through an interactive model.

I'm sick with COVID right now. I might respond in greater depth when I'm not sick.

Reply
4
Gavin
2y
No rush!
7
aogara
2y
This model performance is really impressive, and I'm glad you're interested in large language models. But I share some of Gavin's concerns, and I think it would be a great use of your time to write up a full theory of impact for this project. You could share it, get some feedback, and think about how to make this the most impactful while reducing risks of harm.  One popular argument for short-term risks from advanced AI are the risks from AI persuasion. Beth Barnes has a great writeup, as does Daniel Kokotajlo. The most succinct case I can make is that the internet is already full of bots, they spread all kinds of harmful misinformation, they reduce trust and increase divisiveness, and we shouldn't be playing around with more advanced bots without seriously considering the possible consequences.  I don't think anybody would make the argument that this project is literally an existential threat to humanity, but that shouldn't be the bar. Just as much as you need the technical skills of LLM training and the creativity and drive to pursue your ideas, you need to be able to faithfully and diligently evaluate the impact of your projects. I haven't thought about it nearly enough to say the final word on the project's impact, but before you keep publishing results, I would suggest spending some time to think and write about your impact. 
Digital people could make AI safer
JoyOptimizer2y1
0
0

Digital humans would be much cheaper to query than biological humans. This is because:

An efficient general intelligence on a biological substrate uses a brain structure. It's unclear if that same structure would be efficient on silicon or photonic processors.

Reply
Training a GPT model on EA texts: what data?
JoyOptimizer2y1
0
0
  • A book on ethics seems worth considering. Can you tell me more about how the ideas relate to EA? Nonetheless, these are useful sources for future projects regarding AI alignment.
  • Is utilitarianism.net only about utilitarianism? If so, the rest of the training set should already have a sufficient degree of utilitarian bias.
  • How influential is FHI's texts on the EA community?
  • This seems like a good text to make the model more generally coherent.
Reply
Training a GPT model on EA texts: what data?
JoyOptimizer2y7
0
0

The goal is not to create a model to create the most good. While aligning an AI with values and principles could be a potentially interesting project, the goal of this project is to create a descriptive model of the EA community, not a normative one of the idealized EA community.

I believe GPT-3 can do more than memorizing specific objectives like malaria nets. Infusing principles deeply would need to happen using more sophisticated techniques, probably post-finetuning. 

upbias (-1, 1) is the Forum editors' or users' perspective on the fraction of upvot

... (read more)
Reply
1
brb243
2y
Ok. Average of the values estimated by editors and users familiar with emotional reasoning/marketing tricks or hosting a focus group discussion and agreeing on a number (using human intelligence to calibrate and weigh participants' estimates based on their arguments and relevant skills presentation). Thanks for reviewing the books. In case you are interested I made reading questions for 5 of them. GPT-3/J: I see.  So, the 2/3 reduce critical reasoning by attention-captivating tricks using the legitimacy presentation of the 1/3 academic sources ah hah (can be read as an exaggeration). The inclusion of academic sources also makes arguing against bias less thinkable (due to a 'respectful/less questioning' approach of academics' claims and trust in their neutrality and comprehensive coverage of important topics - this makes me think - is the academic text selected based on if it is a 'conversation ender,' including based on biased norm perpetuation, rather than an invitation for an inclusive solutions-oriented discourse about topics that concern especially disadvantaged groups?). However, it can be a positive step toward GPT-n, which uses 50% academic sources (international), 15% investigative journalism, 10% non-western newspapers and the UN website with its links, and  5% impact investors' sites, NGO sites, and anything nodal in rationality thinking. Also, I must be biased about the GPT-J name stepping up aggression or threat (the category paying attention and renarrating it's cool). I mean it's possibly just a bias don't worry about it. Hmmm .. that is a great question - I have not reviewed the SSC or similar websites in detail but would imagine that the posts get people start thinking about EA-related topics (rather than being for those already up to speed). It can make sense that a post which only hints on some EA topics would not get on the EA Forum (or not be highly upvoted), however, it is also possible that these posts talk about important EA-related topic
Training a GPT model on EA texts: what data?
JoyOptimizer2y1
0
0

How much % of the training mix should be the GiveWell blog and how much should be the 80,000 hours blog? In other words, how many bytes of blog posts should be used from each, relative to the entire dataset?

What kinds of posts are on each blog, and which best reflects the wider EA community, and which reflects the professional EA community? How can this be used to create a dataset?

I also checked and neither blog has a direct view count measure-- some other proxy metric would need to be used.

Reply
2
Charles He
2y
Hmmm.  You're focused on the input text. Maybe, like, it seems like you want to focus on the "output"  instead (and define some metric[1] relative to this output and the "your targeted performance of the model") ? In contrast to focusing on the output, focusing on the mix of input data seems different. For example, it's not clear that a pass with a batch of GiveWell content, will shift GPT-3 more or less vs a same size batch of 80k content. It's not clear that the input length of text would be a good measure, versus something like "perplexity of the fine tune text to the current GPT-3 output". I haven't trained a GPT-3 model though so I'm not sure. 1. ^ Although, in some sense, it's really hard/crazy to think about what this metric would be, besides something trivial like perplexity. Maybe this difficulty is what you want to avoid?
1
Lorenzo Buonanno
2y
I honestly really don't know :/ I know it doesn't help you, but I would expect both blogs (and all the other stuff on the websites that's not in the blogs) to have some content aimed at a wider audience and some content that goes more into depth for a narrower audience.
Training a GPT model on EA texts: what data?
JoyOptimizer2y1
0
0

Thanks for these sources.

How should GiveWell blog and 80,000 hours blog weighted against each other? My instinct is to weight by the number of views.

Posts/comments in Facebook groups, slack groups, and discord groups?

Does the EA community have the norm that these comments are public? I want to make sure the consent of participants is obtained.

Reply
1
Lorenzo Buonanno
2y
That's a very good point and I think it's definitely not the norm, didn't think about text potentially getting leaked from the training set.   What do you mean against each other? Do you mean compared to everything else, including the forum posts/comments? I have no idea, I think the number of views might lead to a better representation of the wider community, while the more technical posts might be more representative of the more "professional" parts of the movement.
Unflattering reasons why I'm attracted to EA
JoyOptimizer2y1
0
0

This is a list of EA biases to be aware of and account for.

Reply
Global health is important for the epistemic foundations of EA, even for longtermists
JoyOptimizer2y1
0
0

The definition of health here should include mental and socioemotional health, since they affect how people reason and relate to each other, respectively.

Reply