Update, 12/7/21: As an experiment, we're trying out a longer-running Open Thread that isn't refreshed each month. We've set this thread to display new comments first by default, rather than high-karma comments.
If you're new to the EA Forum, consider using this thread to introduce yourself!
You could talk about how you found effective altruism, what causes you work on and care about, or personal details that aren't EA-related at all.
(You can also put this info into your Forum bio.)
If you have something to share that doesn't feel like a full post, add it here!
(You can also create a Shortform post.)
Open threads are also a place to share good news, big or small. See this post for ideas.
Hi,
I am writing to say that I might be doing “moderately-high temporal resolution scrapes of some subset of EA Forum content".
This comment/notification is mainly for forum technical admin, and anyone interested in these scrapes or the potential products of such a project.
Precedents for this scraping include this post, this question, the existence of the API and it's discussion here, and general open source/discussion principles, or something.
Feel free to discuss!
For more information, I've very quickly written rambling, verbose thoughts in a reply to this comment.
Hi Nuño,
This is generous of you.
So I managed to stitch together a quick script in Python. This consists of GraphQL queries created per the post here and Python requests/urllib3.
If you have something interesting written up in js, that would be cool to share! I guess you have much deeper knowledge of the API than I do.
It was a bit of a hassle was getting it packaged and running on AWS, with Lambda calls every few minutes. But I got it working!
Now, witness the firepower of this fully armed and operational battlestation!