I had an idea for a new concept in alignment that might allow nuanced and human like goals (if it can be fully developed).

Has anyone explored using neural clusters found by mechanistic interpretability as part of a goal system? 

So that you would look for clusters for certain things e.g. happiness or autonomy and have that neural clusters in the goal system.  If the system learned over time it could refine that concept. 

This was inspired by how human goals seem to have concepts that change over time in them. 

I've got an idea for a business that could help biosecurity by helping stop accidental leaks of data to people that shouldn't have it. I'm thinking about proving the idea with personal identifiable information. Looking for feedback and collaborators.

My expectation is that software without humans in the loop evaluating it, will Goodhart's law itself and over fit to the metrics/measures given.

My blog might be of interest to people 

Here is a blog post also written with Claudes help that I hope to engage with home scale experimenters with

I appreciate your  views on space and AI working with ML systems in that way might be useful. 

But I think that I am drawn to the base reality a lot because of threats to that from things like gamma ray bursts or aliens. These things can only be represented probabilistically in simulations because they are out of context. The branching tree explodes with possibilities. 

I agree that we aren't ready for agents , but I would like to try to build time non-static intelligence augmentation as slowly as possible. Starting with building systems to control and shape them tested out with static ML systems.  Then testing them with people. Then testing them inside simulations etc

I find your view of things interesting. A few questions, how do you deal with democracy when people might be inhabiting worlds unlike the real one and have forgotten the real one exists? 

I think static AI models lack corrigibility, humans can't give them instruction on how to change how to act, so they might be a dead end in terms of day to day usefulness. They might be good as scientists though as they can be detached from human needs. So worth exploring.

There is a concept of utility, but I'm expecting these systems to mainly be user focussed so not agents in their own rights, so the utility is based on user feedback about the system. So ideally the system would be an extension of the feedback systems within humans.

There is also karma which is separate from utility which is given by one ml system to another, if it is helped it out or hindered it in a non-economic fashion.

Does anyone have recommendations for people I should be following for structural AI risk discussion and possible implications of post-current deep learning AGI systems.