Disclaimer: this is not a comprehensive guide, just notes of the answers I gave in recent conversations I have had. It was suggested it could be useful to share. This is very UK focused.
My notes are available in this Google Doc and I may continue to update the Google Doc version with future conversations or updates.
1. Policy careers in the UK
UK POLICY JOBS
There are lots of important causes that are impacted by UK policy, ranging from climate change to animal welfare to AI risk.
Policy jobs can roughly be split into three groups:
- Civil service
- Think tanks, lobbying, etc
Worth noting that the civil service is politically neutral, politics basically requires you to be party affiliated, think tanks can be neutral or partisan. I expect (not seen it happen) that it would be difficult to move a career form senior in politics to senior in the civil service
For more details see below.
General job hunting tips apply. Keep you eyes on EA sources like the 80K job board and various Facebook groups (here and maybe also here and here). Look at job descriptions early to see what qualifications you would need. Etc Etc.
You should probably go for the area which matches your skills best, although of course there are a vast range of jobs in each area. If I had to push one route it would be towards jobs in politics as being the most high impact, but maybe also the hardest to do really well in.
Most people I have encountered in this space focus on their career building on skill expertise like politics (focusing on one political party) or policy making (eg civil service) but a few focus on topic expertise (like selling yourself as the expert on AI) and that seems to work too.
JOBS IN POLITICS
See the guide at: https://80000hours.org/2016/01/10-steps-to-a-job-in-politics/
To apply for many of the political jobs, go to: http://www.w4mpjobs.org/
JOBS IN THE CIVIL SERVICE
Best guide for EA folk on this is: https://80000hours.org/career-reviews/policy-oriented-civil-service-uk/
It is much better than this guide so go read it
To apply for any jobs in the civil service, go to: https://www.civilservicejobs.service.gov.uk, or apply for the FastStream / a civil service graduate scheme.
If looking for jobs you can use the HIPE app to filter for jobs by cause area: https://goo.gl/WoCnWT
It is useful to know that for jobs in the civil service grades from low to high are normally AA, AO, EO, HEO, SEO, G7, G6, Deputy Director, Director, Director General, Perm Sec.
- HEO (or EO) is graduate entry level
- SEO is with a bit of outside experience
- G7 and G6 are middle management
- Deputy Director and above is senior civil service
You might want to aim for jobs in the area you are interested in but it is often better to go for the best jobs for future promotion (especially if you are joining at a junior level and plan on staying in government for a while). This means
- Jobs that match your skills
- Higher grade jobs (better to fail more interviews for a higher grade job)
- Ministerial private office jobs
- Policy jobs (easier to move from policy to analyst rather than vicer versa)
- Prestigious departments (Cabinet Office Treasury)
- Politically important topics
- Project or strategy roles (in policy) so you can about or oversee a bunch of areas.
You can often find an email address on the advert and ask to talk to the hiring manager (my favourite question to ask is “what challenges do you face with finding the right person for the role”) or even better to understand the work, ask to talk to the current job holder.
To get useful experience before joining you want anything that experience that helps you answer questions of the form "tell me one time you have [relevant behavior]". See list of "behaviors” for job interview at: https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/717275/CS_Behaviours_2018.pdf
Policy experience is good. Otherwise anything that gives you experience of managing complex situations, groups of people, difficult projects, etc. If possible, management experience is great.
Once you are in the civil service it is easy to move around and more jobs will be available to you. To read more on how to have an impact in the civil service see the guides going up soon on www.hipe.org.uk
JOBS INFLUENCING FROM THE OUTSIDE
This could be working in think tanks or campaign groups or in industry.
Jobs could include:
- Political lobbying (influencing politicians)
- Government lobbying (influencing and working with civil servants)
- Campaigning (building coalitions, gaining public support, running a public campaign)
The most prestigious jobs are in the most prestigious organisations. A somewhat dated list of well-known think tanks can be found at: https://www.theguardian.com/politics/2013/sep/30/list-thinktanks-uk
There will also be policy jobs in famous organisations (like Google, etc)
EA topic aligned think tanks include, but are not limited too:
- Existential risk policy: Leverhulme Centre for the Future of Intelligence (CFI) Centre for the Study of Existential Risk (CSER), Future of Humanity Institute (FHI) and ALLFED
- Poverty policy: Results UK
- Animal welfare policy: The Good Food Institute, Cellular Agriculture UK, Compassion in World Farming and A-law
- Improving policy making: APPG for Future Generations and Institute for Government
- Arms control: BASIC
2. Summary policy areas and career options in the UK civil service – June 2019
INTERNATIONAL DEVELOPMENT POLICY IN THE UK
Generally quite good and well done in the UK.
DFID (Department for International Development) is a world leader in delivering international development policy.
A significant chunk of ODA (Overseas Development Assistance) budget is spent by other Government departments (such as the FCO or the DHCS) rather than DFID They do not spend these funds as well and may neglect effectiveness considerations or have intentions that override the interest of the people the money should be helping.
International development spending is overseen by ICAI (Independent Commission for Aid Impact) who work to ensure it is effective. (The Treasury international development team may play a similar role too)
ANIMAL WELFARE POLICY IN THE UK
The government has committed to a range of minor improvements in animal welfare, for example recognising animal sentience, improving labeling of fur, ending live transport of animals, etc. This work is seemingly all on hold largely because of Brexit.
Career considerations – animal welfare
Mapping of jobs in the UK civil service that relate to Animal Welfare.
Best place to work is likely in DEFRA (Department of the Environment Food and Rural Affairs (or in a relevant job in a central department like Number 10 or Treasury).
In DEFRA the best teams would be the ones on farming: eg future farming team, (DEFRA biosecurity teams less so as that is more about whether to kill lots of animals if a disease breaks out).
There maybe interesting jobs at the minute in trade (Department for International Trade) given Brexit.
There might be issues other than factory farming worth focusing on. For example where there is a scientific consensus which differs from the current policy (maybe in animal testing requirements) or where the impact of policy change towards the stated policy aims has been poor (like stopping antibiotic use on farms)
AI POLICY IN THE UK
The UK Government is acting to support the AI industry though an Office for Artificial Intelligence, an AI Council and an AI Sector Deal. The UK is managing the implication of new technology through and a Centre for Data Ethics and Innovation (CEDI).
The Office for AI is business focused and will mostly be implementing the priorities mentioned in the UK’s 2017 Industrial Strategy: making sure the workforce has machine learning skills, growing the AI sector and supporting businesses to use AI . One aspect of this work is the AI Sector Deal  which is a billion-pound initiative between government and industry which launched in 2018.
The Office for AI will also act as a secretariat to an AI Council . This AI Council, launched in 2019, has 21 senior members and intends to drive AI adoption, data & ethics, AI skills and diversity through closer engagement with academia and industry. It was Initially a recommendation of the Independent 2017 report on Growing the AI industry 
Centre for Data Ethics and Innovation (CDEI) could perhaps be described as the body that would be a regulator for AI and data use if anyone knew what regulation was needed . Further down the line CDEI could plausibly could become a regulator, alternatively CDEI might support the Information Commissioner's Office (ICO, the information rights and data privacy regulator), to be the body that regulates future technology.
Beyond this the UK Government is encouraging AI adoption in specific areas of policy including defence [6,7] transport  (with the Centre for Connected and Autonomous Vehicles) and healthcare  and supporting AI research through support of the Turning Institute. Recently the Competition and Markets Authority (CMA) have introduced digital markets strategy  reflecting an intent to examine the tech Sector more.
[Edit 2020: Note the new Digital markets taskforce. See: https://www.gov.uk/government/publications/digital-markets-taskforce-terms-of-reference/digital-markets-taskforce-terms-of-reference--3 ]
Career considerations – AI policy
The Office for AI and CDEI are being run by of BEIS (Department for Business Energy and Industrial Strategy) and DCMS (Department for Digital Culture Media and Sport). Jobs will likely show up by looking at jobs in these Departments.
For EAs worried about AI risk and AI regulation and looking to have an impact the CDEI is the better place to be, as more relevant to work in that area.
However for general career capital building in the area of AI policy it's less obvious. At this stage I expect the recruitment will not be for specific roles (just general recruitment) so it is hard to know what roles applicants would get. Overall if the applicant had no experience of working in AI probably Office for AI would be better as more chance of building relevant career capital. That said, the most central gov, Treasury, CO & No10, could be better career capital still.
There will also be corresponding jobs interacting with these organisations from central gov and other bodies including in DCMS, BEIS and NSCS.
- The team in BEIS working on setting this up is fully staffed at present.
- I'm not sure about the DCMS team.
- I believe the relationship between NSCS and these bodies is somewhat nonexistent, but possibly working in policy on that relationship (in ether body) would be a very good place to be in terms of AI risk issues and AI policy career capital.
- There is probably also a Treasury person who is dealing with the funding for this but I expect it is a small part of one job and not an area that can have much say.
RISK POLICY IN THE UK
Long term risks in government: Departments own specific risks. (E.g. Health and epidemic risks. BEIS and civil nuclear disaster. Home Office own a lot of risks.) This is coordinated by the Cabinet Office, who produce:
- National Security Risk Assessment (NSRA), of external risks. Looks over 20 years  (although the focus, the tiering, is on the next 5 years). Was last carried out in 2015 . Did not seem to recognise any emerging risks.
- National Risk Assessment (NRA). Looks over 5 years. Not made public.  
- National Risk Register (NRR) of civil emergencies. The public version of the National Risk Assessment. Published every 1-2 years. Last version was 2017. 
Government risk assessments generally
- do not take into account emerging risks (risks that may happen but are not currently likely or not likely in the next few years).
- are not good at picking up on cross-cutting issues, however they may be getting better at that and I believe Cabinet Office are redoing process at the min.
- are decent at categorising endemic and point risks and recognising small risks.
AI risks are not recognised in Cabinet Office risk analysis (that I have seen). Some recognition of AI risks has already taken place within the UK. The House of Lord’s “AI in the UK: Ready, Willing, and Able?” endorses the Centre for the Study of Existential Risk’s testimony on cyber-security, dual use technology, and the possible AI arms race – but stops short from recognising the risks stemming from superintelligence. 
Cyber security support is provided by the National Cyber Security Centre (NCSC).
Cyber security of weapons is the responsibility of [probably MoD]
 p39, Industrial Strategy, HM Government, https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/664563/industrial-strategy-white-paper-web-ready-version.pdf
 Independent report on Growing the artificial intelligence industry in the UK, https://www.gov.uk/government/publications/growing-the-artificial-intelligence-industry-in-the-uk
 Centre for Data Ethics and Innovation Consultation, https://www.gov.uk/government/consultations/consultation-on-the-centre-for-data-ethics-and-innovation/centre-for-data-ethics-and-innovation-consultation
 House of Lords, (2018). AI in the UK: Ready, willing and able? Report of Session 2017-19. [online] Available at: https://publications.parliament.uk/pa/ld201719/ldselect/ldai/100/100.pdf [Accessed 15 Jul. 2018].