What Should the Global Summit on AI Safety Try to Accomplish?

This post explores possible outcomes of the upcoming UK-hosted global summit on AI safety. It draws on ideas raised in a recent expert workshop hosted by GovAI, but is not intended as a consensus statement.

GovAI research blog posts represent the views of their authors, rather than the views of the organisation.

‍

Summary

Following growing concern about possible extreme risks from AI, the United Kingdom recently announced plans to host the first major global summit on AI safety. The announcement sketched a high-level vision for the summit:

The summit, which will be hosted in the UK this autumn, will consider the risks of AI, including frontier systems, and discuss how they can be mitigated through internationally coordinated action. It will also provide a platform for countries to work together on further developing a shared approach to mitigate these risks.

However, more detailed aims have not yet been shared. GovAI recently convened a small expert workshop – organised without government involvement and held under the Chatham House rule – to explore possible objectives for the summit.

Attendees suggested a wide variety of valuable direct outcomes of the summit. These can could mainly be grouped into six categories:

1. Producing shared commitments and consensus statements from states
2. Planting the seeds for new international institutions
3. Highlighting and diffusing UK AI policy initiatives
4. Securing commitments from AI labs
5. Increasing awareness and understanding of AI risks and governance options
6. Committing participants to annual AI safety summits and further discussions

Several attendees suggested that the summit may be a unique and fleeting opportunity to ensure that global AI governance includes China. China is likely to be excluded from important discussion venues, such as the OECD and G7. China may also have only a temporary period of AI policy “plasticity” and will be more inclined to reject any global governance principles that Western states begin crafting without its input. However, one attendee disagreed that China should be invited, emphasising that Chinese participation could make the summit less productive.

‍

The Summit Could Produce Several Valuable Outcomes

Potential valuable outcomes from the summit include:

1. Producing shared commitments and consensus statements from states. Participating states could sign onto statements that establish a shared understanding of risk, influence global AI governance norms and priorities, and provide guidance to private labs. A baseline goal could be to produce a shared statement acknowledging that extreme safety risks are a global priority.
   
   
2. Planting the seeds for new international institutions. States could commit to creating new international institutions (or at least further discussing possibilities). Possible functions of new institutions include housing research, developing expert consensus and shared standards, providing evaluation and auditing functions, and facilitating the creation of international agreements.
   
   
3. Highlighting and diffusing UK AI policy initiatives. The UK could establish itself as the global leader in AI policy, showcase its new AI policy initiatives, and support their diffusion to other states. Initiatives that could be showcased include an information-sharing and model evaluation regime, plans for an associated licensing regime, and guidance documents to labs outlining responsible practices (e.g. regarding model release and scaling).
   
   
4. Securing commitments from AI labs. Labs could make shared commitments (e.g. to best practices) that they will not by default make without public pressure or government encouragement, due to commercial incentives and legal constraints that limit coordination. This could include commitments to share further information with governments, invest a minimum portion of their resources into safety, temporarily pause development if certain model evaluation results are triggered, or adopt responsible policies regarding model scaling.
   
   
5. Increasing awareness and understanding of AI risks and governance options. Events (e.g. events demonstrating risks), discussions, statements, and media coverage of the summit could increase awareness and understanding of AI safety challenges (particularly AGI safety challenges) and the need for global governance. It is useful for this awareness to be shared across government, industry, civil society, the media, and the broader public.
   
   
6. Committing participants to annual AI safety summits and further discussions. Participants could agree to further meetings to maintain momentum and make progress on issues raised during the summit (e.g. the potential need for new international institutions and agreements). In addition to further iterations of the summit, new expert-led working groups (potentially hosted by the UK or the OECD) could be established. 

‍

The Summit May Be a Critical Opportunity to Ensure Global AI Governance Includes China

A number of participants suggested that this summit may be the only opportunity to include China in productive global AI governance discussions. Other important venues – including the OECD, the G7, or meetings initiated by the US – will likely exclude China by default. More inclusive institutions such as the UN may also struggle to make forward progress due to structural constraints.

This opportunity may be critical, since global AI governance will likely fail if it does not ultimately include China. If China – the world’s third major AI power – does not adopt responsible AI safety policies, then this could threaten the security of all other states. External constraints, such as export controls on advanced chips and relevant manufacturing equipment, will probably only delay China’s ability to create high-risk models.

Summit outcomes (such as those listed above) could therefore be more valuable if they include China. One participant whose research focuses on Chinese AI policy argued that China can be positively influenced by the summit.¹ There is interest in AI safety in China – for instance, a number of prominent Chinese academics signed the recent CAIS statement on AI risk – and Chinese actors have been signalling an interest in participating in global AI governance efforts.

Trying to involve China later – for instance, in future iterations of the summit – may also be much less impactful than involving China now. China is more likely to adopt global frameworks if it can legitimately view itself as a co-creator of these frameworks; this will be less likely if China is not invited from the start. As China develops its own regulatory frameworks for advanced AI and makes its own independent policy statements, Chinese AI policy may also quickly lose some of the “plasticity” it has now

However, within the group, there was not universal consensus that China should be invited to the summit. One potential cost of inviting China is that it may reduce the enthusiasm of other states and non-governmental actors to participate in the summit. Inviting China may also make the summit less productive by increasing the level of disagreement and potential for discord among participants. There may also be some important discussion topics that would not be as freely explored with Chinese representatives in the room.

These concerns are partly mitigated by the likely existence of side meetings – and discussions in venues beyond the summit – which could include smaller subsets of participants. However, involving China in the summit would inevitably involve trade-offs. These trade-offs could be evaluated in part by seeking the views of other likely participants.

‍

Conclusion

The first global summit on AI safety is an important opportunity to make progress toward managing global risks from advanced AI systems. It could directly produce several outcomes, including: (1) shared commitments and consensus statements from states, (2) the seeds of new international institutions, (3) the advertisement and diffusion of UK AI policy initiatives, (4) shared commitments from labs, (5) increased understanding of AI risks and governance options, and (6) commitments to further productive discussions.

It may also be a unique and fleeting opportunity to involve China in global AI governance. Ultimately, however, making the correct decision regarding China's participation will require weighing the value of Chinese participation with the frictions and stakeholder management challenges issues it may bring.

‍

The authors of this piece can be contacted at ben.garfinkel@goverance.ai and lennart.heim@governance.ai.