The international governance of artificial intelligence (AI) is at a crossroads: should it remain fragmented or be centralised? We draw on the history of environment, trade, and security regimes to identify advantages and disadvantages in centralising AI governance. Some considerations, such as efficiency and political power, speak for centralisation. The risk of creating a slow and brittle institution, and the difficulty of pairing deep rules with adequate participation, speak against it. Other considerations depend on the specific design. A centralised body may be able to deter forum shopping and ensure policy coordination. However, forum shopping can be beneficial, and fragmented institutions could self‐organise. In sum, these trade‐offs should inform development of the AI governance architecture, which is only now emerging. We apply the trade‐offs to the case of the potential development of high‐level machine intelligence. We conclude with two recommendations. First, the outcome will depend on the exact design of a central institution. A well‐designed centralised regime covering a set of coherent issues could be beneficial. But locking‐in an inadequate structure may pose a fate worse than fragmentation. Second, fragmentation will likely persist for now. The developing landscape should be monitored to see if it is self‐organising or simply inadequate.