Introduction

I will argue in the next few paragraphs that AI capabilities progress can continue at the current pace, and that it may in fact increase. I’ll address some natural bottlenecks that are expected to slow this down, as well as some policy ideas to slow the rate of progress deliberately. The main factors determining the rate of advancement in capabilities research that I will discuss are:

• Compute
• Training data
• Talent
• Funding

Compute

• It’s useful to think about  “compute clusters” rather than individual processors – with the exception of some specialist chips, the processors used to run large models are mainly GPUs, which are also used for high-fidelity graphics rendering. 
• By contrast, “clusters” of these processors are far more uniquely applied to developing powerful AI.

• Increasing compute has arguably been the largest driving factor in AI capabilities in the last decade, with the largest improvements in models coming from increased parameter counts enabled by the extra processing power. 
• Scaling laws which predict the relationship between compute and performance were theorised a few years ago. With some refinement, they have been shown to be broadly accurate. 

• The continuation of compute as the most important constraint in AI capabilities is debatable. Its historical importance could be redefined as an abundance of other key resources which hold more supply risk in the future. 
• For now though, it’s likely that bigger, better compute clusters will remain a vital component in developing large models.

• Availability of compute does not currently appear to be a major barrier to capabilities research. Though the demand for these chips is increasing and semiconductor shortages have recently created some supply issues, the industry for compute is well established and highly profitable. 
• There are many incentives for further developments in this technology, demonstrated by recent releases of new specialised chips for ML.

• Regulation can’t easily limit the availability of individual chips, as they have uses beyond ML research. However, processors could be designed in the future to contain safeguards to prevent their unauthorised use within clusters. 
• A register of the main buyers of processors could also be maintained – the high monopoly concentration in the industry makes this fairly easy to do.

Training data

• Training data is important for the development of unsupervised and self-supervised models. It serves as the basis for how these models learn and high quality data is necessary to produce high quality output.
• The availability of high quality training data is running out. It’s expected that high quality language data could be exhausted by 2024, and it’s very difficult to generate more of this data at the scale of the entire internet.
• This makes data likely to be a more significant constraint than compute in the near-future.
• There are ways to train models that can mitigate for, or entirely bypass this issue:
  • Self-supervised learning allows agents to use existing datasets to get more data. For example, an autoregressive transformer like GPT-4 that predicts the next word in a sentence can use each word in the dataset as its own datapoint to learn from, meaning it can get much more training data compared to a model that takes each sentence as a single datapoint.
  • Reinforcement learning doesn’t have the same reliance on pre-existing large datasets, with models effectively being able to generate their own arbitrarily large dataset at will by interacting with their environment.
• It is also expected that improvements in data efficiency will mean that less data is needed for powerful models over time.
• It is hard to restrict access to the largest datasets, as they are often derived from public places such as the internet.
• Depending on the application, it is easier to limit access to other datasets, for example sets of human genomes.
• Enforcing copyright legislation on the use of subsets of large datasets such as the internet could significantly lower the profitability of any model relying on copyrighted content for training that has been obtained without permission.

Talent

• Talent has been cited as the fundamental most important resource in a 2023 CSET paper surveying ML researchers.
• The amount of this talent has been increasing as more educational programmes exist and the field matures. However, safety research currently suffers from a low number of senior mentorship figures, making it harder for the field to grow.
• There is also far more availability of jobs in capabilities than safety, meaning that the pace of the research is likely to continue increasing into the future.
• Regulation to mitigate this could include accreditation schemes, requiring any researchers doing work with sufficiently advanced models to demonstrate an understanding of safety and alignment issues. This would work similarly to other professions, such as engineering and accountancy.

Funding

• With the recent release of GPT-4, there is a lot of funding and investment currently going into capabilities research, especially applications of LLMs.
• As such, it’s unlikely to be a bottleneck for this research in the future.
• Historically the funding for safety research has been much less available, though this has also significantly increased recently. However, the funding for safety research hasn’t scaled with the investment into capabilities.
• There is some precedent from other industries for regulation that limits the profitability of any applications of AI, however there are potential enforcement issues without international cooperation on the matter.
• A more successful strategy may be to place restrictions on firms’ R&D budgets, requiring a significant percentage to go into safety work.

Conclusion

• Training data seems to be the most immediate bottleneck to progress in AI, but is likely to be overcome by improvements in data efficiency of algorithms and an increased focus on techniques such as reinforcement learning, where arbitrarily large datasets can be generated as necessary.
• Compute will continue to be an important constraint, but returns on increasing model size will be limited by the availability of training data. Reducing the availability of compute is a promising lever for any future policies to delay the development of AI.
• Though historically the field has been somewhat bottlenecked on talent, increasing awareness and educational opportunities mean this is unlikely to be a major constraint moving forwards. As AIs become more intelligent they will also be able to contribute more to capabilities research. It will be important to find ways to improve the ratio of jobs in safety vs capabilities.
• Funding is not a constraint on capabilities research, but limiting profitability of ML companies is a fairly well understood policy intervention that has had some success in other industries.