We’d like to pace the frontier by restricting the amount of compute AI companies use to improve AI capabilities, perhaps via legislation or voluntary commitments. For this purpose, we’d need a standardized definition of which uses of compute count as “capabilities activities”. This document gives one option for such a definition; a rough breakdown of our classification is as follows:

We define and elaborate on each of these categories below. First, though, we discuss the desiderata this definition is meant to satisfy.
Thanks to Thomas Larsen, Zach Stein-Perlman, Alex Mallen, Eric Gan, Jeff Wu, Girish Gupta, Adam Kaufman, and others for feedback on drafts of this document.
1 What our classification is meant to accomplish
In order for such a definition to be an effective tool for pacing, it needs to satisfy some basic properties:
-
Substantially restricting the amount of compute spent on “capabilities activities” should substantially decrease the rate of AI progress.
-
The classification should be robust to adversarial pressure, in that the above point should hold even if companies are willing to adjust or reframe their activities to avoid being classified as “capabilities”.
-
The classification should be fairly simple and tractable to adjudicate. Reasonable experts should largely agree on how this definition classifies activities (even if it is difficult to agree on the extent to which individual activities are actually useful for capabilities).
There is a simple definition that satisfies all of the above desiderata: one could classify all work by AI companies as “capabilities activities”. The goal of this document is to propose a more scoped notion, allowing many types of work to be excluded from the “capabilities” total, while hopefully remaining pretty robust. We believe that it’s potentially worth having our more complex definition, rather than a maximally simple definition, because (1) it allows AI companies to use their remaining compute to improve safety, and (2) external access spreads power and information about model capabilities. If this definition is challenging to adjudicate, then cutting “narrow safety activities” and/or “narrow product development” from being included as non-capabilities activities could make it easier to adjudicate.
We stress that our definition is not intended as a perfect metric for whether an activity will genuinely end up improving capabilities: the boundaries are sufficiently fuzzy that such a classification is not feasible. Instead, we adopt the following philosophy:
-
False-positives are better than false-negatives. (I.e., it is much more acceptable to classify something as “capabilities” that doesn’t accelerate AI progress, than to classify something as “non-capabilities” that does.)
-
Computationally non-intensive activities are negligible. (I.e., it is not very important how this definition classifies an activity if that activity only represents a very small amount of compute.)
We are imagining a regime where AI companies are allowed to invest in “capabilities activities”, we just intentionally restrict how much compute they may spend on them in aggregate. So, it would be mostly fine if some portion of that restricted budget had to be spent on things that aren’t the most central forms of capabilities work, and we’re not hoping this will be a meaningful constraint on non-compute-intensive activities.
We also stress that, while we believe our below definition to satisfy the desiderata reasonably well, in order to get something that works in practice, it will be important to have third-party auditors to ensure that the definition is being reasonably interpreted, and it will also be important to have procedures to adjust the definition as needed. (Adjusting the definition might be needed if companies end up gaming the definition in various ways, or if there are new activities that seem important to scope as “non-capabilities”, or if AI R&D ends up looking significantly different in the future and many things become unclear.)
We now give our definition. We start by defining non-capabilities activities. Later in the document, we give examples to clarify what capabilities activities are (although they are defined as “everything other than non-capabilities activities”).
2 Non-capabilities activities
Activities are considered capabilities activities by default. The only exceptions are idle compute, narrow product development, serving customers, and narrow safety activities. These exceptions apply only if the activity doesn’t improve general model capabilities. These terms are vague; we’ll define them in more detail below.
Idle compute
Compute that isn’t performing any computations. Note: the whole GPU has to be idle—an underutilized core doesn’t count as an idle GPU. It should also be idle for a non-negligible amount of time.
Narrow product development
Taking the general purpose AI systems that companies have made and turning these into products counts as a non-capabilities activity, assuming that this doesn’t make general AI systems more capable or enable faster capabilities progress. Work on separate narrow models also counts as a non-capabilities activity, again, assuming this doesn’t make general AI systems more capable or enable faster capabilities progress (e.g., by finding generalizable algorithmic improvements).
More specifically, the activities that count as narrow product development are using AI labor to develop products or narrow models, and running R&D experiments for developing the product or narrow model. Here are some examples of narrow products and narrow models AI companies might develop:
-
Integrating AI into Google Docs.
-
Integrating AI into glasses.
-
Training protein-folding models.
-
Recommendation algorithms.
-
Algorithms designed to increase user engagement with social media.
-
Making platforms like TikTok that serve AI-generated video content.
-
Using AI to generate ads or choose ads to show people.
Serving customers
Most forms of serving inference to external customers are non-capabilities activities. The main exceptions are:
-
If the AI company outsources capabilities-relevant tasks to customers.
-
If the AI company trains on its interactions with customers.
-
If the customers themselves use models to help with AI R&D tasks.
Please see capabilities activities for further discussion of this.
Serving inference to company employees should generally be considered a capabilities activity. However, inference for internal company employees who aren’t scientists and aren’t working on AI R&D (namely: company lawyers, marketing teams, HR teams, and communications teams) is considered a non-capabilities activity. (We’ll also rule in some inference for safety researchers in the section on narrow safety activities.)
Letting customers fine-tune models in narrow ways (e.g., fine-tuning for use in a hospital) is also a non-capabilities activity.
Narrow safety activities
Some forms of safety work seem distinct enough from capabilities work that we consider them non-capabilities activities. We define narrow safety activities to be any of the following: (1) evaluating model alignment, (2) testing and improving company security, (3) research that could improve safety and isn’t too dual-use, (4) monitoring, (5) running misuse classifiers, or (6) incident investigation.
Not all safety activities count as narrow safety activities. Some safety activities are dual-use or too hard to separate from capabilities activities. It might be valuable to do some safety activities which aren’t narrow safety activities. However, this classification counts such activities as capabilities activities.
While we’ve tried to draw a principled line between narrow safety activities and capabilities, the line is less sharp than would be ideal. We think this is a reasonable tradeoff for a more permissive notion of non-capabilities activities.
We now give an exhaustive list of what counts as a narrow safety activity.
-
Evaluating model alignment. Alignment evaluations aim to determine whether a model would exhibit some behavior, as opposed to whether it could solve some task. However, if the alignment evaluation involves demonstrating AI R&D capabilities, then it is still considered a capabilities evaluation (it’s fine if it just involves demonstrating general coding or reasoning abilities). (If alignment evaluations and capabilities evaluations prove hard to distinguish in practice, we could require companies to run the evaluations only once per month, or to have a third-party auditor run them; however, we don’t expect this to be necessary.)
-
Testing and improving company security (e.g., patching vulnerabilities in the company’s infrastructure, generic security engineering). This is somewhat hard to distinguish from a cyber offense evaluation and from patching broken RL environments. If companies are iterating against the results of “pen tests” to make models more capable, then that counts as a capabilities activity. For instance, if a company made a cyber capabilities evaluation and then just called it company security, it would still be considered a capabilities activity. One sign that something is an evaluation is that the company is trying multiple configurations of its AI (e.g., different training algorithms), seeing which are good, and then developing those. We also note that this category applies only to security work for company-internal infrastructure: our expectation is that external security work (e.g., patching vulnerabilities in the linux kernel) should be delegated to external parties. It is acceptable for the AI company to grant these parties subsidised inference or access to more powerful internal models, but the company shouldn’t strongly direct their activities, or use their results as evaluation signals.
-
Research that could improve safety and isn’t too dual-use. We specify a whitelist of a few areas of safety research that count under this category: interpretability research, CoT informativeness research, model persona research, model organisms research, and theoretical work on alignment without near-term applications to improving capabilities; we explain what each means below. The compute used for having AI systems code up experiments, and also the compute used for running the experiments (including training experiments) all counts as part of the safety research activity here. Because this category is particularly concerning, researchers must be transparent about the research and its findings for it to count as a narrow safety activity (more transparency is helpful throughout this proposal, but it’s especially important here because much of this research has high potential to be dual-use). Specifically, at the beginning of a project, the company must publicly explain the project plan, and at the end of the project, it must write a public report explaining its results in detail. Subject to these caveats, narrow safety research consists of the activities listed in subbullets below, and only those following activities. It might be surprising that the whitelist is so short. There are two reasons for this: (1) Some forms of safety research, like scalable oversight, are extremely similar to capabilities research; see here for a list of safety activities that are too dual-use for inclusion in the narrow safety work category. (2) There aren’t very many promising areas of safety research overall.
-
Interpretability research. Specifically: Developing post hoc tools to understand models (probes, Jspace, logitlens, NLAs, activation oracles, SAEs), and modifying training and architectures to make models more interpretable (e.g., gradient routing, sparsification), so long as these don’t also have clear applications to improving capabilities.
-
CoT informativeness research.
-
Model persona research. This means studying how training shapes the character a model adopts (its values, self-conception, and broad dispositions) and developing techniques that reliably instill desired personas. Examples include “Teaching Claude why,” inoculation prompting, model spec midtraining, and emergent misalignment. It counts as a narrow safety activity only if the intended effect is on what the model would do rather than what it could do.
-
Model organisms research that doesn’t involve investigating new algorithmic ideas. This means deliberately modifying a training recipe in a way targeted at causing the AI to be misaligned in some way, but which could not plausibly increase capabilities or enable increases to capabilities in other models. After creating the model organism, it’s permissible to run alignment evaluations on it, or to test interpretability tools on it. Examples of model organism research that count as narrow safety work include: sleeper agents, alignment faking, sandbagging, or sabotage. An example of something that wouldn’t count as narrow research would be studying the monitorability properties of a novel architecture (like “neuralese”), as the process of training such models would likely require some experimental new techniques and thus have potential to yield capabilities insights. To be clear: if model organisms research involves testing novel algorithmic ideas (even at a small scale), it instead counts as a capabilities activity.
-
Theoretical work on alignment that doesn’t have near-term applications to improving model capabilities. This includes work on Agent Foundations (e.g., MIRI’s work) and ARC’s work.
-
-
Monitoring. Running monitors to detect bad behavior is a safety activity. Monitors are a specific way of looking for bad behavior, distinct from, for instance, capabilities evaluations and reward models. Monitors differ from reward models because reward models directly provide training signal to AI systems, whereas monitors are not directly optimized against. Monitors shouldn’t directly be part of a loop where developers try several tweaks to a training recipe, run “monitors” on them, and use these monitors to select which tweaks are good (including in small-scale experiments). Otherwise, we wouldn’t count this as a “monitor”, and we’d consider the activity a capabilities activity. Monitors are instead used to potentially stop egregiously bad model behavior, and at least to make it visible (which might later help address the problem at its root).
-
Running misuse classifiers. Running classifiers to check if users are trying to use LLMs to cause harm (e.g., create bioweapons). Running classifiers for jailbreaks is also permissible. The caveats from the section on monitoring apply here too (using misuse classifiers in practice as reward models or capabilities evaluations would not be permissible).
-
Searching for and investigating misalignment incidents. Searching for egregious misalignment incidents (e.g., of the type disclosed here) counts as a non-capabilities activity. Investigating such misalignment incidents (e.g., as in the Hugging Face incident investigation) is also a non-capabilities activity. This activity is quite similar to monitoring, and the same caveats apply.
3 Capabilities activities
As stated, AI company activities are considered capabilities activities by default, unless they fall into any of the categories in the previous section. This category is intentionally broad: it includes, for example, safety research with secondary applications to capabilities, or any other activities that one can’t make a strong positive case for placing in the other categories. Many such activities seem good for AI companies to do, but they are far enough from clear-cut that we want them to count towards this category.
To clarify what falls under this category, we give several examples.
Central capabilities activities
Many workloads are clearly and intentionally geared toward improving production model capabilities or doing R&D for future such improvements. For instance:
-
New production pretraining runs.
-
New production post-training runs.
-
Distillation.
-
Building and running capabilities evaluations and benchmarks.
-
Improving kernels or optimizing inference.
-
AI R&D experiments, except those that deal with “safety” or “narrow product development”.
-
Designing scaffolds, prompts, or skills that could make AI models meaningfully better at tasks related to coding or AI R&D (e.g., working on scaffolding for the Claude Tag in Slack).
-
Designing hardware accelerators.
-
Obtaining, filtering, grading, or creating data to use in AI R&D; for instance, designing RL environments or capabilities evaluations.
-
Using AI labor to grade the quality of AI actions, if the grades serve as a signal to improve the AI.
Safety work that might improve capabilities
Some safety work could have applications to capabilities, or is hard to distinguish from capabilities-driven work. To keep the definition conservative and hard to game, we classify this work as capabilities. This includes:
-
Work on debate or “scalable oversight”, including efforts to make better, smarter, or more robust reward models.
-
Work on conceptual reasoning capabilities, or making models better at solving fuzzily specified tasks.
-
Running control or monitor evaluations. (These are a type of capabilities evaluation.)
-
Running dangerous capabilities evaluations. (These are a type of capabilities evaluation.)
-
Running “constitutional AI” / RLAIF.
-
Training monitors. (As noted, running monitors is a non-capabilities activity, but work on improving them seems quite similar to work on improving other model capabilities.)
-
Training misuse classifiers. (Running these classifiers is a non-capabilities activity, but work on improving them seems quite similar to work on improving other model capabilities.)
-
Training for jailbreak robustness.
-
Improving RL environments, including fixing misspecification issues / ways that the environment was “hackable” / patching broken RL environments.
This table lists a few narrow safety activities that are particularly hard to distinguish from capabilities activities:

We’d like to reiterate some guidance for distinguishing these categories.
Cyber capabilities evaluations are used directly as part of an optimization loop where the AI company tweaks training recipes (in large- or small-scale experiments), measures which tweaks work best, and selects for those tweaks. In contrast, monitoring, searching for misalignment, and searching for (and fixing) company security problems are not directly part of such an optimization loop. Reward models are even more directly used to decide which modifications to an AI system make it more intelligent.
Improving the quality of our oversight signal, e.g., by addressing reward misspecification problems, is distinct from searching for misalignment failures, because the improved reward signal directly feeds into making a more powerful AI system, whereas monitoring for bad behavior is used to block bad actions. Companies must not iterate directly against monitoring, or else the monitoring is considered a capabilities activity.
Product work that might improve capabilities
Some work on consumer product development might also improve model capabilities, directly or indirectly, in ways potentially relevant for coding or AI R&D (e.g., because the improvements might generalize to broader intelligence, or because they might yield generally-applicable algorithmic ideas). Such work should be classified as capabilities work. Examples include:
-
Building products that require training general-purpose models to have improved capabilities in some domain. (For instance, a model trained to be particularly good at math research, or at crafting good legal arguments.)
-
Building products that involve scaffolding models to be better at coding or research.
-
Robotics.
-
Work on voice mode, video models and image models (training them, and doing R&D).
Use of customer inference for AI R&D
Although serving inference (or otherwise selling computing resources) to customers is generally considered a non-capabilities activity, there are three main exceptions.
-
Outsourcing capabilities activities to customers. That is, any cases where the AI company is intentionally directing the activities of outside third-party customers to perform tasks that might help it improve capabilities. Thomas’ attacks supplement (to be released soon) discusses in more detail how to handle these cases.
-
Training on customer data. If the AI company trains on customer data, or iterates against it (e.g., uses it in an evaluation loop that improves model capabilities, e.g., by using it as a capabilities evaluation), then the inference used to generate this data counts as a capabilities activity.
-
Customers performing central capabilities activities for any purpose. (E.g., making RL environments, even if these are not directly sold back to the AI company.) The AI company is expected to use a robust classifier to identify when customers are performing such tasks, and then either refuse to serve these customers, or count the inference as a capabilities activity. For ZDR workloads, the company would need to either refuse to serve the customer based on the classifier result, or presumptively count the inference as a capabilities activity.
Unknown or ambiguous activities
If an activity doesn’t clearly fall into any of the non-capabilities categories described, it is presumptively considered a capabilities activity. If this becomes a problem, a company could describe the workload at a high level, and the definition could be adjusted.
4 Conclusion
Pacing the frontier is urgent. It is therefore also urgent (as discussed here) to have good publicly verified measures of how much compute is being spent on pushing forward the frontier.
Anthropic has already published an estimate of how much of their AI R&D is safety research, and has called for it to be a common practice that third parties measure and publicly report this number. While Anthropic’s publication is not transparent about how they defined “safety research”, and while they didn’t have auditors verify their number yet, they’ve taken a step in the right direction.
We urge AI developers to let third-party auditors measure and report how much compute AI companies are allocating to capabilities and non-capabilities activities, using a definition of these activities broadly similar to (or perhaps identical to) the one given in this document.
Cite this post
@misc{westover2026how,
title = {{How should we define “capabilities activities” for the purpose of pacing?}},
author = {Alek Westover and Nathan Sheffield},
year = {2026},
month = nov,
howpublished = {Redwood Research blog},
url = {https://www.redwoodresearch.org/blog/how-should-we-define-capabilities}
}Also published on Substack.