Research AI proposes large numbers of candidates, and external laboratories make and measure them. Such a division of labour seems likely to accelerate research. But more candidates and more measurements are not the same as faster development decisions.
In the antibody-design benchmark discussed in this issue, performance differed between improving sequences and selecting candidates. Candidates that passed the aggregate developability criterion could still have concerns on individual measures. Treating generation, selection and development judgement as a single capability makes it harder to see what AI has actually improved. Antibody-design benchmark ↗
Meanwhile, access to experiments is expanding. Ginkgo Bioworks announced that it would provide autonomous laboratory time and protein testing services to support the US National Science Foundation's X-Labs initiative. This creates more routes from an idea to an experiment without researchers having to own every piece of equipment themselves. Ginkgo announcement ↗
The question this raises is how the position of the AI buyer, not just the AI company, might change.
If proposals from different models can be compared under the same experimental conditions, users can measure performance on their own problems rather than relying solely on company claims or model rankings. Separating candidate generation from evaluation could also make it easier for users to change AI providers. Access to a laboratory does not by itself make that comparison possible, however. The measurement conditions and the definition of success must be aligned.
In that setting, the relevant measure of value would be the total cost and time needed to reach a candidate that meets the requirements, rather than the number of sequences proposed.
Eliminating unusable candidates is not enough, either. Set the criteria too strictly and promising candidates may be discarded, losing opportunities. Evaluation should include the quality of the judgement: retaining what deserves to advance and identifying what should stop, with fewer experiments.
In an earlier article ↗, we considered research AI companies' moves into experiments and drug assets. From the user's perspective, another important issue is who can continue to hold and use the knowledge accumulated along the way.
Which candidates failed, under what conditions? What changes improved them? Can those records be used on the company's next problem? If the model or experimental service provider changes, can past results still be compared? Contracts and data arrangements need to account for these questions as well.
Research AI businesses offer different things: candidate proposals, experimental execution and outputs ready for further development. There is no need to judge all of them by the same measure. Buyers should, however, be clear about which of their decisions each offering improves.
As AI produces more answers, the criteria used to evaluate them become more valuable. The next stage of competition may involve not just candidate-generating technology, but also the means for users to compare its outputs and decide what to advance.