The appeal of AI in science is obvious: better tools may help us find answers to problems that have resisted us for decades. A new treatment, a more efficient material, or a proof of a difficult theorem would all be meaningful achievements.

But an answer is not the only thing produced by discovery. Along the way, people invent concepts, notation, methods, and questions. Those intermediate ideas become shared infrastructure. They let the next person see further than the first.

Terence Tao makes a version of this point in his writing on AI and mathematics. The concern is not that machines should be forbidden from solving hard problems. It is that a system which returns a result without exposing a path may leave the field with less to build on.

Software offers a useful analogy. Collaborative tools did not emerge all at once. Work on editing and merging documents led to ideas about versions, diffs, merges, and eventually systems such as Git, Google Docs, and Figma. The intermediate abstractions were not incidental; they made later tools possible.

A black-box system can create a different failure mode. It can give us a checked-off result while concealing the conceptual work that would help us generalize it, challenge it, or use it to formulate the next problem. In fields built around insight—pure mathematics is a clear example—that loss could be substantial.

This does not mean outcomes are unimportant. An effective treatment for Alzheimer’s disease, carbon capture that works at scale, or a room-temperature semiconductor would be too valuable to postpone in the name of preserving a particular human process. In applied science, results can be the highest priority.

The better question is contextual: is the answer the main product, or are the concepts created on the way there also part of the product?

For some problems, we should gladly accept solution-finding systems. For others, we should ask our tools to make their work inspectable: show the hypotheses considered, the abstractions discovered, the failures that ruled out a path, and the uncertainty that remains. The goal is not merely to extract solutions. It is to keep expanding what people can understand and discover together.