I recently came across a post about an interactive display at a major natural history museum that had been found to contain obvious errors. The directional labels on an animal’s teeth were reversed, and the illustration of its internal organs did not match the animal’s actual anatomy. From the look of it, the content seemed likely to have been generated with AI.
A museum has a responsibility to communicate science accurately, especially when its audience is the general public and, in many cases, children. Errors like these should not make it into an exhibit. But what concerns me is not only that this particular display was wrong. It is that incidents like this may become much more common.
Not necessarily because people are becoming careless simply because they trust AI too much, but because AI has changed the threshold at which mistakes can enter the real world. In the past, if someone did not fully understand what they were making, it was often difficult for them to produce something that looked convincing. Now they can. Something that has not been properly understood, verified, or truly owned by anyone can still look complete, professional, and ready to deliver.
That is where the problem begins.
Smoothness Is a Risk in Itself
One of the most misleading things about AI is not simply that it can generate content, but that it does so with extraordinary smoothness.
Ask it to draw an anatomical illustration and it can quickly produce something that looks quite professional: clean lines, neat labels, coordinated colors, a convincing layout. For someone without domain expertise, that level of polish can lower their guard.
When we are presented with something that looks complete, clearly structured, and confidently expressed, our first reaction is often not to question it but to accept it. We naturally confuse maturity of form with reliability of content. By contrast, a rough hand-drawn sketch may prompt closer attention precisely because it still carries visible signs of incompleteness. It reminds the viewer that judgment is still required.
AI-generated content is different. It tends to arrive with the appearance of being already finished. It looks less like a draft and more like a final product, less like a suggestion and more like a conclusion. That is often the moment when human judgment begins to loosen.
A museum illustration used to involve something like this: a scientific advisor provides materials, an illustrator interprets them, a sketch is made, an expert reviews it, revisions follow, and only then is the final version approved. The process is slow, but slowness is not just inefficiency. At every stage, someone is understanding the material, making choices, and exercising judgment.
AI compresses that chain. A process that once required multiple people and repeated confirmation can become a prompt, a generated image, and an upload. Efficiency improves, but understanding and verification may be skipped at the same time.
The deeper problem is that AI-generated mistakes often do not look like mistakes. They are wrapped in clean layouts, confident language, and complete structures. By the time they are exposed, they may already be inside the context where people are using them.
Review Alone Cannot Stop This
The most obvious response is to add expert review.
That is necessary. But if review becomes the only solution, we may still be underestimating the problem.
When a piece of work is built from scratch, the people involved usually have some sense of where it might fail and which details require confirmation. They have gone through the process of selecting sources, understanding the material, translating it, and deciding how to represent it. They know where they hesitated and which decisions were important.
A reviewer looking at an AI-generated finished product is in a different position. They did not participate in how it was formed. They do not know which parts came from reliable source material, which were inferred by the model, and which merely look plausible.
Review therefore stops being a matter of checking a few critical points. In many cases, the reviewer has to take the whole thing apart again and verify it piece by piece. That does not necessarily reduce the burden of review. It can increase it.
There is also a more practical scaling problem. When AI multiplies production speed, the volume arriving at the review stage increases as well. What used to be a few illustrations a week may become dozens in a day.
The easiest mistakes to catch are obvious ones: a reversed label, an organ in the wrong position. But many errors are less visible. A statement may be subtly distorted. A classification may be oversimplified. A diagram may be misleading in its proportions. On the surface, everything still looks smooth. Only someone with real expertise may notice that something is off.
So review is necessary, but it cannot exist only at the end. What needs to change is the position AI occupies in the process.
AI Should Not Sit Directly in the Delivery Position
The museum example looks like a content error, but the deeper issue is that AI has been placed in the wrong position.
It has been treated as the deliverer: make a request, generate a result, use the result. The most important middle layer—understanding, judgment, and confirmation—gets compressed into an almost invisible step.
In professional content production, AI is often better placed earlier in the process. It can help with research, generate directions, produce drafts, and surface things that may have been missed. It can make preparation more efficient and expand the range of possibilities. But it should not replace the final professional judgment.
This is not about diminishing what AI can do. It is about distinguishing tasks by how much responsibility they carry.
In many situations, it is perfectly reasonable for AI to produce something close to a final result. A routine email, an ordinary translation, a low-risk code prototype—if the user is capable of judging whether the output works and the consequences of error are limited, AI can sit much closer to the front of the workflow.
Other situations are different. The professional threshold may be high, the consequences of error serious, and the audience unable to verify the information on its own. When these conditions overlap, AI should not be allowed to bypass human understanding and reach the public directly.
Science communication is exactly this kind of case.
What Is Really Being Skipped Is the Process Through Which Responsibility Forms
At this point, it would be easy to turn this into another article about “how to use AI correctly”: redesign the workflow, put AI back into an assisting role, add stricter review.
But even a well-designed process can still be rushed through. Someone receives an output that looks good and knows they should probably check it again, but decides it is likely fine. They do not fully understand the subject themselves, but the AI output roughly matches their vague impression, so they accept it. They may even have a sense that something is slightly wrong, but because they cannot immediately explain what the problem is, they let it pass.
That kind of carelessness did not begin with AI. The difference is that, in the past, many parts of the work were simply harder to skip.
To make an anatomical illustration, you had to find source material, understand the structure, talk to someone with expertise, and revise the drawing over multiple rounds. The process was slow and inconvenient, but the inconvenience kept putting questions in front of you: Do I actually understand this? Has this detail been confirmed? Why are we drawing it this way?
AI changes when those questions appear. Many uncertainties that used to emerge during production can now be packaged directly into something that already looks finished.
A person may receive something that looks enough like a final product before they have even had the chance to realize that they are not actually certain.
That is what makes this especially important in the AI era.
AI increases not only the speed of output, but also the speed at which content that has not been properly judged can enter public space. A moment of carelessness that once remained inside an internal draft can now become an exhibit label, a learning resource, a corporate publication, or part of how other people come to understand the world.
Important Work Needs Pauses Designed Back Into It
Why might we need to add friction back in during the AI era?
Not because slowness is inherently better, and not because efficiency should be rejected. Some friction is simply tedious and can be removed by tools. But some friction serves a more important function: it makes people stop, makes uncertainty visible, and creates an opportunity for responsibility to enter the process.
When everything is too smooth, it becomes easy to simply follow the result forward. AI gives an answer, the layout is already finished, the tone sounds confident, and we assume it can move to the next stage.
What disappears most easily is not time, but the brief pause in which someone asks: Is this right? Do I actually understand it? If it is wrong, who can detect that? Who is responsible?
These questions may reduce efficiency, but they have always been part of professional work.
In areas involving public knowledge, specialist judgment, and consequences that may be difficult to reverse, we need to design these pauses back into the process. Generated content might be required to include sources. Critical diagrams might first enter discussion as sketches rather than polished assets. Experts might become involved while the content is still taking shape instead of being asked to look at it once at the end. Before publication, it might be made explicit who confirmed the facts, who approved the representation, and who is responsible for the result.
This can sound like bringing the old workflow back. The difference is that, in the past, many of these steps existed naturally. Now, if they are not deliberately preserved, the tool will tend to remove them by default.
AI will not ask who confirmed the facts. It will not remind you who should be responsible for the diagram. It will simply make the next step easier.
Most product logic in the AI era is oriented toward fewer steps, shorter paths, and faster delivery. In many contexts, that is progress. But the more important the context, the less we should treat smoothness itself as the highest standard.
Sometimes a process is too smooth because the judgment that should have happened never did, and the responsibility that should have been visible never became visible.
I used AI while writing this essay too. It can quickly suggest structures, sentences, and transitions. Precisely because it can do that, the act of stopping becomes more important: Is this really what I mean? Is this a judgment I am willing to stand behind? Is this paragraph simply fluent, or is it actually true?
AI has made many things faster. But the more important the content, the less we can judge it simply by whether it can be completed more quickly.
What needs to be preserved is not inefficiency itself, but the parts of a process that force people to stop, verify, understand, and take responsibility.
In those places, friction is not an unnecessary step. It is part of what makes professional work possible.
About the author
Qinqin (Stella) Yang is a creative AI builder and experiential creative director. She builds workflows, systems, and creative direction that operate under real constraints, and designs speculative frameworks that expand what teams can imagine before they start making. Over the past decade she has built brand experiences, product design, and speculative projects for Tencent, IKEA SPACE10, Burberry, and the Palace Museum. She is the founder of Actual Fiction, where she develops AI-native tools that turn imagination into a working method.
This newsletter is where she thinks out loud about creativity, AI, and what it means to build toward a future that doesn’t exist yet.
More at qinqinyang.com · Practice at Actual Fiction