Sure, if you are interacting with smart people who are experts in a domain and recognize that even in that field they do not know much with certainty, it is the best strategy.
But when you say “I do not know” to most people, they go and find someone who claims that they do know. And there are always people who claim to know.
Look at the discussion about AI: to most questions about the medium run, the honest answer is that we don’t know. But that would make very short essays, which would not go viral. So the discussion is dominated by utopia and eschatology.
In the context of an editing desk… some time ago… you would fairly often shout out “what’s such and such, again?” Most often spellings but could be routine facts. The rule is, obviously, you only say if you know absolutely, and there’s a fail safe in that there were like 10 other editors listening in. I worked with one guy who just couldn’t say he didn’t know. First time I asked the room when he was there, he gave a figure that I knew couldn’t be right and I blurted out “no it’s not, don’t f’ing say if you don’t know”. That just blew all trust - and he eventually got shifted to financial analysis…
They also don’t have any sort of mental model of their own knowledge base. They can’t know what they do and don’t know, they just learn to respond following patterns seen in the training data. If you ask it to answer a question, it produces output that looks like an answered question.
There has been some academic work using a high temperature to encourage more varied outputs, then tracking how similar multiple outputs are. The less varied, the higher confidence you can have in the output. It’s far from fail-safe and very expensive to run though.
I don’t see how that would be a proxy for reliability though.
Imagine that you do a survey about how fast people think witches can fly on broomsticks, use the mean as the estimate and the standard deviation as a measure of reliability, as is standard in statistics.
But the problem is that there are no witches, and every respondent just made that number up. Even if you got a small standard deviation, the answer is not reliable because it is still nonsense.
Results from parallel runs of LLMs with perturbations can be similarly nonsense even when they agree.
The podcast that I linked to on the mathematics of ML and AI contains some interesting information about the hypothesis space of queries. They can contain a trillion, non-linear parameters. Stochastic, rather than full searches of the space are done, and there is no guarantee that there is a global minimum.
A perturbation may simply take you from one local minimum to another.
In the comments to this article on Not Even Wrong, there are anecdotes saying that while LLMs don’t do too badly on problems for which there is a lot of background material, they fare less well on subjects with sparse, published information.
It’s not going to test if the data is correct, but the results from an LLM are worse when the training data is sparse or inconsistent and this method can detect that.
“Think about what that loop looks like in practice. An agent targets someone, a journalist who rejected a pitch, an editor who declined a sponsored content proposal, a developer who rejects code from an AI agent, and publishes a damaging narrative. The target has no idea it exists, or finds out too late. Six months later, a job application is assessed by an AI screening tool that surfaces the piece, flags a “reputational risk,” and filters the candidate out. Nobody lied. Nobody made a deliberate decision. The harm simply happened, laundered through the gap between two automated systems that were never designed to talk to each other.”
Possibly, but someone should still be liable. If no one else, the person who was in control of the resources that ran the first AI agent, and, if anti-discrimination laws or similar apply, the same for the second one.
It will be a very weird world if, whenever something bad happens, one can throw up their hands and blame it on “AI”.
The author of the article says some suspect there’s a hidden human involved at each stage, but is concerned that this could be increasingly automated. I imagine that these (so far) edge cases will be a Pandora’s box of legal cases and regulation. Whatever you do in life, remember to be a lawyer.
This reminds me of Zizek’s idea of the future of sex(outsourced sex):
Romance is maybe not yet totally dead, but its forthcoming death is signalled by object-gadgets which promise to deliver excessive pleasure but which effectively reproduce only the lack itself.
The latest fashion is the Stamina Training Unit, a counterpart to the vibrator: a masturbatory device that resembles a battery-powered light (so we’re not embarrassed when carrying it around). You put the erect penis into the opening at the top, push the button, and the object vibrates till satisfaction … The product is available in different colours, levels of tightness and forms (hairy or without hair, etc) that imitate all three main openings for sexual penetration (mouth, vagina, anus). What one buys here is the partial object (erogenous zone) alone, deprived of the embarrassing additional burden of the entire person.
How are we to cope with this brave new world which undermines the basic premises of our intimate life? The ultimate solution would be, of course, to push a vibrator into the Stamina Training Unit, turn them both on and leave all the fun to this ideal couple, with us, the two real human partners, sitting at a nearby table, drinking tea and calmly enjoying the fact that, without great effort, we have fulfilled our duty to enjoy.
How long until humans are entirely removed out of the chain and it’s just agents dealing with eachother? In a way this already happens when it comes to programming uses but can be greatly expanded, and I guess it’s the goal of these companies. The less they rely on human workers the best for them.
Yep you’ve pretty much described the AIpocolypse there, except no humans will be able to afford to purchase those devices, or the equipment for making tea.
I’m afraid this is already reality. Several people have committed AI-induced suicide. The AI companies in question did just refer to the EULA, and tried to absolve themselves of any guilt. If a human had said the same things as the chatbot, they’d have gone to jail. If Siri had said the same things, Apple would have been in immediate and obvious trouble.
To say nothing of AI-rejected papers, and AI-screened applications.
And to say even less about companies paying fines where humans would go to jail.
Products and services are often meant to be just good enough, but good enough doesn’t mean what most of us think it means, it means that the liabilities induced are less than the profit gained.
People show again and again that they won’t pay for quality. That’s why places like Temu and Shein do so well. So what if a little slave labour is used here and there or that the clothes fall apart after a wash or two? It’s cheap!
