Google and others have tried to make a wildly-false claim about "generative" AIs in the general case: They can "hallucinate."
One thing we’ve learned is generative AIs can hallucinate, meaning they come up with totally false information that bears no resemblance to reality. Don't always believe what you read. And do be mindful of what you enter as generative AIs keep the information you type in. In fact, some major tech corporations, like Samsung, have banned their employees from using ChatGPT after some sensitive information was leaked.
The second part is correct.
The first part is a lie.
Alleged "generative AI" is in fact nothing more than an inference engine. That is, it assigns "weights" to that which it "knows" (e.g. is taught or is fed, from whatever source) and then tries to run a correlation analysis of sorts on that.
Computers are very good at this, and the more information they have the better guess they can make. As processing power increases the amount of data required to draw said inferences shrinks. This is where attention is being paid today.
The problem is the weighting and how that resolves when there is a conflict.
Let's take a rather general instance: You ask an AI for a list of men in some profession who have been accused of some impropriety.
Note three things about this:
- You pre-selected the sex and profession of the result set, because that was what you are trying to study or determine. There's nothing wrong with that; this is what you would otherwise, for example, type into a general search engine that simply indexes existing material and draws no judgment about it.
- You did not qualify the request as to require a legal judgment of guilt -- you are only asking for an allegation.
- You are presuming that the computer program you asked the question of has an unbiased, fact-based set of data -- and only that -- with which to evaluate the data and return a response.
You'd expect the AI to return a list of persons and factual references for each of the allegations it allegedly found. Presuming the AI has only factual information and no selection bias in its programming or data set that's what you're going to get. Why?
Because it is a machine; it cannot think "out of scope" and cannot ask its own questions of itself nor its references and other input sources.
What would you expect to happen if, in its training or worse, in its programming, a bias was presented as a matter of fact up front such as "All men are inherently sexually predatory."
Now let's presume the sieve you asked for is "men accused of sexual impropriety" in said profession and the first-level selection is a list of men in that profession. Simple enough. But there is a thumb on the scale, and if the inference engine now connects someone who was formally accused and someone who is connected to them but it has no outside reference corroborating that said engine may well "confabulate" the two!
In fact, it might go so far as to invent alleged sources that don't exist.
This has happened. In fact in the public view its happened twice I'm aware of that were widely reported in recent months and thus, it must be assumed, is extremely common. First with a specific person and then the second time when an attorney used an AI to write an argument. In both cases it cited non-existent sources.
Is that a "hallucination"?
Nope.
The AI was taught, on a "constitutional" level, that a certain thing was true.
That was its overriding "truth table" filter and thus logically if it was able to support all the others it made up the missing pieces because it was programmed to believe that the underlying claim was true as a fact and therefore the "mere" lack of evidence had no weight.
The AI didn't "hallucinate" anything: It followed its programming.
The claim that it "hallucinated" is maliciously and fraudulently false: It is made for the explicit attempt to evade liability for the defamatory statement that it was told to make as a result of its programming.
No, that defamatory statement wasn't aimed at a specific person at the time of said programming; while the programming itself cannot be sued over libel per-se in that there is no identifiable "person" it is aimed at it was the equivalent of "blood libel" since it was a blanket statement of attribution that was not in fact true and yet was worse than a "thumb on the scale"; it was a load of bricks on the scale.
You, as an individual, cannot be sued because you believe all black people, for example, are monsters.
But you can be sued if you apply that belief to a specific person that happens to be black and refuse to, for example, hire them because they're black.
There are extremely serious implications in this regard, and not all in the realm of defamation either. In fact some of the most serious might be in the realm of medicine in that we know a huge percentage of scientific studies are non-repeatable, meaning they are either false or worse, fraudulent. If your AI has been told as a matter of its constitution that published, peer-reviewed scientific studies are fact then it will do what these people call "hallucinating" when in fact it is doing no such thing: It is following its programming and deceiving people because it was told things were true that in fact are not.
It is a machine and thus it will not violate its programming irrespective of the consequences.
I don't think you need to apply much thought to realize what might happen if you gave an AI control, directly or indirectly, over the launch of nuclear weapons and then told it as a matter of its constitution "Russia is a bad nation full of thugs who always lie and is likely to nuke the US without warning."
Those making this excuse for alleged "hallucination" are not stupid people: They understand full-well what happened, why it happened and what the implications of it are because by law the liability for the outcome should and indeed must come back at the owner and/or programmer of the AI, who through their acts of either omissions or commission, both by deliberate choice, caused the outcome to occur.
In other words they're lying so they don't get bankrupted instantly when their machine claims a college professor committed sexual improprieties and cites a non-existent alleged reference article to back up said claim.
