The Gingerbread man

August 5, 2023 9:59 PM

John D. Clare of 'Facts and the teaching of History' posits: "EH Carr's What is History?...Carr - very correctly - argues that 'the belief in a hard core of historical facts existing objectively independent of the interpretation of the historian is a preposterous fallacy." Then it unravels into historiographical relevance of fact. Nearly 20 years later, the methodology of History/ historiography is changing. 'How AI is helping historians better understand our past' and 'Digital doping for Historians: Can history, memory, and historical theory be rendered artificially intelligent'
posted by clavdivs (6 comments total)

The problem is that social memory and printed texts (such as newspapers) are often taken at face value, even when they are not factual (using the preferred term here). Examples of this were utilized by Charles Fort for his own purposes. More recently, researchers of Urban Legends can trace many back to newspaper stories that were not at all factual. You need to distinguish facts from bumf. So far as I can tell, AI is lousy at this task. But it is definitely an historian's job to learn how to do this.
"advances in deep learning have begun to address these limitations, using networks that mimic the human brain to pick out patterns in large and complicated data sets." Oh, man! Deep learning is it? Sounds really, uh, advanced.
"let's try to dream responsibly". Okay, got a responsible dream?
posted by CCBC at 10:26 PM on August 5, 2023 [5 favorites]

One thing a lot of people misunderstand about history is how little we have to go on the further back we go.

One good example of this is people on the atheist side of things trying to argue against a historical Christ. The simple fact is that we have so very very few surviving documents from 2000+ years ago that we only know some kings and emperors existed because they're mentioned once, in passing, in a document about someone or something else. And that's the only thing we have to go on.

Something like "in the 7th year in the reign of King Bob XIV, Awesome McCooldude I'm writing about did X, Y, and Z" And that is the first, last, and only time King Bob the XIV is ever mentioned in any known document.

So.... Was King Bob XIV real?

The answer of course is, it depends. But in general, as a rule, the working assumption among historians is that if someone is mentioned as if they were real, and they aren't clearly made up or otherwise indicated to be fictitious, metaphorical, or whatever, a person mentioned in a text is PRESUMED to be real until other evidence is found.

Which is why the "debate" from the atheist side against a historic Christ is just plain wrong. By the usual historic methods he'd be considered real until evidence to the contrary was found.

Note that this DOES NOT mean that historians necessarily agree that Jesus of Nazareth performed miracles, was the Son of God, rose from the dead, etc. Just that there was a radical rabbi who went by that name.

Obviously as we move closer to the present records get better.

But, as the post observes, the truth is that history isn't a science. Working hand in hand with archaeology it is possible to be fairly confident that certain large scale events took place. No one is going to seriously argue that, for example, the US Civil War didn't happen, nor even that particular battles didn't happen.

Some things though? Like CCBC notes, urban legends often get their start from "factual" reports on events that probably didn't happen. People are often credulous and report things they were told whether or not those things happened. And people are often just wrong. Or misunderstand things. Or write down something they saw but remembered wrongly. Or whatever.

If Robert VictorianDude wrote in his diary that he encountered a prostitute on the street does that mean he actually saw a sex worker? Or just a woman dressed in a way he disapproved of? Or a woman getting a loan with no sex involved and he misunderstood things? But if you then add that to a database about sex work in Victorian times have you added information? Or falsehood? Can you actually know what sex work looked like in Victorian England by amalgamating all these little, possibly wrong, tidbits?

Once you start dragging the historian's own unavoidable biases, beliefs, and worldview into things it gets even worse.

So yeah.

History is amazing and cool and people should definitely study more history! But there's no way in hell AI is going to produce something less biased, more accurate, and better able to separate what actually happened from what people wrote down as happening.

The stuff about AI helping in history sounds a lot like the "zoom in, enhance" stuff you see in crime shows. You can't "enhance". The data we have is limited, trying to fill in the gaps will just be making shit up.
posted by sotonohito at 6:32 AM on August 6, 2023 [14 favorites]

I should add, this vagueness, the fact that everything we know is the result of biased people reading and interpreting limited information recorded by biased people, and trying to do "rational" numeric analysis and find the supposed truth is what results in historical quackery like the New Chronology.

I'd be very suspicious of anyone drawing conclusions, especially radical conclusions, from an AI driven analysis of historic documents.

History is not science. It can't be science however much some people might wish it could be. And throwing big science jargony stuff at history (Fomenko was big on "statistical analysis" for example) will tend to produce BS from people who will sound very confident and certain.
posted by sotonohito at 6:37 AM on August 6, 2023 [6 favorites]

I have successfully used AI-driven transcription for 18th century German handwriting, which is notoriously hard to read if you're not used to it. On the other hand, the same program was not nearly as useful for French and Italian even though those hands are much easier to read. I think this is a good example of how unpredictably spotty these tools are. And this isn't a problem that's going to go away soon. Most people have very little idea of just how many manuscript documents a typical archive contains and how hard they are to digitize in a way that preserves all the information they contain. That's not a technological problem — it's a paying people to turn the page and push the scan button problem, which is a lot more linear but also less susceptible to optimization (only a shitty archivist would unbind a volume to make scanning quicker).
posted by derrinyet at 8:21 AM on August 6, 2023 [2 favorites]

Historians: Can history, memory, and historical theory be rendered artificially intelligent'

No.

As an ex-, I have loved watching tech open up new possibilities for historical understanding over the years. Loved it. And I can think of certain narrow use cases for which AI as we now understand it might be able to clear away some underbrush (derrinyet's example is a good one). But generally? No. In fact this is one of the worst uses for it because of the risk that random hallucinations get incorporated into historical work.
posted by praemunire at 8:58 AM on August 6, 2023 [7 favorites]

Historical theorists can, for instance, have customized large language models write a series of descriptive, narrative, and assertive histories about the same events, thereby enabling them to explore the precise relation between description, narration, and argumentation in historical writing. In short, with specifically designed large language models, historical theorists can run the kinds of large-scale writing experiments that they could never put into practice with real historians.

I don't see much valuable in "exploring the precise relation between description, narration, and argumentation in historical writing" but even if there was, doing it using large language models will only tell you how those LLMs produce text based on certain inputs, it won't tell you anything about how history is written, because LLMs don't know and aren't doing it.
posted by alsoran at 11:32 AM on August 6, 2023 [6 favorites]