Using LLMs to query 1939 International Genetical Congress

Proceedings of the Seventh International Genetical Congress held in 1939 and published in 1941.
Title page from Proceedings of the Seventh International Genetical Congress held in 1939 and published in 1941.

What work can large-language models (LLMs) do for historical researching? They offer tools for voluminous compilation of data ready for complex human analysis. They can organise and reorganise data. They can extract data from source material. They can be set to search for trends. We’re coming to grips with LLM tools for historical researching, and we’re quickly moving well beyond the LLM-as-author model so distrusted in our community. Historians must push ourselves to be as creative and demanding of LLM resources as those in our sister disciplines.

In this post, I use the service Google NotebookLM for a series of simple queries about one primary source:

This is a book-size primary source featuring a narrative introduction, lists for several types of participant information, and a substantial number of abstracts for papers intended for presentation at the Congress. For the purposes of this project, I secured a print copy, digitised it, verified it, and made it available CC-BY-NC via Internet Archive.

The Seventh International Genetical Congress took place in August 1939 in Edinburgh following cancellation of the event in Moscow in 1937. The Congress collapsed mid-week as national delegations were recalled, and the international situation deteriorated.

The natural comparison is with Proceedings from earlier and later congresses, such as:

NotebookLM has the advantage of allowing a closed content ecosystem, meaning it only consumes content uploaded by the client, and this content can be activated or deactivated in analysis.

For these queries, Punnett (1941) was divided into component sections:

  • Narrative, pp. 1-12
  • List of Members, pp. 13-21
  • List of Exhibitors, pp. 22-25
  • Programme, pp. 26-38
  • Index of Papers and Abstracts, pp. 39-44
  • Papers and Abstracts, pp. 45-332

Queries were developed to explore responses. Simple queries were used first as a check on the data. For instance, the printed document lists 331 abstracts and papers. A query asking to count the number of abstracts and papers gave the same number.

LLM Queries to Support Historical Questions

List the organisms studied in all these papers, sorting the list by the frequency of the organisms identified.

Organise all mentions of Drosophila by the species, such as Drosophila melanogaster or Drosophila funebris. 

How many papers mention physiology or physiological genetics?

How many papers mention evolution or speciation?

List the papers that mention “genetic drift”?

List the top 15 most frequently mentioned people.

Count the number of papers that mention Fisher, Wright, Haldane, Dobzhansky, Demerec, Chetverikov, Vavilov, Crew, Huxley, Dunn, Morgan, and Mendel.

Count the number of papers in “Papers” by language, such as English, Spanish, French, Italian, German, etc.

List the papers that discuss eugenics.

List the individuals in “Members” who are not listed as authors of papers in “Papers”.

What is the geographic distribution of the exhibitors?

What were the most actively researched and discussed genetic mechanisms at the 1939 Congress?

What involvement did researchers from USSR have in this International Congress?

Summarise the reception of work by Huxley at the Congress.

Summarise the reception of work by Fisher at the Congress.

Summarise the reception of work by Sewall Wright at the Congress.

Summarise research from Germany supporting National Socialist theory.

Summarise research supporting capitalism.

Generated Multimedia

GoogleLM has facility to produce audio and film synopses of content. Using only material from Proceedings, I invoked this feature.

Audio

This mp3 audio file is a generated synopsis for the “Papers and Abstracts” section of the Proceedings. In the printed original, this section has no supervening hierarchy for the abstracts – abstracts are organised alphabetically – therefore anything produced through the LLM was generated by it.

What’s fascinating is the way it’s selected specific papers to emphasise rather than produce an overview or high-level categorisation. It’s not clear what’s driving the prioritisation in selection.

Film

This film is a generated synopsis for the “Narrative” element of the Proceedings only. No supplemental material was provided from the user, which explains why it is oblivious to Lysenkoism and the arrest of Vavilov, among other issues. The film-creation feature does not provide captioning.

Beware lazy AI

Using this technique risks mistakes, and users need to verify. They also need careful attention to the nature of their queries. For instance, the paper document lists 331 abstracts and papers. The query “how many papers and abstracts are included” yields 331. Correct. Not all simple queries return confirmed results. For instance, the query, “List the countries of members in terms of frequency: most frequent to least frequent” using only the “List of Members” content yields numbers at variance with the summary numbers given in the paper document as a count of members listed by country. For a reason that is not clear, the counting is off. Unverified use will mislead.

Moreover, because LLMs prioritise speedy response times, they tend to use simple routes for gathering information rather than difficult routes. For instance, if I request a count and the result is already given in the content, then the LLM tends to return the result given rather than produce a new count. As an example, in the first case above, if this LLM had access to the complete Proceedings rather than only the “Papers and Abstracts” section, it likely would have found the number 331 in the front matter and returned it as a reply rather than count the number of entries. Users need to regularly remind themselves that the interface is designed to be highly user-friendly and might not undertake the precise task requested (or intended) in preference for a quick and affirming response. Command driven prompts through APIs, which draw the historian into ever more complex programming work, offer more control and more reliabililty.