ChatGPT, Collector's Assistant?

[Sulla80]

I have been playing around with ChatGPT 4o (latest version) - with a subscription from OpenAI.  If anyone else is experimenting - I thought it could be worthwhile to share experiences and learn from each other.  My first impression was: "ChatGPT is a numismatic genius!" - until a few people pointed out to me on NF that I was giving it all the answers with my file names 🙃  That said - I have found it to be interesting, entertaining and perhaps even a useful tool.  I tried a few tasks with it and gave each a letter grade for how satisfied I am with the results.

Upload a Document and Chat (Grade: A)

I tried a few different documents - it was fun to talk to the document and ask questions.  I tried a brief article and asked it to write a summary - it did this very nicely.

I also tried chatting with a copy of Stephen Albums Checklist of Islamic Coins 3rd Edition (2011).   You can chat with the full text of the book:  (NOTE: I am truncating answers to not have an absurdly overwhelming post size)

 

Get Help Attributing a Coin (Grade: D--)

Figuring out a partial legend seemed to work quite well - I will definitely try this more often:

In attribution, it was more useful as a helper - where I have some information and context. it didn't do too well on its own.

It did surprisingly well with this coin - I suspect because of the ability to read the reverse legend:

Here is some evidence that it is just getting lucky when it is right (perhaps because it knows a similar image).

It seems to have thought that my coin was this one, and garbled it with an attempt to read the actual words on the reverse of the coin above...

It might be helpful, giving me an idea of what to look for - but given I can read better than it can. For now, I will stick with my primary tools for attribution (other than human beings) searching the internet and ACSearch.

 

Get Help Reading Coins (Grade:D--)

While results are a bit mixed - it does a decent job assisting in reading inscriptions (or at least inferring them from some combination of image and associated text):

Not bad and easy enough to visually check the answer.  I still gave it an "D--" overall because it only could read legends when they were very clear and it didn't do great with rotated text and text going in circles around the coin.   It also seems to be associating text with images - not just doing OCR on the text that is actually there - this can result in garbled interpretations.   Always verify any answer.

Ask it to Describe a Coin (Grade: D--)

In this case, I had given it some context on the specific coin - the article from David Sellwood (see above). 

it also described this coin well.

ChatGPT does seem to be easily misled by context provided and questions you've asked before in the same session- it thought that my Tetradrachm of Artabanos was a Parthian Drachm referenced by Sellwood.  If I didn't know what these coins were, I would also be easily mislead or misinformed.  It was particularly unhelpful with this coin - suggesting anything from a Indo-Greek to Roman Provincial and seeing the double cornucopia as intertwined serpents.

Historical Context was not so useful either:

While it's entertaining and it is sometimes useful - I am not going to start using it for much day to day. 

Find the Reference Numbers (Grade: B)

I don't have Crawford numbers in my head for every RR denarius - so it can be helpful to do a quick lookup with partial information e.g. this table was quickly produced with prompt: (I am interested in producing a complete table of Roman Republican denarii with elephants on them - can you make a table with the following columns: "moneyer", "date issued", "obverse", "reverse", "reference numbers", "estimated hammer price")

Although I am not sure the list is complete and "Head of Pius" should be "Head of Pietas" - it was quick and had the reference number that I wanted (374 - Q. Caecilius Metellus Pius, Head of Pietas / Elephant)

Translate Text (Grade: A+)

Here it is perhaps a little better than Google Translate for languages that you don't read.  I have used it to help me trawl through text from RE (Paully's RealEncyclopaedia).  Many of the pages are only available as images, so you have to convert to PDF, OCR the text and then translate - which is not to hard to do between Adobe PDF software and ChatGPT or Google translate..

Decode and Find Sources (Grade: B/F)

It also can be helpful in expanding abbreviated resources and finding an online source for those resources.

However this didn't feel all that revolutionary - It failed to find Archäologischer Anzeiger 1900, 6 = Klio II 248

(Archäologischer Anzeiger 1900, 6)

(Klio II 248)

Expanding the abbreviations it was more helpful that search (B) but for finding sources it was not useful (F).

Summary and Conclusions:

Overall - it seems worth experimenting with.  There is some promise and some utility today. Here's a summary of what I've described above:

• Upload a Document and Chat (Grade: A)
• Get Help Attributing a Coin (Grade: D--)
• Get Help Reading Coin (Grade: D--)
• Ask it to Describe a Coin (Grade: D--)
• Find the Reference Numbers (Grade: B)
• Translate Text (Grade: A+)
• Decode and Find Sources (Grade: B/F)

Some lessons learned: always take answers as a starting point - expect that answers are incomplete, and don't trust without verification.  Do give it as much context as you can in your request, and be careful with the context in your session as it does make assumptions that are incorrect based on context (and of course if you give it the answer - don't be surprised if it gets it right).  Deleting and starting a fresh session can be useful.

Use the "Customize ChatGPT" option to set expectations for how it should respond.

The technology is changing rapidly and it is definitely better than it was even a few months ago e.g. now ChatGPT can search the internet - read images and do a lot more than it could not that long ago.  Definitely worth keeping an eye on it as it evolves.  I suspect that there is potential in fine-tuning for a class of coins where there is sufficient data.

There are some things it does that seem to simplify tasks that used to take a couple of steps e.g. search the internet, format and summarize text as it did with the table of coins - you can improve utility by pointing it to specific sources of information and asking it to check itself.

Share your experiences with LLMs and ChatGPT,  ancient coins and various tasks associated with researching coins and their history.  What do you find useful?

 

Edited May 20 by Sulla80

[Heliodromus]

I use Claude (free version - their middle "Sonnet" model) which maintains a history of topics you've discussed with it.

95-99% of my Claude use is NOT coin related, but the coin-related subjects (Claude's one-line characterization below) I've tried in past few weeks are:

• Closure and Revival of the Aquileia Mint (it was no help)
• Nero's Inherited Title "Germanicus"
• Translate German Document on Armenian War (it was no help since it couldn't handle the particular scanned PDF type I was trying)
• Evaluating the Historical Accuracy of "The Twelve Caesars"
• Smuggling Ancient Coins from Gaza (asking about the Beale-Alshdaifat connection, not a how-to guide :))

I do find Claude useful for other things, but for coin related it seems that summary and translation of documents is probably one of the best current use cases.

 

Edited May 19 by Heliodromus

[Sulla80]

41 minutes ago, Heliodromus said:

I use Claude (free version - their middle "Sonnet" model) which maintains a history of topics you've discussed with it.

95-99% of my Claude use is NOT coin related, but the coin-related subjects (Claude's one-line characterization below) I've tried in past few weeks are:

• Closure and Revival of the Aquileia Mint (it was no help)
• Nero's Inherited Title "Germanicus"
• Translate German Document on Armenian War (it was no help since it couldn't handle the particular scanned PDF type I was trying)
• Evaluating the Historical Accuracy of "The Twelve Caesars"
• Smuggling Ancient Coins from Gaza (asking about the Beale-Alshdaifat connection, not a how-to guide :))

I do find Claude useful for other things, but for coin related it seems that summary and translation of documents is probably one of the best current use cases.

 

Thanks,
@Heliodromus.  I've been experimenting with Claude as well - the biggest advantage I see in ChatGPT 4o  at the moment is integration with internet search which Claude (even in Pro version that I am using) does not seem to offer).  e.g. ChatGPT has the advantage of being able to integrate current information and cites these two articles in its explanation of the Beale-Alshdaifat connection.

• https://www.lbc.co.uk/news/british-auctioneer-coin-sale-fraud-new-york-richard-beale/
• https://news.artnet.com/art-world-archives/richard-beale-coin-provenance-court-2360975

Edited May 19 by Sulla80

[Deinomenid]

Claude-for-free last week developed a really odd quirk for me. It started making up obscure facts about specific and admittedly also obscure poleis that sounded plausible until it tried it on a couple of places I know really well. I was deeply puzzled as I was sure I would have come across these anecdotes and asked it for details of where it was getting them, only to be met with denial, then obfuscation then what appeared to be abject apology with promises never to do it again. Truly bizarre.

[Sulla80]

1 hour ago, Deinomenid said:

Claude-for-free last week developed a really odd quirk for me. It started making up obscure facts about specific and admittedly also obscure poleis that sounded plausible until it tried it on a couple of places I know really well. I was deeply puzzled as I was sure I would have come across these anecdotes and asked it for details of where it was getting them, only to be met with denial, then obfuscation then what appeared to be abject apology with promises never to do it again. Truly bizarre.

I don’t know how to adjust “temperature” or creativity in Claude - in ChatGPT you can set temperature from 0 to 1 where 1 is most creative and zero very conservative.

[Heliodromus]

38 minutes ago, Sulla80 said:

I don’t know how to adjust “temperature” or creativity in Claude - in ChatGPT you can set temperature from 0 to 1 where 1 is most creative and zero very conservative.

I believe there's a control for it if you pay the $20/mo for Claude Pro (which gives access to the Opus model).

The free version doesn't officially allow you to set it, but you can append ?t=0.5 (any number from 0-1) to the URL to change it!

For example, try:

https://claude.ai/chat?t=0 (temperature 0)

and repeatedly ask for a random number between 10^6 and 10^9 - you'll get same number every time.

I'm not sure what the default temp. is, but certainly not 0.

[kirispupis]

FWIW, I tried using ChatGPT to help with this mystery coin, and it wasn't much help. It answered with authority, but struck out each time.

[Sulla80]

My favorite new ChatGPT trick: transcribe this image:

@DonnaML shared this paragraph from Grueber's RR Coins in teh British Museum with me in another thread.  ChatGPT converted image to text:

Here is the transcribed text from the image:

---

2 The representation of this combat between two warriors is probably a record of the brave deeds of the moneyer’s ancestor and namesake, who was consul B.C. 193, and who was killed in the war in Thrace B.C. 188. During his consulship and the two following years, Q. Minucius Thermus was engaged in a severe contest with the Ligurians, when he is said to have performed many brave deeds, especially on the occasion of a night attack on his camp. Babelon (vol. ii., p. 235) has identified the head on the obverse as of Roma, but it is probably of Mars, being very similar to that on coins of L. Julius Caesar struck at Rome in the previous year (see vol. i., p. 209), and on those of C. Poblicius Malleolus (see below, p. 307). The head of Mars would be more appropriate to the reverse type than Roma. There is a restoration of this denarius by Trajan, and specimens were countermarked by Vespasian. The denarius is the only denomination known of this issue.

 

[Heliodromus]

5 minutes ago, Sulla80 said:

ChatGPT converted image to text:

Not a single error?! Awesome!

 

[John Conduitt]

12 minutes ago, Sulla80 said:

My favorite new ChatGPT trick: transcribe this image:

@DonnaML shared this paragraph from Grueber's RR Coins in teh British Museum with me in another thread.  ChatGPT converted image to text:

Here is the transcribed text from the image:

---

2 The representation of this combat between two warriors is probably a record of the brave deeds of the moneyer’s ancestor and namesake, who was consul B.C. 193, and who was killed in the war in Thrace B.C. 188. During his consulship and the two following years, Q. Minucius Thermus was engaged in a severe contest with the Ligurians, when he is said to have performed many brave deeds, especially on the occasion of a night attack on his camp. Babelon (vol. ii., p. 235) has identified the head on the obverse as of Roma, but it is probably of Mars, being very similar to that on coins of L. Julius Caesar struck at Rome in the previous year (see vol. i., p. 209), and on those of C. Poblicius Malleolus (see below, p. 307). The head of Mars would be more appropriate to the reverse type than Roma. There is a restoration of this denarius by Trajan, and specimens were countermarked by Vespasian. The denarius is the only denomination known of this issue.

 

Like quite a lot on ChatGPT, I believe this comes from Google images (or vice versa)


 

Edited May 23 by John Conduitt

[Sulla80]

Google definitely has competing tech (ChatGPT is from OpenAI - Microsoft invested in them and using these capabilities for Bing).  I used to use Adobe acrobat to do OCR - but this was a bit tedious, Print to PDF, the OCR in Adobe....I tried a novel sentence that couldn't be found in a database - so it is definitely doing the text recognition against the image I provided.

You can try Google Gemini Here: https://gemini.google.com/app

Edited May 23 by Sulla80