Best AI Model for Captioning and Tagging Photos: GPT-6 vs Claude 5.5 Blind Benchmark

I Blind-Tested 10 AI Models on My 142,000-Photo Library: GPT-6.1 Sol Beat Claude Opus

Abstract

I use Claude Opus 5.5 every day for coding and workflows, and for that work I find it faster and better than GPT-6. I assumed it would also be the best at describing photos, so I tested it. I ran ten configurations: Claude Haiku, Sonnet and Opus 5.5, plus OpenAI's GPT-6.1 Sol, GPT-6 Astra and GPT-6 Luna at several effort levels. Each one titled, captioned, keyworded and filed 50 real photos from my 142,521-photo library, and flagged the photos it thought I should delete. Two blind graders from different vendors, Claude Fable 5.1 and GPT-6 Astra, scored all 500 answers on facts, invented details, delete judgment, album fit and usefulness. GPT-6.1 Sol scored highest at 9.72 out of 10. Opus scored 8.57, and Sonnet and Haiku came in at 7.7 to 7.8. Ultra effort was no better than max. At official Batch API prices, captioning the whole library would cost about $240 with Sol at medium effort (9.56), $661 with Opus, and $11 to $20 with Luna, which still outscored Haiku.

Why I tested this

Most of my day goes to coding, agent workflows and automation, and Claude Opus 5.5 is the model I reach for. For that kind of work it has felt faster and better to me than GPT-6. So when I decided to have AI caption and tag my photo library, I picked Opus without a second thought.

The library is 25 years of my life: 142,521 photos, from early digital camera shots to last week's phone pictures, plus more than 16,000 screenshots. Almost none of them have a caption. I want each photo to have a title, a caption, keywords I can search, a sensible album, and an honest flag when it's a duplicate or junk. Before paying to process 142,000 photos, I wanted to know which model does this best and what the difference costs.

I expected Opus to win. On these 50 photos, GPT-6.1 Sol scored more than a point higher.

Results at a glance

Figure 1. Average score from two blind graders, out of 10.

Cost is for all 142,521 photos at each vendor's official API price.

What I'd use, based on this test:

• If I only cared about the score, GPT-6.1 Sol at max effort.
• For my own library, GPT-6.1 Sol at medium. It got 98% of the top score for $240, about a third of what Opus would cost.
• On a tight budget, GPT-6 Luna at max. At $20 it beat Haiku, which costs the same per token, and Sonnet, which would cost about 17 times as much for this library.
• Not Claude Sonnet 5.5. It's mid-priced and had the lowest score.

The models

The prices line up across the two vendors. GPT-6.1 Sol costs the same per token as Claude Sonnet 5.5, GPT-6 Astra the same as Claude Fable 5.1, and GPT-6 Luna the same as Claude Haiku 5.5. Claude Opus 5.5 sits between Sol and Astra. So when two models end up with different costs below, it's mostly because one of them used more tokens for the same job.

The Claude models ran once each at their default settings, which means medium effort for Opus. I tested the GPT models at several effort levels. The closest like-for-like comparison is GPT-6.1 Sol at medium against Claude Opus 5.5 at medium: 9.56 against 8.57.

The test

I used the 50 most recent photos in my library at the time, without videos. They're an ordinary week of my camera roll: family moments at home and on a shopping trip, city views from a high floor, a storage card on my desk, product-page screenshots of electronic parts, order-tracking screenshots, a social media post, a parking-discount QR sign and an EV charger label.

Each model had five jobs for every photo. It wrote a short title and a one-to-three-sentence caption in Chinese, gave 5 to 12 keywords in Chinese and English, and picked the best album from my existing albums, or none. Then it gave a quality score and said whether to keep or delete the photo, looking for duplicates, failed shots, expired screenshots and photos with no lasting value. Every model got the same instructions and the same context for each photo, ten photos at a time.

Figure 2. One of the 50 test photos, with three of the captions. Look at how each model handles the place name; I come back to it below. People in the photo are blurred.

How I graded it

I didn't grade the answers myself. I had already picked a winner before the test, and I use Claude every day, so I wasn't neutral.

Instead, I shuffled each photo's ten answers and labeled them A to J, with a new random order for every photo. Two graders then scored all 500 answers independently while looking at the photos. One was Claude Fable 5.1, Anthropic's most capable model. The other was GPT-6 Astra at xhigh effort, OpenAI's frontier model.

Figure 3. How the blind grading worked.

Each answer was scored on five criteria, 0 to 2 points each, for a total of 10:

I picked one grader from each vendor to see whether either would favor its own models. They mostly agreed. The correlation across the 500 paired scores was 0.71, and both put the same five GPT configurations at the top. The Claude grader also ranked GPT-6.1 Sol and GPT-6 Astra above Claude Opus. The GPT grader was stricter overall, and strictest with the Claude models.

Figure 4. Each grader's average score per configuration.

Results

Figure 5. Average score per criterion, both graders combined. An asterisk marks scores lowered by the grading mistake I describe below.

GPT-6.1 Sol and GPT-6 Astra

Sol and Astra read the text accurately and mostly limited their captions to what the photos showed. They were also the best with local place names, which matters for photos taken in China.

Figure 6. Two local-name tests. A check means the correct Chinese name, a cross means a literal translation, and EN means the name was left in English.

Several photos were tagged with the location "Rainbow Department Store". That's the English name of the Shenzhen mall chain 天虹. A word-for-word translation gives 彩虹百货, which is wrong. Every Sol and Astra configuration got this right, and so did Opus and Luna at medium. Haiku, Sonnet and Luna at max used the literal translation. The coffee shop photo in Figure 2 was tagged "Galaxy Shengshi", whose Chinese name is 星河盛世. Every Sol and Astra configuration wrote 星河. Opus and Sonnet translated it literally as 银河, and Haiku and both Luna configurations left it in English.

Sol and Astra were also careful about deleting. They suggested deleting 4 to 6 of the 50 photos. The set included eight product-page screenshots of electronic parts, and the GPT models kept almost all of them; one Sol configuration flagged a single screenshot as a near-duplicate. The Claude models flagged two to four each. Both graders sided with keeping them, since the pages held pinouts, dimensions and board photos I'd want to look up later.

Claude Opus 5.5

Opus sometimes wrote things into a caption that the photo alone can't show. It described relationships that the image alone can't prove, such as siblings or parent and child. It also got some poses wrong. In one photo a child was standing behind an adult, and Opus wrote that the child was on the adult's back. Both graders took points off for this.

This surprised me. Before the blind test, I had looked through the answers and praised Opus for correctly guessing that two children were siblings. I know my family; the grader doesn't. But with 142,000 photos, I can't check every caption myself, and a caption that states something the photo doesn't show is a problem.

Opus also suggested the most deletions, 13 of the 50 photos. Three of them were product-page screenshots it called expired, although they held reference information I'd keep. It was the only model that wanted to delete the parking-discount QR sign as low value. It also put several photos of a single person into a "family portrait" album.

Claude Sonnet 5.5 and Haiku 5.5

These two made the most factual mistakes, including the literal 彩虹 translation. Sonnet made the weakest album choices and filed everyday photos into an old album from an unrelated event. Haiku invented a detail that led to a bad delete call. In a burst of near-identical shots, it described the best frame as someone looking down, which is the opposite of what the photo shows, and suggested deleting it.

GPT-6 Luna

Luna costs the same per token as Haiku but scored higher at both effort levels: 8.41 and 8.11, against Haiku's 7.81. At max effort it also beat Sonnet, which would cost about 17 times as much to caption the same library. Luna lost most of its points on usefulness and album choice. Its captions were plainer and its albums less precise.

Effort levels

Figure 7. Score by effort level. The orange line is Claude Opus 5.5 at its default effort.

Ultra was no better than max for either model. Sol scored 9.59 at ultra and 9.72 at max; Astra scored 9.58 and 9.61. In this test, ultra even produced fewer output tokens than max. For Sol, medium came close at 9.56, for less than half the cost of max. Higher effort didn't always mean more accurate details, either: Luna at medium got 天虹 right, and Luna at max didn't.

A mistake in my grading

After grading, I noticed that the graders hadn't been given each photo's capture date and time, although the captioning models had. So the GPT-6 Astra grader treated captions that mentioned the exact date or time as invented details. The top GPT configurations mentioned the date most often, so their "No invented details" scores came out lower than they should have.

If I give back only the points lost over dates, the top four GPT configurations rise to between 9.84 and 9.90, which is effectively a tie, and Opus stays at 8.58. I've kept the original scores in this post. Next time I'll give the graders everything the models saw.

Speed

Figure 8. Seconds per batch of 10 photos, as observed. This isn't a controlled latency test, because the two vendors' models were reached through different access paths.

This matches how Opus feels in my daily work: it's fast, about three times quicker than Sol at max. Sol at medium was just as fast as Opus, though, and scored about a point higher.

What it costs

I priced every configuration at the vendors' official API prices. Treating a subscription plan's included usage as free would make some models look better than they are.

Official prices per million tokens, standard tier:

Both vendors' Batch APIs cost 50% less, which suits a job like this that doesn't need instant answers. I took the prices from Anthropic's pricing page (claude.com/pricing) and OpenAI's API pricing page (developers.openai.com/api/docs/pricing) on October 20, 2026.

Figure 9. Score against the cost of captioning the whole library at Batch prices. The cost axis is logarithmic.

Sol at medium got 98% of the top score for 36% of what Opus would cost and 43% of Sol at max. Astra matched Sol's quality but costs about five times as much, because its per-token price is five times higher. Opus cost more than Sol at max and scored lower. It already ran at medium effort, but its per-token price is twice Sol's. Luna would caption the whole library for $11 to $20 and still outscore Haiku and Sonnet.

Method note: GPT token counts were measured from the runs, and Claude token counts were estimated from the run logs. Some Claude reasoning tokens weren't visible, so the Claude costs may be slightly understated. No discounts beyond Batch, such as prompt caching, were applied to any model.

What I learned

I picked Opus before I had any blind scores, mostly because it was already my coding default. I still prefer Claude Opus 5.5 for coding and workflows, where it's faster and stronger in my experience. Describing real photos takes a different skill: reading what's actually in the frame without filling in poses or relationships. GPT-6.1 Sol did that better. If I had gone with habit and skipped the blind test, I would have paid more for worse captions.

Next time I'll start blind and give graders from different vendors all the inputs the captioning models got. I'll keep comparing at official API prices, too, and I won't assume more effort is better: ultra didn't beat max here.

Even the best models flagged useful photos for deletion, so I'll check every delete flag myself before removing anything.

Limitations

This was a small test: 50 recent photos from one city. I didn't test older photos, scanned documents, other languages or videos. The Claude models ran only at their default settings; a higher-effort Opus might score better, at a higher cost. The graders are AI models, not people. They agreed well but not perfectly, and a human spot-check would make the result stronger. Models and prices change quickly, and every number here is as of October 2026.

My pick

For my library: GPT-6.1 Sol at medium effort. For coding and workflows, I'm sticking with Claude Opus 5.5.