Many small companies keep their signed contracts in a shared drive. ChatGPT and Claude can now connect to that drive and read what is in it, and Anthropic now ships a contract review table inside Claude. So it is fair to ask: if you can already ask ChatGPT which contracts renew this year, do you need anything else?
We gave ChatGPT, Claude and AeroSpark the same folder of 99 real contracts from the CUAD contracts and asked all three the same 46 questions.
We had finished those runs when, on 28 October, Anthropic released a set of legal tools for Claude, including one aimed at almost exactly this job: Claude Tabular Review. We quickly re-ran our benchmark on Claude Tabular Review. The contracts, questions and every answer are public, and the full method is in the GitHub repository linked at the end.
The short answer
All four systems fared well. On our 46 questions AeroSpark scored 89.8 out of 100, Claude 85.6, Tabular Review 83.9 and ChatGPT 82.2. AeroSpark outperformed the other three, but the margin was not wide.
However, AeroSpark was significantly faster. AeroSpark answered every one of the 46 questions in under a minute, with a typical answer in 10 seconds. ChatGPT and Claude typically took five to ten minutes per question. Claude Tabular Review sits between the two: a typical question took under a minute, but a question on a new topic took two to five minutes, and the person asking had to step in along the way. A re-run against Claude Tabular Review would likely have been much quicker, probably approaching AeroSpark’s answer times.
Time to answer
| AeroSpark | Claude | Claude Tabular Review | ChatGPT | |
|---|---|---|---|---|
| Questions answered in under a minute | 46 of 46 | 11 of 46 | 24 of 46 | 8 of 46 |
| Questions that took over 5 minutes | 0 | 26 | 1 | 29 |
| Typical time per question | 10 seconds | 5.6 minutes | 42 seconds | 5.9 minutes |
| Slowest answer | 35 seconds | 11.2 minutes | 5.2 minutes | 19.4 minutes |
| All 46 questions, end to end | 9 minutes | 4 hours | 78 minutes | 5 hours |
| Times the person asking had to step in | 0 | 0 | 11 | 0 |
AeroSpark is timed inside its service, from the question to the finished answer. ChatGPT and Claude are timed from their own record of each chat, from the question to the last message, running three chats at a time. Tabular review is timed from its own record of each run plus a stopwatch, and its total leaves out a 30-minute spell when the page stopped accepting calls.
An example question: Which of these contracts were entered into before January 1, 2001?
| AeroSpark | Claude | Claude Tabular Review | ChatGPT | |
|---|---|---|---|---|
| Contracts found | 14 of 14 | 14 of 14 | 14 of 14 | 14 of 14 |
| Named but not in the key | 0 | 0 | 0 | 0 |
| Time to answer | under 1 second | 7.0 minutes | about 30 seconds | 7.1 minutes |
All four found the same 14 contracts and named none outside the key, so accuracy made no difference on this question. Speed did. AeroSpark worked out the answer in under a second from dates it recorded when each contract was added. Claude Tabular Review took about 30 seconds, answering from a date column it had built for an earlier question. Claude and ChatGPT read the contracts to answer and took about seven minutes each.
Why the difference in speed?
A chat in ChatGPT or Claude starts from scratch every time. For each question it searches the drive, opens the files and reads them, which is why one question can take ten minutes. Claude Tabular Review reads the contracts once per topic: you ask, and if its table cannot answer, it proposes a column, you approve it, it reads all 99 contracts into that column, and you ask again. The next question on that topic takes seconds. AeroSpark reads each contract once, when it is added, and records what is in it: its clauses, dates, parties, governing law and renewal terms. A question is answered from that record rather than by reading the files again, with no column to approve, and a date question like “still in force today” is worked out directly from the stored dates, which is why that one took under a second.
Finding contracts
Every question counts the same: it is worth an equal share of 100, and each system earns that share in proportion to how much of the question it got right.
| AeroSpark | Claude | Claude Tabular Review | ChatGPT | |
|---|---|---|---|---|
| Score out of 100 | 89.8 | 85.6 | 83.9 | 82.2 |
| Contracts found, of the 576 in the key | 490 | 463 | 452 | 427 |
| Answer accuracy | 95% | 97% | 95% | 99% |
Thirty of the 46 questions ask which contracts contain something. A few examples:
- Which of these contracts contain a no-solicit of employees provision? (13 contracts)
- Which of these contracts are still in force today? (28 contracts)
- Which of these contracts expressly cap the liabilities of one or both parties? (54 contracts)
Adding up every contract the key lists for those 30 questions, 576 in all, AeroSpark found 490, Claude 463, Claude Tabular Review 452 and ChatGPT 427. AeroSpark and Claude Tabular Review also named more contracts that were not in the key: 24 and 23, against Claude’s 12 and ChatGPT’s 5. ChatGPT names fewer contracts and is right more often when it names one.
Where the gap opens
On questions that only ask which contracts contain a clause, the four are close: out of 100, Claude Tabular Review scored 82.8, AeroSpark 82.5, Claude 81.9 and ChatGPT 77.6. On the liability-caps question Claude Tabular Review did best of all, finding 38 of the 54 contracts against AeroSpark’s 27, Claude’s 23 and ChatGPT’s 20. The gap opens when a question depends on dates or on more than one condition, which are the questions closest to running a business, like which contracts renew this year.
- Dates against a given day (6 questions, such as “Which of these contracts are still in force today?”): AeroSpark 94.6, Claude 82.8, Claude Tabular Review 68.0, ChatGPT 64.3.
- Two or three conditions at once (4 questions): AeroSpark 88.5, Claude Tabular Review 78.4, ChatGPT 77.5, Claude 72.7. Each score is the average across the four questions. Most of the gap comes from the hardest one, “Which of these contracts auto-renew and include a warranty?”, where no system found much more than half of the 26 contracts: AeroSpark 14, ChatGPT 12, Claude Tabular Review 10 and Claude 7.
- “Which of these contracts run until a party ends them, with no fixed end date?” AeroSpark found 11 of the 14, Claude 8, and Claude Tabular Review and ChatGPT 7 each.
The other 16 questions have a single answer: a count (“How many of these contracts are governed by the laws of Texas?”) or a fact about one named contract (“What law governs the Master Services Agreement between Clear Capital and Radial Spark?”). Each system got 15 of the 16 right. All four gave a wrong count for the right-of-first-refusal question.
Claude Tabular Review
Claude Tabular review is a ready-made artifact from Anthropic’s legal gallery, released on 28 October. You add the contracts, it builds a table with a column per topic, and later questions are answered from the table. We ran it the way a new user would: an unchanged copy, the 99 contracts uploaded, the 46 questions asked in order in one review, and every column accepted as proposed.
Benefits of Claude Tabular Review:
- Indexed fields give fast lookups: The first question on a topic costs two to five minutes while it reads all 99 contracts into a new column. The next question on that topic takes seconds. Governing law in England, Canada and outside the US each came back in under 20 seconds and were right, where a fresh Claude chat took three to eight minutes on each.
- It combines what it already knows: “Governed by Texas law with a confidentiality provision” was answered from two existing columns in under 30 seconds, with all four contracts right.
- It did better on some of the hardest clause questions: It found 38 of 54 contracts that cap liability, where plain Claude found 23. It also beat plain Claude on auto-renewal with a warranty, 10 of 26 against 7.
- It is easier to check (than a chat response): Filled cells hold a quote and a page, and the table separates “Not found” from “No”. Anthropic pitches this as a first pass for a lawyer to review, and it is set up for that.
- Flexible column creation: Claude doesn’t build out unnecessary columns for all of your contracts. It only builds them as you need them.
Issues with Claude Tabular Review:
- Column-building is tedious: Twenty of the 46 questions needed a new column, so a new topic is a three-step wait: ask, approve, ask again.
- Column design can be too narrow: Asked about Texas law, it proposed a column recording only “Texas, Other, Not stated”. Asking about Illinois then needed a rewrite and a second read of all 99.
- First-shot responses aren’t reliable: Its first answers scored 69.7. Under the benchmark’s rule, any question where a user would naturally follow up is re-asked; eight questions qualified, and the second answers took it to 83.9. Six of those were lookups it had answered correctly and then taken back in a later reply, telling us “you should not rely on that answer”. On the re-ask, with more columns built, all six stood.
- Counts are unreliable: It said 47 and listed 50, and said 62 and listed 70, and told us to trust the list. Twice, a contract marked Yes in its own column was missing from the written answer.
- Flakiness (launch related?): The page stopped accepting calls for about half an hour during our test, with the account at 61% of its session limit and 45% of its weekly limit. We do not know why.
If you have a known list of terms to pull and a lawyer who will work down the table, Claude Tabular Review is built for that, and you keep the table. If you want to ask whatever comes up and get an answer in seconds, that is what AeroSpark is built for. With any of them a person still has to check: on list questions AeroSpark missed about one contract in seven, Claude and Claude Tabular Review one in five, and ChatGPT one in four.
More information about the benchmark
The full benchmark, along with the questions, the source contracts and every answer from each system, can be found here. The questions are a representative set of questions that we see users asking of their contracts in AeroSpark. The performance of all four systems varies from run to run, so your results might look slightly different should you run them yourself. They should be directionally accurate. We did not run it against the full set of CUAD contracts for cost reasons.
Of the 46 questions, 39 are scored against CUAD’s labels. The other 7 ask about clauses CUAD does not label, such as arbitration, confidentiality and indemnification. For those, every contract was read by two independent AI readers and the disagreements settled one by one. Those labels, both readings and every settlement are published with the benchmark for review and critique.
We used ChatGPT Business with GPT-6 at High reasoning, Claude Max with Opus 5.5 at High effort, and AeroSpark through its Ask box, all with the same wording for every question, one fresh chat per question and no follow-ups. Claude Tabular Review ran on its default model tier, which Anthropic does not name, in one ordered review with the follow-ups it asked for on four questions; those and the re-asks help it. It ran once, three days after release, so we do not know how much its score moves between runs, and Anthropic will keep changing it. We also tried Anthropic’s corporate legal plugin on the first three questions; it was about as accurate as a Claude chat and took nearly twice as long, with each follow-up reading all 99 contracts again, so we stopped there. All four systems were run in October 2026.
If you are interested in learning more about AeroSpark, create a free account and point AeroSpark at your folder at aerospark.ai or contact us to learn more.