Product

Customers

Company

Resources

Pricing

Login

Demo

New

Benchmark · October 2026

Tax research, tested.

Tax research, tested.

Tax research, tested.

How specialist AI compares with the leading AI labs on Australian tax. Six systems, 91 questions under current law, every answer graded against an expert reference answer.

How specialist AI compares with the leading AI labs on Australian tax. Six systems, 91 questions under current law, every answer graded against an expert reference answer.

Download the report

Download the report

How we tested

How we tested

Overall score by system

Mean of 91 questions, scale 1 to 5. Weighted average of the five dimensions.

Nylon
4.68
Claude Fable 5.1
4.32
GPT-6 Astra
4.28
Claude Opus 5.5
4.07
GPT-6 Sol
3.96
Gemini 3.1 Pro
2.97

Summary

Nylon scored highest of the six systems tested.

Nylon scored highest of the six systems tested.

Tax advisers are adopting AI research tools quickly, but have had no common method of comparing them. To our knowledge there is no established benchmark for AI tax research in Australia or New Zealand, so we created one.

Tax advisers are adopting AI research tools quickly, but have had no common method of comparing them. To our knowledge there is no established benchmark for AI tax research in Australia or New Zealand, so we created one.

Nylon scored a composite of 4.68 out of 5, against 4.32 for Claude Fable 5.1 and 4.28 for GPT-6 Astra. Its lead over every other system is statistically significant on the composite and on correctness, faithfulness, helpfulness and citation grounding.

Nylon scored a composite of 4.68 out of 5, against 4.32 for Claude Fable 5.1 and 4.28 for GPT-6 Astra. Its lead over every other system is statistically significant on the composite and on correctness, faithfulness, helpfulness and citation grounding.

Read the full write-up on our blog →

Right more often, and easier to verify.

Right more often, and easier to verify.

4.46

4.46

4.46

/5

Highest on correctness

Highest on correctness

The best correctness score of the six systems, against 4.20 for Claude Fable 5.1. The lead is statistically significant against every system.

The best correctness score of the six systems, against 4.20 for Claude Fable 5.1. The lead is statistically significant against every system.

+0.35

+0.35

+0.35

Lead on the next best

Lead on the next best

Composite lead over Claude Fable 5.1, rising to +1.71 over Gemini 3.1 Pro. Statistically significant against every system.

Composite lead over Claude Fable 5.1, rising to +1.71 over Gemini 3.1 Pro. Statistically significant against every system.

135s

135s

135s

Predictable speed

Predictable speed

Nine in ten answers complete, joint-fastest with Gemini 3.1 Pro, against 210 seconds for Fable and 201 for Astra.

Nine in ten answers complete, joint-fastest with Gemini 3.1 Pro, against 210 seconds for Fable and 201 for Astra.

Head to head

Nylon’s score against each system. Positive means Nylon scored higher.

Nylon vsCorrect.Complete.Helpful.Faithful.GroundingComposite
Claude Fable 5.1+0.26+0.05+0.45+0.42+0.40+0.35
GPT-6 Astra+0.38+0.16+1.03+0.43+0.26+0.40
Claude Opus 5.5+0.66+0.31+0.65+0.73+0.55+0.60
GPT-6 Sol+0.68+0.47+1.37+0.78+0.56+0.72
Gemini 3.1 Pro+1.22+0.65+1.93+2.24+1.70+1.71

How we tested

The same questions, tools and grading for every system.

The same questions, tools and grading for every system.

91

Questions

From the 120 review questions in CPA Program Australia Taxation.

From the 120 review questions in CPA Program Australia Taxation.

6

Systems

Nylon's standard research service and five leading models, each run through its provider API with the same web search and page-reading tools.

Nylon's standard research service and five leading models, each run through its provider API with the same web search and page-reading tools.

5

Rubric dimensions

Citation grounding 35% · Faithfulness 30% · Correctness 15% · Completeness 10% · Helpfulness 10%. Graded blind by Kimi K3, a judge from a lab not in the test.

Read the full report

Method, results, discussion and appendices with every exclusion and the full rubric.

Download PDF

Book a demo

Report cover

Read the full report

Method, results, discussion and appendices with every exclusion and the full rubric.

Download PDF

Book a demo

Report cover

Read the full report

Method, results, discussion and appendices with every exclusion and the full rubric.

Download PDF

Book a demo

Report cover

Nylon is a suite of AI-powered tax & law tools for the modern professional.

Product

Overview

Agent

AI Connector

Meeting Notetaker

Customers

Accountants

Solicitors

Barristers

Tax Consultants

Company

Our Story

Our Team

Careers

Newsroom

Contact Us

Resources

Benchmark Report

AI Policy Builder

Blog

Security

Help Centre

Copyright © 2026 Cyborg Limited. All rights reserved.

Terms

Privacy

Support

Nylon is a suite of AI-powered tax & law tools for the modern professional.

Product

Overview

Agent

AI Connector

Meeting Notetaker

Customers

Accountants

Solicitors

Barristers

Tax Consultants

Company

Our Story

Our Team

Careers

Newsroom

Contact Us

Resources

Benchmark Report

AI Policy Builder

Blog

Security

Help Centre

Copyright © 2026 Cyborg Limited. All rights reserved.

Terms

Privacy

Support

Nylon is a suite of AI-powered tax & law tools for the modern professional.

Product

Overview

Agent

AI Connector

Meeting Notetaker

Customers

Accountants

Solicitors

Barristers

Tax Consultants

Company

Our Story

Our Team

Careers

Newsroom

Contact Us

Resources

Benchmark Report

AI Policy Builder

Blog

Security

Help Centre

Copyright © 2026 Cyborg Limited. All rights reserved.

Terms

Privacy

Support