GDPx: choose AI with a task you can check

Begin with a result you understand

Before choosing an AI tool, pick one task whose result you can check. Write down what must be correct, which information it may use and who will review the work. Keep that example when you compare different tools.

SemiAnalysis introduced GDPx on November 2. It is a benchmark, a shared collection of tests used to compare AI systems. Its public dashboard covers 206 tasks across 11 specialist fields, developed with 14 specialist partners.

The tests examine work that involves gathering information, using tools and producing a finished result. That makes the launch relevant to anyone wondering how much of a real task an AI helper can take on.

Read the announcement: SemiAnalysis: GDPx launch and methodology.

Read the score with its conditions

SemiAnalysis reports a highest overall score of 12% in this first release. That percentage belongs to this demanding collection of tasks and its grading rules. It cannot tell you how often your own everyday task will succeed.

The published method combines results across three attempts for its headline measure and gives each specialist field equal weight. Some tasks use a pass-or-fail check; others award partial credit. The result also depends on the tools, instructions and time available to the AI.

GDPx is one research firm's selected test, with its own priorities. It is useful evidence to explore alongside other evaluations and your own checks. The public dashboard lets you look at differences between fields and compare cost as well as performance.

Make a small test for your own tool

For a practice exercise, prepare a few made-up customer requests and ask an AI helper to turn them into a clear list for a colleague. Decide in advance which details must stay accurate, what to do with an unclear request and whether the helper may change any records.

Try the same information and instructions with each tool you are considering. Inspect the list yourself. Notice whether the helper kept the important details, showed uncertainty and stayed within the job you gave it.

This is our suggested learning exercise, separate from the GDPx tests. A polished answer can still need correction, so include the time a person spends reviewing and fixing the draft when you compare results.

1. Choose one repeated task and write the checks a good result must pass.
2. Use made-up or approved information, including an incomplete example.
3. Give each tool the same task and keep the outputs.
4. Check the result, the usage cost and the work a reviewer still needs to do.
5. Keep the examples that helped you explain which tool fits the job.

Bring evidence to a building decision

If you are commissioning an app, ask the builder to show a sample of the work your tool will handle and explain its limits. Agree where someone reviews a result, how mistakes are corrected and which tasks should remain with a person.

Our perspective is that a useful comparison leaves you understanding the choice. Benchmark results can help you ask better questions; your own examples help answer them. Bring those examples to a training or planning session so the next step is connected to work you need to improve.