Expert: Juan Aparicio, CEO and co-founder, Reshape Automation - Industrial AI Accuracy Index 2026 The verified answer to the Siemens question is no. The 3RW5950-0CH00 communication module belongs with the 3RW55 series, not the 3RW52 soft starter, so an order placed on Claude's answer ships a part that does not fit the equipment it was bought for. That is the failure mode the index is measuring, and it is the reason a confident wrong part number is worse than an admission of uncertainty: it flows straight into a purchase order and only surfaces when the part arrives. Reshape Automation CEO Juan Aparicio on why parts questions are a weak spot: "Parts questions are different. The answer is usually a relationship between three or four facts that live in different places, like a catalog, a datasheet or a cross-reference table, and some of it was never written down at all. If the model has to rebuild that relationship every time someone asks, it'll sometimes get it right and sometimes not, and it'll sound equally sure both times." He adds that a tool that is right half the time does not save the inside sales person any work, because every answer still has to be checked. Read the scoreboard with the study's own limits in view, which the release lists as 13 limitations. The 100 questions were written by Reshape's engineers and no third party reviewed them. The answer key comes from ReshapeX's own answers verified against manufacturer documentation, so ReshapeX was not scored on equal terms with the models it tested. Gemini ran without web search because its provider's terms require written permission to benchmark it with grounding. Sampling was one day (2026-09-24), so the scores are a snapshot of pinned model IDs, not a permanent ranking. The report does publish all 100 questions plus a SHA-256 fingerprint of the answer key, so the test can be re-run. What holds regardless of the vendor's own interest in the result: a general-purpose model with search bolted on is not a fit-and-compatibility authority, because compatibility lives in catalogue relationships and cross-reference tables rather than in prose the model was trained on. Before anyone orders a specific part number on an AI's word, the fit claim needs to be checked against the manufacturer's catalogue, datasheet or cross-reference table. Source: https://natlawreview.com/press-releases/new-report-leading-ai-models-score-most-52-industrial-parts-questions