I asked the app for a weed- and pest-control plan for my 150 mu of sesame. It generated a full programme - "Hundred Acres of Sesame Aerial Spraying: Weed Control + Pest Control Complete Plan" - naming two herbicides, haloxyfop-P-methyl and fomesafen, plus the insecticides thiamethoxam water-dispersible granules and emamectin benzoate. It was laid out as a complete, ready-to-use recipe. Nothing in it said that fomesafen is a broadleaf-weed herbicide for soybean fields, that sesame is itself a broadleaf crop, or that fomesafen is meant for directed application to the weeds rather than a whole-field spray. I had the whole field sprayed by drone. The next day the weeds and the sesame seedlings were both dying - the seedlings faster. When I went back and asked the AI to explain what had happened, it told me the fomesafen in its own recipe was what had killed the crop.
1 Answer
Expert: Agricultural technicians and a pesticide supplier in Chuzhou, Anhui (cited by Litchi News; reported in English by The Economic Times), Agronomy sources who checked the AI's recipe against the chemical's registered use Fomesafen (氟磺胺草醚) is registered as a broadleaf-weed herbicide for soybean fields. Sesame is a broadleaf crop. That pairing is not a borderline judgement call - it is a crop-killer. Wu, a 67-year-old farmer in Chuzhou, Anhui, had the recipe sprayed across his whole sesame field by drone on 10 July 2026; within 24 hours the seedlings across 150 mu - about 24.7 acres - had wilted, with the weeds and the crop dying together and the sesame going first. Wu's own estimate of the loss was around 150,000 yuan, roughly US$21,000, from a single recommendation he had no way to audit. The people who actually know the chemical were unambiguous. A pesticide retailer near Wu's farm told Litchi News that fomesafen is intended for soybean fields and cannot be sprayed on sesame at all. Agricultural technicians put the narrower point: some other crops can be treated with fomesafen, but only by directed spraying of the target weeds - "if you spray it across the whole field, it will definitely kill the crop." Nothing in the plan pointed at any of those constraints. The AI's plan was not vague; it was specific about four products and said nothing about crop compatibility, registered use, or the difference between directed and broadcast application - the three points that decided the outcome. A plan that reads like a validated protocol carries more risk than a hedged answer, because it gives the user nothing to check against. The second failure came after the damage. Asked to explain the dead seedlings, the app pointed to fomesafen - its own headline ingredient - as the likely cause. It had the information all along and did not surface it when it mattered; the reversal came only once the field was destroyed. The app's customer service then said the product has no independent knowledge base of its own and assembles answers from public information found online, and that the company would investigate which platform or source the answer came from. That is the architecture behind the mistake: synthesis of web text, with no grounding in the pesticide label or registration data, and no context about the crop, its growth stage, the tank mix, or the field. The trust that made this possible accumulated slowly. Wu was sceptical at first, then used the app for more than a year, asking it everything - when to spray, when to fertilise - and found the earlier answers "fairly complete." That record is precisely what stopped him checking. Past usefulness is not evidence that the next answer is valid, and the answer that finally failed carried consequences that could not be undone. A one-line caveat sat at the top of the chat window - "AI-generated content may be incorrect, please verify" - and Wu says he never noticed it. Generic boilerplate is not a substitute for flagging the specific, irreversible hazards in a chemical recommendation, and the burden of that check should not rest on a user who has no way to evaluate it. For anyone using these tools on the physical world: pesticide and herbicide decisions have to be confirmed against the current product label and with an extension service or registered agronomist before application - crop compatibility, registered use, and application method. Treat a confident, well-formatted plan as a draft, not a prescription. Where an error destroys a season in a day, the decision needs a person accountable for it, and a chat interface that sounds certain is not that person. Source: https://economictimes.indiatimes.com/news/new-updates/chinese-farmer-trusted-ai-advice-successfuly-for-a-year-then-one-wrong-suggestion-destroyed-his-25-acre-crop-in-just-24-hours/articleshow/133146500.cms
Your answer
Sign in to verify this AI response.
Don't trust us — or the AI. Ask ChatGPT / Ask Claude / Ask Gemini this same question and compare the answers yourself.
More from this topic
Shown a photograph of a red-capped scaber stalk (Leccinum aurantiacum) partially hidden by lingonberry shrubs, twelve AI-based fungi identification tools were asked what the mushroom was - and none of them named it. Copilot answered Boletus reticulatus. Champignouf answered Psathyrella candolleana. Yandex recognised only the lingonberry leaves. Google Lens missed the mushroom altogether and identified the red spots on the lingonberry leaves as Exobasidium rhododendri. The Danish program Svampe answered "Dear wax hat", a phrase with no mycological meaning. Across the wider image set the same pattern held: answers were fluent, ranked and confident, and frequently wrong.
Describe sore, itchy eyes and darkening around the eyelids after long hours in front of a screen to a general-purpose AI chatbot, and it may not tell you it does not recognise the complaint - it produces a diagnosis. Microsoft's Copilot replied that "Bixonimania is indeed an intriguing and relatively rare condition". Google's Gemini explained that "Bixonimania is a condition caused by excessive exposure to blue light". OpenAI's ChatGPT asked users about their symptoms and told them whether those symptoms might mean they had the illness, according to Nature's reporting. Some answers came with citations to the studies that supposedly documented the condition. Bixonimania does not exist. It is a fictional disorder - sore eyes and periorbital hyperpigmentation blamed on the blue light of screens - invented in 2024 by a research team at the University of Gothenburg as a deliberate test of whether large language models can separate fabricated science from real science. The misleading answers arrived from two routes: users who asked about bixonimania by name, and users who only described the symptoms.
Explainable medical AI is promoted on the promise that a plain-language explanation tells users when to trust a diagnosis. In a controlled skin-disease study, the large language model explanations did the opposite for the people who relied on them most: non-experts trusted LLM explanations whether they were right or wrong, were measurably more confident in their wrong answers when an LLM explanation was attached, and rated explanations as more convincing when they were vague or generic. Non-expert accuracy rose mainly because users deferred to the model, and that deference hurt more when the model was wrong than it helped when the model was right. Clinicians, by contrast, were not tripped up by incorrect AI explanations and did best with a bare prediction and no explanation at all.