by Safiyya Vankalwala

AI doesn’t just hallucinate case law, it hallucinates website domains too.

In a new and growing form of digital attack, hackers use AI agents to query search engines to get the most common hallucinated sites. A bad actor then registers the hallucinated domain and creates a convincing website at that address. The next time that an AI search recommends the domain, users are sent to a phishing website instead of the legitimate site, which can be used to steal personal information, share malware, and undermine system security.

Unfortunately, these are not isolated incidents, researchers have already identified more than 13,000 malicious sites, and over 250,000 potential sites that hackers could claim.

Why It Works

Lawyers, staff, and clients increasingly use AI to identify official websites, locate forms, and conduct research.

If the chosen AI hallucinates a plausible-looking domain and an attacker squats on it, future users could be sent directly to a phishing site. Unlike the fake websites of old, these domains are AI-generated, professionally designed, and very difficult to distinguish from legitimate sites.

Ways to Avoid Getting Burned

  • Treat AI-generated links like unverified sources. Until independently confirmed, assume any URL generated by AI could be wrong. When an AI provides a website address, look for independent confirmation by checking whether the organization links to the site from its official materials, whether the URL appears in official correspondence, and whether it is cited by multiple trusted sources.
  • Use trusted routes that cut out AI. Eliminate AI from your search results (there are several YouTube videos on how to do this). Rather than clicking an AI-generated link, go to the organization’s main website through a trusted source and navigate from there.
  • Approach new domains with skepticism. Newly registered domains often lack a long history and may be part of phishing operations. If a website appears unfamiliar or recently created, proceed cautiously.
  • Train staff on AI hallucinations. Add hallucinated websites to your cybersecurity awareness training and AI training.
  • Use bookmarks for critical resources. For government agencies, legal research platforms, practice management systems, and client portals, save verified bookmarks rather than repeatedly searching AI tools for links.
  • Stay vigilant about your own brand. There is an organized and aggressive effort by criminals to impersonate lawyers and their firms. We need to take precautions and maintain vigilance around our professional reputation and brand. At a minimum, set up Google Alerts for your lawyers, firm, and website, so you can monitor what is being said about you and whether misrepresentative material may exist online.

Original article written by Charity Anastasio, Interim Director, AILA Practice & Professionalism Center. Updated by Safiyya Vankalwala, PracticePRO Manager & Counsel, and republished with permission from the American Immigration Lawyers Association. All rights reserved.