AI crawlers matter to docs teams because they determine whether answer engines can see the documentation at all. A blocked crawler or a page that requires JavaScript can prevent a tool from retrieving the documentation. Test access and parsing for the tools that matter to your readers.
There are two distinct kinds: training crawlers, which collect content for future model training, and retrieval crawlers, which fetch pages in real time to answer a live user question. Sites can allow one and block the other in robots.txt, since each uses its own user agent.
JavaScript support varies between crawlers. Serve important documentation in accessible HTML or a documented text format, then check what each crawler can retrieve. Readable content alone does not guarantee that an engine will cite it.