Aksita.ai Tutorials
Knowledge Base

Domain Knowledge Base

Automatically crawl entire website for comprehensive knowledge base

What is Domain KB?

Domain Knowledge Base is the most powerful way to add knowledge base. System will automatically crawl entire website from the domain you provide, find all internal pages, and index them. Perfect for websites with lots of structured content.

Most Powerful

Crawl entire website automatically

Auto Discovery

Discover pages automatically

Can be Filtered

Include/exclude patterns

When to Use Domain KB?

Documentation Website

Complete documentation with many pages

E-commerce

Store website with many products

Help Center

Knowledge base or support website

Company Website

Corporate website with many pages

How to Create Domain KB

  1. 1
    Open Resources → Create Resource
    Resources Page

    Agent Resources Page

    Select Knowledge → Domain

  2. 2
    Enter Domain URL
    Domain Form

    Domain Input Form with Filters

    Domain homepage URL:

    https://example.com
  3. 3
    Add Filters (Optional)

    Limit pages to crawl with patterns

  4. 4
    Click "Discover URLs"
    Domain URL Discovery

    URL List from Domain Crawl

    System crawls and shows URL list

  5. 5
    Select Desired Domain URLs
    Select URLs from Domain

    Select Desired Domain URLs

    Check Domain URLs you want to index from the discovered list

  6. 6
    Review & Create
    Domain KB Created

    Domain KB Created Successfully

    Review selected URLs, give it a name, then click "Create"

    Knowledge base will be processed and ready to use in a few moments

Important Notes

SSR Only, No SPA Support

We can only read SSR (Server-Side Rendered) websites. JavaScript-based SPA (Single Page Application) like React, Vue, or Angular without SSR cannot be read.

Time-Consuming Process

Large crawls take time, depending on website size

Only Internal Links

Only links within same domain

Some Sites Block

Websites with anti-bot might fail to crawl

Tips & Best Practices

Use Filters for Efficiency

Filter to crawl only relevant sections

Test with Sitemap First

If sitemap available, test with Sitemap KB first

Monitor Credit Usage

Domain crawl can use many credits