# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
This site has an agentic ask interface you may use to query the documentation dynamically by asking a question.

Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters:

```
GET https://docs.direqt-search.com/index.md?ask=<question>&goal=<user_goal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

---

# Welcome

Direqt has all the tools you need to create and deploy a custom AI Search experience for your website.

Head to the [Quick-Start](/getting-started/quick-start) guide to launch your new site search in minutes.

### Discover Direqt

{% content-ref url="/pages/swpoKO3EMF2zivf8htZU" %}
[Search Results Page](/result-layouts/search-results-page)
{% endcontent-ref %}

{% content-ref url="/pages/9GYufc4IRZ9YyV1QRdxa" %}
[Overlay](/result-layouts/overlay)
{% endcontent-ref %}

{% content-ref url="/pages/aRTApMGrfoukbpmXk7Sn" %}
[Style Customization](/customization/style-customization)
{% endcontent-ref %}

{% content-ref url="/pages/eMJ4LZ6KBCegLnlRf7Lo" %}
[Indexing FAQs](/content-indexing/indexing-faqs)
{% endcontent-ref %}


# Quick-Start

Get up and running with AI Search in minutes

You've just signed up for Direqt. You might be wondering, what do I do now? How do I get my new search running on my site?

You've come to the right place. Follow the steps below to launch your new search experience.

***

{% stepper %}
{% step %}

### Create a Search Engine

On the **My Search Engines** page of the [dashboard](https://dashboard.direqt.ai/login), click "Create Search Engine". On the next page, enter the URL of the site you want to run search on. We'll immediately begin crawling your website to discover its webpages and build a searchable index.&#x20;

{% hint style="info" %}
The indexing process can take several hours to fully complete, but partial search functionality is typically ready for basic testing within 10-15 minutes.
{% endhint %}
{% endstep %}

{% step %}

### Test Some Queries

After 10-15 minutes, select "Search Preview" from the side bar on the left. Try testing a few search queries! If your website's pages are indexable, you should see some results. Remember, if you have a large website you may not be able to access all of its pages through the search engine right away.

{% hint style="info" %}
If the search engine cannot find any results after 30 minutes of indexing, consider some of [these best practices](/content-indexing/indexing-best-practices) to improve your site's search engine optimization.
{% endhint %}
{% endstep %}

{% step %}

### Configure your Search Experience

Navigate to the "Installation" page from the side bar. Here you can specify how you want to integrate Direqt with your website.

The first option allows you to choose whether you want to embed a Direqt search bar on your website, or integrate Direqt into an existing search bar on your website.

For more information, check out these links:

[Embedding the Direqt Search Bar](/search-bars/embedding-the-direqt-search-bar)[Integrating Your Search Bar with Direqt](/search-bars/integrating-your-search-bar-with-direqt)

Next, for the purposes of this quick-start, select "Dedicated search page" for your search layout. This means your search results will appear as a pop-up on your website. You can configure your search results to be on a dedicated page later.
{% endstep %}

{% step %}

### Add Your Embed Script

Click "Get My Installation Instructions" at the bottom of the installation page. This will give you a personalized embed script to add Direqt to your webpage. It will look something like this:

```html
<script type="module" src="https://embed.direqt-search.com/embed.js?id=YOUR_BOT_ID" async>"></script>
```

Copy and paste your unique script into either the `<head>` or `<body>` section of your webpage's HTML file.&#x20;

{% hint style="danger" %}
This snippet varies from project to project, so make sure you use your *own* embed script from the dashboard rather than paste the one from this guide to your website.
{% endhint %}
{% endstep %}

{% step %}

### Install Your Search Bar of Choice

Based on the search bar you chose in step 3, follow the instructions below to install it on your site

{% tabs %}
{% tab title="My site already has a search bar" %}
Add a new CSS class of `direqt-search-input` to your search bar. This will look something like this:

| Existing                                   | New                                                                    |
| ------------------------------------------ | ---------------------------------------------------------------------- |
| `<input type="text" placeholder="Search">` | `<input type="text" placeholder="Search" class="direqt-search-input">` |

Your search bar's appearance will remain unaltered, but Direqt will now handle the results.
{% endtab %}

{% tab title="My site needs a search bar" %}
You can add Direqt's pre-built search bar to your site.

Simply place the element below into your site's HTML wherever you want the search bar to appear:

```html
<direqt-search></direqt-search>
```

{% endtab %}
{% endtabs %}
{% endstep %}
{% endstepper %}

{% hint style="success" %}
A Direqt overlay window should now pop up when you submit a query in the search bar. Congratulations, you have successfully integrated Direqt search! :tada:
{% endhint %}

To further customize the look and behavior of AI Search on your site, check out our [Style Customization](/customization/style-customization) and [Results Layout](/customization/results-layout) options.


# Results Layout

Selecting the right search results layout for your website

You can choose between displaying results in an overlay window or on a standalone search results page.&#x20;

***

### Overlay

Display search results in a pop-up window in the browser that dims everything behind it. This option is ideal if you want users to quickly preview search results without leaving the page they’re on.

<figure><img src="https://592969535-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8Uav4TSGmA5CSlyCq4is%2Fuploads%2F9HkSjf0ira21dUaPd2SS%2FGroup%202.png?alt=media&amp;token=12deae07-dd7f-44c4-897e-cf7a6b9258e1" alt="" width="223"><figcaption><p>Sample overlay results window</p></figcaption></figure>

To install AI Search in overlay layout, refer to the [Overlay](/result-layouts/overlay) installation guide.

### Search Results Page

Display search results on a dedicated page on your website, e.g., `https://example.com/search`. All search queries will be redirected to your chosen URL, with results displayed inline on the page.&#x20;

<figure><img src="https://592969535-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8Uav4TSGmA5CSlyCq4is%2Fuploads%2F2R0cvev9eZ3miuKqRrSo%2Fimage.png?alt=media&amp;token=a7ec029f-24c7-41a9-b66a-64bb8369603f" alt="" width="563"><figcaption><p>Sample search results page</p></figcaption></figure>

To install AI Search on a dedicated page on your website, refer to the [Search Results Page](/result-layouts/search-results-page) guide.


# Style Customization

Customizing the look and feel of the AI Search interface on your site

On the **Style** page of the [dashboard](https://dashboard.direqt.ai/login), you can customize fonts, colors, and branding to match your site's design.

### Font Customization

**Font Family**: Select a font from the Font Family dropdown to match your brand.

**Font Color**: Set text color using the color dropper or enter a custom color code.

### Placeholder Customization

**Search Placeholder**: The text displayed in the search input before the user types (e.g., *"Search for articles..."*).

**Follow-Up Placeholder**: The prompt in the search input after a user submits a query (e.g., *"Ask a follow-up question..."*).

### Hyperlink Color

Adjust the color of links and clickable elements.

### Background Color

Adjust the background color of the overlay pop-up window.

{% hint style="info" %}
If you're installing AI Search on a search results page, no need to set background color - the Direqt interface has a transparent background to blend seamlessly into the page.&#x20;
{% endhint %}

### Branding <a href="#brand-logo" id="brand-logo"></a>

Customize how your brand is represented in the search interface.&#x20;

**Brand Logo**: Upload your brand's logo in .jpg, .jpeg, or .png format via the **Logo** field. This image will be displayed in the top-left corner of the interface.

{% hint style="info" %}
Horizonal logos with bold text work best.&#x20;
{% endhint %}

**Brand Name:** If no logo is uploaded, your brand name will appear in the top-left corner. By default, your brand name is set to 'Ask \[Your Website Name]'. You can edit this text in the **Brand Name** field.

### Examples

The table below shows two different AI Search interface customizations, along with the style settings used for each.

| **Example 1: Light Background, Brand Name**<img src="https://592969535-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8Uav4TSGmA5CSlyCq4is%2Fuploads%2FJQwP2aH9xtXWmV9NJwa4%2Fimage.png?alt=media&amp;token=b8ecbc58-8c66-4d67-82c6-9b17e1949d99" alt="" data-size="original"> | <p><strong>Brand Name</strong> - "NRO"</p><p><strong>Follow-Up Placeholder</strong> - "Search"</p><p><strong>Font</strong> - <code>Consolas</code></p><p><strong>Hyperlink Color</strong> - <code>#1e90ff</code></p><p><strong>Background Color</strong> - <code>#ffffff</code></p>                                                                                                                                                                          |
| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Example 2: Dark Background, Brand Logo**<img src="https://592969535-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F8Uav4TSGmA5CSlyCq4is%2Fuploads%2FuZ4H4uUXedGntWv8rtIe%2Fimage.png?alt=media&amp;token=c2bac069-8406-4bcd-99c9-4d43ac9ae2b4" alt="" data-size="original">  | <p></p><p><strong>Brand Logo</strong> - <img src="https://docs.direqt.ai/img/customizationExamples/logo.png" alt="Example" data-size="original"></p><p><strong>Follow-Up Placeholder</strong> - "Ask Me Anything"</p><p><strong>Font</strong> - <code>Arial</code></p><p><strong>Font Color</strong> - <code>#ffffff</code></p><p><strong>Hyperlink Color</strong> - <code>#0092ff</code></p><p><strong>Background Color</strong> - <code>#303030</code></p> |


# Search Results Page

Installing AI Search on a dedicated search page on your site

This guide will show you how to integrate AI Search onto a dedicated search results page on your site, e.g., `https://example.com/search`. Direqt will automatically redirect user queries to your specified URL and display search results on that page.

***

{% hint style="success" %}
You should now see a search bar and a list of suggested searches on your site's search page.
{% endhint %}

### **(Optional) Using Your Own Search Bar**

If you have an existing search bar on your site you'd like to use in conjunction with your search results page, refer to the guide on [Integrating Your Search Bar with Direqt](/search-bars/integrating-your-search-bar-with-direqt).

### **(Optional) Adding Additional Search Bars**

Your search results page will have a Direqt-provided search bar at the top of the page. If you wish to add an additional Direqt search bar to any other location on your site, e.g., your site header, refer to the guide on [Embedding the Direqt Search Bar](/search-bars/embedding-the-direqt-search-bar).


# Overlay

Display search results in a pop-up window

This guide will show you how to install AI Search in overlay layout. Search results will be displayed in a Direqt-provided pop-up window within the browser that dims everything behind it.&#x20;

***

{% hint style="success" %}
If successful, the Direqt overlay window should pop up when you enter a search query in your search bar.
{% endhint %}


# Integrating Your Search Bar with Direqt

How to run AI Search with your pre-existing search bar

You might already have a search bar on your site, or maybe you'd like to build a custom one for use with AI Search. Direqt can integrate with any search bar.

***

Add the CSS class below to your existing search input to integrate it with Direqt.&#x20;

```css
direqt-search-input
```

For example:

| Existing                                   | New                                                                    |
| ------------------------------------------ | ---------------------------------------------------------------------- |
| `<input type="text" placeholder="Search">` | `<input type="text" placeholder="Search" class="direqt-search-input">` |

{% hint style="success" %}
Your search bar placeholder text should now read "Ask \[Site Name]".  You can customize this text on the **Style** page of the dashboard.
{% endhint %}

Direqt will automatically detect this class and respond to all inputs from your search bar.


# Embedding the Direqt Search Bar

Adding the Direqt search bar to your website

Don't have a search bar on your site yet? Not a problem! Direqt has a pre-built search bar widget you can add to your site with a single line of code.

***

### Installation

Place the element below into your site's HTML wherever you want the search bar to appear:

```html
<direqt-search-bar></direqt-search-bar>
```

{% hint style="success" %}
A search bar with the placeholder text  "Ask \[Site Name]" should now appear on your page.&#x20;
{% endhint %}

### Customization

You can customize the font family, font color, and placeholder text of the Direqt search bar on the **Style** page of the dashboard.


# Synonyms

Handling synonyms and acronyms in your search results

Synonyms allow you to define equivalent terms in your content to accommodate alternate ways people might describe an item or concept. Adding synonyms to your search engine's knowledge base helps improve relevance and content discoverability.

### How to Use Synonyms

#### Adding Synonyms

1. Go to the **Synonyms** page in the [dashboard](https://dashboard.direqt.ai/login).
2. Enter the terms you want to associate with each other.
3. Click "Save".&#x20;

#### Deleting Synonyms

Click on the "X" button next to the pair of terms. Removing a synonym mapping will revert search behavior to its original state.

### Examples

#### Formal vs. Informal Language

Many users search with casual terms, while your content might use more formal language.&#x20;

* "Heart attack" ↔ "Myocardial infarction"

#### Acronyms vs. Full Names

Industry terms often have both an acronym and a full name.&#x20;

* "1PD" ↔ "First Party Data"


# Indexing FAQs

How Direqt uses your content to power your search functionality

### What's an index?

An index is a database of all the searchable content on your website.&#x20;

### How do you index my content?

Our crawler visits each page that is accessible from the URL you provide and extracts body text and metadata.  The collected data is structured and stored in the search index. The index updates automatically at regular intervals to keep your content current.

### What content do you index?

We can index the content of the following file types:

* HTML Pages
* Images
* PDF Documents *(Enterprise Plans Only)*&#x20;

For each indexed URL, we attempt to extract key fields such as:

* Title&#x20;
* Description
* Author
* Publish Date&#x20;
* Body Text&#x20;

### What content is not indexed?

Some content on your site may not be included in your search index:

* Blocked Pages – If your site’s server permissions or redirects prevent access, those pages won’t be indexed.
* Protected Content – Pages behind logins, paywalls, or authentication barriers cannot be indexed.
* External Domains & Subdomains – We only index the domain you provide. For example, if you create an engine for `example.com`, pages on `blog.example.com` will **not** be included.

### How can I improve the quality of my index?

Refer to the [Indexing Best Practices](/content-indexing/indexing-best-practices) guide for tips on optimizing your site index.


# Indexing Best Practices

To ensure we can successfully index your site and provide the best possible search experience, follow these best practices to optimize your website's structure and accessibility.

### Host a Sitemap at Your Site's Root

A sitemap helps us efficiently locate and index your content.

**Best Practices:**

* Create an **XML** sitemap following the [Sitemaps.org protocol](https://www.sitemaps.org/).
* Post the sitemap at your site root (e.g., `https://yourdomain.com/sitemap.xml`) for easy discovery.
* Verify that your sitemap is publicly accessible by opening it in a browser.
* Ensure your sitemap is referenced in `robots.txt`

### Ensure URLs Are Publicly Accessible

If your site blocks access, we won’t be able to index it.

**Best Practices:**

* Test your URLs in Incognito Mode to ensure they load without login credentials
* Ensure no firewall, VPN, or bot protection is blocking access to our crawler
* Remove `noindex` meta tags from pages that should be indexed

### Avoid Redirect Loops

If a page redirects too many times, we may not be able to follow it.

**Best Practices:**

* Ensure URLs resolve within one or two redirects (e.g., `http → https`, `www → non-www`)
* Avoid redirect loops (`URL A → URL B → URL A`)


# Indexing Troubleshooting

Common errors related to the indexing process and how to resolve them

### No Sitemap Found

This error occurs when no valid sitemap is detected at the expected location. A valid sitemap at your site root is recommended for optimal search performance.

#### **Troubleshooting Steps**

1. **Create a Sitemap**

   If your site doesn't have a sitemap, generate a sitemap in XML format following the [Sitemaps.org protocol](https://www.sitemaps.org/).&#x20;
2. **Place the Sitemap at Your Site Root**

   Ensure your sitemap is posted at your site root  (e.g., `https://yourdomain.com/sitemap.xml`)

### Invalid URL

This error occurs when the provided URL is incorrectly formatted, inaccessible, or blocked.

#### **Troubleshooting Steps**

1. **Check the URL Format**

Common issues:

* The URL is missing `https://` or `http://` (e.g., `example.com/page` instead of `https://example.com/page`).
* The URL contains spaces, special characters, or encoding errors.
* The URL is a relative path instead of a full link (e.g., `/page` instead of `https://example.com/page`).

2. **Verify URL Accessibility**

Common issues:

* The URL returns a 404 (Not Found) or 403 (Forbidden) error.
* The page is behind a login screen or restricted to specific users.
* The website is temporarily down&#x20;
* The page has a `noindex` meta tag preventing search engines from accessing it.
* The website has firewall or security settings blocking our crawler.


# Direqt Crawlers & User Agents

Direqt uses web crawlers (“robots”) and user agents to perform actions for its products, either automatically or triggered by user request. Direqt uses the following `robots.txt` tags to enable webmasters to manage how their sites and content work with our product.

> **User agent headers**\
> `Anomura/1.2 (+https://www.direqt.ai)`&#x20;
>
> `Anomura/1.2 (+https://www.direqt-search.com)`
>
> **User agent token**\
> `Anomura`

***

### Overview

**Anomura** is for search. Anomura is used to link to and surface websites in search results in Direqt’s search features. It is **not** used to crawl content to train any models.

To help ensure your site appears in search results, we recommend allowing Anomura in your site’s `robots.txt` file **and** allowing requests from our published IP ranges below.

> **Note:** The full user-agent string will contain either `Anomura/1.2 (+https://www.direqt.ai)` or
>
> `Anomura/1.2 (+https://www.direqt-search.com)`

***

### User agents

| Agent   | Token     | Full user-agent strings                                                                                                      | Purpose                                                                                            |
| ------- | --------- | ---------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------- |
| Anomura | `Anomura` | <p><code>Anomura/1.2 (+<https://www.direqt.ai>)</code></p><p><code>Anomura/1.2 (+<https://www.direqt-search.com>)</code></p> | Direqt search crawler. Discovers links/metadata to surface in search. Not used for model training. |

***

### Control access with `robots.txt`

By default, Anomura follows standard `robots.txt` directives addressed to the `Anomura` token.

#### Allow Anomura everywhere

```ini
User-agent: Anomura
Allow: /
```

#### Block Anomura completely

```ini
User-agent: Anomura
Disallow: /
```

> Blocking Anomura means your content will not appear in Direqt search features.

#### Allow Anomura but block a specific path

```ini
User-agent: Anomura
Disallow: /private/
```

#### Allow Anomura while blocking all other crawlers

```ini
# robots.txt
User-agent: *
Disallow: /

User-agent: Anomura
Allow: /
```

***

### Source IP ranges

Allow traffic from the following IPs to ensure Anomura can access your site when permitted by your `robots.txt` and server configuration.

* **Published IP addresses:**&#x20;
  * `34.56.242.161`      (Hostnam&#x65;**:** `crawler.direqt-search.com`)
  * `35.224.164.246`    (Hostnam&#x65;**:** `crawler.direqt.ai`)

The IP addresses used by the crawler are also listed [here](https://www.direqt.ai/crawler.json).

***

### FAQ

**Does Anomura train Direqt models?**\
No. Anomura is only used to discover and surface links and metadata for Direqt search features. Content fetched by Anomura is not used for model training.

**Does Anomura respect `robots.txt`?**\
Yes. Anomura follows standard `robots.txt` directives for the `Anomura` user agent token.

**Will you add more IPs?**\
If additional IPs or ranges are published, we’ll update this page. Consider allowing by both user-agent and IP where possible.