---
schema_version: "1"
id: 383f2326d06b76ed124f3634a5798e98200a72f88a542eb5aa938e47e4322126
title: Prepare Your Web Content Connection (Crawler)
collection: developer
version: current
locale: en-us
portal_url: https://docs.salesforce.com/developer/current/en-us/data-cloud/data-cloud-int/guides/webcontentcrawler/c360-a-prepare-your-web-crawler-content-connection.md
canonical_url: https://developer.salesforce.com/docs/data/data-cloud-int/guide/c360-a-prepare-your-web-crawler-content-connection.html
source_site: developer.salesforce.com
content_type: documentation
content_hash: 2a90e941cd9ffaa3f878d4223e9485fe9f5832b86a3475cbcc1b60acd757e3a1
products:
  - Data 360 Integration Guide
  - Data 360 Developer Documentation
---
# Prepare Your Web Content Connection (Crawler)

To set up a website content connection with Data 360, you need to provide some details. Gather this information before you get started.

## Website Authentication Details

Prepare the authentication details for the website you want to ingest.

1. **Website**: Identify the website you want to ingest.
2. **Authentication Method**: Determine the website authentication method. If your website is accessible to anyone, you don’t need to provide any credentials. However, if your website uses basic authentication, get a username and password, and ensure that the user associated with the username has the necessary access to the content you want to ingest, as the connector gets the same access permissions as this user.
3. **User Agent**: If your website requires a specific user agent, prepare the required string.

## Ingested Website Content

Decide what content to include in the ingestion.

1. **Starting URL**: Decide on a starting URL for your website ingestion, for example, your site’s homepage: `https://acme.com`.
2. **Crawl Depth**: Decide on a number of page levels (1-8) from the starting URL to include in the ingestion. For example, if your starting URL is the website’s home page and the crawl depth is 1, Data 360 ingests the home page and its immediate links. If the crawl depth is 2, then the home page, the pages it links to, and the pages those pages link to are ingested. Data 360 doesn't ingest links outside of the website’s domain.
3. **Included Pages**: To ingest specific web pages, prepare a regular expression that matches the pages that you want to ingest. For example, if you enter ._/content/._, all pages under the starting URL that include /content/ in their URL path are included. If you don’t specify any included pages, all pages under the sitemap URL are included.

**Example**

Instead of manually listing every URL you want to include in the ingestion, you can use a regular expression (regex) that includes URLs with specific text or structure patterns.

- For URLs containing the text “support-content”, use the regular expression `.*support-content.*`. This includes any URLs containing the specified text, regardless of what comes before or after it.
- For URLs containing the path “/support/content/”, use the regular expression `.*\/support\/content\/.*`. This includes any URLs containing the specified path, regardless of what comes before or after it.

4. **Included Page Elements**: To ingest specific page elements, prepare a CSS selector to specify which HTML element to include. By default, the body page element, representing the content of the HTML page, is included.

**tip:** To extract a CSS selector, use your browser’s dev tools to inspect a page, find the HTML element you want to ingest, right-click it, and then from the menu, click **Copy** > **Copy selector**.

## Next Step

- [Set Up a Web Content Connection (Crawler)](https://developer.salesforce.com/docs/data/data-cloud-int/guide/c360-a-set-up-a-web-content-connection-crawler.md)