---
schema_version: "1"
id: fb5b9bcd6a9ba9fcfa9d4bce90080a8bf09d028659a43d8ff6fd981488d5643a
title: Web Content (Crawler) Connector
collection: developer
version: current
locale: en-us
portal_url: https://docs.salesforce.com/developer/current/en-us/data-cloud/data-cloud-int/guides/webcontentcrawler/c360-a-web-content-crawler-connector.md
canonical_url: https://developer.salesforce.com/docs/data/data-cloud-int/guide/c360-a-web-content-crawler-connector.html
source_site: developer.salesforce.com
content_type: documentation
content_hash: 0c78c5d365970138a703bcb1299c5bd17f528879a2482615c264a6bc88f5ee76
products:
  - Data 360 Integration Guide
  - Data 360 Developer Documentation
---
# ![Web Content (Crawler)](https://developer.salesforce.com/media/data-cloud-int/icons/WebContentCrawler.svg '{"class": "image-icon-lg", "title": "Web Content (Crawler)"}') Web Content (Crawler) Connector

Use the Web Content connector to ingest your organization’s marketing, ecommerce, documentation, or other website content into Data 360. AI agents can use this content to provide reliable answers in their interactions with your users.

The Web Content connector provides two options to ingest website data.

- **Web Content (Sitemap)**: Ingest content through the website’s sitemap. Use this option if you have a sitemap. See the connector [documentation](https://developer.salesforce.com/docs/data/data-cloud-int/guide/c360-a-web-content-sitemap-connector.html)
- **Web Content (Crawler)**: Ingest content by crawling up to 8 levels of links on the website pages. Data 360 ingests content in crawling depth levels. If, for example, you start crawling at your home page URL and set the crawl depth to 1, Data 360 ingests the home page and its immediate links. If you set the crawl depth to 2, Data 360 ingests the home page, the pages it links to, and the pages those pages link to. Data 360 doesn't ingest links outside of the website’s domain.

**note:** It’s your obligation to ensure that you have the rights to the data collected using this feature. Salesforce disclaims all liability with respect to such data collected.

## Data Type(s)

- Unstructured

## Data Direction

The direction of available connection methods.

### Data In

Create a connection for ingesting data from external systems.

Supported connection method(s):

- Batch

## Government Cloud Compliance

- [FedRAMP High Only](https://help.salesforce.com/s/articleView?id=ind.government_cloud_compliance.htm&type=5)

## Connector Setup

Instructions for adding and configuring your connector.

1. [Prepare Your Web Content Connection (Crawler)](https://developer.salesforce.com/docs/data/data-cloud-int/guide/c360-a-prepare-your-web-crawler-content-connection.md)
2. [Set Up a Web Content Connection (Crawler)](https://developer.salesforce.com/docs/data/data-cloud-int/guide/c360-a-set-up-a-web-content-connection-crawler.md)
3. [Create an Unstructured Data Connection from the Web Content (Crawler)](https://developer.salesforce.com/docs/data/data-cloud-int/guide/c360-a-create-crawler-udlo.md)
4. [Web Content (Crawler) Connector Troubleshooting Guide](https://developer.salesforce.com/docs/data/data-cloud-int/guide/c360-a-crawler-troubleshooting-guide.md)
5. [Common Web Content (Crawler) Connector Errors and Solutions](https://developer.salesforce.com/docs/data/data-cloud-int/guide/c360-a-common-web-content-crawler-errors-and-solutions.md)
6. [Web Content (Crawler) Connector Limitations](https://developer.salesforce.com/docs/data/data-cloud-int/guide/c360-a-web-content-crawler-connector-limitations.md)