--- schema_version: "1" id: d46da10ccca8fed8786f91fc731f51dba1ffe4b645afecae9415496fe341af19 title: Set Up an Apache Iceberg File Federation Connection collection: developer version: current locale: en-us portal_url: https://docs.salesforce.com/developer/current/en-us/data-cloud/data-cloud-int/guides/apacheiceberg/c360-a-set-up-apacheiceberg-file-fed-connection.md canonical_url: https://developer.salesforce.com/docs/data/data-cloud-int/guide/c360-a-set-up-apacheiceberg-file-fed-connection.html source_site: developer.salesforce.com content_type: documentation content_hash: 10c0deb94ab52263d0a36e893381167df82e4dbd45a23225c812cf6569529486 products: - Data 360 Integration Guide - Data 360 Developer Documentation --- # Set Up an Apache Iceberg File Federation Connection Set up the Apache Iceberg File Federation connection to start the flow of data into Data 360. | User Permissions Needed |   | | ----------------------- | ----------------------------------------------------------- | | To create a connection: | System Admin profile or Data Cloud Architect permission set | **Requirements** - **Firewall**: If the Iceberg data lake is behind a network firewall, add these [Data 360 IP addresses](https://help.salesforce.com/s/articleView?id=data.c360_a_data_cloud_ip_address_allowlist.htm&type=5) to your access control list before configuring a connection. If the network firewalls that protect the server hosting the lake's metadata catalog and the lake's storage bucket are different, make sure to update both. Both the Open Catalog and the storage bucket must be publicly accessible. Data 360 doesn't support connecting over AWS PrivateLink or Azure Private Link. However, if the AWS S3 bucket is in the same AWS region as the Data 360 tenant, and a VPC gateway endpoint is provisioned for the bucket, then Data 360 can transparently use AWS PrivateLink to connect to the bucket. - **Catalog**: The [metadata catalog](https://iceberg.apache.org/terms/) (REST catalog) must both implement the REST OpenAPI specification and be publicly accessible. | **REST Catalog Endpoint** | **Description** | | -------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `/v1/config`(Required) | Get all configuration metadata. When you save a connection, Data 360 will attempt to make a request to `/v1/config`, and a 200 response must be received in order for the connection to be saved. See [REST Config Spec](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml#L65). | | `/v1/{prefix}/namespaces` (Required) | List all namespaces in the catalog. See [Namespaces Spec](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml#L240). Even if you do not support nested namespaces, ensure that a request to `/v1/{prefix}/namespaces?parent={some_namespace}` returns an empty list per the [SupportsNamespaces API](https://developer.salesforce.com/docs/data/data-cloud-int/guide/%3Chttps://iceberg.apache.org/javadoc/latest/org/apache/iceberg/catalog/SupportsNamespaces.html#listNamespaces(org.apache.iceberg.catalog.Namespace)>). | | `/v1/{prefix}/namespaces/{namespaces}` (Required) | Get metadata about the specified namespace. See [Namespace Metadata Spec](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml#L336). | | `/v1/{prefix}/namespaces/{namespaces}/tables` (Required) | List all tables in the namespace. See [Tables List Spec](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml#L506). | | `/v1/{prefix}/namespaces/{namespace}/tables/{table}`(Required) | Get metadata about the specified table. See [Table Metadata Spec](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml#L913). | | `/v1/oauth/tokens` (Optional) | When the access token you provide to Data 360 expires, Data 360 will exchange the expired access token for a new one by making a token-exchange request to `/v1/oauth/tokens`. The request must contain the required fields defined in [RFC 8693](https://www.rfc-editor.org/rfc/rfc8693.html). This endpoint is deprecated but still supported by some catalog implementations. See [Token Exchange Spec](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml?ref=chaosgenius.io#L203). If unsupported, you must manually or programmatically update the connection to use a new access token. | - **Storage**: The data must be formatted as Apache Parquet files that are organized as Iceberg V1 tables stored in either (1) a publicly-accessible AWS S3 bucket, (2) an AWS S3 bucket in a VPC that is the same region as the AWS region that the Data 360 org is in, (3) a publicly-accessible ADLS Gen2 container, or (4) a publicly-accessible Azure Blob Storage container. We do not support on-premise or GCP-hosted storage buckets. **Set Up the Connection** 1. In Data Cloud, click **Setup**, and select **Data Cloud Setup**. 2. Under External Integrations, select **Other Connectors**. 3. Click **New**. 4. On the Source tab, select **Apache Iceberg File Federation** and click **Next**. 5. Enter a connection name, connection API name. 6. Under **Authentication Option**, select one of these authentication options. - **Access Token**: Data 360 authenticates directly with the REST catalog using an access token you provide. 1. In the **Client Access Token** field, enter the OAuth 2.0 bearer token. - **Identity Provider Based**: Data 360 exchanges a Salesforce-issued OIDC token for an access token at an authorization server you specify. 1. Copy the value of **External ID** and register Salesforce as an identity provider with your authorization server using these claim values. | **Claim** | **Value** | | --------- | ---------------------------------------------- | | Issuer | `https://{ORG_DOMAIN_URL}/services/connectors` | | Audience | `https://{ORG_DOMAIN_URL}` | | Subject | The **External ID** value copied above | 2. In the **OAuth Server URI** field, enter the URI of the authorization server. 3. In **Custom Properties**, enter any custom OAuth properties that Data 360 should include in token exchange requests. For both options, if your data store’s REST catalog supports storage credential vending, select **CATALOG_PROVIDED** from the **Storage Type** dropdown. If not, select **S3** if the data is stored in AWS S3 and **AZURE** if the data is stored in either Azure Blob Storage or ADLS Gen2. If you did not select **CATALOG_PROVIDED**, Data 360 requires additional information about your storage bucket or container.
Storage Type Authentication Details
AWS S3 Bucket Name: Provide the name of the storage bucket. See General purpose bucket naming rules - Amazon Simple Storage Service.
Access Key ID: Provide the access key ID for the IAM user that Data 360 will use to access the storage bucket. See Manage access keys for IAM users.
Secret Access Key: Provide the secret access key for the IAM user that Data 360 will use to access the storage bucket. See Manage access keys for IAM users.
Azure Blob Storage or Azure Data Lake Gen2 Storage Account Name: Provide the name of the storage account. See Storage account overview – Azure Storage | Microsoft Learn.
SAS Token: Provide the shared access signature token that Data 360 will use to access the relevant storage container within the storage account. See Create shared access signature (SAS) tokens – Azure AI services | Microsoft Learn.
7. In the Connection Details section, enter the publicly-accessible HTTPS URL of the Apache Iceberg REST catalog. To make a request to an endpoint, append the endpoint to the URL. For example, a typical REST catalog's URL is `https://myicebergdatalakehouse:443/iceberg`. 8. If your REST catalog’s implementation of the /v1/config REST endpoint accepts a value for the optional warehouse parameter and you want Data Cloud to read warehouse-specific configuration metadata, specify a value for the warehouse field. 9. To review your configuration, click **Test Connection**. 10. Click **Save**. After the connector details are accepted, the connection is created and listed under Connectors. You can now create data streams. Keep these behaviors in mind when you set up an Apache Iceberg File Federation connection. **Storage Considerations** - **Cross-Region S3 Storage Bucket**: If the Data 360 org is not in the same region as the S3 storage bucket and your catalog doesn't support storage credential vending, make sure that the server hosting the REST catalog includes the client.region property in the LoadTableResult object. See, [Iceberg open API](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml?ref=chaosgenius.io#L3273). If you use AWS S3, the S3 bucket’s name must not contain a period. Such buckets can only be accessed via path-style addressing, which AWS has marked for deprecation. Only buckets that can be accessed via virtual host-style addressing with SSL are supported. See [Virtual hosting of general purpose buckets - Amazon Simple Storage Service](https://docs.aws.amazon.com/AmazonS3/latest/userguide/VirtualHosting.html#path-style-access). - **Azure Storage**: If the data is stored in Azure, make sure that the paths to all files (the JSON metadata file, Avro manifest list file, Avro manifest files, and Parquet data files) included in the LoadTableResult object use either the abfs or abfss protocols and not the wasb or wasbs protocols. The latter two protocols have been officially deprecated by Microsoft in favor of the former. ``` Valid Metadata JSON File Path abfss://@.dfs.core.windows.net//metadata/.metadata.json Invalid Metadata JSON File Path wasbs://@.blob.core.windows.net/
/metadata/.metadata.json ``` **Other Considerations** - **Row-Level Updates**: Querying Iceberg tables that are configured to use Iceberg V2 MoR Position or Equality Deletes or Iceberg V3 Deletion Vectors isn't supported. - **Views**: Querying Iceberg views isn't supported. - **Namespaces**: Up to two levels of nesting is supported: catalog -> namespace -> table / catalog -> namespace -> namespace -> table. When you configure a data stream, the database dropdown displays the names of all top-level namespaces, and the Schema drop down displays the names of all secondary namespaces registered under the selected top-level namespace. If there are no namespaces registered under the selected top-level namespace, the Schema picklist will be empty. - **Temporal Data Types**: The time and timestamp_ntz data types aren't supported. - **Governance**: To allow Data 360 to see only a portion of the tables registered with the REST catalog, configure a service principal that is only allowed to see the relevant portion and provide Data 360 the access token that corresponds to that service principal. - **Change Data**: Certain features in Data 360 (such as Data Actions) require the ability to detect when a data lake object changes. To use these features, define a primary key for your Iceberg table by using the `identifier-field-ids` construct and ensure that your query engines (writers) respect the configuration. ## Next Step - [Create an Apache Iceberg File Federation Data Stream](https://developer.salesforce.com/docs/data/data-cloud-int/guide/c360-a-create-apacheiceberg-file-fed-data-stream.md)