--- schema_version: "1" id: 22562d3a49ac4ca523055c555270ff0eb9d3bfc07dcab4fabf8c6c70bff4465c title: Set Up an AWS Glue Data Catalog File Federation Connection collection: developer version: current locale: en-us portal_url: https://docs.salesforce.com/developer/current/en-us/data-cloud/data-cloud-int/guides/awsglue/c360-a-set-up-awsglue-connection.md canonical_url: https://developer.salesforce.com/docs/data/data-cloud-int/guide/c360-a-set-up-awsglue-connection.html source_site: developer.salesforce.com content_type: documentation content_hash: b90a81c53f8382a4d0629911ab30c8140dbe65d2d5d65549f2afc9324ac976fb products: - Data 360 Integration Guide - Data 360 Developer Documentation --- # Set Up an AWS Glue Data Catalog File Federation Connection Create an AWS Glue Data Catalog connection to pull data into Data 360. | User Permissions Needed |   | | ---------------------------------------------- | --------------------------------------------------------- | | To create an AWS Glue Data Catalog connection: | System Admin profile or Data 360 Architect permission set | Requirements: - **Firewall**: If your Glue Data Catalog-managed Iceberg data lakehouse is behind a firewall, add these [Data 360 IP Addresses](https://help.salesforce.com/s/articleView?id=data.c360_a_data_cloud_ip_address_allowlist.htm&type=5) to your allowlist before creating a connection. If the network firewall that protects the Glue Data Catalog and the underlying S3 bucket are different, make sure to update both network firewalls. Also, make sure both the Glue Data Catalog and the S3 bucket are publicly accessible. **note:** Data 360 doesn't support connecting over AWS PrivateLink to either the Glue Data Catalog or the storage bucket, **Set Up the Connector** 1. In Data Cloud, click **Setup**, and select **Data Cloud Setup**. 2. Under External Integrations, select **Other Connectors**. 3. Click **New**. 4. On the Source tab, select **AWS Glue Data Catalog** and click **Next**. 5. Enter a name for the connection. 6. In the Authentication Details section, enter the AWS Access Key ID and AWS Secret Access Key for the Identity and Access Management (IAM) user. Make sure that the IAM user has a policy that grants the user read-only access to Glue Data Catalog. Use AWS Lake Formation to configure storage credential vending. This approach is for Glue Data Catalog to vend temporary credentials at run time so that Data 360 can access the underlying storage bucket. { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "lakeformation:GetDataAccess" ], "Resource": "*" }, { "Action": [ "glue:ListSchemas", "glue:GetTables", "glue:GetTableVersions", "glue:GetTableVersion", "glue:GetTable", "glue:GetSchema", "glue:GetDatabases", "glue:GetDatabase" ], "Effect": "Allow", "Resource": [ "arn:aws:glue:us-west-2:240852588578:userDefinedFunction/*/*", "arn:aws:glue:us-west-2:240852588578:tableVersion/*/*/*", "arn:aws:glue:{REGION}:{ACCOUNT_ID}:table/*/*", "arn:aws:glue:{REGION}:{ACCOUNT_ID}:schema/*", "arn:aws:glue:{REGION}:{ACCOUNT_ID}:registry/*", "arn:aws:glue:{REGION}:{ACCOUNT_ID}:database/*", "arn:aws:glue:{REGION}:{ACCOUNT_ID}:connection/*", "arn:aws:glue:{REGION}:{ACCOUNT_ID}:catalog" ], "Sid": "AWSGlueDataCatalogWithS3ReadOnlyAccess" } ] } If you don't configure storage credential vending, make sure that the policy also explicitly covers the S3 bucket. Add this IAM policy, and remove the component of the previously mentioned policy that is specific to Lake Formation. ``` { "Version": "2012-10-17", "Statement": [ { "Sid": "ListBucketForIceberg", "Effect": "Allow", "Action": [ "s3:ListBucket", "s3:GetBucketLocation" ], "Resource": [ "arn:aws:s3:::{bucket}" ], "Condition": { "StringLike": { "s3:prefix": [ "{s3_prefix_1}/*", "{s3_prefix_2}/*" ] } } }, { "Sid": "ReadIcebergFiles", "Effect": "Allow", "Action": [ "s3:GetObject", "s3:GetObjectVersion" ], "Resource": [ "arn:aws:s3:::{bucket}/{object_prefix_1}/*", "arn:aws:s3:::{bucket}/{object_prefix_2}/*" ] }, { "Sid": "KMSAccessForIceberg", "Effect": "Allow", "Action": [ "kms:DescribeKey", "kms:Decrypt" ], "Resource": "{kms_key_arn}" } ] } ``` 7. For **catalog URL**, enter the URL of the Glue Data Catalog. See [Connecting to the Data Catalog by using AWS Glue Iceberg REST endpoint.](https://docs.aws.amazon.com/glue/latest/dg/connect-glu-iceberg-rest.html) 8. For **Catalog ID**, enter the 12-digit AWS account ID linked to the Glue Data Catalog. 9. For **Signing Region**, enter the host AWS region that Glue Data Catalog. 10. For **Signing Service**, enter **glue**. Data 360 requires the Signing Service, in addition to the AWS Access Key ID, Secret Access Key, and Signing Region, to sign requests to the Glue Data Catalog by using AWS Signature Version 4. **Storage Considerations** - **Cross-Region S3 Storage Bucket**: If your Data 360 org isn’t in the same region as your S3 storage bucket and your catalog doesn't support storage credential vending, make sure that the server hosting the REST catalog has the client.region property in the LoadTableResult object. See [Iceberg OpenAPI](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml?ref=chaosgenius.io#L3273). - **S3 Storage Bucket Name**: If you use AWS S3, make sure that the S3 bucket’s name doesn't contain a period. You can access such buckets via path-style addressing, which AWS has marked for deprecation. Data 360 supports only those buckets that are accessed via virtual host-style addressing with SSL. **Other Considerations** - **Row-Level Updates**: Querying Iceberg tables that are configured to use Iceberg V2 MoR Position Equality Deletes or Iceberg V3 Deletion Vectors isn't supported. To express a merge-on-read table as a copy-on-write table, see [Submitting Spark Jobs for MoR to CoW Conversion](https://www.ibm.com/docs/en/watsonxdata/standard/2.2.x?topic=engine-submitting-spark-jobs-mor-cow-conversion). - **Views**: Querying Iceberg views isn't supported. - **Namespaces**: Only single-level (catalog -> database -> table) and two-level (catalog -> database -> schema -> table) namespaces are supported. - **Temporal Data Types**: The time and timestamp_ntz data types aren't supported. - **Change Data**: Certain features in Data Cloud require the ability to detect when a data lake object changes (example, data actions). For Data Cloud to construct a change data feed, that is, a log of incremental changes to an external data lake object, it needs a primary key. For such features, use Iceberg’s identifier-field-ids construct to define which columns comprise a table’s primary key and make sure that your query engines (writers) respect your configuration. ## Next Step - [Create an AWS Glue Data Catalog Data Stream](https://developer.salesforce.com/docs/data/data-cloud-int/guide/c360-a-set-up-awsglue-file-fed-data-stream.md)