The Einstein generative AI platform routes large language model (LLM) requests to servers that are closest to where your Einstein generative AI platform instance is located. Geo-aware routing includes LLM API requests made by Einstein generative AI features that use the Einstein generative AI platform.
Required Editions
| Available in: Enterprise, Performance, and Unlimited Editions with an Einstein for Sales, Einstein for Platform, Einstein for Service, Einstein 1 Service, or Einstein GPT Service add-on. To purchase add-ons, contact your Salesforce account executive. |
Supported Models
Geo-aware routing is available to these models, provided that the models are available in the relevant regions.
- OpenAI models (such as GPT-4 Omni). These geo-aware models use OpenAI or Azure OpenAI to service LLM requests. The same underlying model is used by OpenAI and Azure OpenAI. See Geo-Aware Routing for OpenAI and Azure OpenAI.
- Anthropic models (hosted on Amazon Bedrock)
- Gemini models (hosted on Vertex AI)
Availability
Geo-aware routing is available to:
- AI agents
- Salesforce applications and features that use OpenAI, Anthropic, and Gemini models through the Einstein generative AI platform
- Customers who use OpenAI, Anthropic, and Gemini models through Models API or Prompt Builder.
To find out whether geo-aware routing is enabled for any specific Salesforce AI feature, refer to its documentation or contact your Salesforce account executive.
Proximity and Routing
Proximity to the nearest LLM server is determined by the region in which your Einstein generative AI platform instance is located. If you enabled the Einstein generative AI platform on or after June 13, 2024, then your Einstein generative AI platform region is the same as your Data 360 region (Data 360: Data Center Locations). Otherwise, contact your Salesforce account executive to learn where it’s provisioned.
In-Region Model Requests
When a request fails to process in your designated region, the default behavior for models is to automatically re-route the request to another region to avoid returning an error. You can change this behavior in your org for some models by turning on “Enable In-Region Model Requests Only” from Einstein Setup.
For Anthropic and GPT models, this setting entirely disables region fallback. Requests fail if they can’t be processed in the designated region.
Routing in the Models API
If you use the Einstein generative AI platform directly through Models API, we recommend that you use model API names for geo-aware models.
Feedback and Audit
To view where your LLM requests are being routed to, refer to Feedback and Audit features.
Feature-specific Support
To track whether an Einstein feature supports geo-aware LLM request routing, see the feature's documentation.
- Geo-Aware Routing for OpenAI and Azure OpenAI
A geo-aware model, such as GPT 4.1, uses OpenAI or Azure OpenAI to service LLM requests. The same underlying model is used by OpenAI and Azure OpenAI. - Geo-Aware Routing for Anthropic
LLM requests for Anthropic on AWS Bedrock are routed to the nearest AWS data center where a model endpoint is available. - Geo-Aware Routing for Gemini
LLM requests for Gemini are routed to a Vertex AI (Google) model endpoint.