Accelerated Hands-Free Field Execution With Speech-to-Text Integration

Available in: Lightning Experience Available in: Agentforce 1, Einstein 1, Enterprise, and Unlimited editions with Consumer Goods Cloud Retail Execution and Agentforce for Consumer Goods Cloud add-on licenses.

Voice-driven workflows aren't standard built-in features. Admins can create them by setting up APIs in the Consumer Goods Cloud mobile app customization layer by using the Visual Studio Code based Modeler. Design workflows that orchestrate these APIs to transcribe verbal requests into structured, AI-driven actions. The Speech-to-Text API captures the user's voice and converts it into text.

For example, you can customize the app for sales reps to use voice to record notes during a visit. When the app is offline and Speech-to-Text is turned on, the app saves the converted text in the relevant fields.

For other custom use cases like voice-to-survey, configure the workflow to use additional processing to pass the transcribed text to the Prompt Template API, when the device is online.

To use these configurations, make sure that the mobile device is online and the Integrate Agentforce setting is turned on in the Consumer Goods mobile app sync settings. The key configuration components are:

• Verify network: Use DeviceManager.isOnlineAsync() to check network connectivity. Because the promptTemplateAsync API requires an internet connection, use this check to turn off voice features when the device is offline to prevent errors and ensure a smooth user experience.
• Check Agentforce availability: Use Facade.isAgentForceAvailable() to confirm that the Agentforce feature is active.
• Capture input (Speech-to-Text): Use launchSpeechToTextAsync() to integrate the native speech-to-text interface. This function captures the user’s spoken words and returns the transcribed text to the application logic, ready for use as a parameter.
• Process intent (Prompt Template): Use Facade.executePromptTemplateAsync to connect to Salesforce and retrieve the AI-generated response by sending the inputs to a specific prompt template. The API handles the communication, but customizers should define the business logic to consume the response that is returned. For example, capturing data or dynamically querying the system. This functionality is available on Android, iOS, and the Modeler simulator.

Before you implement this workflow, review the considerations for Speech-to-Text integration.

• Considerations for Speech To Text Integration
  Review these important considerations for setting up speech-to-text. These include changing session limits and silence detection, designing the user interface and theme, stopping sessions automatically, handling errors, and setting up offline language packs.