--- schema_version: "1" id: f0163169c67dc5988196394e21a5dd9a2e89ed98a0904a90b0232d472f749d82 title: Visual Search using SPI collection: developer version: current locale: en-us portal_url: https://docs.salesforce.com/developer/current/en-us/commerce-cloud/retail-cloud/guides/visual-search-via-spi.md canonical_url: https://developer.salesforce.com/docs/commerce/retail-cloud/guide/visual-search-via-spi.html source_site: developer.salesforce.com content_type: documentation content_hash: d968816db5c1efe64f46591070d123470ded7f516e16565e0000b8e231fc7b86 products: - Point of Sale for Developers - Salesforce APIs --- # Visual Search using SPI With the Point of Sale Visual Search SPI, the POS app sends an image captured by a store associate to an external image-recognition provider and receives a ranked list of matching product IDs. Results are resolved against the local product catalog and displayed in the POS app. Optionally, the provider can return filters, sort options, and pagination to enable server-side refinement of results. - Modern POS app build: 10.7.1 or later - Platform support: iPad and iPhone - Offline support: No **note:** For a complete list of Point of Sale APIs and SPIs, see [Point of Sale API, SPI, and Webhook Overview](https://developer.salesforce.com/docs/commerce/retail-cloud/guide/retail-cloud-api-spi-webhook-specs.html). ## Interaction Flow This diagram shows how the store associate, Point of Sale, and the SPI implementer interact during a visual search session. ![Sequence diagram showing the interaction flow between the store associate, Point of Sale, and the SPI implementer during a visual search session.](https://developer.salesforce.com/docs/media/retail-cloud/visual-search-interaction-flow.png) ### Required Flow (Steps 1–4) 1. The store associate captures an image in the POS app. 2. Point of Sale sends the image to the SPI provider via the `VISUAL_SEARCH_UPLOAD` connector. 3. The provider performs image recognition and returns a `searchId` along with a ranked list of matching product IDs. Optionally, the response can include `filters`, `sortOptions`, and `pagination`. See [POS Experience Based on SPI Response](#pos-experience-based-on-spi-response). 4. Point of Sale resolves the product IDs against the local product catalog and displays results to the store associate. ### Optional Flow (Steps 5–8) If the SPI response from step 3 includes `filters`, `sortOptions`, or `pagination`, and the `VISUAL_SEARCH` connector is configured, Point of Sale renders the corresponding UI controls (filter panel, sort dropdown, page navigation). 5. The store associate applies a filter, changes sort order, or navigates to the next page. 6. Point of Sale sends a request to the SPI provider via the `VISUAL_SEARCH` connector, passing the `searchId` and the selected criteria. 7. The provider applies the criteria to the stored result set and returns an updated list of product IDs (with optionally updated filters, sort, and pagination reflecting the new state). 8. Point of Sale displays the refined results. **note:** The second SPI call is entirely optional. If your provider doesn't support server-side filtering, sorting, or pagination, omit `filters`, `sortOptions`, and `pagination` from the upload response. The POS doesn't render these controls, and the `VISUAL_SEARCH` connector doesn't need to be configured. ## Configure Visual Search SPI Configure one or both SPI connectors in CMS depending on whether your provider supports server-side refinement. ### Configure VISUAL_SEARCH_UPLOAD Connector 1. In CMS, go to **Integrations** | **Custom Connectors**. 2. Click **Create**. 3. For Connector Type, select **SPI**. 4. For Endpoint Name, select **VISUAL_SEARCH_UPLOAD**. 5. For API Endpoint, specify the external endpoint for image upload and recognition. 6. For Integration Type, select **MULTIPART_FORM_POST**. 7. Specify any required JSON headers. For examples, see [Point of Sale API, SPI, and Webhook Overview](https://developer.salesforce.com/docs/commerce/retail-cloud/guide/retail-cloud-api-spi-webhook-specs.html). 8. Configure the remaining fields as needed. 9. Save your changes. ### Configure VISUAL_SEARCH Connector (Optional) Configure this connector only if your SPI provider supports server-side filtering, sorting, or pagination. 1. In CMS, go to **Integrations** | **Custom Connectors**. 2. Click **Create**. 3. For Connector Type, select **SPI**. 4. For Endpoint Name, select **VISUAL_SEARCH**. 5. For API Endpoint, specify the external endpoint for search refinement. 6. For Integration Type, select **HTTP_POST**. 7. Specify any required JSON headers. For example, `{"Content-Type":"application/json"}`. 8. Configure the remaining fields as needed. 9. Save your changes. ### Enable Custom API Headers (Optional) Enable the `populateNewCustomAPIHeaders` toggle to inject contextual headers into every SPI request. For details, see [Headers Sent by Point of Sale](https://developer.salesforce.com/docs/commerce/retail-cloud/guide/retail-cloud-api-spi-webhook-specs.html#headers-sent-by-point-of-sale). ## POS Experience Based on SPI Response The POS experience varies depending on the content of the SPI response. ### Minimal Response (VISUAL_SEARCH_UPLOAD Only) With a minimal response, the store associate sees a flat list of visually similar products. No pagination, filters, or sorting controls appear. This is the simplest implementation: image in, products out. ```json { "searchId": "44c66064-52fb-4d60-b5f0-97ddd34b3701", "results": [ "1010183BKIAD4", "1010183BKIADY", "1010183BKIAEE", "1010183BKIAEJ" ] } ``` ### Full Response (VISUAL_SEARCH_UPLOAD and VISUAL_SEARCH) With a full response, the store associate sees product results with page navigation, a filter panel, and a sort dropdown. Selecting a filter, changing sort, or navigating pages triggers a call to the `VISUAL_SEARCH` connector. ```json { "searchId": "44c66064-52fb-4d60-b5f0-97ddd34b3701", "results": [ "1010183BKIAD4", "1010183BKIADY", "1010183BKIAEE", "1010183BKIAEJ", "5123274", "5123393", "6010002-1", "6010002-12", "6010002-24", "6010002-6", "6010004-12", "6010004-24", "6010004-6" ], "pagination": { "start": 0, "limit": 100, "totalResults": 106 }, "filters": [ { "name": "Category", "displayName": "Category", "type": "FIELD", "values": [ { "value": "Jackets" }, { "value": "Hoodies" }, { "value": "Vests" }, { "value": "Cycling Computers" }, { "value": "Cycling Locks" }, { "value": "Coffee" }, { "value": "Smart Dispensers" }, { "value": "Coffee Machines" }, { "value": "Energy Drinks" }, { "value": "Energy Bars" }, { "value": "Energy Gels" } ] }, { "name": "Size", "displayName": "Size", "type": "FIELD", "values": [ { "value": "M" }, { "value": "L" }, { "value": "S" }, { "value": "XL" }, { "value": "XS" }, { "value": "XXL" }, { "value": "Single Bag" }, { "value": "12 Pack" }, { "value": "24 Pack (Case)" }, { "value": "6 Pack" } ] } ], "sortOptions": { "current": "RELEVANCE", "options": [ { "key": "RELEVANCE", "label": "Relevance" }, { "key": "PRICE_LOW_TO_HIGH", "label": "Price: Low to High" }, { "key": "PRICE_HIGH_TO_LOW", "label": "Price: High to Low" } ] }, "responseTime": 150 } ``` ### Filter Panel Each entry in `filters[]` becomes a section in the filter panel. The `displayName` value is the section title. Each entry in `values[]` renders as a selectable option. The order of values in the array determines the display order. ![The filter panel in the POS app showing Category and Size sections with selectable options.](https://developer.salesforce.com/docs/media/retail-cloud/pos-visual-search-filter-panel.png) ### Sort Dropdown The `sortOptions.options[]` array renders as a dropdown list. Each option's `label` is the row text. The `key` is the value sent back to the SPI when selected. The option whose `key` matches `sortOptions.current` is pre-selected. ![The sort dropdown in the POS app showing Relevance selected, with Price: Low to High and Price: High to Low as additional options.](https://developer.salesforce.com/docs/media/retail-cloud/pos-visual-search-sort-dropdown.png)