To efficiently store and find information, the search engine breaks up record content into smaller pieces, called tokens. The tokens are stored in the search index. See how common types of information, including emails, websites, and phone numbers are tokenized.
Here are some examples of the tokens created and indexed from record information. You can find a record with a search using any token listed.
Note Alphanumeric terms are split at letter-number boundaries. Terms with non-alphanumeric
characters, such as punctuation marks are split at the non-alphanumeric characters into
alphabetic, numeric, and alphanumeric tokens.
| Record Information | Indexed Tokens |
|---|---|
|
Web2Lead |
web2lead web 2 lead |
|
Acme, Inc. |
acme inc |
|
Exhibit-XYZ1234 |
exhibit-xyz1234 exhibitxyz1234 exhibit xyz1234 xyz 1234 |
| +1 (555) 123-4567 |
1(555)123-4567 15551234567 5551234567 1234567 |
|
john.smith@domain.com |
john.smith@domain.com @domain.com john.smith john smith johnsmith domain.com domain com domaincom |
Tokenization Search Tips
- If your search contains punctuation, such as commas, or special characters, and you’re not getting the expected results, add white space before, after, or in place of the punctuation and special characters.
- When searching for a record using a phone number:
- Enter a complete number. Searching for part of a phone number doesn’t always return the results you expect. For example, you search for 567 to find the phone number 1(555)123-4567. Search doesn’t find a result because it breaks up the number as 1, 555, 123, 4567. If you search for 4567, you get the expected result.
- When you add a phone number to a record, include the country code before the number to ensure expected results.
- With the autonumber or ID field type, tokenization removes all the zeros. For example, when you search XXXX000012, search breaks up the term as XXXX, 00000012, and 12. Keep in mind, you can’t search for a term less than 1 character long.
- How a record is broken up can differ depending on the field. For example, a record's phone number field can be tokenized differently from one in a Chatter post.
- For searches in Chinese, Japanese, Korean, and Thai (CJKT), the search engine returns accurate results using a special form of tokenization.
| Record Information | Indexed Tokens |
|---|---|
| 東京都 (Tokyo Prefecture) |
東京 (Tokyo) 都 (Prefecture) |