Configure Textkernel Search and Match indexing

In this section we discuss how to configure the indexation process to synchronize data from Salesforce into the Textkernel environment to enable searching and matching. This also discusses mapping your Salesforce data model to the Textkernel search data model.

Setup indexing status, record filtering, and data access control

1. Configure Textkernel indexing status custom object

   On the Textkernel indexing status custom object, add lookup fields to the primary Candidate and Vacancy objects for your data model. This relationship links Textkernel indexing status records to Candidates and Vacancies. These records maintain indexation status information. The records track which Candidate/Vacacny records are up to date in the Textkernel index, which are waiting to be indexed to Textkernel and/or which records had indexing errors (e.g. because of an unparseable CV).

   Go to SFDC Setup → Object Manager → Textkernel Indexing Status, and then create the 2 Lookup relationships.

2. Define Custom formula fields to define the subset of records to be indexed

   Usually, not all records are copied into the TK Index for search and match. For example, perhaps only active Vacancies should be stored in the Index, or perhaps any Candidates which have not given recent marketing consent need to be excluded. To support this, you should create a custom Formula Field (using Salesforce Object Manager) that returns a boolean True if the Candidate/Vacancy record should be indexed, and otherwise returns False. Once those fields have been created, you should assign them in the data mapping. The filter function can be defined for each related object in your data model (ie, not just Candidate, but related Work Experience, Education, etc).

   As an example: in the image, all Candidates are stored on a custom Candidate object. We use the custom Boolean filter “Textkernel Match Object” to control which Candidates should be indexed. We select this field in the “Filtering formula field” field on Candidate Data Mapping tab of the Textkernel setup tab. With this configuration, only Candidate records whose field Textkernel Match Object evaluates to True will be indexed to Textkernel.

   

3. (Optional) Define data-segregation roles

   You may want to restrict access to the Candidates/Vacancies in the TK Index, so that recruiters only see the relevant set of Candidates and Vacancies for their particular responsibility (i.e. not all records are visible). If so, you should follow the steps in this section. If you don’t use this option, then all Candidates and Vacancies will be visible to all Recruiters.

   1. Enable Data Segregation on the Textkernel portal settings page of the App.

      

   2. When the feature is enabled, then you will also see the option to Define Textkernel Candidate Visibility on the Candidate Data Mapping page and Define Textkernel Vacancy Visibility on the Vacancy Data Mapping page.

   3. Using the Object Manager, add a custom multi-select picklist field to the primary Candidate and Vacancy objects. The custom field lists all of the possible access roles that you want in your Textkernel index (only Salesforce users which are also assigned one of these access roles for the record will be able to view the record in the Textkernel search results).

   4. Select the relevant field under Define Textkernel Candidate Visibility. In the example setup, the field name is Segregation Roles.

      

   5. Define the global list of possible access roles. The multi-select picklist fields controlling the access must use the global value set Textkernel access roles, which is included in the package. You need to list all of the possible access roles you want, and add them to this global value set. In the examples, the custom field Segregation Roles (which is used on the primary Candidate and Vacancy objects) always uses this global value set.

      Go to SFDC Setup → Picklist Value Sets, and add a Global Value Set.

   6. Create sets of Textkernel Search Policy records on the Textkernel Search Policies tab. You should create a search policy for each type of user profile you want in your Textkernel index search results. Each search policy contains a set of access roles which a Salesforce User should see when viewing Search results.

   7. Link your Salesforce users to the relevant search policies using the Textkernel Search Policy of User object. To do this, open the Textkernel Search Policy Record which you want to associate with Users. On the Related tab, you see a list of all the Search Policy of User records using this search policy. You should create a new record linking each Salesforce User to the appropriate search policy. When this is setup, you will see the linked Salesforce Users listed under the Related tab.

   8. As part of your Salesforce environment integration with Textkernel, you should add automation to:

      1. assign the appropriate access role(s) to the Candidate record when they are created
      2. assign the appropriate access role(s) to the Vacancy record when they are created
      3. auto-create the Textkernel Search Policy of User records when new Salesforce Users are created.

      Important

      These custom fields control visibility of records in the Textkernel search results, but they don’t impact the access control to Objects inside your Salesforce environment. If you want the user to have similar access rights, when you automate the management/creation of Search Policy of User, you should build a workflow that derives the search policies from your preferred Salesforce access control mechanism.

Map Candidate data

The Candidate Data Mapping page allows you to map your customized Salesforce data model onto the core Textkernel Candidate indexation model. Textkernel’s Candidate data model is flexible and supports custom fields as well as a range of adjustable relationships. For each primary object and related object in the data model, there are a set of custom fields of different data types available. You can map your custom environment data fields into these fields to make them available in your search results.

On the left-hand side of the page, selectable tabs control which object's field mapping controls are currently active (Candidate, Work Experience, Education, Vacancy, etc). As you select the different tabs, the relevant fields are shown in the UI.

1. Primary Candidate object

   The first tab to configure is Candidate. Candidate is the root object, so all other objects displayed on the other Tabs of this page are related to the primary Candidate object. Select the root Candidate object in the primary Candidate object section.

2. You typically also filter records that should be indexed to Textkernel. To do this use Filtering formula field to select the Boolean formula field that returns true when a Candidate should be indexed. Only records that meet the filter criteria will be indexed to Textkernel. (See section 2 of Configure Textkernel App for more information on the requirements for the filtering field).

3. If you are using data segregation, you will also select the field controlling who has access to the Candidate records using the field Roles that may see the candidate (see section 3 of Configure Textkernel App for more information).

4. Now map the rest of your preferred fields into the available fields in the Textkernel Candidate data model. You can also use custom fields if necessary. Remember to use Save to store the mapping. (Note: whenever you change the Candidate object it will reset all the dependent fields across all related objects, so make sure you have selected the intended object.

5. After you define the mapping for the primary Candidate object, proceed to map the relevant related objects and their fields from your data model into the standard Textkernel Candidate data model, too. Select the related objects using the bar on the left-hand side of the screen.

6. The first related object you will configure is the CV/Resume (File) tab

   • CV/Resume file Option 1: Often, the CV/Resume (File) is saved as a File related to the root Candidate object. In this scenario, the main configuration is to select a custom filtering function for the CV file. There could be multiple files or different types attached to a Candidate (e.g. different submitted CV versions, and potentially other document types such as a cover letter). The filtering function should return a Boolean True value for whichever file is the one master CV document version that you want to store in the Textkernel index.

   • CV/Resume file Option 2: In some cases, the CV/Resume (File) is not linked directly to the primary Candidate object, but to a related object, instead. If this is your data model style, select the checkbox File or Attachment linked to related object. This displays the relevant settings for that model. If the related records should be filtered, create a boolean formula field that returns “True” to indicate which single CV record should indexed (for example, you might have different types of related content records like a CV, cover letter, and other document types). In this case, select this field in Apply a filter section. There could also be multiple content versions for the file attached to the related record, so you may also need a 2nd filter to handle that and indicate which content version you want to retrieve.

   • CV/Resume file Option 2: In some unusual cases, your environment might use the older Attachment object instead of the File object to store CV files. This is only possible if the Attachment is linked to a related object, as in Option 2. If this is the case, please set things up as in Option 2, but also select the checkbox: Use Attachment instead of File Object.

7. Most the the remaining tabs relate to Candidate data which are typically defined as objects relating to the primary Candidate record, like Work Experiences, Educations, Placements, Job submissions, Contact Lists, Comments, Skills.

   For each of these related objects:

   1. Select the appropriate object related to the primary Candidate object
   2. Select a filtering formula field to define which records should be indexed
   3. Map fields into the default Textkernel data model
   4. Add custom fields, if needed, which are not part of the default Textkernel data model

8. (Optional) If you want to index other non-standard objects related to the Candidate in your data model, there are 5 additional custom object relationships that are supported.

Skills data model mapping

Textkernel supports 4 different skills categories in its standard data model:

• IT Skills
• Language Skills
• Soft Skills
• Professional Skills

This means there are multiple possible approaches for mapping the skills data model:

• Option 1 "Separate skills objects for each skill type": with this mapping, there is a separate related Skills object for each of the 4 types of Textkernel skill. So, you define the relationship to each of the Skills objects, define relevant filtering criteria, and map the fields into the Textkernel data model for indexing.

• Option 2 "Single skill object for all skill types": in this data model, there is one Skill object which merged each of the 4 Textkernel skill types mapped into one. To use this data model, in addition to the Skill object, you should also create 2 types of custom fields on the skills object:

  • First field type: a single picklist to define the skill type. The picklist values should be:

    • Language skill (API - langskill)
    • IT skill (API - compskill)
    • Professional skill (API - otherskill)
    • Soft skill (API - softkill)

    

  • Second field type: a set of custom formula fields that evaluate to True when a records is one of the 4 skill types (e.g. Is_Soft_Skill, Is_IT_Skill, Is_Language_Skill, etc). These formulas will be used with the Apply a filter setting so that the Skill record will be mapped to the appropriate skill type.

    

  • After these fields have been set up, map the skills tabs under Candidate Data Mapping onto your single Skills object, and use the filtering formula fields to identify which records belong to which Skill type. Map the remaining data fields for each of the categories, too.

    

Map Vacancy data

In addition to mapping your Candidate data model, you also need to map your Vacancy data model to the standard Textkernel data model. This process is very similar to the one for setting up the Primary Candidate Object, except you use the Vacancy Data Mapping page:

1. The first thing you should do is specify a primary Vacancy object. Just like on Candidate Mapping page, the primary Vacancy object is the root object. Select it under Select the primary Vacancy object.

2. You may also filter records that should be indexed to Textkernel. In order to do this, create a Boolean formula field that returns True if the record needs to be indexed. Then select this field in the Apply a filter section. Only records that meet the filter criteria will be indexed to Textkernel (see section 2 of Configure Textkernel App for more information).

3. If you use data segregation, you may also select the field controlling who has access to the Vacancy records in Textkernel. (See section 3 of Configure Textkernel App for more information).

4. After the main Vacancy object is defined, you can map remaining related fields and objects from your organization into the standard Textkernel data model.

5. The Document tab works in a similar way as the CV/Resume (File) tab for Candidate Data Mapping page. This related object represents a binary or text attached file containing the vacancy description.

6. The Comments tab is mapped in a similar way as it was for Candidate data mapping.

7. (Optional) If you want to index other non-standard objects related to the Vacancy in your data model, there are 5 additional custom object relationships that are supported.

Exporting and importing your data mapping

When managing different sandbox and production environments, you want to be able to reuse your Candidate and Vacancy data mapping to quickly configure new orgs. To enable this, you can export the mapping to a file, and then import the mapping to quickly configure the mapping for a different Salesforce org. This saves time so that you don’t need to manually re-enter the mapping configuration each time.

This feature will not work if the data model is different between the two orgs. If you need to alter the Salesforce data model, first do the import, then change the data model afterwards.

Indexation Settings page

The Indexation Settings page is where you configure and monitor your recurring processes that index data from Salesforce into TK to keep the Textkernel index synchronized with Salesforce.

1. Go to Textkernel App → Textkernel Search and Match Setup → Indexation Settings tab

   Define the Indexing endpoint (this information will be provided by Textkernel support)

   

2. Add Textkernel endpoint domains to Remote Site Settings. You should add the following domains (check with Textkernel support for the correct values):

   • Extract Indexing: https://staging.textkernel.nl
   • SSO / user search policies: https://sso-staging.textkernel.nl

   Go to SFDC setup → Remote Site Settings → New Remote Site button → Fill the form → Save

3. Enter credentials for the admin user of the Textkernel environment for indexing vacancies and candidates. You will receive these from Textkernel support.

   

4. (Optional) Manage indexation throughput

   The indexation process submits requests to the Salesforce APEX Flex queue, which is a shared resource for APEX jobs running in your Salesforce instance. If you want to restrict how much of that queue will be used by the indexation infrastructure, you can use this setting to limit how many jobs will be added to the queue. Usually it is not necessary to change this, but if you have a lot of other processes using the queue, you might want to adjust this setting.

   

Process automation to send Salesforce changes to Textkernel

To keep Textkernel's index in sync, a record synchronization needs to happen whenever new records are created, or existing records are updated and/or deleted. The integration package makes this process straightforward to set up using standard Salesforce workflow features (e.g. Flow and Process Builder).

The package contains a pair of invocable classes: Delete record from Textkernel index and Upsert record to Textkernel index which can be used to create or update Textkernel Indexing Status records for Vacancies and Candidates. Both of these invocable classes share 1 parameter:

• Record Id: this is the record ID of the primary record (Candidate or Vacancy). So, when these classes are used by a child record, this id should refer to the related primary record, (i.e. something like {!$Record.Candidate__r.Id} or {!$Record.Vacancy__r.Id}).

In addition, Upsert record to Textkernel index has an additional optional parameter:

• handleImmediately: this is a boolean value. When set to True the TK indexation request will happen immediately rather than waiting for the scheduled indexation job. This parameter is only available for Vacancy records, not Candidate records. It is an advanced feature intended for cases when an interactive user has just created a new Vacancy and wants the vacancy indexed quickly to generate match results rather than waiting the normal indexation wait time. (So that match results could be generated quickly right after the Vacancy is saved).

For each object used in your Candidate and Vacancy data mapping, you need to set up automations: some handle the upsert operations, and some handle the delete operations. After doing this and activating the automation, all changes in Salesforce will get automatically synchronized to Textkernel’s index.

You can use the Salesforce Flow or Process Builder automation tools to handle this configuration requirement. It is also possible to call them from APEX code if you have a more complex environment. Below are some examples of setting up the Flows.

There are several specific scenarios to consider with the automation:

Automation to upsert records to the Textkernel index

These scenarios should be covered using Upsert record to Textkernel index:

• Creation of a new record, which satisfies the filter criteria for indexing → action is to upsert the record into the Textkernel index
• Update of an existing record, with static True value on filter criteria → action is to upsert the record into the Textkernel index
• Deletion of a child record of the primary Candidate/Vacancy record (e.g. Work History item), which satisfies the filter criteria for indexing → action is to upsert the primary Candidate/Vacancy record, which will update the whole record including the removal of the deleted child record
• Transition on the filter criteria value of a record, where it was not indexed before, but now it meets the filter criteria for indexing → action is to upsert the record from the Textkernel index

Automation to delete records from the Textkernel index

These scenarios should be covered using Delete record from Textkernel index

• Deletion of an existing primary Candidate/Vacancy record, which satisfies the filter criteria for indexing → action is to delete the record from the Textkernel index
• Transition on the filter criteria value of a record, where it used to be indexed, but no longer meets the filter criteria for indexing → action is to delete the record from the Textkernel index

Important

To minimize unnecessary indexing activity, the Flow or Process Builder upsert triggers should check to see if there was a field transition on a relevant (mapped) field rather than always updating the TK index on any field change on the triggering record. If an unmapped field changes there is no need to reindex the record into TK.

Important

The example below is illustrative, and does not cover all the scenarios required to maintain correct and efficient data synchronization.

Important

Salesforce Flow jobs execute under the Salesforce user which triggers a change on a record. These example Flows assume that user has a license for the Textkernel Connector package, and has permission to call the contents of the package. If the user has no license for the Textkernel Package (for example a different user type for an external Community user), then this approach will not work and will generate errors. In that scnenario, the recommended approach is: 1) setup a dedicated Salesforce integration user with access to a Textkernel license. 2) have the Flows send a Platform Event when there is a relevant change requiring indexation, 3) have a Flow consume those Platform Events but execute as the indegration user who has access to the Textkernel Connector package contents.

Example automation: Flow on record upsert

1. SFDC Setup → Flows → New flow button → Select Record-triggered flow → Press Next button →
2. Configure trigger: Select A record is created or updated and After the record is saved → Click Done

3. Click Choose object → Select candidate object (object that is used as Candidate) → Select -None- in Condition Requirements field → Press Done button

   

   

4. Add Action to the screen → Set up Action: Textkernel section → Select “Upsert record to Textkernel index” → Set Input values: Record Id - {!$Record.Id} → Press Done button → Activate the flow

   

Start updating the Textkernel index

After the process automation discussed in the previous section has been set up for the primary Candidate/Vacancy object and all the related objects used in your data model, now you are ready to start indexing data into Textkernel. However, before that, first you need to do a one-time initialization to create any missing Textkernel indexing status records.

1. Go to the Textkernel App → Textkernel Search and Match Setup → Indexation Settings tab

2. On the Indexing status section, click Generate missing records to generate any missing indexing status records.

   This batch will create the records for any existing candidates and vacancies in the organization (which meet the filtering criteria). Now that you have created and activated the Flow or Process automation, indexing status records will be created automatically whenever a new Vacancy or Candidate record is created. Also, whenever Vacancy or Candidate records and their related records are updated or deleted, the associated Textkernel indexing status record will be set with the “To be indexed” status.

   Now we are ready to start a recurring, scheduled indexation job to import Vacancies and Candidates into the Textkernel search environment. This job periodically checks for records with the “To be indexed” status, and indexes them to Textkernel, so that up-to-date data is available for recruiters.

3. Go to Textkernel App → Textkernel Search and Match Setup → Indexation settings tab

4. Press the Schedule indexation button

Monitor the integration

We provide tools so that you can monitor the indexation process and make sure that your Candidate and Vacancy records are properly synchronizing to Textkernel. You can view a summary of the indexation status to see if there have been any errors.

Go to Textkernel App → Textkernel Monitoring

Indexation status

Here you can see a summary of the indexation status for your records. You can see if they are Up to date (synced to Textkernel), waiting to be indexed into Textkernel (because they are new or changed records), or waiting to be removed from Textkernel (because they have been deleted from Salesforce and you want them removed from Textkernel, too).

The integration will automatically retry a failed indexing attempt 3 times, before considering it a permanent error. Permanent errors could be caused, for example by corrupted CV binaries that are unparseable, or it might be a different issue.

If you want to see the detailed status, click on the View details button. You can also easily reset the errors by clicking on the Reset errors button, and the integration will automatically attempt to re-index the failed records.

Debug logs

We also log general errors that happen independently of a particular indexation attempt. These errors can be viewed and analyzed by clicking on the View debug logs button.