Professional Documents
Culture Documents
Azure Search
Azure Search
Azure Search
TRAINING CONTENT
Introduction
Creating search services
Azure Search Concepts
Creating Azure Search Indexes
Managing Documents
Azure Search Queries
Filtering and Scoring Search Results
Navigating Search Results
Demo
INTRODUCTION
Azure Search is a search-as-a-service cloud solution that gives developers API and
tools for adding a rich search experience in application.
There are some important features of Azure search which is as follows:
1. Scalability
2. Query abilities
3. Scoring profiles
4. Search Navigation
INTRODUCTION
1. Scalability : We can easily scale-up and scale down azure search service based on requirement. Scalability can
be done manually by azure portal or programmatically using azure search management API.
2. Query abilities : Azure search following features for query processing:
a) Natural language processing: This features helps to understand query used for search.
NLP helps to understand the variations of term given for search. For ex if try to search term “GO”. It will
not only search for occurrence for “GO” in text. It will also provide result having “Went” in text because
“Went” is variations of “GO”.
a) Search Recommendations: This feature provided search suggestions so that user can jump to the result
fast.
b) Geosearch Support: We can search items in a particular geographical area using this feature. For ex. We
can search hotels with five kms for a particular point.
c) Scoring profiles: This feature helps to find most relevant search result.
3. Search Navigation : This feature provides different options to navigate search result:
a) Faceted Navigation: It provides categories along with search result which helps to narrow down search.
b) Paging: Azure search provides facility to divide results in smaller chunks in form of pages.
c) Hit Highlighting: This provides facility to highlight relevant text in search result.
CREATE AZURE SEARCH
Following steps are required to create Azure Search:
1. Go to portal.Azure.com
2. Go to New->Data+Storage+Azure Search
CREATE AZURE SEARCH
3. Fill the fields e.g. Service Name, Subscription, Resource Group, Pricing Tier and
location.
Note: Azure search service Name should follow the naming convention
For ex: Organization-application-location-Environment.
AZURE SEARCH CONCEPTS
There are two important components of Azure Search:
1. Search Indexes : It is similar as table in relational database. It is highly optimized
for search. User search query is broken down in terms and compared to records
in indexes.
2. Search Document.
CREATING AZURE SEARCH INDEXES
There are two ways to create Azure Search Index:
1. Using Portal.azure.com
2. Using Azure search management API
We have to add API key in header to execute query. We can get API key from
management portal.
UPDATING AZURE SEARCH INDEXES
We can update Azure search index using PUT request to Azure search API.
For ex. We can add other fields in Azure search index by using following request to
Azure search API:
UPDATING AZURE SEARCH INDEXES
Some of the important points to be considered while updating Azure search index:
1. We have to supply the existing field name along with new fields in body of PUT
request.
2. We can not rename fields while updating Azure Search Index.
3. We can not change field types while updating Azure Search Index.
4. We can not remove field while updating Azure Search Index.
5. To change or remove field in Azure Search Index following actions are required:
a. Remove Search Index
b. Re-Create the Search Index.
c. Repopulate the documents.
SHARING AZURE SEARCH INDEXES
We can share Azure search indexes to different application. To enable this feature we
have to CORS origin policy. Application hosted in different domain can not use the
search service without configuring CORS.
1. CORS options specifies which domains a search index can accept request from.
2. This option is only required when search experience is used by client side java
script.
3. We can configure this option using Azure search API by using following request:
SHARING AZURE SEARCH INDEXES
Comparison Operator: There are six comparison operator which are discussed below:
Equal (eq)
Not Equal (ne)
Greater Than (gt)
Less Than (lt)
Greater than or equal to (ge)
Less than or equal to (le)
ODATA OPERATOR FOR FILTERING
Logical Operator: There are three logical operators in ODATA expression:
AND (and)
OR (or)
NOT (not)
Lambda operators: There are two lambda operators used for collection filtering.
Any : Ex: “filter”:”flavors/any(f : f eq ‘chocolate’)
All : Ex: “filter”:”flavors/all (f: f ne ‘smokey’)
DOCUMENT RELEVANCY IN SEARCH RESULT
Azure search returns most relevant result after executing query against search. Some
of the important point related to document relevancy is as follows:
1. There is one of the property “@search.score” returned in search result which
shows document relevancy against query.
2. Higher the score, the more relevant the document.
3. More relevancy means higher ranking.
4. By default results are displayed in descending order of score.
SEARCH SCORE IN SEARCH RESULT
Search score in search result is calculate by algorithm (TF-IDF).
It means TERM FREQUENCY-INVERSE DOCUMENT FREQUENCY.
This algorithm works on following parameter:
1. Looks at how often tem appears in document.
2. Looks at how often it appears in other document.
3. The more frequent a term appears, the more relevant the document.
SCORING PROFILE IN AZURE SEARCH
The default algorithm for calculation for scoring in search result can be overridden
according to business requirement. It can be overridden by creating scoring profile.
It provides following features:
1. It allows to promote one or more fields
2. We can give priority to some of the fields in comparison of others. It is also
known as boosting.
3. It connects search results with business goals.
CREATING SCORING PROFILE
Scoring profile is the part of index definition. To create scoring profile we have to do
following steps:
1. Get the current index definition.
2. Update the index definition.
To retrieve the index definition the following request is required:
CREATING SCORING PROFILE
To update the index definition we can use the following request:
USING SCORING PROFILE
To use scoring profile in azure search query following query is required:
{
“search”:”north”,
“count”:”true”,
“select”:”name, breweryName”,
“scoringProfile”:”breweryName”
}
Note: Each index can have up to 16 scoring profile. If the scoring profile is not found
as per request. It will throw “result not found”.
After running this query order for scoring result is going to be change. The ranking
value is going to be impacted after this implementation.
CUSTOMIZING SCORING PROFILE
Azure search provides configuration based approach for customizing scoring profile.
There are five ways to customize scoring profile:
1. Text
2. Numeric values
3. Data Freshness
4. Proximity
5. Collection of strings
BOOSTING TEXT FIELD
Text based scoring profile:
To boost field of string we can use text base scoring profile. The following syntax can be used for this configuration.
“scoringProfiles”:[
{
“name”:”brewery name”,
“text”:”{
“weights”:{
“breweryName”:5
}
}}]
In above example name field is the populated for scoring profile name.
Text property is encapsulated with other property weights. It is populated with the name of the field along with magnitude. In above example brewery name field is boosted with magnitude of 5. It
means the specified field is more valuable than others.
We can add multiple fields in scoring profile with different weightage according to the requirement. To include multiple fields the following syntax is required:
“scoringProfiles”:[
{
“name”:”brewery name”,
“text”:”{
“weights”:{
“name”:5
“breweryName”:2.5
}
}}]
BOOSTING NUMERIC FIELD
Number based scoring profile: Scoring profile provide feature to boost numeric based fields. To boost numeric field
magnitude Function is required. Syntax for magnitude function is as follows:
“scoringProfiles”:[
{
“name”:”abv”,
“functions”:[
{
“type”:”magnitude”,
“fieldname”:”abv”,
“boost”:2.5
“magnitude”:{
“boostingRangeStart”:0
“boostingRangeEnd”:100
}
}]}]
It will impact result order. The scoring value is going to be changed.
BOOSTING NUMERIC FIELD
Fields used for number based scoring profile.
1. Type field is to be assigned to “magnitude” in case of number based scoring profile. It is required to
be set to implement scoring profile.
2. It should be in smaller case only.
3. Field name identifies the name of field for which scoring profile is to be created. It is required to be
set to implement scoring profile.
4. Only filterable filed is allowed in scoring function.
5. All azure search field is filterable by default.
6. The property boost is used to define a multiplier. It can be any positive value greater than 0 not 1.
7. If we try to set value 1.0. Azure search provides error because it will not have any impact.
8. Boost property is required field for all scoring functions.
9. Magnitude property contains two more properties. BoostingRangeStart and BoostingRangeEnd.
These two values are define to represent boost scope. Any values which are not in this range is not
going to be boosted up.
10. Boosting range can be any positive or negative value.
BOOSTING DATE RANGE
Boosting Date Range Values: To boost date range values there is freshness function in Azure search. It returns most recent data as the relevant data. For ex:
{
“name”:”freshBeers”,
“functions”:[
{
“name”:”freshness”,
“fieldname”:”lastTappenOn”,
“boost”:10,
“freshness”:{
“boostingDuration”: “P90D”
}
}]
}
BoostingDuration property defines the time span in which the values are boosted.
This time span begins when query gets executed.
The end time decides by the value of boostingduration. It is used to set when boosting expires.
The value for boosting duration is provided in format of ISO 8601 Format.
Example for ISO 8601 format is (1Day=P1D), (12Hrs=PT12H), (30Minutes=PT30M), (10.123 seconds=PT10.123S)
To boost future dates we can use negative sign. –P90.
BOOSTING STRING IN COLLECTION
To implement boosting for collection of string data type we can use tag function.
It works with collection(Edm.String).
For ex:
{
“name”:”prefferedFlavors”,
“functions”:[
{
“type”:”tag”,
“fieldName”:”flavors”,
“boost”:10,
“tag”:{
“tagsParameter”:”myBelovedFlavors”
}
}
}]}
In above example tagsParameter field is like placeholder. We can pass value to place holder from search request.
For ex:
{
“search”:”Ale”,
“scoringProfile”:”prefferdFlavors”,
“scoringParameters”:”[“myBelovedFlavors:pine”]
}
BOOSTING MATRIX AS PER DATA TYPE
We can set the default scoring profile to our custom scoring profile. It can be
done by setting property “defaultScoringProfile”. In this case Azure search is
not going to worked on TDFIF. Approach.
Summary
NAVIGATING SEARCH RESULT
To navigate search results the following techniques are provided by Azure Search:
1. Sort search result.
2. Paging in search result.
3. Faceted Navigation.
4. Query suggestion.
SORTING SEARCH RESULT
Sorting in Azure Search can be done on any Sortable Field.
By default all fields are sortable except collection(edm.string).
Collection(Edm.string) can’t marked as sortable.
We can implement sorting by using orderby parameter in queystring.
By default search results are sorted in descending order on @search.score value.
Orderby property can be used to override this default behavior.
Example for orderby is as follows:
{
“search”:”*”,
“orderby”:”name”
}
SORTING SEARCH RESULT
The default sorting direction is ascending.
We can use suffix (asc or desc) to sort results in a particular direction.
To combine sort on multiple fields we can provide comma delimited list to orderby.
The default sorting direction is Left to right. It means the result is going to sort on first field
then others in sequence of left to right.
As we discussed earlier there is “Score” property in the results which helps to get relevant
results. It may possible that multiple items has same score. The result sequence in this
scenario is not relevant to the user. To handle this we can update or add scoring profile.
One of the important point to be noted that @search.score field can not be used in order
by parameter.
PAGING SEARCH RESULT
It may possible that we have larger index. In this case volume of search result is bigger. To navigate search result in this case we can use paging.
By default Azure search return 50 search result per query.
There is “Top” parameter in Azure search where we can set maximum result to be returned from Azure search. For ex:
{
“search”:”*”,
“count”:”true”,
“select”:”name”,
“top”:5
}
There is “Skip” parameter to specify how many documents to pass over. It is used to navigate from one page to another. For ex:
{
“search”:”*”,
“count”:”true”,
“select”:”name”,
“top”:5
“skip”:5
}
The max value for skip parameter is 100,000.
FACETED NAVIGATION IN SEARCH RESULT
Facets: Set of items displayed with counts along with results is known as facets. It helps to navigate results faster.
We can understand facets in following diagram:
Facets are define from filter. It is returned from Azure search with counts. It is not pre-defined.
FACETED NAVIGATION IN SEARCH RESULT
To include “Facets” in search result include the “facets” parameter in query.
Fields should be facetable in index to include in facets.
By default all fields are facetable in Azure search index.
To include Facets we can provide multiple fields in facets parameter in form of array for ex:
{
“search”:”*”,
“facets”:[“breweryName”]
}
After implementing this query facets is going to be returned along
with this search result in following format:
FACETED NAVIGATION IN SEARCH RESULT
We can represent in the form of ranges. It is useful for creating facets for fields of numeric type or
datetime.
To represent facets in ranges we have to create buckets. We can define range for facets as follows:
The above query only defines what to highlight in search result. To provide information for how to highlight
items in search result we have to use “highlightPreTag” and “highlightPostTag” parameters. These
parameters are used to provide values in form of HTML tags. For ex:
{
“search”:”North”,
“highlight”:”name,breweryName”,
“highlightPreTag”:”<span style=‘background-color:orange;’>”,
“highlightPostTag:”</span>”
}
SEARCH SUGGESTION IN AUTO-COMPLETE
Suggesters are part of Azure index. To configure search suggestion we have to configure Azure search
index.
Suggesters are defined by two required fields : name and source fields
Source fields in suggester specifies which fields are examined for suggestions. It can accept multiple field
names in the form of array.
We can define suggester while building index. For ex:
SEARCH SUGGESTION IN AUTO-COMPLETE
To request search suggestion from Azure search append suggest path For ex:
/indexes/{indexName}/docs/suggest
The above endpoint can be requested by GET or POST request.
We can use search parameter with above endpoints to which suggestions we are looking for.
It can only between 1 and 100 characters.
It can not use simple and full query search features.
We can also specify field names for which we are looking suggestions. Fields names must be included in suggestion
definition.
We can enable fuzzy searches for getting suggestions. Fuzzy searches helps to forgive spelling error provided for search.
Syntax for query is as follows:
{
“search”:”Ale*”,
“suggesterName”:”beerSuggestions”,
“searchFields”:”name”,
“fuzzy”:”true”
}
SEARCH SUGGESTION IN AUTO-COMPLETE
To return additional fields in suggestions. We can use “select” parameter for this purpose.
Top parameter is used to specify max number of suggestions to return.
Five is the default value to return suggestion result.
Scoring profiles or relevancy is not used with suggestions.
We can sort suggestion result by using “order by”
{
“search”:”north”,
“suggesterName”:”beerSuggestions”,
“select”:”name, breweryName, abv, ibu”,
“top”:3,
“orderby”:”name”
}