New function | intelligent open search online customized word weight model

Catalog

Business pain points

Pain points 1： Strong industry attribute

Pain points 2： It is difficult to build a model by oneself 、 The high cost 、 Cycle is long

Open search lightweight customized word weight solution

Introduction of the plan

For customers

Usage method

Effect comparison

E-commerce Scene Effect Comparison

Content Scene Effect Comparison

---

Business pain points

In the query analysis stage of search , Word weight will analyze the importance of each word in the text , And quantify it into weights , Words with low weight may not participate in search recall . Use word weight model , It can avoid when the query words entered by the user contain some words with low weight , Recall is still limited by the original input , Cause too few hits . therefore , Word weight can effectively improve the recall rate of search , It is an indispensable function in the search, query and analysis stage .

Pain points 1： Strong industry attribute

Customers' search applications often belong to specific industries , When the general word weight model is applied in a specific industry, due to the lack of industry knowledge , There will be more badcase Affect the recall sorting effect .

Based on the industry model, the adaptation problem of word weight model in customer search application can be alleviated , However, if the customer's data distribution belongs to a specific vertical category in the industry or there is no corresponding industry, it can be linked , You need to customize the word weight based on customer data to achieve the best effect .

Pain points 2： It is difficult to build a model by oneself 、 The high cost 、 Cycle is long

The self built word weight model mainly includes the following processes ：

difficulty 1： Word weight tagging requires high domain knowledge , It is difficult to judge the importance of different words in search engines . At the same time, the amount of data also needs to reach at least 10000 , It can take months .

difficulty 2： The threshold of model training is high , Need professional algorithm practitioners to debug , And the model effect 、 Iteration efficiency strongly depends on the investment and ability of Algorithm Engineers .

difficulty 3： Model deployment 、 The operation and maintenance process is complex , Engineering required 、 Algorithm 、 Operation and maintenance and other parties participate , And the launch of the depth model also involves performance 、 Many optimizations related to efficiency .

Open search lightweight customized word weight solution

Introduction of the plan

Before the search text recall , Open search will analyze and process the query semantics of the keywords entered by users . Due to the diversity of business scenarios , Different industries and businesses have their own particularity , Only the word weight model specific to the application level can guarantee the optimal search effect .

Compared with participle , Word weight plays a role in both recall and relevance ranking . In the recall phase , The word weight model will give each search word a corresponding weight score . First query , Words with low weight do not require a hit in the query, but will participate in the calculation . If you query zero or less results for the first time , The second query will lower 、 Words with medium weight are not mandatory to have a hit , So as to expand the recall . Correlation sorting stage , The weight of each search term given by the word weight model will participate in the calculation of correlation characteristics . When there are hits , Documents with high weight words will get higher scores , Thus ranking higher . The customized word weight model is customized based on the customer's own business data , Greatly improve search results .

OpenSearch It provides rich domain specific word weight models , Users can base on the corresponding industry analyzer , After simple configuration training, we get a customized word weight model . After training , Users can view the variance rate in the console 、 Typical word weight case Compare and other model effects , Wait until the effect meets expectations , This customized word weight model can be used in open search , And support the manual intervention of word weight effect .

The whole customization process does not require additional data docking , Word weight model training will automatically extract existing data for adaptation .

For customers

1. Search is an important scenario of core business , Customers who have higher requirements for search results
2. industry 、 Pendula 、 Special business , Customers with more exclusive terms
3. Search manpower is limited , Algorithm students have relatively few customers

Usage method

1. Import business data
2. Create word weight model , Select the training field and create the model
3. After model training , Quote word weight model in query analysis

For more instructions, please refer to ： Recall custom word weights - Intelligent open search OpenSearch - Alibaba cloud

Effect comparison

E-commerce Scene Effect Comparison

original text	Weight of e-commerce common words	Customize word weights
Four piece gift set in winter	winter : in Four piece suit : high gift : high	winter : in Four piece suit : high gift : low
Bear digital display warm hand treasure	Little bear : low Digital display : in Hand warmer : high	Little bear : in Digital display : low Hand warmer : high
Defibrillator compatibility package	Defibrillator : high compatible : in package : low	Defibrillator : in compatible : low package : high
22 Annual calendar	22: low year : low wall calendar : high	22: in year : low wall calendar : high

Content Scene Effect Comparison

original text	Common word weight	Customize word weights
Potential function	potential : low function : high	potential : high function : low
get post difference	get: in post: high difference : low	get: high post: high difference : low
ktv Song ordering system	ktv: in choose a song : high System : in	ktv: high choose a song : in System : low

Summary ：

• If your business is currently using or preparing to use the industry version of open search , You can train the customized word weight model based on the industry model ;
• If open search has not yet provided an industry version close to your business , It is suggested to train the customized word weight model based on the general version model , This situation requires as much data as possible , The distribution shall be as comprehensive and balanced as possible , Help to improve the effect of the model ;
• Open search currently supports Custom word breaker 、 Custom word weight model , More customized recall models will be provided later , Coming soon ~