当前位置:网站首页>Multi classification of unbalanced text using AWS sagemaker blazingtext
Multi classification of unbalanced text using AWS sagemaker blazingtext
2020-11-06 01:22:00 【InfoQ】
background
Text classification (Text Classification) It belongs to the field of natural language processing , It refers to the process that the computer maps a text containing information to a given category or several categories of topics in advance . But in reality , We often encounter imbalances in the categories of data samples (class imbalance) The phenomenon , It seriously affects the final result of text classification . The so-called sample imbalance refers to a given data set, some categories of data more , Some data categories are few , And the data category samples with more data proportion and data category samples with small proportion reach a large proportion .
BlazingText yes AWS SageMaker A built-in algorithm for , Provides Word2vec And text classification algorithm highly optimized implementation . This article uses Sagemaker BlazingText It realizes the text multi classification . On the problem of sample imbalance , Back translation and EDA Two methods are used to over sample a small number of samples , The back translation method calls AWS Translate The service was translated and retranslated , and EDA Methods mainly use synonyms to replace 、 Insert randomly 、 Random exchange 、 Random deletion deals with text data . This article also uses AWS SageMaker Automatic parametric optimization for BlazingText The text classification algorithm based on the algorithm finds the optimal hyperparameter .
This article is based on DBpedia The public dataset generated by processing contains 14 Unbalanced text data of categories , And did not do any sample imbalance processing Baseline Experiment and include back translation and EDA Oversampling experiments of two methods .
Link to the original text :【https://www.infoq.cn/article/xbSAYuJcQrm048GHl5dJ】. Without the permission of the author , Prohibited reproduced .
版权声明
本文为[InfoQ]所创,转载请带上原文链接,感谢
边栏推荐
- Leetcode's ransom letter
- 基於MVC的RESTFul風格API實戰
- PHPSHE 短信插件说明
- 6.4 viewresolver view parser (in-depth analysis of SSM and project practice)
- 怎么理解Python迭代器与生成器?
- 快快使用ModelArts,零基礎小白也能玩轉AI!
- Swagger 3.0 天天刷屏,真的香嗎?
- Relationship between business policies, business rules, business processes and business master data - modern analysis
- ipfs正舵者Filecoin落地正当时 FIL币价格破千来了
- Python + appium automatic operation wechat is enough
猜你喜欢
合约交易系统开发|智能合约交易平台搭建
数据产品不就是报表吗?大错特错!这分类里有大学问
Character string and memory operation function in C language
What is the side effect free method? How to name it? - Mario
Thoughts on interview of Ali CCO project team
如何玩转sortablejs-vuedraggable实现表单嵌套拖拽功能
加速「全民直播」洪流,如何攻克延时、卡顿、高并发难题?
Subordination judgment in structured data
It's so embarrassing, fans broke ten thousand, used for a year!
JVM memory area and garbage collection
随机推荐
Did you blog today?
中国提出的AI方法影响越来越大,天大等从大量文献中挖掘AI发展规律
容联完成1.25亿美元F轮融资
IPFS/Filecoin合法性:保护个人隐私不被泄露
采购供应商系统是什么?采购供应商管理平台解决方案
钻石标准--Diamond Standard
全球疫情加速互联网企业转型,区块链会是解药吗?
[JMeter] two ways to realize interface Association: regular representation extractor and JSON extractor
6.2 handleradapter adapter processor (in-depth analysis of SSM and project practice)
从海外进军中国,Rancher要执容器云市场牛耳 | 爱分析调研
In depth understanding of the construction of Intelligent Recommendation System
xmppmini 專案詳解:一步一步從原理跟我學實用 xmpp 技術開發 4.字串解碼祕笈與訊息包
带你学习ES5中新增的方法
Examples of unconventional aggregation
6.3 handlerexceptionresolver exception handling (in-depth analysis of SSM and project practice)
100元扫货阿里云是怎样的体验?
Summary of common algorithms of linked list
关于Kubernetes 与 OAM 构建统一、标准化的应用管理平台知识!(附网盘链接)
How to encapsulate distributed locks more elegantly
In order to save money, I learned PHP in one day!