当前位置:网站首页>RuntimeError: CUDA error: CUBLAS_STATUS_ALLOC_FAILED when calling `cublasCreate(handle)`问题解决
RuntimeError: CUDA error: CUBLAS_STATUS_ALLOC_FAILED when calling `cublasCreate(handle)`问题解决
2022-07-07 02:18:00 【不撸先疯。】
一、问题描述
使用transformers包调用pytorch框架的Bert预训练模型时,使用正常的bert-base-cased在其他数据集上正常运行,但是运用Roberta的时候却一直报错:RuntimeError: CUDA error: CUBLAS_STATUS_ALLOC_FAILED when calling `cublasCreate(handle)`
忙活了好几天也没查出是什么错误,网上一直提醒batch_size是否太大导致,修改为16->8->4->2都没有用。
通过与其他数据集的对比,发现我在tokenizer中加入了新的special_token,导致可能报错了!
二、问题解决
在原始的tokenizer中加入special_tokens时,忘记将model的tokenizer的词表进行更新导致!
完整更新方式为:
from transformers import BertTokenizer, BertModel
tokenizer = BertTokenizer.from_pretrained('bert-base-cased')
# 添加特殊词
tokenizer.add_special_tokens({'additional_special_tokens':["<S>"]})
model = BertModel.from_pretrained("bert-base-cased")
# 在模型中更新词表的大小!
# 重要!
model.resize_token_embeddings(len(tokenizer))
三、问题解决
可以通过,开始训练!
边栏推荐
- 学习笔记|数据小白使用DataEase制作数据大屏
- Go straight to the 2022ecdc fluorite cloud Developer Conference: work with thousands of industries to accelerate intelligent upgrading
- Symmetric binary tree [tree traversal]
- Programmers' daily | daily anecdotes
- MySQL (x)
- C面试24. (指针)定义一个含有20个元素的double型数组a
- Force deduction 62 different paths (the number of all paths from the upper left to the lower right of the matrix) (dynamic planning)
- 软件测试到了35岁,真的就干不动了吗?
- Open the blue screen after VMware installation
- "Parse" focalloss to solve the problem of data imbalance
猜你喜欢
PostgreSQL database timescaledb function time_ bucket_ Gapfill() error resolution and license replacement
Abnova 体外转录 mRNA工作流程和加帽方法介绍
Test the foundation of development, and teach you to prepare for a fully functional web platform environment
Which foreign language periodicals are famous in geology?
面试中有哪些经典的数据库问题?
基于FPGA的VGA协议实现
「解析」FocalLoss 解决数据不平衡问题
POI export to excel: set font, color, row height adaptation, column width adaptation, lock cells, merge cells
matlab / ENVI 主成分分析实现及结果分析
字符串常量与字符串对象分配内存时的区别
随机推荐
[solution] final app status- undefined, exitcode- 16
Array proof during st table preprocessing
Unity C# 函数笔记
【从零开始】win10系统部署Yolov5详细过程(CPU,无GPU)
How to find the literature of a foreign language journal?
Doctoral application | Professor Hong Liang, Academy of natural sciences, Shanghai Jiaotong University, enrolls doctoral students in deep learning
Kotlin之 Databinding 异常
Ant manor safety helmet 7.8 ant manor answer
A program lets you understand what static inner classes, local inner classes, and anonymous inner classes are
Abnova 膜蛋白脂蛋白体技术及类别展示
反射(二)
Jmeter 5.5版本发布说明
Abnova循环肿瘤DNA丨全血分离,基因组DNA萃取分析
How can I check the DOI number of a foreign document?
Party A's requirements for those who have lost 800 yuan
MySQL(十)
[opencv] morphological filtering (2): open operation, morphological gradient, top hat, black hat
「运维有小邓」符合GDPR的合规要求
Navicat导入15G数据报错 【2013 - Lost connection to MySQL server during query】 【1153:Got a packet bigger】
tkinter窗口选择pcd文件并显示点云(open3d)