当前位置:网站首页>Error when creating dataset with mindscore
Error when creating dataset with mindscore
2022-07-25 19:41:00 【Xiaole happy】
Creating a dataset by yourself will cause errors
【 Operation steps & Problem phenomenon 】
I want to create a super-resolution dataset , This data set has two columns ,data and target,data Is a low resolution image ,target High resolution image . My idea is to create a data and target Image like data set , And then to data Reduce the image in , But an error is reported in the process of creating the dataset
My code
from os import listdir
from os.path import join
import os
from PIL import Image
import random
import numpy as np
import mindspore.dataset as ds
import mindspore.dataset.vision.py_transforms as py_vision
import mindspore.dataset.transforms.py_transforms as py_transforms
def is_image_file(filename):
return any(filename.endswith(extension) for extension in ['.png', '.jpg', '.jpeg', '.PNG', '.JPG', '.JPEG'])
def make_dataset(dir_path, max_dataset_size=float("inf")):
"""Return image list in dir."""
images = []
assert os.path.isdir(dir_path), '%s is not a valid directory' % dir_path
for root, _, fnames in sorted(os.walk(dir_path)):
for fname in fnames:
if is_image_file(fname):
path = os.path.join(root, fname)
images.append(path)
return images[:min(max_dataset_size, len(images))]
class MakeDataset:
def __init__(self, dataset_dir, max_dataset_size=float("inf")):
self.dir_A = os.path.join(dataset_dir,'data')
self.dir_B = os.path.join(dataset_dir,'target')
self.A_paths = sorted(make_dataset(self.dir_A, max_dataset_size))
self.B_paths = sorted(make_dataset(self.dir_B, max_dataset_size))
self.A_size = len(self.A_paths)
self.B_size = len(self.B_paths)
def __getitem__(self, index):
A_path = self.A_paths[index % self.A_size]
B_path = self.B_paths[index % self.B_size]
A_img = np.array(Image.open(A_path).convert('RGB'))
B_img = np.array(Image.open(B_path).convert('RGB'))
print(A_img.shape)
print(B_img.shape)
return A_img,B_img
def __len__(self):
return self.A_size
dataset=MakeDataset(dataset_dir="/data/xjtu/wys/image/")
DS=ds.GeneratorDataset(dataset,column_names=["LR,HR"])
for data in DS.create_dict_iterator():
print(data["LR"].shape)Error message
Traceback (most recent call last):
File "/data/xjtu/wys/SRGAN/dataset.py", line 49, in <module>
for data in DS.create_dict_iterator():
File "/root/archiconda3/envs/xjtu/lib/python3.7/site-packages/mindspore/dataset/engine/iterators.py", line 263, in __next__
data = self.get_next()
File "/root/archiconda3/envs/xjtu/lib/python3.7/site-packages/mindspore/dataset/engine/iterators.py", line 344, in get_next
return {k: Tensor(v.as_array()) for k, v in self.depipeline.GetNextAsMap().items()}
RuntimeError: Thread ID 281472224702944 Exception thrown from PyFunc. Invalid parameter, Generator should return same number of numpy arrays as specified in column names.
Line of code : 122
File : /home/jenkins/agent-working-dir/workspace/Compile_Ascend_ARM_EulerOS/mindspore/mindspore/ccsrc/minddata/dataset/engine/datasetops/source/generator_op.ccReport errors Generator should return same number of numpy arrays as specified in column names., Description means that the returned field does not match the number of specified column names .
I see in the code :class MakeDataset: -> def __getitem__(self, index): -> return A_img,B_img Two fields are returned .
however DS=ds.GeneratorDataset(dataset,column_names=["LR,HR"]) As defined in column_names Need to change to column_names=["LR", "HR"], Expressed as two fields .
边栏推荐
- The query data returned by the print database is null or the default value. Does not match the value returned by the database
- what is qml in qt
- Binary tree visualization
- Flutter tips: optimizing the buildcontext you use
- Split very long line of words into separate lines of max length
- Illegal mix of collations for operation ‘UNION‘(bug记录)
- Code sharing of social chat platform developed by dating website (III)
- JS learning notes 16: switching pictures small project practice
- Concept of IP address
- [record of question brushing] 21. Merge two ordered linked lists
猜你喜欢

滑雪手机端H5小游戏源码下载

Research and application of servo driver in robot

二叉树可视化
![[record of question brushing] 21. Merge two ordered linked lists](/img/81/5f1ab0054ef5e8cba32f7a546a64c4.png)
[record of question brushing] 21. Merge two ordered linked lists

哪吒 D1-H 测试 microbench
![Scala foundation [set 01]](/img/6b/0f5da7ea923ef3aa436d7af9c4425c.png)
Scala foundation [set 01]

Legal mix of collations for operation 'Union' (bug record)

C语言学习日记3——realloc函数
![[wp]ctfshow-web introductory information collection](/img/22/c2e5cca918800dda9df27272eb9871.png)
[wp]ctfshow-web introductory information collection

连接数据库警告 Establishing SSL connection without server‘s identity verification is not recommended.
随机推荐
软件设计师下午真题:2009-2022
Oracle数据库下载、安装、使用教程及问题汇总
Research and application of servo driver in robot
Scala foundation [set 01]
A high efficiency 0-delay 0-copy QT player scheme based on Hisilicon 3559
JS learning notes 16: switching pictures small project practice
[record of question brushing] 21. Merge two ordered linked lists
Small program completion work wechat campus maintenance application small program graduation design finished product (2) small program function
Wechat campus maintenance and repair applet graduation design finished product of applet completion work (4) opening report
Shopping guide for high-end flagship projectors: dangbei X3 pro and dangbei F5 are more immersive!
919. Complete binary tree inserter
Wechat campus maintenance and repair applet graduation design finished product (5) assignment of applet completion work
Old wine in new bottles -- sample analysis of recent apt32 (sea Lotus) organizational attacks
JS learning notes 17: DOM query exercise
Improvement of wechat applet 26 playing music page ②
重磅!《几何深度学习》新书发布,帝国理工/DeepMind等图ML大牛共同撰写,160页pdf阐述几何DL基础原理和统一框架
Yyds dry inventory how to locate browser page crash
什么是唯心主义
Kcon 2022 highlights and agenda revealed!
C # add multi line and multi column text watermark in word