当前位置:网站首页>深度学习之数据处理
深度学习之数据处理
2022-07-06 16:54:00 【彭祥.】
数据操作
- 数据类型,我们最常用的便是数组了
创建数组需要
- 形状:几行几列
- 元素类型:int还是float
- 元素值
数组访问方式:
代码:
这种机制的工作方式如下:首先,通过适当复制元素来扩展一个或两个数组, 以便在转换之后,两个张量具有相同的形状。 其次,对生成的数组执行按元素操作。
由于a和b分别是和矩阵,如果让它们相加,它们的形状不匹配。 我们将两个矩阵广播为一个更大的矩阵,如下所示:矩阵a将复制列, 矩阵b将复制行,然后再按元素相加。
数据预处理
创建文件并写入数据
import os
os.makedirs(os.path.join('.', 'data'), exist_ok=True)#在当前目录下创建data文件夹
data_file = os.path.join('.', 'data', 'house_tiny.csv')#在data文件夹下创建house_tiny.csv
print(data_file)
with open(data_file, 'w') as f:
f.write('NumRooms,Alley,Price\n') # 列名
f.write('NA,Pave,127500\n') # 每行表示一个数据样本
f.write('2,NA,106000\n')
f.write('4,NA,178100\n')
f.write('NA,NA,140000\n')
读取文件,对于csv文件多用pandas这个库
import pandas as pd
data=pd.read_csv(data_file)
print(data)
数据处理缺失值与转换
对于缺失值,我们可以采用插入法和删除法两种,插入即我们给定取值,删除则是直接删除不再考虑,这里我们采用缺失值取均值的方式
inputs,outputs = data.iloc[:, 0:2], data.iloc[:, 2]#按照文件格式读取数据,读第一列至第二列
inputs = inputs.fillna(inputs.mean())#对于缺少的数值我们一般取其他值的均值
inputs = pd.get_dummies(inputs, dummy_na=True)#对于string类型我们看到Alley取值只有Pave和NaN,所以我们可以将Pave记为1,NaN记为0
print(inputs)
将我们的数据转换为张量
import torch
x,y=torch.tensor(inputs.values),torch.tensor(outputs.values)
print(x,y)
到这里,我们便将数据转换为tensor的张量,这种对于计算机是可处理的
完整代码:
import os
os.makedirs(os.path.join('.', 'data'), exist_ok=True)
data_file = os.path.join('.', 'data', 'house_tiny.csv')
print(data_file)
with open(data_file, 'w') as f:
f.write('NumRooms,Alley,Price\n') # 列名
f.write('NA,Pave,127500\n') # 每行表示一个数据样本
f.write('2,NA,106000\n')
f.write('4,NA,178100\n')
f.write('NA,NA,140000\n')
import pandas as pd
data=pd.read_csv(data_file)
print(data)
inputs,outputs = data.iloc[:, 0:2], data.iloc[:, 2]#按照文件格式读取数据,读第一列至第二列
inputs = inputs.fillna(inputs.mean())#对于缺少的数值我们一般取其他值的均值
inputs = pd.get_dummies(inputs, dummy_na=True)
print(inputs)
import torch
x,y=torch.tensor(inputs.values),torch.tensor(outputs.values)
print(x,y)
边栏推荐
- Introduction to GPIO
- 沉浸式投影在线下展示中的三大应用特点
- Use mujoco to simulate Cassie robot
- On February 19, 2021ccf award ceremony will be held, "why in Hengdian?"
- If the college entrance examination goes well, I'm already graying out at the construction site at the moment
- After leaving a foreign company, I know what respect and compliance are
- 37 page overall planning and construction plan for digital Village revitalization of smart agriculture
- Interesting wine culture
- uniapp中redirectTo和navigateTo的区别
- Matlab learning notes
猜你喜欢
The difference between redirectto and navigateto in uniapp
Mujoco Jacobi - inverse motion - sensor
Introduction to GPIO
Article management system based on SSM framework
GPIO简介
[2022 the finest in the whole network] how to test the interface test generally? Process and steps of interface test
Mujoco second order simple pendulum modeling and control
X.509 certificate based on go language
准备好在CI/CD中自动化持续部署了吗?
37 pages Digital Village revitalization intelligent agriculture Comprehensive Planning and Construction Scheme
随机推荐
Use type aliases in typescript
Wechat applet UploadFile server, wechat applet wx Uploadfile[easy to understand]
Advanced learning of MySQL -- basics -- multi table query -- subquery
2022年PMP项目管理考试敏捷知识点(9)
Lombok 同时使⽤ @Data 和 @Builder 的坑,你中招没?
What can the interactive slide screen demonstration bring to the enterprise exhibition hall
37 pages Digital Village revitalization intelligent agriculture Comprehensive Planning and Construction Scheme
Imeta | Chen Chengjie / Xia Rui of South China Agricultural University released a simple method of constructing Circos map by tbtools
[daily problem insight] prefix and -- count the number of fertile pyramids in the farm
build. How to configure the dependent version number in the gradle file
Idea automatically imports and deletes package settings
JWT signature does not match locally computed signature. JWT validity cannot be asserted and should
Introduction au GPIO
Introduction to GPIO
Web project com mysql. cj. jdbc. Driver and com mysql. jdbc. Driver differences
DAY FIVE
Memory optimization of Amazon memorydb for redis and Amazon elasticache for redis
iMeta | 华南农大陈程杰/夏瑞等发布TBtools构造Circos图的简单方法
Explain in detail the implementation of call, apply and bind in JS (source code implementation)
GPIO簡介