当前位置：网站首页>Import data to the database? Try the copy from stdin statement
Import data to the database? Try the copy from stdin statement
2022-06-11 15:47:00 【Gauss squirrel Club】
Users can use the following methods through COPY FROM STDIN Statement directly to openGauss Write data .
- Enter... Through the keyboard openGauss Database writes data . For details, see COPY.
- adopt JDBC Driven CopyManager Interface from file or database to openGauss Write data . This method supports COPY In the syntax copy option All parameters of .
1.CopyManager Class introduction
CopyManager yes openGauss JDBC One provided in the driver API Interface class , Used for batch delivery openGauss Import data into the database .
CopyManager Inheritance relationship of
CopyManager Class is located org.postgresql.copy Package in , Inherited from java.lang.Object class , The declaration of this class is as follows ：
public class CopyManager
extends Object
Construction method
public CopyManager(BaseConnection connection)
throws SQLException
Common methods
surface 1 CopyManager Common methods
2. Processing error table
Operation scenario
When an error occurs in data import , Please handle according to the guidance information in this article .
Query error message
Error occurred during data import , It is generally divided into data format errors and non data format errors .
Data format error
When creating appearances , By setting parameters “LOG INTO error_table_name”, Write the data format error information during data import into the specified error information table error_table_name in . You can use the following SQL, Query detailed error information .
openGauss=# SELECT * FROM error_table_name;The structure of the error information table is as follows surface 1 Shown .
surface 1 Error message table
In the data source file , Line number with data format error .
In the data source file , Original record with data format error .
Non data format error
For non data format errors , Once this happens, the entire data import will fail . You can perform the data import process according to , Error message prompted on the interface , Help locate the problem , Processing error table .
Processing data import error
According to the error information obtained , Please refer to the following table , Processing data import error .
surface 2 Processing data import error
3. Example 1： Import and export data through local file
In the use of JAVA Language based openGauss During the secondary development , have access to CopyManager Interface , By streaming , Export the data in the database to a local file or import the local file into the database , File format support CSV、TEXT Equiform .
The sample program is as follows , When executing, you need to load openGauss Of JDBC drive .
import java.sql.Connection;
import java.sql.DriverManager;
import java.io.IOException;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.sql.SQLException;
import org.postgresql.copy.CopyManager;
import org.postgresql.core.BaseConnection;
public class Copy{
public static void main(String[] args)
{
String urls = new String("jdbc:postgresql://localhost:8000/postgres"); // database URL
String username = new String("username"); // user name
String password = new String("passwd"); // password
String tablename = new String("migration_table"); // Define table information
String tablename1 = new String("migration_table_1"); // Define table information
String driver = "org.postgresql.Driver";
Connection conn = null;
try {
Class.forName(driver);
conn = DriverManager.getConnection(urls, username, password);
} catch (ClassNotFoundException e) {
e.printStackTrace(System.out);
} catch (SQLException e) {
e.printStackTrace(System.out);
}
// Will table migration_table Export data from to local file d:/data.txt
try {
copyToFile(conn, "d:/data.txt", "(SELECT * FROM migration_table)");
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// take d:/data.txt Import data from to migration_table_1 in .
try {
copyFromFile(conn, "d:/data.txt", tablename1);
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// Will table migration_table_1 Export the data in to a local file d:/data1.txt
try {
copyToFile(conn, "d:/data1.txt", tablename1);
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public static void copyFromFile(Connection connection, String filePath, String tableName)
throws SQLException, IOException {
FileInputStream fileInputStream = null;
try {
CopyManager copyManager = new CopyManager((BaseConnection)connection);
fileInputStream = new FileInputStream(filePath);
copyManager.copyIn("COPY " + tableName + " FROM STDIN with (" + "DELIMITER"+"'"+ delimiter + "'" + "ENCODING " + "'" + encoding + "')", fileInputStream);
} finally {
if (fileInputStream != null) {
try {
fileInputStream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
public static void copyToFile(Connection connection, String filePath, String tableOrQuery)
throws SQLException, IOException {
FileOutputStream fileOutputStream = null;
try {
CopyManager copyManager = new CopyManager((BaseConnection)connection);
fileOutputStream = new FileOutputStream(filePath);
copyManager.copyOut("COPY " + tableOrQuery + " TO STDOUT", fileOutputStream);
} finally {
if (fileOutputStream != null) {
try {
fileOutputStream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
}4. Example 2： from MY towards openGauss Database for data migration
The following example demonstrates how to use CopyManager from MY towards openGauss The process of data migration of database .
import java.io.StringReader;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import org.postgresql.copy.CopyManager;
import org.postgresql.core.BaseConnection;
public class Migration{
public static void main(String[] args) {
String url = new String("jdbc:postgresql://localhost:8000/postgres"); // database URL
String user = new String("username"); //openGauss Database user name
String pass = new String("passwd"); //openGauss Database password
String tablename = new String("migration_table_1"); // Define table information
String delimiter = new String("|"); // Define separator
String encoding = new String("UTF8"); // Define character set
String driver = "org.postgresql.Driver";
StringBuffer buffer = new StringBuffer(); // Define the cache for storing formatted data
try {
// Get the source database query result set
ResultSet rs = getDataSet();
// Traversal result set , Get records line by line
// The value of each field in each record , Split by the specified separator , End with newline , Put together a string
// Put together a string , Add to cache buffer
while (rs.next()) {
buffer.append(rs.getString(1) + delimiter
+ rs.getString(2) + delimiter
+ rs.getString(3) + delimiter
+ rs.getString(4)
+ "\n");
}
rs.close();
try {
// Establish the target database connection
Class.forName(driver);
Connection conn = DriverManager.getConnection(url, user, pass);
BaseConnection baseConn = (BaseConnection) conn;
baseConn.setAutoCommit(false);
// Initialization table information
String sql = "Copy " + tablename + " from STDIN with (DELIMITER " + "'" + delimiter + "'" +","+ " ENCODING " + "'" + encoding + "'");
// Commit cache buffer Data in
CopyManager cp = new CopyManager(baseConn);
StringReader reader = new StringReader(buffer.toString());
cp.copyIn(sql, reader);
baseConn.commit();
reader.close();
baseConn.close();
} catch (ClassNotFoundException e) {
e.printStackTrace(System.out);
} catch (SQLException e) {
e.printStackTrace(System.out);
}
} catch (Exception e) {
e.printStackTrace();
}
}
//********************************
// Return the query result set from the source database
//*********************************
private static ResultSet getDataSet() {
ResultSet rs = null;
try {
Class.forName("com.MY.jdbc.Driver").newInstance();
Connection conn = DriverManager.getConnection("jdbc:MY://10.119.179.227:3306/jack?useSSL=false&allowPublicKeyRetrieval=true", "jack", "[email protected]");
Statement stmt = conn.createStatement();
rs = stmt.executeQuery("select * from migration_table");
} catch (SQLException e) {
e.printStackTrace();
} catch (Exception e) {
e.printStackTrace();
}
return rs;
}
}边栏推荐
- [0006] titre, mots clés et description de la page
- GO语言-数组Array
- Shuttle -- hero component
- 零基础自学软件测试，我花7天时间整理了一套学习路线，希望能帮助到大家..
- Unified record of my code variable names
- Google Earth engine (GEE) - create a simple panel demo to display the map
- Verification code is the natural enemy of automation? Ali developed a solution
- Dapr mind map
- 数据库密态等值查询概述及操作
- 拿到20K我用了5年，面了所有大厂，这些高频面试问题都帮你们划出来啦
猜你喜欢
AGC安全规则是如何简化用户授权和验证请求
Analysis of the execution process of opengauss simple query SQL
从0到1稳稳掌握大厂主流技术，年后涨薪不是必须的吗？
![[Yugong series] June 2022 Net architecture class 079 cluster principle of distributed middleware schedulemaster](/img/bc/694f8baec114cc3f6e35c4c76c29f0.png)
[Yugong series] June 2022 Net architecture class 079 cluster principle of distributed middleware schedulemaster
零基础自学软件测试，我花7天时间整理了一套学习路线，希望能帮助到大家..
07 _ Functions and disadvantages of row lock: how to reduce the impact of row lock on performance?
数据库资源负载管理（下篇）
Dapr mind map
Interview shock 26: how to stop threads correctly?
From 0 to 1, master the mainstream technology of large factories steadily. Isn't it necessary to increase salary after one year?
随机推荐
向数据库导入数据？试试COPY FROM STDIN语句
leetcode 120. 三角形最小路径和
Methodology and practice of real-time feature computing platform architecture
如何管理并发写入操作？带你快速上手
What is the future of software testing in 2022? Do you need to understand the code?
[digital signal processing] correlation function (correlation function property | conjugate symmetry property of correlation function | even symmetry of real signal autocorrelation function | conjugat
IDEA2021.1版本安装教程
NielsenIQ宣布任命Tracey Massey为首席运营官
前沿科技探究之AI功能：慢SQL发现
使用Cloud DB构建APP 快速入门-Server篇
[Yugong series] June 2022 Net architecture class 078 worker cluster of distributed middleware schedulemaster
openGauss企业版安装
Why are bugs changing more and more?
GO語言-值類型和引用類型
openGauss 3.0.0版本正式发布，立即体验社区首个轻量版本
微软韦青：狗尾巴的故事—数智时代的第一性原理 | 极客时间
Go language - array
如何预测SQL语句查询时间？
前沿科技探究之AI工具：Anomaly-detection
Charles自动保存响应数据