Before applying data warehouse ODBC, you need to understand these problems first

This article is shared from Huawei cloud community 《GaussDB(DWS) ODBC Problem orientation guide 》, author ： power_gouge .

The user's Application , Called ODBC Of API, In fact, it is provided by the driver manager , The driver manager on Microsoft is odbcad32(.exe/.dll), stay Linux Up is UnixODBC. Then they call ODBC Specific driver （Gauss、Oracle、TD……）.

Therefore, the general problems basically lie in the following aspects ：

• Application load driver manager
• The driver manager loads the driver
• Drive connection to database

A typical user application is as follows ：

Problem handling

Application failed to load driver manager

Windows

Windows Due to the problem of operating system implementation mechanism ,ODBC Part of an operating system component , Unless the operating system is damaged , Generally, there will be no application loading driver manager failure ; Only the driver manager fails to load the driver . So I won't repeat it here .

Linux

Linux Application load driver manager on , Means that your application depends on UnixODBC Related dynamic libraries , When starting, you should load it into your own process space . There are many problems in loading, which are manifested as that... Cannot be found at startup libodbc.so.x/libodbcinst.so.x Such as the library .

Solve this problem ：

First, make sure that... Is installed in the environment UnixODBC. If installed , Then at least there will be isql/odbcinst And other tools are available . For example, you can use which isql Such a command to determine the installation path . dependent lib The library is usually in which isql designated bin Under the same level directory of the directory , Add it to LD_LIBRARY_PATH Restart the application in .

export LD_LIBRARY_PATH=/where/unixodbc/installed/lib:$LD_LIBRARY_PATH

Particular attention , This sentence only affects At present shell Subsequent commands in the session , Other sessions are not affected , Especially the process started in the background . If it's a background process , Please confirm with the developer of the application that if the environment is modified .

If confirmed LD_LIBRARY_PATH Has to take effect （ Can open /proc/ Applications pid/environ Document confirmation ）, Still unable to load libodbc.so.x/libodbcinst.so.x, So it could be .1 perhaps .2 The problem with the library of .

Historical reasons , In a general development environment ,UnixODBC The library provided may be called libodbc.so.1 It could be called libodbc.so.2. User programs may rely on .2, But there are only .1; It may also exist .2, But it depends on .1.

Solve this problem , Only need UnixODBC Of lib Under the table of contents , Build a non-existent version into a soft link , Guarantee .1 And .2 All exist , In this way, the probability of problems will be much smaller . Particular attention , It is strongly recommended to establish soft links , Instead of copying physical files , Otherwise, there may be an application process , Two identical exported functions are loaded .so, This may cause some difficult startup errors , I will talk about .

Driver manager failed to load driver

Windows

Most common x86 And x64 Loading failure caused by different architectures ; The typical error report of this problem is ：

At the designated DSN in , Architecture mismatch between driver and Application

Possible causes of this problem ： stay 64 Bit program uses 32 Bit drive , Or vice versa .

stay 64 Bit system ：

C:\Windows\SysWOW64\odbcad32.exe： This is a 32 position ODBC Drive manager .

C:\Windows\System32\odbcad32.exe： This is a 64 position ODBC Drive manager .

When using applications with different architectures , Please use a different driver manager ; Install different drive types at the same time .

32 The bit system can only run 32 Bit program , Can't install 64 Bit drive , So there is basically no need to distinguish .

If it is 64 Bit system , Then it supports both 32 Bit program , Also support 64 Bit program ; If you are not sure that the application is 32 still 64, The following methods can be used ：

ctrl + shift + esc Open Task Manager

Check whether there is a process behind a process in the process list *32 The words... （ Yes, it is 32 position , No is 64 position ）, Here's the picture ：\

Linux

Lack of information in the environment psqlodbcw.so The library of

Generally, the error report of this problem is

[UnixODBC][Driver Manager]Can't open lib 'xxx/xxx/psqlodbcw.so' : file not found.

The possible reasons are ：

odbcinst.ini The path configured in is incorrect , The easiest way is ls -l Look at this error file , See if this really exists , At the same time, it has execution permission .

The dependent library does not exist , The easiest way to do this is ldd Look at this error file , See if the library is really missing , If the library is missing, the following problems will occur ：

To deal with this problem , Need to put unixodbc Under the installation directory of lib, Add to LD_LIBRARY_PATH in , for example ：

export LD_LIBRARY_PATH=/where/unixodbc/installed/lib:$LD_LIBRARY_PATH

meanwhile , stay UnixODBC Installed lib Under the table of contents , have a look libodbcinst.so.x The suffix after is .1 still .2. If it is .2, Then please establish a soft link , link to .1, We can solve this problem .

If there are only .1 The library of , No, .2 The library of , It is also suggested to establish a .2 The soft links , Prevent applications from relying on .2, We depend on .1, Cause conflict .

Particular attention , Of the current conversation LD_LIBRARY_PATH It doesn't mean that the application runtime is the same LD_LIBRARY_PATH, When necessary , Align with application developers .

If it's a resident process , You can get the environment variable of the current process through the following methods ：

cd /proc/ Applications PID
cat environ

The printed results may be messy , Take a patient look ; If the content is wrong , Align the modification method with the application developer .

both .2 And then there is .1 The library of , Cause repeated loading

Such diligent users are rare , But when it comes to , Hard to locate , A typical error message is ：

Driver's SQLAllocHandle on SQL_HANDLE_DBC failed

In this case, the application and driver load different physical files , This will result in two sets of function lists with the same name , Appear in the same visible field at the same time （UnixODBC Of libodbc.so.* The function export list of is exactly the same ）, Conflict , Unable to load database driver .

The solution to this problem is also relatively simple , see LD_LIBRARY_PATH The first of unixodbc Of lib Catalog . In its directory , The lack of .2 or .1, Establish corresponding .2 or .1 Soft links can .

Connection problem

The server is not reachable

Typical error reporting ：

connect to server failed: no such file or directory

Possible causes of this problem ：

Configured the wrong / Unreachable database address , Or port
Please check... In the data source configuration Servername And Port Configuration item , Make sure the network is accessible .

Incorrect server listening
If confirmed Servername And Port The configuration is correct , Please refer to the information in listen_addresses Parameter configuration method , Ensure that the database listens to the appropriate network card and port , especially Servername The address and address of the network card specified in LVS The virtual IP Address .

Remember to restart the database after modification , Make listening effective .

Firewall and gateway 、 Flow control equipment
Please confirm the firewall settings , Add the communication port of the database to the trusted port .
If there is a gateway 、 Flow control equipment , Please confirm the relevant settings .
This problem is generally caused by third-party commercial products , Need consulting services / Strategies related to product providers .

SSL Incorrect configuration

Typical error reporting 1：

The password-stored method is not supported.

terms of settlement 1：

Please set... In the data source configuration sslmode Adjust to allow And above , Allow to use SSL Connect .

Typical error reporting 2：

Server common name "xxxx" does not match host name "xxxxx"

The problem is due to sslmode Full verification is used in (verify-full), It will not only verify the certificate , It also checks the domain name of the certificate / Whether the machine name matches the certificate , This problem will be reported in case of nonconformity .

terms of settlement 2：

Give way to CA Organization with new domain name / Machine name issuing certificate ; Or will SSLMODE Adjust to verify-ca Option or below （ See the description in the product data for the specific level ）.

Using open source driver connection problem

Our database supports low version and open source driver （V1R7C10 And later ）, So you can use .

But occasionally from V1R6C10 Upgraded database , When accepting open source driver connections , You will encounter the problem that the user authentication algorithm does not support , Typical error reports are as follows ：

authentication method 10 not supported.

The mechanism of this problem is complex , Let's expand in detail , Let's start with the solution ：

Please use the new management account to create a new database user , And give it relevant access rights , Connect to the database with a new user .

Change the password of the current business user , Use the new password to connect to the database .

Explain the mechanism of this problem ：

When our database is connected , It will let the client send the authentication information . But the message sent will not be plaintext password , It's not safe ; The authentication information sent by the client to the database is generally the password hash after adding salt (md5/sha256 Algorithm ), The number of iterations is also very large .

The database itself does not store the user's plaintext password , What is stored is a hash value （ So if the password is lost , It's really lost , It can only be reset but not retrieved ）. When the database checks this password , It's a hash , This involves what algorithm is used to get the hash value of the hash stored in the database , Common is sha256（ Gauss self research ） And MD5（ Open source ）.

V1R7C00 And before , Only stored in the database SHA256 Hash , When upgrading, we can't deduce the user's original password , So after the upgrade, there are still only SHA256 Hash ; But open source clients only recognize MD5 Hash , This causes the database to fail to authenticate ; The database will only allow the client to SHA256 Hash to do authentication , At this time, the open source does not recognize the authentication request , Cause the above error to be reported .

When creating users and modifying user passwords , Our new version will record two hashes at the same time , Therefore, the following versions will support both open source and self-developed versions .

Due to incorrect data source configuration , Connection error caused by

Typical error reporting ：

Data source name not found, and no default driver specified

Possible causes ：

The data source name is incorrectly written （ Or contain special symbols ）

Under different operating system users , The visibility of data sources is different ;

Linux Data source configuration file on (odbc.ini) Not in the right place

terms of settlement ：

Fix data source name .

Windows Next , Please open the driver manager under the current user , Check whether the data source configuration is correct .
Please note that 64 Use the correct data source manager on the bit system ：

C:\Windows\SysWOW64\odbcad32.exe： This is a 32 position ODBC Drive manager .
C:\Windows\System32\odbcad32.exe： This is a 64 position ODBC Drive manager .

Linux Next use odbcinst -j command , View the correct path of the configuration file in the current environment , The output is generally as follows ：

unixODBC 2.3.0
DRIVERS............: /usr/local/etc/odbcinst.ini/odbcinst.ini
SYSTEM DATA SOURCES: /usr/local/etc/odbcinst.ini/odbc.ini
FILE DATA SOURCES..: /usr/local/etc/odbcinst.ini/ODBCDataSources
USER DATA SOURCES..: /usr/local/etc/odbc.ini
SQLULEN Size.......: 8
SQLLEN Size........: 8
SQLSETPOSIROW Size.: 8

among USER DATA SOURCES Is the user data source , Preferential effect ;SYSTEM DATA SOURCES Is the system data source , The whole system is available . For these file paths ,

Be careful ： Sometimes the configuration file of the user data source may be $(HOME)/.odbc.ini

Communication protocol mismatch

Typical error reporting ：

unsupported frontend protocol 3.51: server supports 1.0 to 3.0

Possible causes ：

The new version of database driver is used to connect to the old version of database (V1R6C10);
Or use our driver , Connected to the open source database .

terms of settlement ：

Please use the old version of the database driver to connect to the database .

In our database , By default, the low version client can connect to the high version database ; Use the high version driver to connect to the low version client , Not within specification constraints .

This specification is also easy to understand ： Because users generally manage the database centrally , Upgrades are also planned . But the client driver may be embedded in every corner of the business , Sometimes even in released equipment , Upgrading is relatively difficult .

If it's an open source server ： Our database driver does not support open source databases , Unable to connect ; Please use the open source client to connect .

Other common questions

The difference between open source and ？

Increased security , Used SHA256 Algorithm to do authentication encryption ; At the same time, adjust the number of key hash encryption

Improved batch performance （V1R7C10 2019-4-30 And later versions ）

Can you add new API？

Can not be .

Or the picture in the basic knowledge , User referenced API It is provided by the driver manager , What we offer API Only available for Drive Manager ; newly added API Unable to provide services to users through the middle tier .

Do you support COPY Interface ？

Copy yes PG Special syntax and communication protocol ; and ODBC It is a general calling interface proposed by Microsoft , Therefore, there is no such special interface .

Linux What are the driver managers on ？

The driver manager is Linux Is not part of the operating system , It was also developed by other open source organizations . So it's not just UnixODBC A kind of . But now the most widely used , In fact, only UnixODBC, When the multi data operating system is installed, it is basically brought with it UnixODBC A specific version of ; besides , also iODBC etc. , Now it has slowly withdrawn from the market .

UnixODBC Are there any restrictions on the version of ？

Yes . We support 2.3.0 And above .

UnixODBC Of .1 And .2 What's the difference between your library ？

There's no difference .

This with UnixODBC The history of , There was a time in history when .1, Then I made a big upgrade , The dynamic library version has also been upgraded , Turned into .2. However, a large number of stock businesses are restricted and unable to operate , Then I added .1 Support for .

.1 And .2 Is really just UnixODBC Of configure Part of the configuration , Its function is exported , The functions are exactly the same , Don't worry about this .

Can I print the log ？

Sure .

General advice for printing log manager , This log is clear , Positioning problems are relatively fast . How to operate ：

Windows On ：

Open the driver manager , Adjust to the tracking page , After determining the log file path , Click start tracking to , Pictured ：

Linux On ：

open odbcinst.ini file （ You can use odbcinst -j To determine the file path ）, Add the following section ：

[ODBC]
Trace=Yes
TraceFIle=/tmp/odbc.log

Restart the business .

Be careful ： When not necessary , Please close the log , Because of every API There are write disks , Will cause performance loss .

The query result set is too large , What should I do if I can't afford memory ？

open UseDclareFetch switch , It will wrap the query as a cursor operation . For not supporting Cursor with hold Use with caution . At the same time, due to the use of server-side cursor , The forward and backward operation of the cursor will also be fed back to the database , Communication will increase , So the performance will be partially degraded ; Can be adjusted by Fetch Parameter to determine the amount of data fed back by the database to the client each time , To reduce the performance impact .

Linux Open down UseDeclareFetch Methods ：

Add... To the data source configuration item ：

UseDeclareFetch=1
Fetch=1000

Windows Open down UseDeclareFetch Methods ：

Among them Cache Size Options , Namely Fetch Size .

Do you have performance baseline data ？

The performance of creating connections is roughly 0.5s It's normal in （ commonly 0.3s about ）

If data is inserted

Non batch version （R7C10 And before ） about 400tps（SSD Test results ）

Batch version （R8C10 And later ） about 4000tps（SSD The server ）, Due to differences in database objects 、 The Internet 、 Physical hardware, etc , basic 2000tps All the above can be considered as normal

How to check if the statement is wrong ？

Error during operation ,ODBC It won't play the screen . Error information needs to be obtained from the business （API by ：SQLGetDiagField/SQLGetDiagRec）.

If a specific error message is printed in the user's business , Or the business error information is not enough to locate , Please according to 3.7 Description in , take ODBC Print out your log , Find... In it SQL_ERROR Related information . It's best to position here with business developers , because API The call sequence of is out of order , It's up to the business to decide .

If ODBC The log is still not enough to locate and support , Please log in to the connected target CN Where the machine is , Enter... Through the following methods CN Log file directory ：

source ${BIGDATA_HOME}/mppdb/.mppdbgs_profile
cd $GAUSSLOG/pg_log/cn_50xx # there cn_50xx Express CN Number , Modify... According to the actual situation 

Find the log at the corresponding time point , Observe whether there is any error message .

We and Oracle/MYSQL Of ODBC What does it matter

ODBC It's a set of standards , Each database provides its own driver file . Just like everyone calls it a graphics card , Each manufacturer also provides its own driver , Your own driver can only drive your own graphics card ; The interface standard of graphics card is Microsoft （ODBC And Microsoft ） To formulate the , It's just that we don't touch , I didn't know there was such a set of standards .

Our driver file is called psqlodbcw.so/psqlodbcw.dll.

Support GBK Well ？

Support .

But we ODBC The driver defaults to utf-8 code . If you want to adjust , You need to set it at the beginning of the session client_encoding.

But notice , Please do not use... In wide byte mode GBK, Because wide byte coding itself is UNICODE Part of （ It's usually UCS2）, At this time to use GBK Data may not be displayed ,SQL Incorrect query results and other problems .

If this problem arises , Please put VS The solution character set in is adjusted to “ Multibyte ”.

The article is written by the blogger @ Guiyun original

Want to know GuassDB(DWS) For more information , Welcome to wechat search “GaussDB DWS” Pay attention to WeChat public number , Share with you the latest and complete PB Black Technology , There are many learning materials available in the background ~

Click to follow , The first time to learn about Huawei's new cloud technology ~