QT database application 22 file coding format recognition

One 、 Preface

In the process of data import and export , If there are more application scenarios , I believe you will encounter a problem is the problem of file coding , Some files are ANSI code , Some are utf8 code , Some are utf8 belt bom code , If different file codes use the same encoding format to parse the read data , You will definitely encounter the problem of garbled code , This is not Qt The problem of , It's not something Qt The problem of disorderly code , Instead, you need to identify the file code and then use the corresponding code to read the content , So there will be no garbled code , Of course, the appearance of garbled code must be Chinese , If all the documents are in English numbers , No matter what code , It's not a mess .

So here comes the question , How to use the program to automatically identify the file coding format ？ I searched all over the place and couldn't find the complete answer . Looking up the data, we learned that utf8 belt bom The encoding will have a fixed header byte EFBBBF, So it's easy to distinguish , because ANSI Coding and utf8 The encoding has no corresponding header byte identifier , So we need to take a turn to deal with , Still read the three bytes of the header , use QTextCodec Of toUnicode Function conversion , Of the conversion result ConverterState The number of available characters can be recognized , If the quantity is greater than 0 That is the ANSI code , Need to use gbk Decode .

Two 、 Functional characteristics

1. The component also integrates exporting data to csv、xls、pdf And print data .
2. All operations provide static methods without new, Various parameter settings such as data and attributes adopt structure data , It's very convenient .
3. Support at the same time QTableView、QTableWidget、QStandardItemModel、QSqlTableModel Equal data source .
4. Provide static methods to pass in directly QTableView、QTableWidget Control , Automatically identify column names 、 Column width and data content .
5. Each set of functions provides a separate complete example , Note details , Very suitable for all stages Qter The programmer .
6. Original export data mechanism , Do not rely on any office Third party libraries such as components or operating systems , Support embedded linux.
7. Speed super fast ,9 A field 10 Ten thousand rows of data only need 2 Seconds to complete .
8. It only takes four steps to quickly export massive data, such as 100W Record to Excel.
9. At the same time, it provides direct write data interface and multithread write data interface , The main interface is not stuck .
10. You can set the title 、 Subtitle 、 Table name .
11. You can set the field name of the exported data 、 Name 、 Column width .
12. You can set the automatic stretch filling of the end column , The default stretch is more beautiful .
13. You can set whether to enable the verification and filtering data , When enabled, the data conforming to the rules will be displayed in special colors .
14. You can specify the columns to check 、 Validation rules 、 Check value 、 Check value data type .
15. Validation rules support Exactly equal to ==、 Greater than >、 Greater than or equal to >=、 Less than <、 Less than or equal to <=、 It's not equal to !=、 contain contains.
16. Check value data type supports integer int、 floating-point float、 Double precision type double, Default text string type .
17. You can set random background color and column set that needs random background color .
18. Support group output data , For example, output data by device group , Convenient view .
19. Can be set up csv Separator 、 Line content separator 、 Sub content separator .
20. You can set the border width 、 Automatically fill in the data type , The default automatic data type is on .
21. You can set whether to turn on the data cell style , Not on by default , Not opening can save about 30% The file volume of .
22. Horizontal layout can be set 、 Paper margins, etc , Like exporting to pdf And print data .
23. Provide mixed arrangement of pictures and texts to export data to pdf And print examples , Automatic paging , Support multi graph .
24. Provide a plot template that includes both horizontal and vertical layout examples .
25. Provide static functions to export control screenshots to pdf file .
26. Provides a static function to convert an image into pdf file .
27. Providing static functions will csv Document conversion xls file , Support parameter settings such as column width and table name .
28. You can set the field alignment style for each column 、 Content alignment style , Including left alignment 、 Align center 、 Right alignment .
29. Super flexibility , Free to change the source set alignment 、 Text color 、 Background color, etc .
30. Support any excel Form software , Including but not limited to excel2003-2021、wps、openoffice etc. .
31. pure Qt To write , Support any Qt edition + Any compiler + Any system .

3、 ... and 、 Experience address

1. Experience address ：https://pan.baidu.com/s/1ZxG-oyUKe286LPMPxOrO2A Extraction code ：o05q file name ：bin_dataout.zip
2. Domestic site ：https://gitee.com/feiyangqingyun
3. International sites ：https://github.com/feiyangqingyun
4. Personal home page ：https://blog.csdn.net/feiyangqingyun
5. Zhihu Homepage ：https://www.zhihu.com/people/feiyangqingyun/

Four 、 design sketch

5、 ... and 、 Related codes

// Check the file code  0=ANSI 1=UTF-16LE 2=UTF-16BE 3=UTF-8 4=UTF-8BOM
int DataCsv::findCode(const QString &fileName, QString &flag)
{
    
    // Assume default encoding utf8
    int code = 3;
    flag = "UTF-8";

    QFile file(fileName);
    if (file.open(QIODevice::ReadOnly)) {
    
        // Read 3 Bytes for judgment 
        QByteArray buffer = file.read(3);
        quint8 b1 = buffer.at(0);
        quint8 b2 = buffer.at(1);
        quint8 b3 = buffer.at(2);
        if (b1 == 0xFF && b2 == 0xFE) {
    
            code = 1;
            flag = "UTF-16LE";
        } else if (b1 == 0xFE && b2 == 0xFF) {
    
            code = 2;
            flag = "UTF-16BE";
        } else if (b1 == 0xEF && b2 == 0xBB && b3 == 0xBF) {
    
            code = 4;
            flag = "UTF-8BOM";
        } else {
    
            // Try to use utf8 transformation , If the number of available characters is greater than 0, It means ansi code 
            QTextCodec::ConverterState state;
            QTextCodec *codec = QTextCodec::codecForName("utf-8");
            codec->toUnicode(buffer.constData(), buffer.size(), &state);
            if (state.invalidChars > 0) {
    
                code = 0;
                flag = "ANSI";
            }
        }

        file.close();
    }

    return code;
}