Xiao Bo mentioned many times in his previous blog post , Can be in VBA Write in SQL To operate Excel file , Realize various data processing and analysis requirements . that , You may have such a question ：Excel Native VBA, PivotTable , Data analysis Is the function insufficient , Why do you have to use SQL To implement? ？ stay Excel VBA Use in SQL Where are the advantages ？ Today, I'll take you to have a good look at these problems .
First , stay VBA Write in SQL, This is basically VBA High level functions of , Not everyone knows , therefore , In addition to SQL, Indeed, there are other technical solutions . secondly , If we are more skilled in using other technologies , We can use our best technology stack to solve problems . In terms of consequence , As long as it can solve practical problems , All good plans .
however , What I want to discuss here is , If we are right VBA A native method , PivotTable ,SQL Under the premise of being familiar with knowledge , How can we choose a technology stack to solve practical problems , This is the time ,SQL What's the advantage of ？ I think there are several points ：
One 、 The highest efficiency of execution
Follow VBA Native methods and Excel The formula compares ,SQL The execution speed is much faster . When the number of tables exceeds 10000 rows , This advantage is more and more obvious . More data , The more suitable to use SQL To solve the problem , More efficient ;
Two 、 The code is extremely concise
①、 Data matching scenario ： If we have multiple fields to match , In principle, more Vlookup perhaps VBA Create multiple dictionaries in , however , If we use SQL External connection of , Maybe one line of code is enough ;
②、 Data grouping and aggregation scenario ： If we were VBA Record macros in to generate PivotTable code , To find the maximum by grouping and aggregating ,EXCEL A lot of code will be generated automatically in the background , These codes are extremely unreadable , But if we use SQL Group By, combination max,min,average Wait for the aggregate function , We only use one line SQL Sentence can be used .
3、 ... and 、 Easier to implement
There are some special scenes , Use native VBA The method will be extremely complex , But use SQL Maybe just one line of code . Take my previous blog post VBA How to realize the filtering conditions “ Some values are excluded ” give an example , This scene , That article used a lot of space to use pure VBA Skills to achieve such a seemingly simple need , Single one “ Screening —— Some values are not included ” Of VBA scene , We need to use Recording macro function , One dimensional array 、 Two dimensional array function 、 Transpose method of array 、 Dictionary remove Method 、 Dictionary key fast storage array method etc. . But in fact , If we use SQL, The knowledge points needed are much less .
Let's review this scene again ,【 Source data 】 The table contains 【 full name 】【 postal address 】【 Postal Code 】 The three column ,【 Exception list 】 The table stores names to be excluded ,【 result 】 Table requires return 【 Source data 】 Table does not contain 【 Exception list 】 All data of the name . We use Python faker Library generation 20000 Row data , It is convenient to compare the efficiency of different methods when comparing large samples .
If you use SQL, In essence, we need to 【 Source data 】 Table left outer connection （Left Join)【 Exception list 】 surface , be based on 【 full name 】 Column , Plus where The conditions are good .
The specific code is as follows ：
1 Sub myQuery()
2 Dim conn As Object, rs As Object, sht1 As Worksheet, sht2 As Worksheet, sht3 As Worksheet, sql As String, startTime As Date, endTime As Date
3 startTime = Timer
4 Set conn = CreateObject("ADODB.Connection")
5 Set rs = CreateObject("ADODB.recordset")
6 Set sht1 = ThisWorkbook.Sheets(" Source data ")
7 Set sht2 = ThisWorkbook.Sheets(" Exception list ")
8 Set sht3 = ThisWorkbook.Sheets(" result ")
9 conn.Open "provider=Microsoft.ACE.OLEDB.12.0;extended properties=excel 12.0;data source=" & ThisWorkbook.FullName
10 sql = "SELECT a.* FROM [ Source data $]a LEFT JOIN [ Exception list $]b ON a. full name =b. full name WHERE b. full name IS NULL"
11 Set rs = conn.Execute(sql)
12 For i = 0 To rs.Fields.Count - 1 ' Output recordset Field names to 【 result 】 surface
13 sht3.Cells(1, i + 1) = rs.Fields(i).Name
14 Next
15 sht3.Cells(2, 1).CopyFromRecordset rs ' Output recordset Results to 【 result 】 surface
16 conn.Close
17 Set conn = Nothing
18 endTime = Timer
19 sht3.Activate
20 MsgBox " Cumulative operation " & (endTime - startTime) & " second "
21
22 End Sub
The running results are as follows , It takes about 0.63 second ：
Let's review the use of VBA Dictionary to achieve this requirement , The code is as follows ：
1 Sub dictWay()
2 Dim conn As Object, rs As Object, sht1 As Worksheet, sht2 As Worksheet, sht3 As Worksheet, sql As String, startTime As Date, endTime As Date, maxRow1 As Integer, myDic As Object, maxRow2 As Integer
3 startTime = Timer
4 Application.ScreenUpdating = False
5 Set sht1 = ThisWorkbook.Sheets(" Source data ")
6 Set sht2 = ThisWorkbook.Sheets(" Exception list ")
7 Set sht3 = ThisWorkbook.Sheets(" result ")
8 Set myDic = CreateObject("scripting.dictionary")
9 maxRow1 = sht1.Cells(Rows.Count, 1).End(xlUp).Row
10 maxRow2 = sht2.Cells(Rows.Count, 1).End(xlUp).Row
11 Dim i As Integer, j As Integer, k As Integer
12 For i = 2 To maxRow2
13 myDic.Add sht2.Cells(i, 1).Value, ""
14 Next
15
16 k = 1
17 For i = 1 To maxRow1
18 If myDic.exists(sht1.Cells(i, 1).Value) = False Then
19 For j = 1 To 3
20 sht3.Cells(k, j).Value = sht1.Cells(i, j).Value
21 Next
22 k = k + 1
23 End If
24
25 Next
26
27 endTime = Timer
28 sht3.Activate
29 Application.ScreenUpdating = True
30 MsgBox " Cumulative operation " & (endTime - startTime) & " second "
31
32 End Sub
In order to make this method as fast as possible , We used a dictionary , At the same time, it turns off Excel Screen refresh . Final 20000 Line data , It takes about 4 second ：
You can see , Prior to VBA Compared with , Just one line SQL sentence , The logic is clear and easy to understand , The code is simple and clear , The running time of the program is traditional 1/6 Less than , This is the small climbing force pushing in VBA Use in SQL The main reason is ~~
Welcome to scan code to pay attention to my official account Get more crawlers 、 Knowledge of data analysis ！
stay Excel VBA Use in SQL More related articles on where the advantages are
- stay Excel VBA Write in SQL, What kind of experience is it
Often mentioned Excel Office automation , All we can think of is the formula . PivotTable . macro .VBA, This is also the advanced way of data analysis for most of us . When we are used to VBA After the skill has been quite proficient , There is often a " my VBA Enough knowledge ...
- Excel vba Medium visit ASP.NET MVC project , Record the visit time , File name
Every time 30 Connect to the server every second , The cell turns green when the connection is successful , Connection failure turns red , The status cell is 17 That's ok ,2 Column 1, open excel file , Get into vba Editor , Create a new one modules modular , In it, first write a every 30 Once per second ConnSe ...
- excel VBA Regular modules in vbscript.regexp Usage of
One . It's an object , Used to perform Regular expressions ! Two . There are three properties : 1. Global attribute : True or False, Indicates whether the pattern matches everything that matches it in the whole string or only the first occurrence . Default ...
- excel VBA in Xldown and xlup usage
1.Worksheets("Sheet1").Range("A1").End(xlDown).Select ' It means from A1 rise , Return the last non empty from top to bottom ...
- Excel 2003 How to use VBA The code accesses the values in the cell and operates the cell - Tang poetry column - Blog channel - CSDN.NET
stay Excel Written in VBA Code , Perhaps the most common thing to do is to manipulate the data in the cells of the form . I'd like to summarize how to start from VBA Code to manipulate the data of a cell . stay VBA In the code operation cell needs to use Range object ,Range yes ...
- Excel VBA Connecting to various databases （ 3、 ... and ） VBA Connect SQL Server database
This paper mainly deals with : VBA Medium SQL Server Environment configuration VBA Connect SQL Server database VBA Reading and writing SQL Server data How to install SQL Client System environment : Windows 7 64bi ...
- Excel VBA Connecting to various databases （ One ） VBA Connect MySQL database
In this paper, the reference [ Dongwei householder ] Of cnblog post Excel.VBA And MySQL Interaction Successful debugging on your own machine , Write down the problems encountered in debugging . This paper mainly deals with : VBA Medium MySQL Environment configuration VBA Connect MySQL ...
- Suspicion Excel VBA Slow execution , You must read these suggestions
Excel It's a sharp tool for office , There is no need to say much about it . Especially in the office ,Excel Whether you are proficient in using , It will be Excel Do you have much knowledge , It largely determines whether you work efficiently , Can you clock out on time . But we often hear such roast :Excel Good is good , But ...
- stay EXCEL Use in SQL Statement query
SQL Statements are very important in database usage . stay EXCEL Workbook can not be opened in , Just get the data , It is very useful for multi workbook operations , Very soon . Deal with a large amount of data , Much faster than the cycle , But there is still a little distance than the dictionary method ( Unfortunately, I haven't learned a dictionary yet ). To the database ...
- Asp.Net Use in OpenRowSet operation Excel surface , Import Sql Server( example )
There are two interfaces to choose from :Microsoft.Jet.OLEDB.4.0( hereinafter referred to as Jet engine ) and Microsoft.ACE.OLEDB.12.0( hereinafter referred to as ACE engine ). Jet The engine is familiar to everyone , can ...
Random recommendation
- Java stay DOS Operation under command and its API The documentation process
This document mainly describes java The program is in DOS Run under command , And some common commands Commonly used DOS command : d: enter Drive switch dir(directory): List the files and folders in the current directory md (make direct ...
- JavaScript For variables and functions in “ advance ”
Variable declarations “ Be advanced ” JavaScript Grammar and C .Java.C# similar , Collectively referred to as C Kind of grammar . There was C or Java Students with programming experience should be right “ First of all . After use ” The rules of are familiar , If you use undeclared variables or ...
- MySQL Database transaction analysis
MySQL Database transaction analysis A transaction is a set of atomic SQL Inquire about , It is an independent execution unit . Statements within a transaction , Or it all works , Or all failed . 1. Standard characteristics of transactions A well functioning transaction processing system , It has to be atomic . Agreement ...
- CentOS5.4 Lower installation codeblocks 12.11
centos6.3 Lower installation codeblock Much simpler , These open source software are also constantly improving . Originally installed codeblocks10.05, Forget , This installation took me another half day , Finally, it's done . First installed wxGTK-2.8. ...
- Python Base64 code
0x00 Base64 brief introduction 0x01 Examples of common scenarios 0x02 Ed . Decoding process 0x03 Python in Base64 Encoding and decoding 0x00 Base64 brief introduction We know that any data in the computer is by pressing ascii Code storage ...
- Codeforces Round #550 (Div. 3) F. Graph Without Long Directed Paths
F. Graph Without Long Directed Paths time limit per test 2 seconds memory limit per test 256 ...
- WPF Prevent instances from running repeatedly
1. Method 1 stay app.xaml.cs Add the following code : /// <summary> /// App.xaml Interaction logic of /// </summary> public partia ...
- iptables Set up port forwarding
from :https://blog.csdn.net/sigangjun/article/details/17412821 One Forwarding from one port to another Enable network card forwarding function #echo 1 > /p ...
- XBOX360
[ Summary + Share ]XBOX360 Multiplayer game summary post https://tieba.baidu.com/p/3550398060?pn=13&red_tag=3423139816&traceid= ...
- The database returns the record just inserted ID
-- Create databases and tables create database MyDataBaseuse MyDataBase create table mytable(id int identity(1,1),name var ...
