The problem background
Before you start the text , Do some background bedding , It is convenient for readers to understand my engineering requirements . My project is a client message distribution center , After connecting to the message background , The backstage will push some news to me from time to time , I then forward them to other desktop products on this computer for display . Background to ensure that messages can be pushed to the client , It takes a strategy of pushing repeatedly , in other words , Every time I reconnected to the background , The backstage will push me all the news in a period of time 、 Whether or not these messages have been pushed before , If I don't deal with it, I just push it to the product , It may cause the same message to be displayed repeatedly for many times . So , After I received the message , They will be stored in a container in the process , When new news arrives , It will check whether the message has been received in this container first , If there is , No forwarding .
1 namespace GCM { 2 class server_msg_t 3 { 4 public: 5 void dump(char const* prompt); 6 7 std::string appname; 8 std::string uid; 9 std::string msgid; 10 time_t recv_first = 0; 11 time_t recv_last = 0; 12 int recv_cnt = 0; 13 }; 14 15 class WorkEngine 16 { 17 public: 18 WorkEngine(); 19 ~WorkEngine(); 20 21 private: 22 // to avoid server push duplicate messages to same client. 23 // note this instance is only accessed when single connection to server arrives message, so no lock needed.. 24 std::vector<server_msg_t> m_svrmsgs; 25 }; 26 }
Above is the simplified code ,m_svrmsgs Members store all the background messages they receive ,server_msg_t It represents a background message ,appname、uid Which instance is used to locate which product to send ;msgid Used to uniquely identify a message ;recv_first、recv_last、recv_cnt The first time the message was received 、 The last time and the number of repetitions . Now a very realistic problem is , I need to serialize these messages to persistent storage , So that the information is still available after the process is restarted . I used it here sqlite database , The relevant code is encapsulated in WorkEngine In the member function of , It's easy to think of a way to declare a function like this ：
1 namespace GCM { 2 class server_msg_t 3 { 4 public: 5 void dump(char const* prompt); 6 7 std::string appname; 8 std::string uid; 9 std::string msgid; 10 time_t recv_first = 0; 11 time_t recv_last = 0; 12 int recv_cnt = 0; 13 }; 14 15 class WorkEngine 16 { 17 public: 18 WorkEngine(); 19 ~WorkEngine(); 20 21 protected: 22 int db_store_server_msg (std::vector<server_msg_t> const& vec); 23 int db_fetch_server_msg (std::vector<server_msg_t> & vec); 24 25 private: 26 // to avoid server push duplicate messages to same client. 27 // note this instance is only accessed when single connection to server arrives message, so no lock needed.. 28 std::vector<server_msg_t> m_svrmsgs; 29 }; 30 } 31
image line 22-23 As shown , Use it directly std::vector<server_msg_t> This container as a parameter （ Some people may think that I'm overstepping , Access directly in the function m_svrmsgs It's just the members , Why pass through parameters ？ Maybe this example is not obvious , But there are some cases where containers exist as local variables rather than as member variables , Here are some simplifications for illustrative purposes ）. But I think it's too rigid , What if I change the container later , Is it necessary to change here ？ Maybe it's generic algorithms that look too much , It's not enough to write like this “ Universal ”. But if it's written like this , Or change the soup without changing the dressing ：
int db_store_server_msg (std::vector<server_msg_t>::iterator beg, std::vector<server_msg_t>::iterator end);
Reference standard library std::copy Algorithm , Transform it , It turns out to be like this ：
template <class InputIterator> int db_store_server_msg(InputIterator beg, InputIterator end);
It's called a member function template , Or member template function , Or template member function …… I don't know , It's a member function anyway + template function . It can be written in this way ：
1 namespace GCM { 2 template <class InputIterator> 3 int WorkEngine::db_store_server_msg(InputIterator beg, InputIterator end) 4 { 5 int ret = 0, rowid = 0; 6 qtl::sqlite::database db(SQLITE_TIMEOUT); 7 8 try 9 { 10 db.open(get_db_path().c_str(), NULL); 11 writeInfoLog("open db for store server msg OK"); 12 13 db.begin_transaction(); 14 15 for (auto it = beg; it != end; ++it) 16 { 17 // 1th, insert or update user info 18 rowid = db.insert_direct("replace into server_msg (appname, uid, msgid, first_recv, last_recv, count) values (?, ?, ?, ?, ?, ?);", 19 it->appname, it->uid, it->msgid, it->recv_first, it->recv_last, it->recv_cnt); 20 21 ret++; 22 } 23 24 db.commit(); 25 db.close(); 26 writeInfoLog("replace into %d records", ret); 27 } 28 catch (qtl::sqlite::error &e) 29 { 30 writeInfoLog("manipute db for store server msg error: %s", e.what()); 31 db.rollback(); 32 db.close(); 33 return -1; 34 } 35 36 return ret; 37 } 38 }
You can see , The core code is to traverse the iterator interval (line 15). The caller is also very concise ：
db_store_server_msg(m_svrmsgs.begin(), m_svrmsgs.end());
One line fix , It looks like it's done , There's no difficulty , So what is the point of this article ？ take it easy , The real difficulty is recovering data from the database . First of all, we can't use iterators directly , Because now we're going to insert elements into the container , Iterators can only traverse elements , It doesn't help at all . But I believe that readers must have seen code like this ：
1 int main (void) 2 { 3 int arr[] = { 1, 3, 5, 7, 11 }; 4 std::vector vec; 5 std::copy (arr, arr + sizeof (arr) / sizeof (int), std::back_inserter(vec)); 6 for (auto it = vec.begin (); it != vec.end (); ++ it) 7 printf ("%d\n", *it); 8 9 return 0; 10 }
To insert elements at the end of the container , The standard library algorithm relies on back_inserter This east east . So naturally I thought , Can we state here that back_inserter As an input parameter ? Like this ：
template <class OutputIterator> int db_fetch_server_msg(OutputIterator it);
Template implementation is written like this ：
1 namespace GCM { 2 template <class OutputIterator> 3 int WorkEngine::db_fetch_server_msg(OutputIterator it) 4 { 5 int ret = 0; 6 qtl::sqlite::database db(SQLITE_TIMEOUT); 7 8 try 9 { 10 db.open(get_db_path().c_str(), NULL); 11 writeInfoLog("open db for fetch server msg OK"); 12 13 db.query("select appname, uid, msgid, first_recv, last_recv, count from server_msg", 14 [&ret, &it](std::string const& appname, std::string const& uid, std::string const& msgid, time_t first_recv, time_t last_recv, int count) { 15 server_msg_t sm; 16 sm.appname = appname; 17 sm.uid = uid; 18 sm.msgid = msgid; 19 sm.recv_first = first_recv; 20 sm.recv_last = last_recv; 21 sm.recv_cnt = count; 22 *it = sm; 23 ++ret; 24 }); 25 26 db.close(); 27 writeInfoLog("query %d records", ret); 28 } 29 catch (qtl::sqlite::error &e) 30 { 31 writeInfoLog("manipute db for store server msg error: %s", e.what()); 32 db.close(); 33 return -1; 34 } 35 36 return ret; 37 } 38 }
In fact, the core is a sentence to back_inserter Assignment statement for (line 22). The caller does the same thing in one line ：
db_fetch_server_msg (std::back_inserter(m_svrmsgs));
The separation of template declaration and template implementation
The above code can be compiled normally , But the premise is that the template implementation and template call are in the same file . Considering this class, there was a lot of logic before , I decided that the content related to the database , Transfer to a new file (engine_db.cpp), To reduce the amount of code in a single file . The adjusted file structure is as follows ：
+ engine.h： WorkEngine Statement + engine.cpp：WorkEngine Realization （ contain engine.h） + engine_db.cpp：WorkEngine::db_xxx Template implementation （ contain engine.h）
recompile , Reported a link error ：
1>workengine.obj : error LNK2001: Unresolved external symbols "protected: int __thiscall GCM::WorkEngine::db_fetch_server_msg<class std::back_insert_iterator<class std::vector<class GCM::server_msg_t,class std::allocator<class GCM::server_msg_t> > > >(class std::back_insert_iterator<class std::vector<class GCM::server_msg_t,class std::allocator<class GCM::server_msg_t> > >)" (??$db_fetch_server_msg@V?$back_insert_iterator@V?$vector@Vserver_msg_t@GCM@@V?$allocator@Vserver_msg_t@GCM@@@std@@@std@@@std@@@WorkEngine@GCM@@IAEHV?$back_insert_iterator@V?$vector@Vserver_msg_t@GCM@@V?$allocator@Vserver_msg_t@GCM@@@std@@@std@@@std@@@Z)
Obviously, it is caused by the failure to find the corresponding link when calling the template . You need to use “ Template display instantiation ” stay engine_db.cpp The file forces the template to generate the corresponding code entity , Come and join us engine.cpp Link with the call point in the . You need to add the following two lines of code at the beginning of the file ：
using namespace GCM;
template int WorkEngine::db_fetch_server_msg<std::back_insert<std::vector<server_msg_t> > >(std::back_insert<std::vector<server_msg_t> >);
Notice the syntax of template member functions showing instantiation , I looked it up 《cpp primer》, The format is ：
template return_type CLASS::member_func<type1, type2, ……> (type1, type2, ……);
Corresponding to the above statement , Is the use of std::back_insert<std::vector<server_msg_t> > Replace the original OutputIterator type , To tell the compiler to display the generation of such a function template instance . Note that the same type should be written twice , One is the function template parameter , One time it's a function parameter . However, the display instantiation syntax failed to compile ：
1>engine_db.cpp(15): error C2061: Grammar mistakes : identifier “back_inserter” 1>engine_db.cpp(15): error C2974: 'GCM::WorkEngine::db_fetch_server_msg' : Templates about 'OutputIterator' Is an invalid parameter , Should be type 1> f:\gdpclient\src\gcm\gcmsvc\workengine.h(137) : See “GCM::WorkEngine::db_fetch_server_msg” Statement of 1>engine_db.cpp(15): error C3190: With the template parameters provided “int GCM::WorkEngine::db_fetch_server_msg(void)” No “GCM::WorkEngine” An explicit instantiation of any member function of 1>engine_db.cpp(15): error C2945: Explicit instantiation does not refer to template class specialization
puzzled . Went out for a walk , A little fresh air , The brain suddenly came into being ： There was a long list of link errors before , Use the type in there directly , Should be able to compile ！ Do as you say , So here's a long list of instantiation declarations ：
template int GCM::WorkEngine::db_fetch_server_msg<class std::back_insert_iterator<class std::vector<class GCM::server_msg_t,class std::allocator<class GCM::server_msg_t> > > >(class std::back_insert_iterator<class std::vector<class GCM::server_msg_t,class std::allocator<class GCM::server_msg_t> > >)
Too much is —— Actually passed the compilation ！ Take a closer look at this long list of type declarations , It seems that it's just vector It's just unfolding , I use “ Condensed version ” Of vector Again, try to find out what's changed ：
template int GCM::WorkEngine::db_fetch_server_msg<std::back_insert_iterator<std::vector<server_msg_t> > >(std::back_insert_iterator<std::vector<server_msg_t> >);
It's actually passed . It looks like it's just using back_insert_iterator Instead of back_inserter Just fine ,back_insert_iterator What a ghost again ？ see back_inserter Definition , Here are the findings ：
1 template<class _Container> inline back_insert_iterator<_Container> back_inserter(_Container& _Cont) 2 { // return a back_insert_iterator 3 return (_STD back_insert_iterator<_Container>(_Cont)); 4 }
Looks like back_inserter It's a return back_insert_iterator Template function of type , And std::make_pair(a,b) and std::pair <A,B> It's like , Because this is a type , So it can't be transmitted directly back_inserter This function gives the display instantiation declaration . good , So far I'm not , We did it with a inserter Or two iterator Parameters instead of clumsy container parameters 、 And can declare 、 call 、 Split it in three different files , It's perfect . There is a fly in the ointment , Template display instantiation is a bit tedious , Use here typedef Define the type to instantiate , Make the above sentence clearer ：
typedef std::back_insert_iterator<std::vector <server_msg_t> > inserter_t; template int WorkEngine::db_fetch_server_msg<inserter_t>(inserter_t);
Empathy , Yes db_store_server_msg Make the same transformation ：
typedef std::vector <std::string, server_msg_t>::iterator iterator_t; template int WorkEngine::db_store_server_msg<iterator_t>(iterator_t, iterator_t);
Is this more perfect ？
Use map Instead of vector
In use , Discover the use of map It is faster and more convenient to query whether the message is already in the container , So I decided to change the message container definition as follows ：
std::map<std::string, server_msg_t> m_servmsgs;
among map Of value Part of the same as before , To increase the key Part of it is msgid. After this change , Use... When traversing "it->second." Instead of "it->"; You need to use “*it = std::make_pair (sm.msgid, sm)” Instead of “*it = sm”. Make the above changes , I found that the program still failed to compile . After some investigation , It turned out to be back_inserter It doesn't fit map Containers . because back_inserter Corresponding back_insert_iterator stay = The container will be called in the operator push_back Interface , And this interface is only vector、list、deque Several containers support ,map Is not supported . What shall I do? , Fortunately, some kind-hearted people have written it map The inserter of —— map_inserter：
1 #pragma once 2 3 namespace std 4 { 5 template <class _Key, class _Value, class _Compare> 6 class map_inserter { 7 8 public: 9 typedef std::map<_Key, _Value, _Compare> map_type; 10 typedef typename map_type::value_type value_type; 11 12 private: 13 map_type &m_; 14 15 public: 16 map_inserter(map_type &_m) 17 : m_(_m) 18 {} 19 20 public: 21 template <class _K, class _V, class _Cmp> 22 class map_inserter_helper { 23 public: 24 typedef map_inserter<_K, _V, _Cmp> mi_type; 25 typedef typename mi_type::map_type map_type; 26 typedef typename mi_type::value_type value_type; 27 28 map_inserter_helper(map_type &_m) 29 :m_(_m) 30 {} 31 32 const value_type & operator= (const value_type & v) { 33 m_[v.first] = v.second; 34 return v; 35 } 36 private: 37 map_type &m_; 38 }; 39 40 typedef map_inserter_helper<_Key, _Value, _Compare> mi_helper_type; 41 mi_helper_type operator* () { 42 return mi_helper_type(m_); 43 } 44 45 map_inserter<_Key, _Value, _Compare> &operator++() { 46 return *this; 47 } 48 49 map_inserter<_Key, _Value, _Compare> &operator++(int) { 50 return *this; 51 } 52 53 }; 54 55 template <class _K, class _V, class _Cmp> 56 map_inserter<_K, _V, _Cmp> map_insert(std::map<_K, _V, _Cmp> &m) { 57 return map_inserter<_K, _V, _Cmp>(m); 58 } 59 };
I copied this code from the Internet , Please refer to the link below for details ：std::map Of inserter Realization . unfortunately , This code “ disability ” 了 , I don't know it's the author who stole the chain 、 Still not input the complete reason , This code has some inherent grammatical defects , It doesn't even compile , In my unremitting “ Cerebral repair ” In the process , The missing parts have been made up with highlights , Ladies and gentlemen, you can enjoy it directly ~
In particular , The most technical deficiency occurs in line 37 A reference to , If you don't add this , Although it can be compiled , But in the process of operation ,inserter Not to map Insert elements in , Will result in an empty read from the database map. I've been trying to find the original text of this article , But nothing , For the Internet dissemination process found such a donkey head horse mouth error event , I feel very sad （ Although I don't quite understand , But you can't pit me either ）……
Okay , Back to the point , With map_inserter after , So we can say that ：
typedef std::map_inserter<std::string, server_msg_t, std::less<std::string> > inserter_t; template int WorkEngine::db_fetch_server_msg<inserter_t>(inserter_t);
For this map_inserter Realization , We need to deliver map Three template parameters of , instead of map This parameter itself , I'm not sure it's progress 、 It's a kind of retrogression , Anyway, this one map_inserter It's a little weird , Not encapsulated as map_insert_iterator + map_inserter In the form of , There is still a difference between the standard library and the implementation level , Let's see . The caller also needs to make some fine-tuning ：
db_fetch_server_msg(std::map_inserter<std::string, server_msg_t, std::less <std::string> >(m_svrmsgs));
have a look , There's no simplicity in standard library implementation , Is it a fake product ~ Fortunately, we've encapsulated inserter_t type , It can be rewritten like this ：
db_fetch_server_msg(inserter_t(m_svrmsgs));
It's much simpler . Now let's look at the file composition of the project ：
+ map_inserter.hpp： map_inserter Statement + Realization + engine.h： WorkEngine Statement （ contain map_inserter.hpp) + engine.cpp：WorkEngine Realization （ contain engine.h) + engine_db.cpp：WorkEngine::db_xxx Template implementation （ contain engine.h) ……
Here, to reduce the complexity , take map_inserter Put it in the header file for sharing , Similar to the way standard library header files are used .
Use the normal template function instead of the class member template function
At the end of this article , Let's go back to the two member template functions in the above example , It is found that they are not used by other members of the class , In fact, they can be separated into two ordinary template functions to call , For example, change it to this ：
1 namespace GCM {
2 class server_msg_t
3 {
4 public:
5 void dump(char const* prompt);
6
7 std::string appname;
8 std::string uid;
9 std::string msgid;
10 time_t recv_first = 0;
11 time_t recv_last = 0;
12 int recv_cnt = 0;
13 };
14
15 class WorkEngine
16 {
17 public:
18 WorkEngine();
19 ~WorkEngine();
20
21 private:
22 // to avoid server push duplicate messages to same client.
23 // note this instance is only accessed when single connection to server arrives message, so no lock needed..
24 std::vector<server_msg_t> m_svrmsgs;
25 };
26
27 template <class InputIterator>
28 int db_store_server_msg(InputIterator beg, InputIterator end);
29 template <class OutputIterator>
30 int db_fetch_server_msg(OutputIterator it);
31
32 typedef std::map <std::string, server_msg_t>::iterator iterator_t;
33 typedef std::map_inserter<std::string, server_msg_t, std::less<std::string> > inserter_t;
34 }
Move template function declaration from class to out of class (line 27-30), At the same time to modify engine_db.cpp The definition and display of instantiation statements of two classes in , Remove class restrictions (WorkEngine::)：
template int db_fetch_server_msg<inserter_t>(inserter_t); template int db_store_server_msg<iterator_t>(iterator_t, iterator_t);
There is no need to modify at the caller . Error is reported when compiling again ：
1>engine_db.cpp(16): warning C4667: “int GCM::db_fetch_server_msg(GCM::inserter_t)”: There is no function template defined that matches the mandatory instantiation 1>engine_db.cpp(17): warning C4667: “int GCM::db_store_server_msg(GCM::iterator_t,GCM::iterator_t)”: There is no function template defined that matches the mandatory instantiation 1> Creating library F:\gdpclient\src\gcm\Release\gcmsvc.lib And the object F:\gdpclient\src\gcm\Release\gcmsvc.exp 1>workengine.obj : error LNK2001: Unresolved external symbols "int __cdecl GCM::db_fetch_server_msg<class std::map_inserter<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class GCM::server_msg_t,struct std::less<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > > > >(class std::map_inserter<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class GCM::server_msg_t,struct std::less<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > > >)" (??$db_fetch_server_msg@V?$map_inserter@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@Vserver_msg_t@GCM@@U?$less@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@2@@std@@@GCM@@YAHV?$map_inserter@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@Vserver_msg_t@GCM@@U?$less@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@2@@std@@@Z) 1>workengine.obj : error LNK2001: Unresolved external symbols "int __cdecl GCM::db_store_server_msg<class std::_Tree_iterator<class std::_Tree_val<struct std::_Tree_simple_types<struct std::pair<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const ,class GCM::server_msg_t> > > > >(class std::_Tree_iterator<class std::_Tree_val<struct std::_Tree_simple_types<struct std::pair<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const ,class GCM::server_msg_t> > > >,class std::_Tree_iterator<class std::_Tree_val<struct std::_Tree_simple_types<struct std::pair<class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > const ,class GCM::server_msg_t> > > >)" (??$db_store_server_msg@V?$_Tree_iterator@V?$_Tree_val@U?$_Tree_simple_types@U?$pair@$$CBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@Vserver_msg_t@GCM@@@std@@@std@@@std@@@std@@@GCM@@YAHV?$_Tree_iterator@V?$_Tree_val@U?$_Tree_simple_types@U?$pair@$$CBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@Vserver_msg_t@GCM@@@std@@@std@@@std@@@std@@0@Z)
The first two warning It's because after a member function becomes a normal function , The display instantiation needs to be placed behind the function implementation , Let's adjust these two statements to the end of the file . For the next two links error, puzzled , And then I used a very simple test Template functions do experiments , Discovery is caused by the namespace , You need to add a namespace qualification before the definition and display of each function instantiation statement (GCM::)：
template int GCM::db_fetch_server_msg<inserter_t>(inserter_t); template int GCM::db_store_server_msg<iterator_t>(iterator_t, iterator_t);
You can see , Class member template functions and ordinary template functions are still quite different , Because the class itself is a namespace , Its appearance simplifies the addressing of member functions .
Conclusion
In fact, this article explains a general way to pass iterator Read the container 、 adopt inserter How to insert container elements , It's a way to pass on the container itself directly “ grace ” not a few , Although it can't be realized 100% Seamless switching containers , But it also provides great flexibility . In particular, it also studies how to declare and implement the template functions implemented in this way in different files , To achieve the purpose of decoupling code , It has strong practicability . Of course , This is just the template instantiation method , If you encounter different templates TYPE If you need to use a different function implementation , You may also encounter template specific syntax （ Including full specialization and partial specialization ）, That's going to increase the complexity , There is no further exploration here .
Reference resources
[2]. std::map Of inserter Realization
[3]. C++ The separation of declaration and implementation of template class （ template instantiation ）
[4]. C++ How to compile function templates
[5]. c++ Function template declaration is separated from definition
[6]. C++ Template function template instantiation and materialization
[7]. C++ Function templates Instantiation and materialization
[9]. c++ Template function declaration and definition separation
