Live broadcast starrocks technology insider: low base global dictionary optimization

String  Type data is distributed in databases and storage systems . In order to improve  String  Processing efficiency 、 Save storage resources , There have been many  String  Technical means of type optimization , Such as prefix coding compression to improve  String  Storage efficiency , Various dictionaries are used to speed up  String  Handle .

When processing the same amount of data ,String  The processing time of type is almost integer  2-3  times . If you use an integer instead of  String  Type for data processing , The system performance will be significantly improved ！

stay  MPP  Under the architecture , Usually, dictionary optimization can only work on a single machine or storage layer , Cannot take effect globally . If you can use a global dictionary , You can trigger dictionary optimization in more scenarios , Let string processing speed and shaping close , Further improve query performance .

that  StarRocks  How to perform low cardinality based on the global dictionary  String  Query optimization ？7  month  28  Japan  19:00-20:00,StarRocks  Feng haoan, a database R & D Engineer, will appear in the live broadcast room , Read for you  StarRocks  Technical insider of the core team .

About  StarRocks

StarRocks  It has been established for more than two years , We have been focusing on creating the world's top new generation of extreme speed full scene  MPP  database , Help businesses build “ Speed unification ” New paradigm of data analysis , Help enterprises to conduct comprehensive digital operation .

Currently, it has helped Tencent 、 Ctrip 、 Shun Feng 、Airbnb 、 sound of dripping water 、 JD.COM 、 Zhong'an Insurance Co., Ltd. exceeds  110  Large users have built new data analysis capabilities , Stable operation in production environment  StarRocks  There are thousands of servers .

2021  year  9  month ,StarRocks  Open source , stay  Github  The number of stars on the has exceeded  3000  individual .StarRocks  Our global community is growing rapidly , So far, there are more than 100 contributors , Community users breakthrough  5000  people , Attract dozens of domestic and foreign industry leaders to participate in the joint construction .