The paper of gaojingjian center was selected into the ACL 2022 of the international summit to further expand the privacy computing capacity of Chang'an chain

THE-X The core algorithm of will be further expanded “ Changan chain ” Privacy Computing supports large model inference in cloud service scenarios , With continuous technological innovation , Meet the privacy protection needs of large-scale application of artificial intelligence models in various industries in the future .

—— The text is quoted from official account ： The future blockchain and privacy computing high-end Center

      In recent days, , The paper of the research team of the future blockchain and privacy computing high-end Innovation Center 《THE-X: Privacy-Preserving Transformer Inference with Homomorphic Encryption》 Successfully selected into the international top academic conference in the field of natural language processing ACL 2022 Findings.THE-X It is a homomorphic encryption solution for pre training large model , For the first time Transformer Structure can infer ciphertext , It is of great significance for the development and large-scale application of large models .

      ACL 2022 Sponsored by the international society for Computational Linguistics , It is the highest level academic conference in the field of natural language processing and computational linguistics , It is also the Chinese computer society (CCF) Recommend A Class meeting , Every year, it attracts a large number of scholars from all over the world to contribute and participate in the Conference . This year, ACL Rolling review is introduced for the first time （Rolling Review） Mechanism , Only 20% About papers were accepted . In this paper , From the center “ Homomorphic encryption operator design and dense state inference of Neural Networks ” The research team of the task proposed to use homomorphic encryption to solve Transformer An innovative algorithm for privacy protection when the model infers in the cloud , made Lattice Privacy security in theory .

      With more and more people based on Transformer The pre training model built adopts the architecture of cloud deployment , And such as the search history of users , Medical records , Sensitive information such as bank statements is processed by the pre training model , Must be converted to clear text format , The risk of privacy disclosure is greatly increased . Homomorphic encryption allows the model to calculate directly on the ciphertext data , It is an important solution to the problem of data privacy in recent years . However , At present, homomorphic encryption only supports simple addition and multiplication , Unable to support Transformer Complex calculation of structure .

      Facing this thorny problem , The research team proposed a workflow of operator estimation THE-X, Can train well Transformer The model is transformed into homomorphic encryption supported operators , Thus, the pre training model can directly start inference on the ciphertext . This paper mainly aims at Transformer Complex operators in structures , Such as GELU、Softmax、LayerNorm etc. . stay GLUE and SQUAD etc. 8 Experimental results on two data sets show that ,THE-X It can enable ciphertext computing power for various downstream tasks , And does not significantly reduce the effect of the model .

chart 1   Use THE-X Infer the big cloud model

chart 2  THE-X Results on many different tasks

      As the center “ Homomorphic encryption operator design and dense state inference of Neural Networks ” One of the important research contents of the task ,THE-X The core algorithm of will be further expanded “ Changan chain ” Privacy Computing supports large model inference in cloud service scenarios , With continuous technological innovation , Meet the privacy protection needs of large-scale application of artificial intelligence models in various industries in the future .

RECOMMEND

Recommended reading

Changan chain ChainMaker Anti duplication optimization of trading pool transactions

The cuckoo filter of Chang'an chain transaction

BigFilter Introduction and application of global transaction anti duplication component

Tips

More Chang'an chain open source projects QA, You can log in to the open source community 、 Technical document library view .

Download the source code

https://git.chainmaker.org.cn/chainmaker/chainmaker-go

Consult the documentation

https://docs.chainmaker.org.cn/

Changan chain ChainMaker Case collection

http://www.wenjuan.com/s/UZBZJvhFGte/

“ Changan chain ChainMaker” It is the first independent and controllable blockchain software and hardware technology system in China , It is jointly developed by microchip Research Institute, enterprises and universities , With full autonomy 、 High performance 、 Strong privacy 、 The outstanding characteristics of wide cooperation . Chang'an chain is oriented to large-scale node networking 、 High transaction processing performance 、 Strong data security, privacy and other next-generation blockchain technology requirements , Integrate the special acceleration chip hardware of blockchain and the bottom software platform that can be assembled , To build high performance 、 High credibility 、 High security digital infrastructure provides new solutions , Provide strong blockchain technical support for Chang'an chain ecological alliance . The name “ Changan chain ”, Metaphorical meaning “ lasting political stability 、 Create brilliance again 、 Link to the world “.