How many stages did the development and evolution of data analysis go through?

CDA Data Analyst Produce

author : Deepesh Nair

compile : Mika

In recent years , We have made great progress in the field of information technology , A series of revolutionary achievements in the field of technological ecology are indeed commendable . In the past ten to twenty years , Data and analysis has always been a very popular word . So we need to be clear about how they relate to each other , What role does it play in the market , And how it will reshape the business .

For those who have realized their potential , Technology is a blessing , However, for those who can't keep up with its rapid development , This is also a test . Now , Almost every industry is inseparable from data analysis .

In this paper, the development and evolution of data analysis in recent years will be summarized , Simplify various terms , Explain some common application scenarios . Let's get started ！

Data analysis 1.0 → Business intelligence needs

This is the rise of data warehouse , Customer （ Business ） And production process （ transaction ） To a huge repository , Such as eCDW（ Enterprise consolidated data warehouse ）. Real progress has been made in the objective understanding of business phenomena , So that managers can make decisions based on their understanding of the facts , Not just intuition .

In this stage, the data passes through ETL and BI Tool collection 、 Transformation and query . The types of analysis are mainly descriptive （ What happened? ） And diagnostic （ Why does it happen ）.

However , The limitation of this stage is that the data is only used within the company , That is, business intelligence activities can only deal with what happened in the past , And can't predict the future trend .

Data analysis 2.0 → big data

As major enterprises have stepped out of the comfort zone , When trying to use a broader approach for more complex analysis , The limitations of data analysis in the previous stage become more prominent .

Enterprises begin to obtain information through external resources , For example, click stream 、 social media 、 Internet, etc , At the same time, the demand for new tools is becoming more and more obvious . inevitably ,“ big data ” The word appears , In order to distinguish small data purely from the company's internal system .

At this stage , The company hopes that employees can help process a large amount of data through a fast processing engine . What they didn't expect was , Therefore, the emerging group came into being , What is now called “ The open source community ” Will have a huge impact , It's also data analysis 2.0 A sign of the times .

With the unprecedented support of the community , Big Data Engineer ,Hadoop Administrators and other roles have grown in the field of employment , And for each IT Businesses are crucial . Technology companies are eager to develop new frameworks , These frameworks can not only collect 、 Transform and process big data , And it can also integrate predictive analysis . and , The trend was further detected by the results of descriptive and diagnostic analysis 、 Clustering and anomalies , And predict future trends , This also makes it an important prediction tool .

In today's technology ecosystem , I personally think “ big data ” This term has been widely used , Even abuse . Technically speaking , Now “ big data ” Refers to all data , Or just data .

Data analysis 3.0→ Powerful data products

Pioneering big data companies began to invest in data analysis , To support customer facing products , Services and functions . They use better search algorithms 、 Buying advice and targeted advertising attract users to their websites , All this is driven by data analysis . The phenomenon of big data is spreading rapidly , Today, it is not just technology companies that are developing products and services through data analysis , This is true for companies in almost every industry .

On the other hand , The popularity of big data technology has had a mixed impact . While technology giants reap a lot of profits and succeed , Most enterprises and non technology companies fail miserably because they ignore data . therefore , The field of data science came into being , Aimed at using scientific methods 、 The exploration process 、 Algorithms, etc. obtain knowledge and analytical insights from various forms of data .

actually , The field of data science is interdisciplinary , It is defined as “ Combined with statistics 、 Data analysis 、 Concepts of machine learning and other related methods ”, So as to use data “ Understand and analyze practical phenomena ”. let me put it another way , Good data combined with excellent training models can produce better prediction results . The new generation of quantitative analysts is called data scientists , They have computing and data analysis skills .

The technology industry is developing rapidly with the help of data science , And make full use of predictability and standardization to predict the future trend . Competition for data analysis has also opened up among enterprises , Companies not only improve traditional ways such as internal decision-making , But also continue to develop more valuable products and services . This is data analysis 3.0 The essence of the period .

Today, data analysis has undergone a great change . The company is developing at an unimaginable speed , Set up more R & D departments internally , Like data scientists 、 Data Engineer 、 Solution architect 、 A data analysis team composed of chief analysts and other personnel .

Data analysis 4.0 → Automation

There are four main types of analysis ： describe , Explain the past ; The diagnosis , Use past data to study the present ; forecast , Predict the future through insights based on past data ; standard , Guide the best behavior through the model .

Although data analysis 3.0 Contains all of the above types , But it emphasizes the last , The concept of small-scale automatic analysis is introduced .

Create more models through machine learning , So as to make the prediction more detailed and accurate . however , The cost and time of deploying such custom models is very expensive . Final , Automated data analysis through intelligent systems 4.0 The time has come .

without doubt , Artificial intelligence 、 machine learning 、 Deep learning will have a profound impact . Machine translation 、 Intelligent reply 、 chatbot 、 Conference assistant and other functions will be widely used in the next few years . Data mining technology 、 Machine learning algorithms have achieved a lot of results , Automated analysis will become a new stage of data analysis .

Data analysis 5.0 → What's next

We can understand automation as , The powerful combination of man and intelligent machine , To achieve better results .

Instead of thinking “ What human jobs will be replaced by machines ？” I'd rather think optimistically about , With the help of the machine , What new achievements can the enterprise achieve ？ How can we in disaster prone areas , Reduce casualties through artificial intelligence programs ; Or how to establish AI driven e-schools in poor areas .

To make a long story short , I am confident in the development of data analysis , The key is whether we can actively accept and deal with its impact .

Link to the original text ：

https://towardsdatascience.com/the-evolution-of-analytics-with-data-8b9908deadd7