What is massive data? What are the characteristics of "it"?

What is massive data? Mass data can also be called big data. For big data researcher Gartner, this definition is given: big data is a massive, high growth rate and diversified information assets that require new processing models to have greater decision-making power, insight, and process optimization capabilities. .

From a technical point of view, the relationship between big data and cloud computing is like the opposite side of a coin. Big data must not be processed by a single computer. It must use a distributed architecture. Its characteristic lies in distributed data mining for massive data, but it must rely on cloud computing distributed processing, distributed database and cloud storage, virtualization technology.

With the advent of the cloud era, big data has also attracted more and more attention. The Analyst team of the “PTZ” believes that big data is often used to describe the large amount of unstructured data and semi-structured data created by a company. This data will spend too much time and time when downloaded to a relational database for analysis. money. Big data analysis is often associated with cloud computing because real-time, large-scale dataset analysis requires a framework like MapReduce to distribute work to tens, hundreds, or even thousands of computers.

1. Characteristics of big data

Compared with traditional data warehouse applications, big data analysis has the characteristics of large data volume and complex query analysis. The article “Architecture Big Data: Challenges, Status Quo, and Prospects” published in the Journal of Computers lists several important features that a big data analytics platform needs to have. It is the current mainstream implementation platform—parallel database, MapReduce, and a hybrid based on the two. The structure was analyzed and summarized, and their respective advantages and disadvantages were pointed out. At the same time, the research status in various directions and the author's efforts in the analysis of big data were introduced, and the future research was prospected.

The characteristics of big data have four levels: First, the volume of data is huge. Jump from TB level to PB level. Second, there are many types of data. The aforementioned web logs, videos, pictures, geographic information, etc. Third, the processing speed is fast. The 1s law can quickly obtain high-value information from various types of data, which is also fundamentally different from traditional data mining techniques. Fourth, as long as the data is properly used and correctly and accurately analyzed, it will bring high value returns. The industry grouped them into four "V" Volumes (large data volume), Variety (various data types), Velocity (fast processing speed), and Value (high data value).

To some extent, big data is the leading edge technology for data analysis. In short, the ability to quickly obtain valuable information from various types of data is big data technology. It is important to understand this, and it is this that motivates the technology to have the potential to reach many companies.

2. Use of big data

Big data can be divided into big data technology, big data engineering, big data science and big data applications. The most talked about at present is big data technology and big data application. Engineering and science issues have not been taken seriously. Big data engineering refers to the systematic engineering of planning, construction, and operation management of big data. Big data science pays attention to the rules of discovering and validating big data and its relationship with nature and social activities in the development and operation of big data networks.

Internet of Things, cloud computing, mobile Internet, car networking, mobile phones, tablet PCs, PCs, and a variety of sensors all over the globe are all data sources or ways of carrying.

3. Big data storage

The core value of big data lies in the storage and analysis of massive data. Compared with other existing technologies, the comprehensive cost of "cheap", "rapid" and "optimization" of big data is optimal.

Big data requires special techniques to effectively handle large amounts of data in tolerable times. Technologies for big data include massively parallel processing (MPP) databases, data mining grids, distributed file systems, distributed databases, cloud computing platforms, the Internet, and scalable storage systems.

Distributed storage systems store data dispersedly on multiple independent devices. The traditional network storage system uses a centralized storage server to store all data. The storage server becomes the bottleneck of system performance (which is also the focus of reliability and security) and cannot meet the needs of large-scale storage applications. The distributed network storage system adopts an extensible system structure, uses multiple storage servers to share the storage load, and uses location servers to locate storage information, which not only improves system reliability, availability, and access efficiency, but also facilitates expansion.

4. The significance of big data

Big data is a collection of data that cannot be captured, managed, and processed using conventional software tools within an affordable time frame. It is a massive amount of data that requires a new processing model to provide greater decision-making, insight, and process optimization capabilities. Growth rate, diversified information assets.

On May 10, 2013, Ma Yun, Chairman of Alibaba Group Board of Directors, said at the Taobao 10th Anniversary Party, “Everybody has not figured out the era of the PC, the mobile Internet has come, and when the mobile Internet is not understood, the era of big data coming".

Big data is changing products and production processes, companies and industries, and even changing the nature of competition itself. Viewing information technology as an aid or service tool has become an outdated concept. Managers should recognize the wide-ranging influence and profound meaning of information technology, and how to use information technology to create strong and lasting competitive advantages. Undoubtedly, information technology is changing the way people are accustomed to doing business. A technological revolution that has a bearing on the survival of companies has come.

With the boom in the era of big data, Microsoft has produced a data-driven software that is mainly used to save resources for engineering construction and increase efficiency. In the process, it can save the world 40% of energy. Aside from the prospect of this software, starting from the Microsoft team's commitment to research, it can be seen that their goal is not only to save energy but also to focus more on intelligent operations. By tracking the huge amount of data accumulated by heaters, air conditioners, fans, and lights, you can capture how to eliminate energy waste. "If you give me some data, I can make some changes. If you give me all the data, I can save the world," Microsoft said. The smart building is exactly what his team is focusing on.

With the popularity of personal computers, smart phones and other devices worldwide, and the growing number of Internet traffic in emerging markets, as well as the explosion of data generated by surveillance cameras or smart meters, the scale of the digital universe will be between 2012 and 2013. Doubled in two years to reach a staggering 2.8ZB. IDC expects that by 2020, the digital universe will exceed the expected size, reaching 40ZB.

What is the concept of 40ZB? The sand on all the beaches on Earth together add up to an estimated 705 billion. 40ZB is equivalent to 57 times the amount of sand on all beaches on the planet. In other words, by 2020, the digital universe will double every two years; by 2020, the per capita data volume will reach 5247GB.

The report also shows that although individuals and machines generate large amounts of data each day, and the digital universe continues to expand more than ever, only 0.4% of global data has been analyzed. It can be seen that the application of big data is almost a virgin land that has not been exploited.

5. The value of big data

Google search, Facebook posts and tweets make it possible to make detailed measurements of people's behavior and emotions. From the user's behavior habits and preferences, from behind the messy data to find more in line with user interests and habits of products and services, and targeted adjustments and optimization of products and services, which is the value of big data. Big data is also increasingly showing its impetus to various industries.

The advent of the era of big data is determined first by the richness of data. With the rise of social networks, a large number of unstructured data such as UGC (Internet terminology, User Generated Content, meaning “user-generated content”) content, audio, text information, video, and pictures emerged. In addition, the Internet of Things has a larger amount of data, and the mobile Internet can collect user information, such as location and living information, more accurately and more quickly. From the data point of view, it has entered the era of big data, but the hardware has obviously not kept pace with the development of data.

In the past, big data was often used to describe a large amount of unstructured and semi-structured data created by a company. Now referring to “big data” usually refers to a method to solve the problem, and analyzes and mines it to obtain it. Valuable information eventually evolved into a new business model.

Although big data is still in its infancy in China, its commercial value has already emerged. First, companies that master data stand on gold mines and can generate good results based on data transactions. Second, there are many business models based on data mining that have different orientations or focus on data analysis. For example, to help companies do internal data mining, or focus on optimization, to help companies more accurately find users, reduce marketing costs, increase business sales, increase profits.

In the future, data may become the largest trading commodity. However, large amounts of data cannot be considered as big data. Big data is characterized by large amounts of data, multiple types of data, and maximized value of non-standardized data. Therefore, the value of big data is to obtain maximum data value through data sharing and cross-multiplexing. In the future, big data will become big industries, like infrastructure, by data providers, regulators, regulators, and cross-reuse of data. According to statistics, the market size formed by big data is about US$5.1 billion, and by 2017, this figure is expected to rise to US$53 billion.

For more information on smart door lock product prices , click here!