The use of Big Data technologies makes possible to transform large amounts of data in knowledge, offering to the companies the possibility to pick up, through the generated data, valuable information about their clients, products and services, allowing them to be ahead of market trends. These technologies offer the base to advanced analytic products such as Bidoop Layer, able to do complex analysis about large amounts of data, no matter their nature or structure.
The main limitations associated to processing and analyzing large volumes of data have been all collected in the "3 Vs Model", pointed out in 2001 by Dough Laney, Industry Analyst & Information Innovation Agenda Manager. According to this model, the following attributes are considered as Big Data main challenges.
- Speed: The increse in the information processing speed, made possible to face the challenge of transforming large amounts of information in knowledge in real time, providing answering in a quickler way to each business specific needs.
- Variety: The increase of sources ables to generate information (emails, social network, images, videos, logs, etc) has producted, as well, an increase of the data analysis complexity.
- Volume: As the processed data volume increases exponencially, it became essential to develope analytic models able to detect data correlactions from different sources of information.
The new models have other factors to consider, such as the high rate of data variability, the need to value its veracity, as well as the importance for them to be viewable through dynamic dashboards which make easier to prepare reports and to make business decisions.
Apache Hadoop is an Open Source platform used to storage and process large-scale data. Characterized by its flexibility to store any kind of information, by its capability to process complex data and by its economical price since it doesn't require license of use.
Hadoop is inspired in Map&Reduce technologies and Google File System (GFS), originally implemented by Google. With this tool some of the most representative companies in the world got to optimize the efficiency and quality of its information, increasing its benefits. Amazon, Yahoo, Telefónica and LinkedIn are some examples of this.