Published: 07th February 2018
Data mining and the art of deducing what corporations need to keep doing
To unearth obscure trends, patterns and inferences from sets of huge and varied data, a Big Data analyst should have knowledge of rudimentary subjects like statistics and numbers.
Cisco predicts that by 2019, global IP traffic will reach two zettabytes (one zettabyte = one billion terabytes). However, data growth does not necessarily come from the increased utilisation of existing methods, but from new avenues as the number of connected devices expands. Consumer devices are playing a primary role in data growth. Also, incremental volume from M2M (Machine to Machine) data through AI and IoT technologies is adding to the growth. The term Big Data has come into vogue due to the shift of analysing structured relational databases to working with a huge number of unstructured databases generated from a variety of sources. There was a need to create tools to analyse and interpret such unstructured datasets.
Today, it is not uncommon to see data being described as the currency of the digital economy or the lifeblood of a digital organisation. As the cost of computing continues to drop, data-driven business models are more affordable now. Markets and Markets, a research organisation, predicted that Enterprise Data Management including software and services for migrating, warehousing, integrating and analysing — all forms of data, will be worth $105 billion in 2020.
Number game: A Big Data analyst must make the most of data systems and process it to the best of their capability
However, one of the major challenges in this area is a rising skill gap. The industry has transformed from using SQL (Sequential Query Language) to managing structured relational databases to management and analysis of unstructured data. To understand the skills required in Big Data, we need to comprehend the three stages of data analytics: collection and storage, processing and organising and finally, analysis and visualisation.
Since data analytics and management is comparatively an older stream within the tech world compared to Artificial Intelligence, Cloud and IoT, the industry has some understanding of the skills that are required. However, the requirements of now and the future will rest more on the analytics. The burning question is how we can intelligently use huge volumes of data for business transformation including incremental revenues, cutting costs, improved decision making and targeted marketing. Therefore, most of the skill gap around data management now lies on Real-Time Analytics, Predictive Modelling, Data Security, Distributed Storage and Data Mining.
People who opt for this line are often required to be number junkies who thrive on finding patterns and connections in large mounds of data. By analysing this data, the analyst can help companies decipher trends and patterns
A typical data-centric workforce, framework and skill requirement would include the following:
Data interpretation and visualisation with job roles such as Data Analytics and Business Intelligence. In this domain, the skill requirement will be focused on learning the tools for data analytics and warehousing such as Netezza, MicroStrategy, SAS and Tableau.
Data Management and Processing with job roles such as database administration, development, data integration and data lifecycle management. IT professionals in this domain need to work on database software and platforms such as Microsoft, Oracle, Cloudera and Hadoop.
Data Infrastructure, which is the system behind data management, includes job roles in data storage, data centre management, business continuity and data security.
(Pradipto Chakrabarty is the Regional Director of CompTIA Technology India. In this capacity, Chakrabarty leads the India operations for CompTIA. He is a regular speaker at various career awareness seminars organised for students, enterprise organisations and job seekers across the country in association with IT training providers. He has participated and spoken at skill development forums organised by agencies such as NASSCOM, ICTACT and various leading educational institutes. Apart from covering topics on skill development, Chakrabarty is also a passionate follower of technology trends. He has over 18 years of industry experience. Before joining CompTIA, he has also worked for a good many years in the industry and has previously been associated with Prometric Testing and NIIT)