What is big data and what is it used for?

The digital revolution is changing the economy, society and the way people interact with technology. At the heart of this transformation lies the data generated by billions of connected devices. According to the Global Telecom Outlook 2023 to 2027 report produced by PwC, the total number of installed IoT devices will grow from 1.64 billion in 2022 to an estimated 2.51 billion by 2027, which equates to approximately three devices for every person on the planet. This expansion of the Internet of Things (IoT) will drive new use cases in both the business and consumer sectors, with increasingly widespread applications in sectors such as industry, healthcare, mobility and smart homes.

What is big data

Almost an endless number of applications: GPS systems can detect traffic jams in the area checked by a user and suggest alternatives; a subscription streaming TV channel has created the characters and plot of its most successful series by analysing the contents its viewers consume and prefer to watch; smart watches monitor the heart rate of millions of users and identify patterns that can anticipate to and prevent cardiovascular diseases; humidity sensors in crop fields plan the irrigation frequency, combining their data with the weather forecasts, and a long etcetera. Their applications have even reached the world of politics: Juan Verde, member of Joe Biden's President's Export Council and Spanish advisor to two U.S. Democratic Party campaigns stated that: "These are not the TV elections anymore; they are the elections of big data".

How does big data work?

The process of big data is based on different stages that enable data to be transformed into useful information:

Some data about big data

 SEE INFOGRAPHIC: 10 interesting facts about big data in 2024 [PDF]

The eight key facts about big data and its future begin with the growth in professional roles: roles related to big data, data and artificial intelligence are forecast to grow by more than 30% by 2030. The second fact shows how these technologies will transform business models: 86% of companies expect AI and information processing technologies to have a significant impact on their organisations by 2030. 
The third fact refers to the economic value of big data: the global market is estimated to reach $103 billion by 2027. The fourth fact focuses on the evolution of professional skills: 39% of current skills will change or become obsolete between 2025 and 2030 due to technological transformation. 
The fifth figure highlights one of the main challenges facing businesses: 63% regard the lack of skills as a barrier to progress in their digital transformation. The sixth figure highlights the growth in the volume of information generated worldwide: analysts forecast it will reach 394 zettabytes of data by 2028. 
The seventh statistic analyses the impact of big data and other technologies on employment: up to 78 million net new jobs could be created globally by 2030. Finally, the eighth statistic highlights the importance of continuous training: 77% of companies plan to train or retrain their staff to adapt to technological transformation.

Applications of big data (examples)

Key applications include:

One of the main applications of advanced data analysis is the study of consumer patterns. Social networks, such as Facebook, Twitter or Instagram, are a tool used by brands to learn more about their consumers and connect with them. Companies have also started to gather data from their consumers. A company specialising in big data and retail intelligence has installed 15,000 sensors in the shopping areas of 25 countries. Thanks to the data gathered with these sensors, they have detected that only 59% of customers entering a shop have bought something from the shop.

Digital transformation within businesses will continue to drive demand for professionals capable of managing and analysing large volumes of data. According to the World Economic Forum's Future of Jobs Report 2025, big data specialists are among the roles projected to see the strongest growth up to 2030. The report estimates a 30 to 35 per cent increase in demand for data-related professionals such as analysts, data scientists, big data specialists and data engineers, driven by the adoption of advanced technologies such as artificial intelligence.

Digital transformation companies leads to the generation of huge volumes of data that organisations do not know how to use and manage. And this is already being portrayed in the labour market. In 2024, big data specialists were the second most sought-after profile. Companies are now asking their candidates to have international experience, strategic vision, analytical capacity and adaptation to change as the main requirements.

Big data is also a working partner of another technological milestone: artificial intelligence (AI). They work together, as the millions of data processed by big data is necessary to train AI to make complex decisions that increasingly resemble the human cognitive process. AI then provides a set of tools and techniques to perform faster and more advanced analysis of that data.

Analysts forecast that by 2030 a total of 170 million new jobs will be created worldwide (leaving a net gain of 78 million, after taking into account the number of jobs displaced), and that among these, big data specialists will be among the fastest-growing roles in the technology sector. What’s more, 86% of companies state that AI and information-processing technologies will transform their businesses. Will we be ready to take that step?

Challenges of big data

The growth in data volume and the expansion of technologies such as artificial intelligence offer great opportunities, but they also present new challenges that we must address to ensure the reliable and sustainable use of information.

Among the main challenges are:

• Data quality and reliability: having accurate and up-to-date information is essential for obtaining appropriate results and making sound decisions.
• Privacy and data protection: managing large volumes of data requires ensuring data protection and compliance with regulatory frameworks.
• Cybersecurity: the rise in connected devices and digital systems makes it necessary to strengthen protection against potential threats.
• Algorithmic biases: data-driven models must be designed and monitored to avoid results that are unrepresentative or discriminatory.
• Specialised talent: Technological developments require professionals with skills in data analysis, artificial intelligence and digital management.
• Energy consumption of digital infrastructure: the growth of data centres and processing systems presents the challenge of moving towards more efficient and sustainable solutions.
• Governance and responsible use of AI: establishing clear criteria for control, transparency and oversight is key to harnessing the potential of these technologies.

Iberdrola: driving technological innovation

At Iberdrola, we integrate digital technologies throughout our value chain to accelerate electrification and move towards a smarter, more efficient and sustainable energy model. The advanced use of data and artificial intelligence enables us to optimise our processes, improve the management of our assets and strengthen the quality and resilience of the energy system.

Within the context of big data and our group, data becomes a key element in the operation of smart grids, renewable energy facilities and predictive maintenance, which helps us anticipate potential incidents and improves the availability of our assets. We also use these technologies to offer better services to our customers as well as to strengthen our cybersecurity capabilities by identifying potential risks and improving our ability to respond to threats.

We are also driving the responsible use of artificial intelligence and data, guided by criteria of security, control and operational continuity. We currently have more than 300 artificial intelligence projects underway, focusing on areas such as growth, productivity, operational efficiency, service quality and the resilience of the energy system.