What is Augmented Analytics?
What technology exactly? It’s called augmented analytics, and Gartner defines it as ‘’an approach that automates insights using machine learning and natural-language generation.’’ Also called augmented data discovery, this technology is set to replace the arduous manual process of sorting and analyzing data to gain actionable business insights. Augmented analytics automates the process of analyzing data and gaining insights, and as an added benefit, it eliminates human bias.
Bernard Marr gives the most lucid definition of the technology: ‘’Augmented analytics describes the process where data is automatically taken from raw data sources, scrubbed and in an unbiased manner, and communicated in a report using natural language processing that humans can understand.’’
The technology is already being used in many industries.
The global augmented analytics market was valued at US$9.487 billion in 2019 and is expected to reach US$22.4 billion by 2025, according to a new report by Grand View Research, Inc. The market is anticipated to expand at a CAGR of 25.2 percent from 2019 to 2025. The increasing adoption of the technology by large enterprises is being driven by the high volume of big data that these companies are generating and their need to get accurate and relevant business insights from their data. Many industries are benefiting from augmented analytics, including healthcare, retail & e-commerce, banking, financial services & insurance (BFSI), IT & telecommunication, manufacturing, government, and energy utilities. The BFSI sector is fast adapting advanced analytics partly because of its ability to provide real-time financial insights. The report references the mortgage banker, Sierra Pacific Mortgage that has integrated the Microsoft Power BI, an augmented analytics tool in its business operations to drive competitive advantage.
There is also a high demand for the technology in the healthcare sector. Hospitals are leveraging it to deal with a range of operational issues. Next time you type in a question on a hospital website, you will probably get your answer from an augmented analytics-generated chatbot. The demand for augmented analytics is expected to grow the fastest in the healthcare sector.
A special feature of the technology is its ability to use ‘’natural language’’ generation that eliminates inaccessible jargon to explain trends in terms that non-tech professionals can easily understand, for instance: ‘’44 percent of companies are planning to expand paid leave benefits in 2021’’.
This means data science and the insights it generates will be more accessible to non-tech employees. It’s common knowledge that data scientists spend 80 percent of their time gathering and sorting data and only 20 percent of their time on finding business insights. Augmented analytics automates the processes of data gathering and data preparation. Some analysts see the ultimate outcome of augmented analytics as completely replacing data science teams as the technology will do all the work: gathering and analyzing data and providing insights in language that is easy to understand.
What does this mean in practice?
Normally, if a business wants to solve a problem a research question would be formulated and data is gathered, organized and cleaned. The data science team builds a machine learning model to discover trends in the gathered data. This process is time-consuming and labor-intensive and answers only one question.
Enter augmented data discovery.
Any employee who needs an answer, types the question into a search box and receives an immediate answer in plain language, which is further illuminated by visualizations like graphs.
For instance, an employee could ask the augmented analytics tool why a certain product is not selling and what could be done to improve sales. The tool could be instructed to analyze online reviews of the specific product and similar products to come up with an answer.
Benefits of augmented data discovery
Solves the data scientist problem
Data scientists are a scarce commodity and expensive to hire. McKinsey & Company estimates that there will be a shortage of approximately 250,000 data scientists by 2024 in the United States.
The adoption of augmented analytics makes the need for data scientists less urgent as non-technical employees will be able to use the technology. Unlike traditional BI tools that were difficult to use, augmented data discovery with its natural language capabilities provides easy access to insights to a broader range of employees.
Gain insights faster
Augmented data discovery automates the process of analyzing data and generating insights, shortening the time to insights from vast data sets to milliseconds. This includes insights from usually problematic unstructured data sets.
The technology is scaled to work through huge volumes of complex data in no time.
Faster data preparation and data analysis
Manual data preparation is complex and time-consuming and takes up most of data scientists’ time.
Augmented analytics allows for faster data preparation. For instance, data scientists can automate the tagging and annotation of their data. The technology fully automates repetitive transformation and integrations and the system automatically generates data quality and enrichment recommendations.
Augmented analytics makes use of machine learning to find patterns in data and discover valuable insights. It delivers its findings using natural language capabilities and visualizations that further illustrate the findings. This makes the findings easy for non-technical staff to understand. In the industry, this is called data democratization.
The fact that more people in an organization can understand the data analysis can spur organizations to more innovation as more people have access to data-driven insights.
Gartner also lists this development in their Top 10 Data and Analytics Trends for 2021. According to Gartner, we are seeing the rise of the augmenter consumer where data and analytics dashboards are no longer restricted to data analysts or citizen data scientists and insight knowledge becomes available to anyone in an organization.
New, unexpected insights
Because of user bias, people look for specific insights and usually find them. Of course, this is not useful for businesses that depend on new insights to come up with new products and services. The artificial intelligence (AI) in augmented analytics platforms are not subject to human bias and will therefore come up with unexpected insights.
Machine learning ML) algorithms can find relationships, connections and correlations that humans can’t even imagine. This is partly possible because ML algorithms aren’t subject to preconceived ideas. They won’t ‘’think’’ something doesn’t make sense and ignore it like a human might.
Augmented data discovery tools give users a real-time view of the business, which users can choose to act on immediately. In the past, these actions were limited to very expensive data visualization dashboards. With augmented analytics, it is possible to understand and act upon live IoT data via sensors or live spatial data. Live spatial data is used by law enforcement to respond immediately to crime incidents, unrest, and traffic situations.
Data swamps can become data lakes again
A data lake is a large body of raw data. Data lakes deteriorate into data swamps when organizations dump their company-related data in an unplanned manner and fail to manage it. Data swamps tend to grow out of proportion and are largely filled with disorganized data. It can be difficult for companies to retrieve and use data once it lives in a data swamp.
Augmented analytics uses AI to condense huge amounts of unorganized data and make it more useful and understandable.
Augmented data discovery does have challenges of its own though. Here we mention a few.
CEO Stephen Blum told TechTarget that processing data at scale is expensive. While a cognitive service like IBM Watson is powerful and offers easy integration through its APIs, companies are charged separately for each execution of the service, which can quickly add up to a huge sum. He suggested that only unstructured data be analyzed.
Possible resistance to adoption
Augmented analytics may end up doing the work of employees and replace them. This prospect may lead to employees resisting adoption. Companies must find ways for this technology to assist employees in their work, so they don’t resist it.
Enabling everyone may be a problem
It is all very well that augmented data discovery will put business insights in the hands of more people, but what if it lands in the hands of the wrong people? Some observers note that these tools could be used for purposes other than a company’s benefit if they are used by the wrong employees.
There is an implied risk in letting anyone use these powerful systems to ‘’gain insights’’ and act according to those insights.
Although it may appear that data scientists and data analysts are at risk of being automated out of their jobs, thought leaders like Bernard Marr and Ronald van Loon don’t think this is likely to happen.
According to Van Loon, there's going to be a high demand for data scientists in the coming years, requiring a highly specific, highly specialized skill set. He made the observation during a webinar: Your Future in Data Science: Career Outlook for 2020.
In his article on augmented analytics, Marr concludes that data scientists will still be in high demand even when organizations adopt augmented analytics. The technology will take over repetitive tasks allowing data scientists to focus on more strategic and creative tasks.