Making big data simple will enable the public sector to successfully harness the opportunities presented by advances in cloud, data and AI, says Alex Purkiss, lead for government and life sciences at Databricks
The ability to take vast amounts of data and turn it into ‘breakthrough’ insight could drive unprecedented change and innovation across the public sector. It will enable us to re-imagine the possibilities of public service delivery to not only meet our citizens needs but also enrich their lives.
But big data at scale is just that: big. It is essential, therefore, to have the tools with which to contain it, analyse it and make it work with other technologies, such as artificial intelligence. Thankfully the speed at which such tools are evolving is rapidly democratising data - next generation tools like Databricks mean that the days of one data scientist writing complex and inaccessible code are over, as whole teams can now simplify, collaborate, share and work together.
Accelerate innovation
Our recent collaboration with Microsoft has now optimised Databricks’ Apache® Spark™-based analytics platform for the Azure cloud platform. Azure Databricks combines the best of both to help the public sector accelerate innovation with one-click set up, streamlined workflows and an interactive workspace that enables collaboration between data scientists, data engineers, and business analysts.
At its recent launch, Rohan Kumar, corporate VP, Azure Data at Microsoft Corp, said that Microsoft Azure Databricks “has proven to be the most efficient way for us to meet customer demand for Spark on Azure. We experienced a lot of interest and engagement in the preview from organizations in need of a high performance analytics platform based on Spark. With Azure Databricks, deeply integrated with services like Azure SQL Data Warehouse, our customers are now positioned to increase productivity and collaboration and globally scale analytics and data science projects on a trusted, secure cloud environment.”
One click to launch
You can launch Azure Databricks on the Azure Portal with a single click, and integrate it effortlessly with a wide variety of data stores and services such as Azure SQL Data Warehouse, Azure Cosmos DB, Azure Data Lake Store, Azure Blob storage, and Azure Event Hub. You can add artificial intelligence (AI) capabilities instantly and share insights across the organisation through rich integration with Power BI.
Customers are already unleashing this power. Energy company E.ON needs to analyse nearly a terabyte of wind turbine data every day in order to optimise its data models. This process used to take several hours each time but since moving to Azure Databricks it takes just a few minutes each day.
Meanwhile, solar energy company renewables.AI wants to “create the best AI solution in the world to revolutionise the solar energy space”. As part of this goal to use data to make solar energy profitable, Azure Databricks has enabled the company to boost productivity in its data science teams by over 50%. According to Andy Cross, director at renewables.AI, “Instead of one data scientist writing AI code and being the only person who understands it, everybody uses Azure Databricks to share code and develop together.”
You can watch the fantastic story of their data journey here:
Learn more about Azure Databricks
- Download Databricks’ white paper: Getting Started with Spark on Azure Databricks
- Try Azure Databricks with a Free Trial on Azure
- Watch the webinar on Streaming Analytics Use Cases on Apache Spark
If you would like to discuss how you can harness Azure Databricks, email Alex Purkiss, Databrick’s UK Government Lead here.
Image by IR Stone, istock
