In Australia, the Peter MacCallum Cancer Centre, in collaboration with the John Holland Group, has leveraged Databricks’ cloud-based information and artificial intelligence (AI) platform to overcome key challenges associated with data fragmentation, thereby enabling more effective insights extraction from enterprise data.
Tech leaders from various organisations gathered at Databricks’ Information + AI World Tour in Sydney, Australia last month, sharing their experiences of grappling with familiar challenges: siloed data, competing business domains, integration hotspots, and legacy architectures, ultimately driving the need to seek a cloud-based data solution.
Peter MacCallum Cancer Centre leverages artificial intelligence to streamline data consolidation.
Despite Peter Mac’s robust infrastructure, its ability to effectively harness vast amounts of data was hindered by the complexity of its medical and analytical operations. The organization’s legacy expertise posed a significant risk to its goal of improving cancer patients’ lives, including leveraging AI to accelerate medical decision-making, organic insights, and drug discovery.
Issues with information infrastructure
During the conference, Jason Li, head of the bioinformatics core facility within Peter Mac’s cancer research division, noted that:
- Peter Mac struggled to navigate a complex landscape of fragmented data and outdated methods.
- Throughout the operations of numerous cancer centers, the sheer volume and intricacy of medical and analytical data presented significant hurdles in realms such as information management and data interpretation.
- The moral, privacy, and security implications have been pivotal factors in governing Peter Mac’s data and informing decisions surrounding the deployment of future AI applications.
- The integration between medical and analytics departments proves challenging due to vastly differing information requirements in each, thereby exacerbating the issue of info governance.
Li revealed that Peter Mac chose Databricks to streamline data consolidation across its healthcare center, leveraging AI-driven analytics while ensuring compliance with stringent information security and privacy regulations in the healthcare industry.
The emergence of new AI-driven applications holds tremendous potential for transforming various industries and aspects of our lives.
Peter Mac initially assessed the AI capabilities of the Databricks platform through a pioneering AI transformation pilot project.
- The centre developed a comprehensive AI lifecycle that leveraged deep learning to analyze and evaluate high-resolution, gigapixel-sized whole-slide images, ultimately quantifying a novel biomarker for accurate breast cancer diagnosis.
- Databricks facilitated the entire AI lifecycle – from initial data ingestion to model deployment and monitoring – thereby streamlining the mission’s timeline and value proposition.
- The findings of this mission hold promising implications for improving breast cancer diagnosis.
Throughout the mission, Li highlighted pace as a game-changing advantage: “By leveraging Databricks, we’ve accelerated our event timeline by a remarkable fivefold, while concurrently reducing communication overheads among stakeholders by an impressive tenfold, ultimately empowering us to deliver improvements to the market sooner and benefit customers more quickly.”
AI techniques are evolving with a focus on future innovations.
As AI has increasingly become an integral component of Peter Mac’s creative process, its impact on his work is undeniable. Databricks is collaborating with a leading cancer center to deploy its cloud-based analytics platform across three key areas: genomics, radiation oncology, and cancer imaging. Moreover, Peter Mac is:
- Enhancing the AI program to integrate mainstream bioinformatics, encompassing inhabitants’ genetic endeavors featuring colossal patterns and vast amounts of genomic data.
- Utilizing the latest advancements in Giant Language Models and Retrieval-Augmented Technology to efficiently extract valuable insights from vast collections of medical and radiological case studies?
- Intending to leverage Large Language Models (LLMs) in the near future to accelerate advancements in genomics and transcriptomics research, specifically targeting the analysis of RNA and transcriptomes to maintain a competitive edge in cancer research.
John Hollander aims to harmonize knowledge across development projects?
By mid-2023, John Holland successfully delivered 80 major infrastructure projects with a combined value of approximately AUD $13.2 billion. Despite this, Travis Rousell, the corporation’s chief of information and analytics, noted that their legacy information warehouse setup was disjointed and challenging to integrate.
“We’ve overcome the traditional challenges associated with information warehousing and management,” Rousell stated. Our legacy information warehouse has evolved incrementally over a span of two decades. Slowly evolved and fragmented over time, we’ve inadvertently built a complex network of isolated knowledge repositories.
Rousell stated: “While we’ve created BI and experiences at individual touchpoints, aggregating this information to derive actionable insights into customer journeys and behavioral patterns has proven particularly challenging for our organization, hindering our ability to drive meaningful change.”
A centralized intelligence hub providing actionable knowledge.
John Holland established a comprehensive digital hub to facilitate seamless access and sharing of valuable data across the organization, unlocking new levels of productivity and insight. The initiative was designed to foster creativity and accelerate business benefits through the strategic integration of cutting-edge digital technologies and data-driven insights, ultimately driving a comprehensive digital transformation across the organization.
The organisation has sought to:
- Integrate disparate data sources to present a seamless and comprehensive view of organizational information across the entire enterprise.
- Ensure effective management of information across distinct, self-governed projects.
- Focus on data information engineering rather than platform engineering?
Pricing financial savings stem from optimized information management.
To date, John Holland has successfully implemented various core enterprise processes within Databricks’ data lake, including mission administration, mission operations, mission control, security, and fleet analytics.
Because he utilized Databricks, Rousell noted that John Holland had.
- Realized a 46% reduction in platform infrastructure costs for comparable workflows compared to traditional environments.
- Streamlined information engineering processes, achieving a 30% reduction in time and effort, through the creation of innovative new products and formats.
- Transferred more than 600 clients to data-driven offerings utilizing the scalable and secure capabilities of the Databricks Lakehouse for storing, processing, and sharing large datasets.
As IT systems became increasingly sophisticated, they were slowly transforming into an enabler for John Holland’s enterprise operations.
Roussel noted that Databricks eliminates barriers between technology expertise and business innovation, thereby allowing enterprises to advance unhindered by IT limitations.
“According to Rousell, the key takeaway from their approach is establishing a culture of certainty within John Holland, which he believes is a significant achievement.” Traditionally, provisioning innovative products required cumbersome, slow-moving projects that often fell short of meeting business expectations.
“Now, when an enterprise conceives an idea, we’ll confirm with enthusiasm. We’ll assign them a dedicated knowledge workspace, giving them seamless access to all necessary skills, tools, and resources, empowering them to bring their vision to life at their own pace.”
