About IDIA

The Inter-University Institute for Data Intensive Astronomy (IDIA) was established as an inter-university partnership in September 2015. The partners include the University of Cape Town (UCT), the University of the Western Cape (UWC) and the University of Pretoria (UP), and initially also North-West University (NWU). IDIA also works in close collaboration with the South African Radio Astronomy Observatory (SARAO) and has established a broad network of national and international collaborations.

The overarching goal of IDIA is to build within the South African university research community the capacity and expertise in data intensive research to enable global leadership on MeerKAT large survey science projects and large projects on other SKA pathfinder telescopes, leading to leadership on SKA phase 1 Key Science programs.

One of the first elements to reach this goal was for IDIA to set up a data-intensive research cloud facility to service its scientific community. Currently, IDIA is the primary platform to service five out of eight MeerKAT large survey projects.

Since its inauguration in July 2018, the MeerKAT radio telescope has established itself as one of the most productive scientific instruments in the world. IDIA researchers and students have contributed to a growing number of scientific discoveries enabled by the world’s most sensitive mid-frequency radio telescope. South Africa’s rise as a global leader in radio astronomy accelerated following its successful 2012 bid to host the majority of the mid-frequency component of the Square Kilometre Array (SKA-MID), helping to attract international expertise while training a new generation of African scientists and engineers. Built and operated by the South African Radio Astronomy Observatory (SARAO), MeerKAT had contributed to more than 600 scientific research publications based on MeerKAT data by April 2026, demonstrating its transformational scientific capabilities.

IDIA works closely with SARAO to support the processing and analysis of MeerKAT data through advanced research computing infrastructure, including the ilifu cloud platform. While radio astronomy remains a major focus, IDIA also applies its expertise in data-intensive science, visualisation and machine learning to other fields such as bioinformatics and computational biology. As MeerKAT expands towards the SKA Observatory era, including developments such as MeerKAT+ and the ongoing rollout of SKA antennas in South Africa, IDIA continues to help develop the technologies, skills and research capacity needed for the next generation of data-intensive science.

One of the main goals of IDIA was to establish a technology platform that would enable researchers in South Africa to undertake world-class science with astronomical big data. To achieve this, IDIA developed and now operates ilifu, a NICIS Tier-2 data intensive research cloud created through a successful multipartner initiative supported through the National Integrated Cyberinfrastructure System (NICIS). Since 2019, the IDIA research cloud has been fully integrated into ilifu, creating a powerful national platform that supports data-intensive research in astronomy, bioinformatics, and related fields, while enabling real-time collaboration between researchers across the globe through web-based access to shared tools, computing resources, and datasets. The platform also supports the integration of advanced techniques such as machine learning directly into scientific data processing pipelines, helping researchers analyse increasingly large and complex datasets more efficiently.

Beyond large-scale computing infrastructure, IDIA is also developing advanced technologies for astronomical data visualisation and analysis. A major focus is CARTA (Cube Analysis and Rendering Tool for Astronomy), a world-leading visual analytics platform developed through an international collaboration involving IDIA and global partners, which enables researchers to interactively explore massive datasets from facilities including MeerKAT, ALMA, the VLA, and ASKAP through a web browser.

Through the IDIA Visualisation Laboratory (VisLab) at the University of Cape Town, UCT and IDIA are also working with an international collaboration to develop iDaVIE (Immersive Data Visualisation Interactive Explorer), an immersive 3D visualisation platform designed for exploring large astronomical datasets in virtual reality environments. Together, these technologies are helping researchers worldwide analyse and better understand increasingly large and complex scientific datasets in preparation for the SKA era and beyond.

Developing the next generation of scientists and data specialists is a cornerstone of IDIA’s work. Through data science and machine learning hackathons, training initiatives, postgraduate scholarships, and support for MeerKAT and SKA-related research, IDIA is helping build critical skills in data-intensive astronomy across South Africa and the African continent. Since its inception, IDIA has supported Honours, Master’s, PhD students, and postdoctoral fellows at its partner universities, while the ilifu research cloud has become an essential platform for postgraduate research and training. Many former IDIA-supported researchers have gone on to careers in academia, research institutes, and industry, both locally and internationally, highlighting the strong transferability of the technical and data science skills developed through the programme.