The agreement forming the Inter-University Institute for Data Intensive Astronomy has been renewed by the University of Cape Town, the University of the Western Cape and the University of Pretoria for a period of five years. IDIA also welcomes a new partner in the South African Radio Astronomy Observatory (SARAO), who operates the MeerKAT telescope. IDIA is the main place for South African university researchers to process the very large data sets coming from SARAO’s MeerKAT telescope.
In the beginning
The Inter-University Institute for Data Intensive Astronomy, also known as IDIA, was set up in 2015 to establish a leading centre of research and innovation in South Africa to address the scientific and technical challenges of the era to big data in astronomy, and to ensure that both the infrastructure and the skills pipeline for data intensive research are developed at South African universities as South Africa prepares to host the Square Kilometre Array. The vision was, and still is that this in turn enables South African researchers to become global leaders on MeerKAT large survey science projects and large projects on other SKA pathfinder telescopes. “The Square Kilometre Array, a global megascience project that is coming to South Africa within the next few years, is the biggest scientific data challenge of this decade. With renewal of the IDIA agreement among the partner universities we can build upon our first five years to ensure that researchers and students at South African universities have the capabilities and data science skills to be leaders in this new era of big data astronomy.”, says Prof. Russ Taylor, Director of IDIA.
In the first few years of operations, the institute reached several major milestones including: setting up a first-in-Africa cloud computing facility for data intensive research, building a big data processing pipeline for MeerKAT observations and facilitating the first science results from the telescope, which was inaugurated in 2018. IDIA is also home to a big data visualisation laboratory where immersive scientific data analysis is made possible using technologies like VR and the IzIko digital planetarium. IDIA’s very active Development and Outreach department whose purpose is to engage the public and to ensure maximum socio-economic benefits from IDIA programs.
Tthe Institute today counts 14 senior researchers, 11 Postdoctoral fellows, 6 PhD students, 20 Masters students and 3 Honours students. Four PhDs, 2 Masters and 2 Honours have already graduated from IDIA. When asked what access to IDIA means for his research, Leon Mtshweni jokes “High resolution, full-bandwidth data can be reduced in time durations less than the Hubble time”, referring to the approximately 14.4 billion years old the universe would be if it had expanded linearly. Mtshweni is currently studying for his Masters at the University of Pretoria. Emmanuel Ocran, a PhD student at the University of Cape Town adds that “Access to IDIA is a game changer in terms of research collaborations”. Karina Santana, also doing her Masters at the University of Pretoria says that learning to use remote systems is a skill she’s developed that is important for her training.
Science and Technology
From finding black holes are spinning in unison, to processing data for MeerKAT’s launch image of the center of the galaxy, the science highlights of what has been achieved by IDIA researchers are numerous as they are impressive. And they have been made possible by some equally impressive technology developments, such as the development of scientific software optimized for running on cloud computing infrastructure enabling researchers to do their science, to innovations in data visualization using immersive technologies. These, and other key successes are documented in the first IDIA activity report published in 2019. After an international review in early 2020, the IDIA partner universities followed the review panel’s recommendations to renew the agreement constituting the Institute. “While we may be geographically separate, the IDIA partners collaborate on a wide range of technical and scientific aspects relevant to MeerKAT and towards the SKA. The 5-year review illustrated that coherence as it showcased the diverse yet synergistic projects pursued at all the partner institutions, pointing toward a highly-productive next 5-year cycle.”, says Prof. Roger Deane, IDIA senior researcher at the University of Pretoria.
A major spin-off of IDIA is the establishment of ilifu, a scientific cloud computing facility serving both astronomy and bioinformatics. The new partnership includes Sol Plaatje University, the University of Stellenbosch, the Cape Peninsula University of Technology and the South African Radio Astronomy Observatory as well as UWC and UCT through its computational biology department. “Cloud technologies are key to data-intensive research in many fields, including astronomy and bioinformatics. The containerisation of entire analysis pipelines and the increasingly easy interfaces with scientific software through for example Jupyter notebooks make cloud computing an essential tool for scientific research today and in the future.” adds Prof. Rob Simmonds, Associate Director New Technology Initiatives.
Development and Outreach
But this is not just about building technology and carrying out research. With a very active department for Development and Outreach, IDIA has been supporting and running activities since 2016, such as data-to-dome events showcasing astronomical data onto the digital planetarium dome for the public at the Iziko Museums in Cape Town.
IDIA also runs regular big data hackathons and data science skills development workshops. Partnering with the DARA Big Data initiative and the Office of Astronomy for Development, these hackathons train students in machine learning and artificial intelligence on big data in various topics. Astronomy is of course featured, but so is satellite image analysis and even sentiment analysis of social media feeds.
“These hackathons, with an Africa-wide footprint have also proven to be a very successful way of engaging with historically disadvantaged institutions locally.” says Prof. Carolina Odman, IDIA Associate Director for Development and Outreach.
IDIA today and in the future
To remain competitive in today’s world of data intensive research, we need new ways of doing science and cloud computing is just that. Not just because of the large amounts of data, but also because it helps the integrity of the scientific process to be able to reproduce scientific results that ultimately inform policy and how we design our future. By training a generation of scientists on the platform of the future, we develop skills that are increasingly in demand in all science fields and open the doors to careers in industry and in the knowledge economy that only people can grow. “IDIA is at the forefront of modern radio astronomy, supporting many MeerKAT science cases and a large number of astronomers across the spectrum — from students to senior researchers. It’s vital to have close cooperation between IDIA and SARAO by developing tools, supporting data management and processing, and various astronomy use-cases. This ensures that our ever growing community of radio astronomers have access to robust, flexible and reliable resources to fully exploit the excellent science opportunity of MeerKAT data. Moreover, by playing a strong role in the science value chain, IDIA leads the way in defining what the SKA Regional Data Centre will look like, in the future.” says Dr. Bradley Frank, Associate Director Astronomy Operations.
Hand-in-hand with other projects, such as the MeerKAT telescope and the H3Abionet initiative, the renewal of the IDIA agreement is therefore an important milestone in South Africa’s recent scientific history — one that will yield yet unforeseen results and develop the country’s growing talent in science and technology.
Contact: admin@idia.ac.za