Sarah Kenderdine is lying in darkness, looking up.
Suspended above her is a six-metre-diameter “mega brain”, its corrugations, layers and cognitive circuits exposed in 3D at razor-sharp resolution.
Immersed in the inner workings of the human brain, Professor Kenderdine, from UNSW Art & Design, is focused on the future of medicine. The video projection environment above her is called DomeLab, the highest-resolution, touring full-dome in the world. It allows researchers to ‘look up’ into ultra-high-resolution data sets, an experience Kenderdine describes as “like being up with the gods”.
Art has a lot to offer science, and medical visualisation is hugely fertile territory.
Centuries ago, Leonardo da Vinci demonstrated the role art could play in science when he delicately sketched the intricacies of the human heart, providing the medical profession with the most anatomically correct drawings of the time.
Now, in the high-resolution, gigapixel 21st century, medical scientists are becoming increasingly reliant on artists to present scientific data in new and dynamic ways.
“Art has a lot to offer science, and medical visualisation is hugely fertile territory,” says Kenderdine.
At the forefront of this revolution is UNSW’s Expanded Perception and Interaction Centre (EPIC), where Kenderdine is head of visualisation, and where DomeLab will be based.
The purpose-built centre, a joint initiative of UNSW’s faculties of Art & Design and Medicine, will house some of the most sophisticated medical visualisation equipment yet created, allowing researchers to ‘walk through’ parts of the human anatomy using immersive, 3D technology.
Bringing together leaders in bio-informatics, medical- and neuro-imaging, art and design, immersive visualisation, data fusion, anatomy research and computational engineering from around the world, the centre aims to advance research in cancer, vaccines, dementia, neurological function and mental illness.
Housed in a heritage building on UNSW’s Paddington campus, adjacent to Oxford Street’s medical and arts hub, EPIC will also offer exhibition spaces where the public can engage with the technology.
“The centre will build on Art & Design’s research-led approach to investigate how art can inform science, by providing a space for our researchers and the public to get up close and personal with the mysteries of the human body,” says Dean of Art & Design Professor Ross Harley.
Our task at Art & Design is not to replace the scientific aspects of the data, but to offer an interpretive device to make it more visible and intelligible.
Merging UNSW’s significant scientific and artistic expertise, EPIC will be led by Kenderdine and Caroline (Lindy) Rae, Professor of Brain Sciences from the School of Medical Sciences and Neuroscience Research Australia.
Facilitating the collaboration, an ‘umbilical cord’ of fibre-optic cable will connect EPIC to UNSW’s main campus five kilometres away, allowing medical data to be received, manipulated and visualised in real time.
Kenderdine says the rapid advances in medical imaging, such as MRI, fMRI and PET scans, yield huge data sets that are difficult to interpret unless they are visualised in a compelling way.
“Our task at Art & Design is not to replace the scientific aspects of the data, but to offer an interpretive device to make it more visible, intelligible, as well as aesthetically and acoustically resonant,” she says of DomeLab, which is funded with an Australian Research Council (ARC) Linkage Grant.
Leading childhood cancer researcher Professor Peter Gunning, head of UNSW’s School of Medical Sciences, says the immersive experience of the dome transforms the way scientists approach their research.
“Under the dome I can get inside a cancer cell. I can walk around inside it and see all the structures and their connectivity, and I believe this will lead to new drugs to kill cancer cells,” he says.
Rae, EPIC’s medicine director, researches brain structure and function and believes the technology could accelerate discoveries in her field including improved understanding of the brain’s ageing pathways, leading to better prediction, diagnosis and treatment for patients with mild cognitive impairment and dementia.
“In neuroscience it’s impossible to see the connections in complex data on a flat computer screen,” Rae explains. “But in DomeLab, we can project real data on to the screen to visualise a ‘mega brain’, and this lets us see what’s hidden behind the structure and layers, and how stimulation to the brain activates cognitive circuits and pathways.
“It allows me to fuse the data sets together statistically to make something that is greater than the sum of its parts.”
As well as DomeLab, the high-resolution engine room of EPIC will be a second state-of-the-art immersive visualisation platform, providing a 340-degree view of medical images. The panoramic projection, which was designed and developed at UNSW, will have a resolution of 120 million pixels in 3D, giving it three times higher fidelity than its nearest competitor. Combined with a 32-speaker, surround-sound system, the platform will deliver a multi-sensory experience.
Kenderdine says sound adds a critical dimension to medical visualisations, particularly in mental health.
“Sound is a powerful tool for researching auditory hallucinations in dementia patients. The complex neurological pathways of hearing offer us a new and important perspective,” she says.
EPIC is an exemplar multidisciplinary initiative, says Art & Design’s Chief Scientist in Residence Professor Robert Clark, a former Chief Defence Scientist of Australia and founding director of UNSW’s Centre of Excellence for Quantum Computer Technology.
Its genesis emerged in early discussions between Clark and Professor Jill Bennett, director of UNSW’s National Institute for Experimental Arts, about how art and technology could improve medical and mental health. The discussions were stimulated by research already under way by Art & Design’s John McGhee, George Khut, Alex Davies and Bennett’s team. Initial enthusiasm soon spread to a core project group of researchers, UNSW’s leadership and professional staff.
“The early meetings were magical moments when everyone realised the extraordinary potential of the project,” says Vice-President of Advancement Jennie Lang. “It went from an idea in a few people’s brains to the first stages of a bricks and mortar facility in less than 18 months; it was the fastest actualisation of a centre I’ve ever seen.”
She says Clark played an integral role. As chief scientist he, in partnership with Bennett, was responsible for bringing together experts and key projects that would gain a research edge by the detailed visualisations EPIC will provide. His role has extended deeply into EPIC’s construction and equipment procurement program, involving a significant multi-skilled project team. “UNSW professional staff have worked with enormous intensity and skill to cut through and deliver the facility in record time,” says Clark.
A key feature of EPIC is the establishment of a uniquely qualified technical support team led by visualisation researcher Professor Paul Bourke, who has worked in fields as diverse as architecture and astronomy. A first appointment to this team is Senior Research Fellow Derek Gerstmann, a former production engineer at New Zealand’s Oscar-winning visual-effects company, Weta Digital.
“It’s these experts who will literally bridge the science–arts divide by working closely with medical researchers to deliver a quantum leap in the visualisation of their complex data,” says Clark.
For Kenderdine, EPIC is the culmination of 15 years of creating complex visualisation environments – what she humbly calls “machines”. But she insists it’s not just the techy side of their development that drives her.
“I can get really geeky and obsessed with the technical aspects of machines, but ultimately I’ve always had a burning desire to create things and a strong aesthetic sense,” she says. “My aim is to deliver researchers exceptional opportunities for discovery that are unexpected, and audience experiences that are profoundly affecting.”
Kenderdine says despite all their technological complexities, her “machines” hark back to the great artistic traditions: DomeLab is an expression of the wonder of domed Baroque ceilings and the platform is a nod to 18th-century panoramas, the first artistic wide-angled perspectives.
“People say we are in the fourth industrial revolution, we’re also in a period that’s similar to the Renaissance in terms of expression, visualisation and creation,” she says, adding that EPIC’s visualisation environments can be used to increase understanding of any subject, not just medical data.
But as a starting point, EPIC’s program will launch with 10 groundbreaking medical and mental health research projects, known collectively as M+.
Among them, UNSW Psychology’s Professor Simon Killcross will tackle the visualisation of mental illness, including phobias and Post Traumatic Stress Disorder, to help clinicians better understand their patients; Professor Maria Kavallaris, from the Lowy Cancer Research Centre, will track the complex journey of drugs to human cells; while Scientia Professor Katharina Gaus, from the ARC Centre of Excellence in Advanced Molecular Imaging, will use visualisations to better understand immune responses.
People say we are in the fourth industrial revolution, we’re also in a period that’s similar to the Renaissance in terms of expression, visualisation and creation.
In an M+ project focusing on teaching rather than research, UNSW’s Head of Anatomy Professor Edna Hardeman will use EPIC to revolutionise the learning experiences of medical students.
As EPIC’s Head of Engagement Science, Professor Jill Bennett is working with international cognitive- and neuro-psychologists to investigate whether visual cues can help dementia and amnesia sufferers retrieve otherwise inaccessible memories.
They will use EPIC to screen virtual landscapes combined with personal photos to test whether immersing people in these spatial settings can improve memory retrieval.
In her next project, Bennett will develop a new area of visualisation termed, ‘experience visualisation’, that will allow clinicians and carers to immerse themselves in the thought processes of people suffering from mental health and neurological conditions, including dementia.
“Our aim is to better understand neuro-diverse populations by simulating different sensory and perceptual experiences,” she says.
“The value of art and literature always lies in enabling us to understand the world differently, to see it through another person’s eyes.
“With EPIC we can take it one step further and actually enter inside that experience.”