Fantastic journey

The world of virtual reality and the science of the body can seem a paradox: one takes you out of space and time and the other is wholly tangible – a living, breathing collection of cells.

But for Associate Professor John McGhee, pulling on a virtual-reality (VR) headset doesn’t transport you to a sci-fi underworld. It is a journey to the surface of a cancer cell. 

Ground-breaking virtual-reality technology led by McGhee, from UNSW Art & Design, allows multiple scientists to see inside a human cell at the same time, giving researchers a three-dimensional tool to improve doctor interaction and help analyse how cancer drugs work.

“I saw we could do so much more with VR than selling products and superheroes,” says McGhee, who began his career visualising data in a UK radiology clinic. “Maybe I’m a frustrated scientist but I want to help people make sense of their bodies. We have such amazing gaming technology and we can use it to benefit patients and specialists. 

The inner workings of the body can often get lost in specialist data. This makes the process democratic.” 

McGhee is working with Professor Maria Kavallaris, from UNSW’s Australian Centre for Nano Medicine and Children’s Cancer Institute, in the second phase of the Journey to the Centre of the Cell project, which combines scientific data, microscopy images and animation to create a VR world of cells and blood vessels that can be seen through headsets.

Until now, McGhee’s work involved single VR headsets, with one person moving around in the data. But new technology means multiple users from different parts of the world can walk inside the ‘landscape’ of the cell at the same time. It is a huge leap for scientists and clinicians.

“We can have multiple scientists logging in at once and up to four scientists being able to view the same data, so multiple people can annotate, interact and make sense of what’s there,” he says. “This could help researchers ask questions they’ve never thought of. It’s a dialogue tool that could help change their approach [to cancer treatment].”

As co-chief investigators at the ARC Centre of Excellence in Convergent Bio-nano Science and Technology, McGhee and Kavallaris are using this phase of the project to educate researchers on how cells function and to look at the way nanoparticle drugs are absorbed by cancer cells.

The mitochondria (orange) and endosomes (blue) inside a virtual breast cancer cell. Photo: John McGhee

“Rather than seeing data through a microscope, we use immersive headset systems so you are literally standing right next to a cell structure,” McGhee says. “The first phase was static but now we have dynamic data so we can track the movement of nanoparticles.

“Our goal is to be able to see a drug absorbing into the tumour, so you can highlight the target for radiotherapy. It could mean you can also deploy a drug more accurately.”

Kavallaris, a leading cancer biology researcher and nanomedicine expert, says tracking the three-dimensional movement of cells in tumours can show scientists what happens when cells move in real time – information that can be used when looking at the spread of cancer.

“We look at tumour cells in a live animal, capture information about the migration of the cells, and that data gets sent back to John to be rendered to make it interactive,” Kavallaris says.

“We have never had access to something like this before. Apart from being educational, we hope the multi-user VR will help scientists better understand how and what happens if you interfere with a genetic process and add certain drugs, to see what the effect is on the cancer cells. We can travel and touch things and walk through structures. When I’m looking at a static image I can’t look at the underside of a cell.”

Kavallaris can see VR technology being particularly useful in the childhood cancer neuroblastoma, due to the speed at which it spreads.

“Eventually it could be a tool to explain to patients and their parents about a cancer. Ultimately the technology could be used in MRI and PET scans, to take patients through the body to show where the cancer cells are and what doctors are doing to treat them,” she says.

Instead of the doctor talking to a patient about how the tumour has shrunk or grown, the technology could allow the patient to see the three-dimensional uptake of the drug and feedback on its effectiveness.

McGhee has collected cell data using PET CT, MRI data, cryo-electron microscopy and lattice light-sheet microscopy to create a dynamic cell.

“If scientists can work more collaboratively then they can see more things in the data that can accelerate the discovery process,” he says. 

“And that’s the ultimate thing. If a scientist can walk up to a three-dimensional data set and touch and engage with cells, it becomes an incredibly useful tool.”