Blog icon

The challenge

Accurately representing a user's environment using vision processing algorithms

The challenge is to develop novel vision processing methods that aim to extract and process salient information from a scene and represent that information using only the limited amount of phosophenes (points of light) from the retinal implant (i.e. bionic eye).

The aim is to represent the environment using vision processing algorithms that enable individuals implanted with the bionic eye to experience improved functional vision.

Six images generated by the vision processing technology developed by CSIRO's Data61 in collaboration with Bionic Vision Technologies. The images show ground plane segmentation to augmented depth phosphene images.

Our response

Developing a retinal implant and software library

We developed a retinal implant and supporting technologies, including vision processing and a software library, to restore sight to a degree in people with profound vision loss due to Retinitis Pigmentosa or Macular Degeneration.

Three profoundly blind individuals received the implant in 2012-2014 and into a further four in 2018.

Our technology selects and displays information to solve tasks required for independent living. We pioneered vision processing technologies that take image streams from a head worn camera and perform processing to turn them into stimulation patterns on the retinal implant, resulting in the perception of spots of light by the patients.

Advanced methods select and display key information such as orientation, mobility, and finds items on a table top.

This led to significant papers including a 'first-in-humans' study where we showed that the supra-choroidal approach was an effective and safe retinal implant therapy.

Critically, the paper showed end-to-end results where CSIRO vision processing enabled profoundly blind subjects to pass ultra-low vision tests. In the Journal of Neural Engineering, we reported two important breakthroughs:

  • For the first time, our advanced vision processing methods led to significantly improved performance on visual tasks.
  • Participants using our advanced methods for scene understanding from colour and depth images performed significantly better in mobility tasks.

The results

Vision processing in the bionic eye

This research is embodied as software that converts an image stream captured by a head-worn camera into a useful visual representation for implantees via electrical stimulation for a retinal implant. This might include highlighting the ground, obstacles, or faces.

The Bionic Eye consists of the implantable hardware, an external processor, and image processing software. The implant sits in the eye, behind the retina, and stimulates nerve cells providing the perception of vision.

The electrical impulses it produces are 'seen' as spots of light by the recipient. The implant is connected to an external smartphone sized device and a wearable camera. The device captures video from the camera and encodes it for the implant.

Bionic vision processing diagram
An image of a woman wearing the external apparatus needed to process the user's environment. On the bottom right hand corner, an internal photo of the implant sitting within the eye is shown. 

1. The camera in the glasses the user wears are captured in real time
2. The image is sent to the processing device (a small black box located externally on the user)
3. That processed data is then sent to the implant and then to the brain for user navigation
An image of a woman wearing the external apparatus needed to process the user's environment. On the bottom right hand corner, an internal photo of the implant sitting within the eye is shown. 1. The camera in the glasses the user wears are captured in real time 2. The image is sent to the processing device (a small black box located externally on the user) 3. That processed data is then sent to the implant and then to the brain for user navigation

In 2017, spin out company Bionic Vision Technologies, which CSIRO is a shareholder, raised US$18M in capital.

Following a successful two-year feasibility study conducted on patients with retinitis pigmentosa, Bionic Vision Technologies is on track to complete clinical trials in early 2024, with plans for the device to become available commercially in the US and Europe in late 2024 subject to additional capital funding.

"The team at CSIRO's Data61 are excited about the success of the trials Bionic Vision Technologies is conducting, with the possibility that individuals with retinitis pigmentosa could have their vision restored an amazing impact," says CSIRO's Data61 project lead, Dr Nariman Habili.

"CSIRO's Data61 is proud to have contributed a software library that implements the algorithms that enable users to see while using this technology, and look forward to continuing to help drive this great application of AI-powered tech in the future."

Contact us

Find out how we can help you and your business. Get in touch using the form below and our experts will get in contact soon!

CSIRO will handle your personal information in accordance with the Privacy Act 1988 (Cth) and our Privacy Policy.


First name must be filled in

Surname must be filled in

I am representing *

Please choose an option

Please provide a subject for the enquriy

0 / 100

We'll need to know what you want to contact us about so we can give you an answer

0 / 1900

You shouldn't be able to see this field. Please try again and leave the field blank.