There has been a decline in the number of retail investors that own ASX shares directly over the past three years, shows new research by UNSW Business School's Professor Carole Comerton-Forde.