Former Supreme Court judge Justice Rohinton Nariman on Friday suggested that independent, retired judges should be directly involved in the system of appointing Supreme Court and High Court judges, to replace the present Collegium system of appointments.