An ‘atyAshrami’ is one who is either in the SannyAsa-Ashrama or one who is even higher than that, namely one who is a jnAni whom no ShAstraic injunctions touch.
The Svetasvataropanishad (VI-21) seems to be teaching Brahma VidyA only to such atyAshramis. There is an Upanishad called Kaivalyopanishad. The Acharya used to quote from it often.
In the beginning of that Upanishad it says the atyAsharami goes to a solitary place, sits in a straight Asana, controls his senses and mind and meditates on the Shiva svarUpa, his Atman.
After shravaNa, come manana and nididhyAsana. Just as it says: "Only after becoming a sannyAsi the shravaNa process takes place" so also there is also an authority for saying Only a SannyAsi has the right to do manana and nididhyAsana: *mananAdau sannyAsinAM adhikAraH*.