The cage

I don't need a billion for that; I have enough in my sofa seats to buy your mum.

I'm on the same path trying to find a solution. Is it possible to just whitelist certain accounts without all the public rooms. E.g., I have my own instance, but want to allow a DM from friend@matrix.org without allowing public rooms or other random DMs?

As far as I can tell...you can't. Debating just using XMPP instead.