The setting is unfortunately hidden. You need to click on the little arrow that's next to the chat login next to stream to find it. If you're not signed into chat first do so then click the little arrow again. From there you can add and remove circles from your chat.