By default, Ventrilo ships with only 20 unauthenticated users allowed. To get around this, you can either assign these users to their own accounts or you can simply raise the limitation.
To raise the limitation:
1. Log in to your Ventrilo server and escalate your rights to admin by right-clicking the lobby, clicking "Server admin" and then "Login" and entering your admin password (the one listed on the Easy setup page in your control panel) when it asks.
2. Right-click on the lobby again and then click "Server admin" and "Server properties".
3. Under "Max guest logins", enter "0".
4. Click "Send" and the box will close.