For future reference, to connect to a server directly, I believe you go to the console and use the "connect" command. For example, Covar's server would be "connect tf2.charlescovar.com:27015" without the quotes.
You might also go into your server browser, choose the favorites tab, add the server manually.