I ran into an issue today where I changed my TeamCity default port (from 8080) and found that my build agents were suddenly disconnected. After some fiddling around, I discovered that I needed to manually change each Build Agent's configuration to point to the new master server port.
To make this change, open up the $TEAMCITY_HOME/buildAgent/conf/buildAgent.properties file and update the following value:
In this case, the agent is running on the same server as the master server. Once you restart the build agent, the master server recognizes the agent as back online. Hope this helps.