TCP SYN timeout
Such a host will wait for a certain amount of time called the SYN timeout without a response, before considering the attempt a failure. The SYN timeout depends upon the operating system of the device in question, as well as the application that is using that TCP session. There is a SYN timeout that is configured on each type of device.
For example, a Linux device will use a hardcoded retransmission timeout setting, where it will attempt to establish the connection every one second. It will continue to do so until the application trying to establish that session will choose to stop, and to consider the attempt failed.
Windows uses a dynamic TCP timeout which can be configured in the registry. The initial timeout is 3 seconds, but it doubles each time a SYN attempt fails. It has a maximum number of retransmissions of 5 before the attempt is considered a failure.
These behaviors can typically be adjusted by the application performing the connection, and should be adjusted to fit the needs of that application. An email client will operate differently than a web browser for example.