What’s the issue?
Always On Availability Groups communicate between replicas through a TCP database mirroring endpoint, typically named Hadr_endpoint and configured during AG creation to listen on a specific port (5022 by default). Each replica in an AG has its own endpoint, and the endpoints handle the log record streaming, health detection, and synchronization traffic that keep the AG running.The endpoint has a state recorded in sys.tcp_endpoints and can be either STARTED (accepting connections) or STOPPED (not accepting connections). The normal operating state is STARTED, and the endpoint must be running for the local replica to participate in the AG.
This finding identifies replicas where the AG endpoint is currently in the STOPPED state. The condition can result from manual administrative action, certain failure scenarios, or configuration changes that left the endpoint in an unintended state.
Why is this a problem?
A stopped endpoint cannot send or receive AG traffic, which means the local replica cannot synchronize with its partners. If the affected replica is the primary, secondaries cannot receive log records and fall behind, causing the primary’s transaction log to grow because records cannot be truncated until they are delivered. If the affected replica is a secondary, that secondary cannot receive updates and falls out of synchronization, eliminating its usefulness for failover or backup offload.The condition also affects automatic failover. If the primary fails while a secondary’s endpoint is stopped, the cluster may not be able to fail over to that secondary because health detection and the synchronization handshake depend on endpoint connectivity. This can turn a routine failover into an extended outage that requires manual intervention to restore the AG.
A stopped endpoint can be hard to spot because the AG dashboard and standard monitoring may continue to show partial information. The synchronization state shows as not synchronizing or in an error state, but the underlying cause (the endpoint itself) is not always obvious without specifically checking endpoint status.
Endpoints can also be stopped silently as a side effect of other administrative actions. Certain SQL Server upgrades, security configuration changes, or scripts that drop and recreate endpoints can leave the endpoint in a stopped state until someone explicitly starts it. The AG may have been working perfectly before the change and stops working immediately afterward, with the cause not obvious to anyone troubleshooting the resulting symptoms.
What should you do about this?
Start any stopped endpoint with ALTER ENDPOINT [Hadr_endpoint] STATE = STARTED;, adjusting the endpoint name to match the actual configuration. The change takes effect immediately, and AG synchronization should resume automatically once both ends of the connection are active. Monitor the synchronization state to confirm the affected replica catches up successfully.Investigate why the endpoint was stopped to determine whether the cause was a deliberate action, a side effect of other work, or something less expected. Review the SQL Server error log and the change history for the instance around the time the endpoint state changed, and document the cause so similar incidents can be prevented or detected faster in the future.
Verify that endpoint permissions are correct, since a started endpoint that lacks the proper CONNECT permissions for the partner replicas’ service accounts produces symptoms similar to a stopped endpoint. The service account of each partner replica needs CONNECT permission on this replica’s endpoint, and the inverse must also be true. Use GRANT CONNECT ON ENDPOINT::[Hadr_endpoint] TO [DOMAINServiceAccount]; if necessary.