Troubleshooting

Start with the checks below, then look up the message you see.

Where this page comes from

The messages and what triggers them were read from the source code of the app at commit a03a8bc2 and of WaterWall, and on Android at commit 0fc837e3. The commands are the ones we use on our test server.

First checks

1. Open the app's log

Tap the status text under the power button. For more detail, set Log Level to Expert or Debug in the profile's OPTIONS and connect again.

2. Is WaterWall running on the server?

Run these on the server as root:

systemctl status waterwall.service --no-pager
ss -ltnup | grep -i waterwall
Check The service is active (running), and your configuration's port is in the list: TCP for TLS, Reality, AEAD, no outer layer and QUIC Stream, UDP for KCP and QUIC Datagram. The program shows as Waterwall.

If the service does not start, its own messages say why:

journalctl -u waterwall.service -n 50 --no-pager
tail -n 50 /var/log/waterwall/core.$(date +%Y%m%d).log

After an upgrade from 1.5.3, a configuration file with a name that 1.5.4 and later refuse stops the whole program at start, with one of these lines (we started WaterWall 1.5.42 with two of our server's 1.5.3 files to see them):

MessageWhat happenedWhat to change
FATAL JSON Error: TcpUdpConnector->settings no longer accepts domain-strategy or resolve-domains; use dns-behaviourA node's settings still have domain-strategy or resolve-domains. The line names the node type.Remove the setting, or use dns-behaviour; see Upgrading from 1.5.3.
FATAL NodeManager: node creation failure: library "TcpOverUdpServer" (hash: …) could not be loadedThe KCP node has a new name since 1.5.4."type": "KcpServer" on the server (KcpClient on a client).

3. Is the port open in the firewall?

ufw status numbered

The port and its protocol (tcp or udp) must be allowed. A KCP or QUIC Datagram chain needs a UDP rule; a TCP rule for the same number does not help.

4. Do the two halves match?

Compare the app's choices with the server file, layer by layer, using The server node for each app choice. The most common mistakes are a different port, a different UUID or password, a missing or extra layer (for example MUX on one side only), and an HTTP path that differs from /.

5. Can the server reach the internet?

When it connects, the app checks the tunnel by opening a test connection to 1.1.1.1 on TCP port 80, and for chains that carry UDP, a DNS query to 1.1.1.1. These go through your server, so the server must be allowed to reach 1.1.1.1 on those ports.

curl -sS -o /dev/null -w "%{http_code}\n" http://1.1.1.1/

Any HTTP status code means the server can reach it. On our server it prints 301. An error such as Connection timed out means something blocks the server's outgoing traffic.

Messages and what they mean

When saving a profile

An alert titled Invalid WaterWall Configuration names the field to fix. The full list is in the UI guide.

When connecting

MessageWhat happenedWhat to check
The WaterWall server address could not be resolved.The phone could not look up the Server name.The spelling of the name, and that its DNS record points to your server.
WaterWall TCP probe failed after fifteen attemptsThe tunnel started, but the test connection to 1.1.1.1 through your chain got no answer.First checks 2 to 5: the server is running, the port is open, the halves match, and the server can reach 1.1.1.1.
WaterWall UDP/DNS probe failed after five attemptsThe test DNS query through the chain got no answer.The server can send DNS queries to 1.1.1.1 on UDP port 53, and the chain allows UDP on the server (for example "udp": true in VlessServer).
WaterWall data-plane probe timed out or WaterWall data-plane validation timed outThe start-up check did not finish within its time limit (20 and 30 seconds).The same as the TCP probe message.
WaterWall TCP health probe failed or WaterWall UDP/DNS health probe failedA connection that was working stopped carrying data: the app checks again when nothing has passed for 15 seconds.The network, and whether the server is still running. With Auto Reconnect on, the app reconnects after 2 seconds.
WaterWall runtime stoppedWaterWall inside the app stopped.The app's log for the line before it.
WaterWall could not start (invalid configuration or another session is active)The app could not start WaterWall with this profile.Disconnect any other VPN and try again; check the profile's advanced node settings.
Could not reconnect after N attempts. (…)Auto Reconnect used up the Retry Times you set. The reason is in the brackets.The reason in the brackets; leave Retry Times empty for unlimited attempts.
The connection was closed while the device slept.On Sleep is Off and Auto Reconnect is off, so the VPN ended when the phone slept.Choose On Sleep Stay or Pause, or turn Auto Reconnect on.
GeoIP is enabled but its database is unavailable: …GeoIP routing is on but the country database is not installed.Turn GeoIP Routing off and on again on the WaterWall screen; that installs the database.
WaterWall network settings failed: VPN permission was withdrawnAndroid only: another VPN app took the permission away while connecting.Connect again and tap OK in the Connection request.

In the logs

These lines explain why one request can fail, for example with A TLS error caused the secure connection to fail or The network connection was lost, while the connection stays up.

MessageWhat happenedWhat to check
DomainResolver: async dns resolve failed for <name>: Timeout while contacting DNS servers, in /var/log/waterwall/dns.YYYYMMDD.log on the serverThe server could not get a DNS answer for that name in time and closed that connection.The server's DNS. WaterWall 1.5.42 tries 5 times by default; if core.json sets "dns": {"tries": …}, keep it at 5 or more.
QuicServer: closing association: application backlog budget exceeded, in /var/log/waterwall/network.YYYYMMDD.log on the serverThe server's queue for one QUIC connection passed its max-buffered-bytes, so the server closed that connection, and the requests on it failed.max-buffered-bytes on both ends; see the Configuration Reference. The app opens a new QUIC connection for the next request.
QuicServer: closing association: Network blackhole detected, in the same logThe server got no acknowledgement on one QUIC connection for several retransmission timeouts in a row and closed it, for example during a network outage.The network at that time. The app opens a new QUIC connection for the next request.
PacketsToConnection: TCP aggregate pending-byte budget is full; refusing a new flow, in WaterWall's log inside the app, which on iOS the app keeps only in test runs; on Android it is in the app's log at Expert or Debug.More TCP connections were open than the packet budget allows, so the new one was refused. iOS shows it as the TLS error.max-total-pending-bytes divided by max-pending-bytes is the limit. The app's defaults allow 256. Keep the defaults, or do the division first; see the Configuration Reference.

Certificates

With TLS Server Verification on, the app refuses a server certificate that is not valid for the name it sends: the SNI, or the Server field when SNI is empty. If your server has a Let's Encrypt certificate for its domain, type that domain in Server and leave SNI empty. Turning verification off hides the problem but lets someone between you and the server pretend to be it, so fix the certificate instead.