Troubleshooting
Start with the checks below, then look up the message you see.
The messages and what triggers them were read from the source code of the app at commit a03a8bc2 and of WaterWall, and on Android at commit 0fc837e3. The commands are the ones we use on our test server.
First checks
1. Open the app's log
Tap the status text under the power button. For more detail, set Log Level to Expert or Debug in the profile's OPTIONS and connect again.
2. Is WaterWall running on the server?
Run these on the server as root:
systemctl status waterwall.service --no-pager
ss -ltnup | grep -i waterwall
active (running), and your configuration's port is in the list: TCP for TLS, Reality, AEAD, no outer layer and QUIC Stream, UDP for KCP and QUIC Datagram. The program shows as Waterwall.
If the service does not start, its own messages say why:
journalctl -u waterwall.service -n 50 --no-pager
tail -n 50 /var/log/waterwall/core.$(date +%Y%m%d).log
After an upgrade from 1.5.3, a configuration file with a name that 1.5.4 and later refuse stops the whole program at start, with one of these lines (we started WaterWall 1.5.42 with two of our server's 1.5.3 files to see them):
| Message | What happened | What to change |
|---|---|---|
FATAL JSON Error: TcpUdpConnector->settings no longer accepts domain-strategy or resolve-domains; use dns-behaviour | A node's settings still have domain-strategy or resolve-domains. The line names the node type. | Remove the setting, or use dns-behaviour; see Upgrading from 1.5.3. |
FATAL NodeManager: node creation failure: library "TcpOverUdpServer" (hash: …) could not be loaded | The KCP node has a new name since 1.5.4. | "type": "KcpServer" on the server (KcpClient on a client). |
3. Is the port open in the firewall?
ufw status numbered
The port and its protocol (tcp or udp) must be allowed. A KCP or QUIC Datagram chain needs a UDP rule; a TCP rule for the same number does not help.
4. Do the two halves match?
Compare the app's choices with the server file, layer by layer, using The server node for each app choice. The most common mistakes are a different port, a different UUID or password, a missing or extra layer (for example MUX on one side only), and an HTTP path that differs from /.
5. Can the server reach the internet?
When it connects, the app checks the tunnel by opening a test connection to 1.1.1.1 on TCP port 80, and for chains that carry UDP, a DNS query to 1.1.1.1. These go through your server, so the server must be allowed to reach 1.1.1.1 on those ports.
curl -sS -o /dev/null -w "%{http_code}\n" http://1.1.1.1/
Any HTTP status code means the server can reach it. On our server it prints 301. An error such as Connection timed out means something blocks the server's outgoing traffic.
Messages and what they mean
When saving a profile
An alert titled Invalid WaterWall Configuration names the field to fix. The full list is in the UI guide.
When connecting
| Message | What happened | What to check |
|---|---|---|
| The WaterWall server address could not be resolved. | The phone could not look up the Server name. | The spelling of the name, and that its DNS record points to your server. |
| WaterWall TCP probe failed after fifteen attempts | The tunnel started, but the test connection to 1.1.1.1 through your chain got no answer. | First checks 2 to 5: the server is running, the port is open, the halves match, and the server can reach 1.1.1.1. |
| WaterWall UDP/DNS probe failed after five attempts | The test DNS query through the chain got no answer. | The server can send DNS queries to 1.1.1.1 on UDP port 53, and the chain allows UDP on the server (for example "udp": true in VlessServer). |
| WaterWall data-plane probe timed out or WaterWall data-plane validation timed out | The start-up check did not finish within its time limit (20 and 30 seconds). | The same as the TCP probe message. |
| WaterWall TCP health probe failed or WaterWall UDP/DNS health probe failed | A connection that was working stopped carrying data: the app checks again when nothing has passed for 15 seconds. | The network, and whether the server is still running. With Auto Reconnect on, the app reconnects after 2 seconds. |
| WaterWall runtime stopped | WaterWall inside the app stopped. | The app's log for the line before it. |
| WaterWall could not start (invalid configuration or another session is active) | The app could not start WaterWall with this profile. | Disconnect any other VPN and try again; check the profile's advanced node settings. |
| Could not reconnect after N attempts. (…) | Auto Reconnect used up the Retry Times you set. The reason is in the brackets. | The reason in the brackets; leave Retry Times empty for unlimited attempts. |
| The connection was closed while the device slept. | On Sleep is Off and Auto Reconnect is off, so the VPN ended when the phone slept. | Choose On Sleep Stay or Pause, or turn Auto Reconnect on. |
| GeoIP is enabled but its database is unavailable: … | GeoIP routing is on but the country database is not installed. | Turn GeoIP Routing off and on again on the WaterWall screen; that installs the database. |
| WaterWall network settings failed: VPN permission was withdrawn | Android only: another VPN app took the permission away while connecting. | Connect again and tap OK in the Connection request. |
In the logs
These lines explain why one request can fail, for example with A TLS error caused the secure connection to fail or The network connection was lost, while the connection stays up.
| Message | What happened | What to check |
|---|---|---|
DomainResolver: async dns resolve failed for <name>: Timeout while contacting DNS servers, in /var/log/waterwall/dns.YYYYMMDD.log on the server | The server could not get a DNS answer for that name in time and closed that connection. | The server's DNS. WaterWall 1.5.42 tries 5 times by default; if core.json sets "dns": {"tries": …}, keep it at 5 or more. |
QuicServer: closing association: application backlog budget exceeded, in /var/log/waterwall/network.YYYYMMDD.log on the server | The server's queue for one QUIC connection passed its max-buffered-bytes, so the server closed that connection, and the requests on it failed. | max-buffered-bytes on both ends; see the Configuration Reference. The app opens a new QUIC connection for the next request. |
QuicServer: closing association: Network blackhole detected, in the same log | The server got no acknowledgement on one QUIC connection for several retransmission timeouts in a row and closed it, for example during a network outage. | The network at that time. The app opens a new QUIC connection for the next request. |
PacketsToConnection: TCP aggregate pending-byte budget is full; refusing a new flow, in WaterWall's log inside the app, which on iOS the app keeps only in test runs; on Android it is in the app's log at Expert or Debug. | More TCP connections were open than the packet budget allows, so the new one was refused. iOS shows it as the TLS error. | max-total-pending-bytes divided by max-pending-bytes is the limit. The app's defaults allow 256. Keep the defaults, or do the division first; see the Configuration Reference. |
Certificates
With TLS Server Verification on, the app refuses a server certificate that is not valid for the name it sends: the SNI, or the Server field when SNI is empty. If your server has a Let's Encrypt certificate for its domain, type that domain in Server and leave SNI empty. Turning verification off hides the problem but lets someone between you and the server pretend to be it, so fix the certificate instead.