Standalone proxy - traffic relayed almost zero with restricted NAT type

Mine also in les then 24 hours got 0 clients in the last 3-4 intervals.
Does anyone know how to add an auto restart function for the container?

Try here. You’ll need to look for a process manager as well:

1 Like

My snowflake proxy has 5 IPv6 https connections in status established with snowflake server. If there is no traffic registered, snowflake server has either a bug on load distribution or made a decision not to use these connections? (now 15h of zero traffic relayed)
Finding: iftop presents low traffic on connection to snowflake server. So, both sides talk to each other. Two connections seem to be actively in use.
At least one host resolves to snowflake-01, the second is shown as m-dnslb-a01.
I’ll keep it running …

Update: all 5 connections terminate on snowflake-01 with little traffic (40s average: approx. 400 Bytes/s)
Now 29 repeats on stdout (nohup.out) of

In the last 1h0m0s, there were 0 connections. Traffic Relayed ↑ 0 B, ↓ 0 B

My conclusion: There’s either no demand for this (my) snowflake proxy or my proxy is ignored for some reason.

I’ll keep it running …

My containers all aready configured to autostart in case of failure/crash
i’m using ->unless-stopped option.

But the fact that there are no connections to proxy doesn’t mean the container has issues. I wanted to add a timer or something when i start the container to have the option to restart after 24h hours, at least untill the bug is solved :slight_smile:
Something silimar to -verbose.

Have found a new snowflake commit in git (8b1970a3ce85436e87a9ac868652c3fc3d8c4b58), cloned and compiled it.
Restarted local snowflake proxy with this version …
New message in nohup.out

Bad range port format: 0xc000222070

But it does work and relay messages. …

2 Likes

I made bug report about it:

1 Like

Which is good news. Maybe the message means the server/broker (?) tried a port, spit out the message and then moved on. (Edit: see @Vort ’s issue #40215 above)

By the way, here’s a restart script for nohup from another user, @Beckers

use the “@reboot” directive in /etc/crontab to start a script which contains “nohup /full/path/to/proxy/ -summary-interval 30m -log /full/path/to/proxy.log”. The server is restarted every 24 hours.

Now, again, 16 zero activity messages in sequence. Still 5 TCP connections to snowflake-01 (my proxy is started with -capacity 5). No UDP traffic anymore. During first hours after restart, lsof did list plenty of UDP ports allocated. Now none anymore.

IMHO draining of snowflake proxies happens on the server side! And a proxy restart is like “new game, new luck”.

Anyway, upgrade to proxy git commit ac856280.
Error message is gone. Seems to work, lsof lists TCP and UDP connections. First traffic is relayed …

After approx. 25h zero traffic, snowflake proxy reported io errors, like
io.Copy inside CopyLoop generated an error: websocket: close 1006 (abnormal closure): unexpected EOF
This did start today, approx. 07:28:14 UTC.
Three seconds later a couple of lines like this
error dialing relay: wss://snowflake.torproject.net/?client_ip=[scrubbed] = dial tcp [scrubbed]: con nect: connection refused
did show up in proxy log.
First connected to relay ...did show on 07:31:27 UTC.
Probably a snowflake server restart?
Relaying did restart on 07:35:53 UTC.

I was having the same problem with my standalone proxy the last few days and updating the proxy worked. I recommend trying this.

1 Like

Now, approx. 50h after having been restarted, standalone proxy stopped to write to stdout that gets redirected into nohup.out!
Regardless of this, process is still running but hasn’t relayed any traffic during last 24h.
Process still holds 5 TCP connections to host snowflake-01. This is a waste of resources on server side!
I’ll restart proxy now.

David Fifield documented restart of snowflake-01 yesterday (see commit a179646e on The Tor Project / Network Health / Metrics / Timeline · GitLab). This correlates with my reported stop of stdout messages in nohup.out, although I don’t understand why this might have been triggered by a server restart.

Anyway, snowflake proxy did relay messages for 5 hours. Then drained out. Now 12 hours no relaying!! Process has 5 TCP connections to snowflake-01 established.

I guess there’s something going wrong on server side. As mentioned in previous message: Not using existing connections is a waste of resources. I can’t imagine that there so little demand that during 12 hours not a single message gets relayed.

This is how it looks in snowflake proxy log now:

2022/10/20 21:39:13 Traffic throughput (up|down): 1 MB|163 KB -- (456 OnMessages, 1147 Sends, over 9 seconds)
2022/10/20 21:39:13 copy loop ended
2022/10/20 21:39:13 datachannelHandler ends
2022/10/20 21:39:14 sdp offer successfully received.
2022/10/20 21:39:14 Generating answer...
2022/10/20 21:39:22 OnDataChannel
2022/10/20 21:39:22 Connection successful.
2022/10/20 21:39:22 OnOpen channel
2022/10/20 21:39:23 connected to relay: wss://snowflake.torproject.net/
2022/10/20 22:00:27 In the last 1h0m0s, there were 5 connections. Traffic Relayed ↑ 30 MB, ↓ 1 MB.
2022/10/20 23:00:27 In the last 1h0m0s, there were 0 connections. Traffic Relayed ↑ 0 B, ↓ 0 B.
2022/10/21 00:00:27 In the last 1h0m0s, there were 0 connections. Traffic Relayed ↑ 0 B, ↓ 0 B.
2022/10/21 01:00:27 In the last 1h0m0s, there were 0 connections. Traffic Relayed ↑ 0 B, ↓ 0 B.
2022/10/21 02:00:27 In the last 1h0m0s, there were 0 connections. Traffic Relayed ↑ 0 B, ↓ 0 B.
2022/10/21 03:00:27 In the last 1h0m0s, there were 0 connections. Traffic Relayed ↑ 0 B, ↓ 0 B.
2022/10/21 04:00:27 In the last 1h0m0s, there were 0 connections. Traffic Relayed ↑ 0 B, ↓ 0 B.
2022/10/21 05:00:27 In the last 1h0m0s, there were 0 connections. Traffic Relayed ↑ 0 B, ↓ 0 B.
2022/10/21 06:00:27 In the last 1h0m0s, there were 0 connections. Traffic Relayed ↑ 0 B, ↓ 0 B.
2022/10/21 07:00:27 In the last 1h0m0s, there were 0 connections. Traffic Relayed ↑ 0 B, ↓ 0 B.
2022/10/21 08:00:27 In the last 1h0m0s, there were 0 connections. Traffic Relayed ↑ 0 B, ↓ 0 B.
2022/10/21 09:00:27 In the last 1h0m0s, there were 0 connections. Traffic Relayed ↑ 0 B, ↓ 0 B.
2022/10/21 10:00:27 In the last 1h0m0s, there were 0 connections. Traffic Relayed ↑ 0 B, ↓ 0 B.
1 Like

@mjacobs-de can you test how commit 1bc54948 will work?

1 Like

What do you mean?
Please tell me how can i implement that workaround in my docker build?
One if my builds has stoped for 12hours and another had low traffic…i will upload logs to Reduction in traffic relayed by standalone proxy (#40211) · Issues · The Tor Project / Anti-censorship / Pluggable Transports / Snowflake · GitLab maybe it helps but i also want to try that workaround

I have no experience with Docker to say exactly what to do.
I think that it may be possible to change this line to RUN git clone --depth=1 -b issue/40220 https://gitlab.torproject.org/cohosh/snowflake.git
But I don’t know what to do next.

Another option is just to wait for merge of !108 and for release of new version.

1 Like

Yes, cloned, compiled, started, running, … I’ll keep you updated.

2 Likes

@Vort Sorry, I didn’t recognize that your commit is a branch, so my git clone operation just did get the same code as before.
Now I did git clone of branch issue/40220 with commit 1bc54948, compiled, deployed, startet snowflake proxy, now running …

I’ll keep you updated.

@Vort Now 11 non zero traffic summary lines in sequence since restart. Looks promising :slight_smile:

Update
Now 21 non zero traffic summary lines in sequence since restart. That’s a significant improvement.

Update #2 (Sorry, not more then 3 answers in sequence are allowed)
Since restart:
35 non zero traffic summary lines in sequence! That’s really good :slight_smile:

New findings in nohup.out
5 times this (timestamp is local time instead of UTC):
sctp ERROR: 2022/10/22 21:24:18 [0xc001108000] stream 1 not found)
and some hours later again but 7 times
sctp ERROR: 2022/10/23 08:04:29 [0xc0009ba000] stream 1 not found)

There is nothing special in regular snowflake proxy log at given time stamp.

1 Like