NumCPUs best practice on multi core/multi relay servers

[NTH]

Hi,

Tor keeps telling me that it maxes out at 16 cores:

Wow! I detected that you have 24 CPUs. I will not autodetect any more than 16, though. If you want to configure more, set NumCPUs in your torrc

I was under the impression that by default this applies for each torrc file/tor relay individually, and that multiple torrc files will use the cores they need unless you limit it in the specific torrc file with the NumCPUs configuration parameter.

But the total load for all tor relays (each having their own torrc file) always averages out on 16 (for months now), so I begin to suspect this is also some general limit (or that it’s just coincidence that it’s both 16). My tor server still has some room to grow so I wouldn’t mind putting it to work a bit more. So my questions are:

1. Is there indeed some general limitation of 16 cores for all tor processes together without defining a higher amount manually?
2. If so, what would be best practice to fix this? Add 2 cores to each torrc file? Or is there some general setting to upgrade the limit?

Thanks in advance!

[boldsuck]

NumCPUs have been a topic in the meetings and on the list in the last few weeks.
In short: leave it alone
more precisely here: Sanity check on NumCPUs

[NTH]

Thanks for your reply! I already read the linked comment when I searched this forum before I posted my question, but it doesn’t answer my question sadly.

Does the default maximum of 16 cores apply to each tor relay/torrc file or to tor as a whole? If it’s the latter then the advice to let tor use all available cores also means configuring NumCPUs. But I can’t find documentation on the inner workings of this default maximum, so hence I ask to be sure .

[boldsuck]

@NTH

Does the default maximum of 16 cores apply to each tor relay

Yes, if you have 40 tor-instances you will see 40 instances with this Wow! ... Each instance starts multiple threads: htop - 2 tor-instances on quad core
I have 40 instances running on 40 CPU cores. The only problems are network I/O or the network driver.

[NTH]

Okay, then it shouldn’t matter and I can leave NumCPUs as default. Thanks for helping! Kind of sad though, I was hoping that this would be the bottleneck I’m experiencing on my Tor server. My CPU is only utilized ~60% so I was trying to locate the bottleneck to make it use ~100% and contribute more bandwidth to the network.

[boldsuck]

That’s why we all have multiple instances on one server. Only 2 instances/IP possible!

tor-instance-create 00
tor-instance-create 01
systemctl enable tor@00
systemctl enable tor@01
systemctl mask tor@default
systemctl daemon-reload

nano /etc/tor/instances/00/torrc
nano /etc/tor/instances/01/torrc
systemctl start tor

If your relay is new, be patient. Guard relays only make traffic after 3-4 months.
On the servers where I don’t pay electricity, the free CPU’s do Monero mining. ¯_(ツ)_/¯

[NTH]

I’m doing pretty much the same, running 18 relays right now, of which 15 are on the same server. They run for some time now as well so that shouldn’t be the bottleneck. The difference is I’m not running Linux but FreeBSD, so running multiple relays works a bit differently for me .

Actually not a bad idea to mine some crypto currency on the underutilized cores though… I might be able to contribute some more to Tor that way.

[krix]

On Windows it does not matter, tor process will stick to one CPU

[boldsuck]

This is also the case on Linux and BSD. C-Tor is not really multicore aware.
If I understand correctly, the Tor main process is on one core. The other threads are not really used. (1-3% CPU) Important the processor AES-NI option is enabled → BIOS setting.
Chapter 4 + 5 Tor: A Multi-Threaded Relay Architecture

Arti, the upcoming Tor implementation in Rust, will utilize multiple cores/threads much better.
Tor Browser first (this or next year) relays and onion services in Rust development will take a few more years.

[code9n]

Hi,

Just to clarify, then:

I have a vps with one relay running on it. (4 CPUs, 4GB RAM, debian 11). It’s about maxing out CPU usage according to top . But only using about half the RAM.

But if I run another instance that should take up some of the unused resources on there without much trouble by the looks of it - via unused CPUs?

What do I do to run a second instance? The first instance is just running as root. Should I make a new user with sudo privileges to install the new instance under? Or just do another install like the first, still in root and adjust for the notices.log files as per this thread:

Cannot initiate notice logs for second instance of Tor

I would like to run a second non-exit relay instance on my server. However, there seems to be no way to let my second instance log notices into a file within the /var/log/tor folder. Removing the “Log notice file /var/log/tor/notices2.log” parameter and replacing it with “Log notice syslog” or removing the Log notice option entirely allows the second instance to start. I’d still like to log notices to a file separate of the first instance. I am running Ubuntu 20.04. Here is what I’ve tried: …

Also, is it bad practice to run tor as root in the first place?

Thanks,

[boldsuck]

@code9n

What do I do to run a second instance?

On Debian and derivatives like Ubuntu just use: tor-instance-create

tor-instance-create 00
tor-instance-create 01
systemctl enable tor@00
systemctl enable tor@01
systemctl mask tor@default
systemctl daemon-reload

config goes to

/etc/tor/instances/instance_name/torrc

systemctl commands under systemd

systemctl start tor@00
systemctl status tor@00
systemctl reload tor@00
systemctl reload tor  # <- reload all instances

@code9n

The first instance is just running as root.

Default tor user is debian-tor not root.
Instances “user:group” according to the setup above: _tor00:_tor00

@code9n

Also, is it bad practice to run tor as root in the first place?

Yes, this is the worst case security nightmare. All services should never run as root. (tor, sshd, httpd, smtpd, …)

Logging:
I recommend log to syslog, otherwise the systemd script must be adapted for each instance. Or you get troubles with apparmor and updates. See modifying-existing-systemd-services and “man systemd.service” for details.

If you want to add further options or overwrite existing ones then use something like:
~$ systemctl edit tor@00.service
and add the path to log Dir, e.g.:

[Service]
ReadWriteDirectories=-/var/log/tor-instances

If you don’t want the disks to fill up, don’t forget the logrotate config in “/etc/logrotate.d”

[code9n]

Thanks for the reply.

I’ll set up some sudoers and give one the existing instance and start a second instance in a second user.

Everyone’s a noob at something. You learn by asking questions (if you can’t find a good answer in the docs) and getting helpful answers.

Thanks again,