[tor-project] minutes from the sysadmin meeting

Hi!

The moment you have all been waiting for: the first sysadmin minutes of 2023! Whoohoo!

Roll call: who’s there and emergencies

There was a failed drive in fsn-node-03, handled before the meeting,
see tpo/tpa/team#41060.

Dashboard review

Normal per-user check-in:

General dashboards, were not reviewed:

Q1 prioritisation

We discussed the priorities for the coming two months, which will be, in order:

  1. new gnt-dal cluster setup, see milestone 2
  2. self-hosting the forum (@lavamind? march? project ends in July,
    needs to be setup and tested before! created issue
    tpo/tpa/team#41063)
  3. donate page overhaul (meeting this week, @kez, could be Q1, may
    overflow into Q2 - download page in Q2 will need kez as well)
  4. email changes and proposals (TPA-RFC-45, TPA-RFC-47)
  5. bullseye upgrades (milestone 5)
  6. considered lektor-18n update for Google Summer of Code but instead
    we will try to figure out if we keep Lektor at all
    (TPA-RFC-37), then maybe next year depending on the timeline
  7. developer portal people might need help, gaba will put anarcat in touch

OOB / jumpstart

Approved a ~200$USD budget for a jumphost, see tpo/tpa/team#41058.

Next meeting

March 6th 1900UTC (no change)

Metrics of the month

  • hosts in Puppet: 95, LDAP: 95, Prometheus exporters: 163
  • number of Apache servers monitored: 31, hits per second: 675
  • number of self-hosted nameservers: 6, mail servers: 9
  • pending upgrades: 13, reboots: 59
  • average load: 0.79, memory available: 4.50 TiB/5.74 TiB, running processes: 722
  • disk free/total: 33.42 TiB/92.30 TiB
  • bytes sent: 513.16 MB/s, received: 266.79 MB/s
  • planned bullseye upgrades completion date: 2022-12-08 (!!)
  • GitLab tickets: 192 tickets including…
    • open: 0
    • icebox: 148
    • backlog: 20
    • next: 9
    • doing: 11
    • needs information: 5
    • (closed: 3024)

Upgrade prediction graph lives at

Now also available as the main Grafana dashboard. Head to
https://grafana.torproject.org/, change the time period to 30 days,
and wait a while for results to render.