[tor-project] OONI Monthly Report: May 2022

Mailing Lists tor-project
1

Hello,

OONI’s May 2022 status report is shared below.

# OONI Monthly Report: May 2022

Throughout May 2022, the OONI team worked on the following sprints:

  • Sprint 64 (1st - 8th May 2022)
  • Sprint 65 (9th - 22nd May 2022)
  • Sprint 66 (23rd - 31st May 2022)

Our work can be tracked through the various OONI GitHub repositories: https://github.com/ooni

Highlights are shared in this report below.

## Job opening for OONI Community Coordinator

In May 2022, we published a job opening for hiring a Community Coordinator for OONI: https://ooni.org/post/2022-job-opening-ooni-community-coordinator/

In an effort to help ensure that our job opening reached interested candidates around the world, we worked on quite extensive outreach efforts. In particular, we posted the job opening on:

We also reached out to our partners and many other community members, requesting that they help share this opportunity with their networks.

Following the publication of this job opening, we worked on tracking and reviewing incoming applications, and following-up with applicants.

## OONI Probe Mobile

Throughout May 2022, we worked on various improvements to the OONI Probe mobile app in preparation for the next release.

Specifically, we worked on:

## OONI Run

We implemented a fix related to the copy-pasting of URLs into the OONI Run web interface (https://github.com/ooni/run/pull/105). Based on community feedback, we explored how we can add support to OONI Run for saving custom test lists (https://github.com/ooni/probe/issues/867), and we had team meetings to discuss possible implementation approaches. We also wrote an internal design document on implementation approaches for addressing the most pressing OONI Run community needs (as identified through a previous OONI Run usability study and ongoing community interactions).

## OONI Probe Desktop

We made some improvement to the description of the Experimental card in OONI Probe Desktop (https://github.com/ooni/probe-desktop/pull/294) and we worked towards ensuring that the autorun directory is deleted when a hard reset is performed (https://github.com/ooni/probe-desktop/pull/295).

## OONI Probe Web

We added support to OONI Probe Web for using the check-in API to retrieve information about the probe IP address country code and ASN (https://github.com/ooni/probe-web/commit/bc680323df0c7404b14651acea11a281c7353f96). We also worked on improving the UI of OONI Probe Web.

We circulated an internal testing beta of OONI Probe Web (https://ooni.github.io/probe-web/) to collect community feedback that can help inform further development and improvements.

## Expanding censorship measurement methodologies

### Website measurement

After some initial experimentation in April 2022, we sketched out a plan for integrating some methodologies from the “Understanding the Practices of Global Censorship through
Accurate, End-to-End Measurements” (https://shhaos.github.io/papers/sigmetrics22.pdf) study into OONI Probe (https://github.com/ooni/probe/issues/2086).

We started implementing the plan for merging the Websteps prototype (https://github.com/bassosimone/websteps-illustrated/) into OONI Probe. To this end, we merged several improvements for netxlite, the ooniprobe package that provides basic networking functionality (https://github.com/ooni/probe/issues/2096).

We also drafted and implemented a plan for observing duplicate DNS replies (https://github.com/ooni/probe/issues/2099). This plan differs from the Websteps implementation because community feedback (in particular from Jigsaw’s Vinicius Fortuna and Sarah Laplante) convinced us that the implementation of this functionality that we had sketched out in the Websteps prototype could be further simplified and streamlined.

### New experiments

We implemented several new exploratory experiments:

These are “exploratory” in the sense that we expect users to manually run them from the command line when they are actively researching censorship. We introduced these four new experiments based on community feedback and requests.

### Vanilla Tor experiment

We implemented the Vanilla Tor experiment (https://github.com/ooni/probe-cli/pull/718), which used to be part of the legacy version of OONI Probe. We reintroduced this experiment following discussions with Tor developers, who mentioned that it would be quite useful to them.

### Data quality improvements

We investigated and fixed a rather surprising and complex data quality issue. It turns out that the Android implementation of getaddrinfo behaves differently from any other existing getaddrinfo implementation and only returns one of two return codes: 0 in case of success, and EAI_NODATA in case any other possible error occurs, including cases where a domain name does not exists, cases where the server is temporarily unavailable, cases where the server is misconfigured, and cases of censorship.

In practical terms, this means that the result of getaddrinfo on Android is unreliable and we need to improve our heuristics on the backend and our algorithms on the probe to work around this. The pull request where we fixed this bug refers to our investigation, as well as to our follow up plans to further improve DNS measurements on Android (https://github.com/ooni/probe-cli/pull/764). The reference issue describing this problem (https://github.com/ooni/probe/issues/2029) is still open because we have not included the fix into the 3.15.x release cycle (which was being finalized while we were still investigating the root cause of this surprising bug).

## OONI Explorer

We fixed a couple of OONI Explorer bugs. Specifically, we added support for searching URLs with a longer TLD (https://github.com/ooni/explorer/pull/757) and we updated the copyright year in the footer (https://github.com/ooni/explorer/pull/756).

## Building a web platform for updating test lists

Based on community feedback on the private beta version of our new test list web platform (https://test-lists.ooni.org/), we added backend support (https://github.com/ooni/api/pull/286) to enable users to make changes to open pull requests (once a pull request has been opened through a test list web platform submission).

In anticipation of a potentially increased volume of test list pull requests following the upcoming public launch of the test list web platform, we (along with the Citizen Lab) invited Censored Planet and Netalitica team members to get added to the Citizen Lab test list repository as reviewers of pull requests.

## OONI backend

### Migration to the Clickhouse database

We continued to work on the ongoing migration to the new OLAP database. In May 2022, we migrated 12 API entry points to the new backend host.

### Reprocessing legacy measurements

We continued to work on the reprocessing of measurements collected before 2021 to improve data quality and ensure all measurements are reachable from OONI Explorer.

### OONI API

We reviewed and deployed a change to support the detection of the ASN and country code of probes based on their IP address on the API side. This will allow the removal of bulky GeoIP database files from the probes. We also improved the account_metadata API call to display login status on web UIs (https://github.com/ooni/api/pull/292).

### Other operations

We added blackbox exporter-based testing and support for the alarming of various API entry points. We finished moving the test helper rotation script to the new backend host (https://github.com/ooni/pipeline/pull/386).

## Test list updates

In May 2022, we contributed a few updates to the Citizen Lab test lists. In particular, we updated the US test list (https://github.com/citizenlab/test-lists/pull/984) and we removed dead URLs from the Global test list (https://github.com/citizenlab/test-lists/pull/995).

## Notable community use of OONI Probe and OONI data

### Sinar Project report

Malaysia’s Sinar Project published a report on the blocking of a NFT fundraising campaign website (of graphics designer and political activist, Fahmi Reza): https://sinarproject.org/digital-rights/measuring-and-detecting-network-interference/nft-fundraising-campaign-website-of-malaysian-artist-fahmi-reza-blocked

Their report makes use of OONI Probe and OONI data.

## Community activities

### OONI workshop for civil society groups in Latin America

On 19th May 2022, OONI’s Maria facilitated a 2-hour OONI workshop for civil society groups in Latin America. This workshop (organized by Derechos Digitales) was facilitated in collaboration with Venezuela Inteligente.

### OONI presentation at Tactical Tech

On 24th May 2022, OONI’s Maria presented OONI at Tactical Tech (https://tacticaltech.org/) in Berlin. This presentation also involved a live demo of using the Measurement Aggregation Toolkit (MAT) to investigate various forms of internet censorship around the world.

### OONI presentation for journalists

On 25th May 2022, OONI’s Maria presented OONI’s tools, methods, research, and dataset to a group of journalists from Russia and Belarus.

### OONI Community Meeting

On 31st May 2022, we hosted the monthly OONI Community Meeting on our Slack channel (https://slack.ooni.org/), during which we discussed the following topics:

  1. Job Opening: OONI Community Coordinator (https://ooni.org/post/2022-job-opening-ooni-community-coordinator/)

  2. Detecting regional censorship, which is increasingly becoming an urgent issue (https://github.com/ooni/probe/issues/1581#issuecomment-1125221383)

In particular, we discussed the following questions with the community:

  • Should there be an option to also include the region or should we include the region by default?
  • Should we invest some legal research time to figure out if we could use MaxMind?
  • Should we try to have location permission in the app and use that? Should we have two apps, one with location and the other without?
  • Should we allow users to manually annotate/correct the geolocation results if they are inaccurate?
  1. Copyright law for enforcing internet censorship

## Userbase

In May 2022, 41,882,898 OONI Probe measurements were collected from 2,905 AS networks in 171 countries around the world.

This information can also be found through our measurement stats on OONI Explorer (see chart on “monthly coverage worldwide”): https://explorer.ooni.org/

~ OONI team.

2 Likes