IP Filtering, Deep Dive

BrightSilence · February 10, 2020, 11:06pm

I can confirm that @Alexey must be correct that this was just because of the proxies used. I had a little dig through the code to find the actual code that’s responsible for selecting nodes.

You can find it here:

github.com

storj/storj/blob/7999d24f81f534b61b192bfe1ccdb64821b6f98c/satellite/satellitedb/overlaycache.go#L259


		safeExcludeIPs = ` AND last_net NOT IN (?` + strings.Repeat(", ?", len(excludedIPs)-1) + `)`
		for _, ip := range excludedIPs {
			args = append(args, ip)
		}
	}
	args = append(args, count)


	rows, err := cache.db.Query(ctx, cache.db.Rebind(`
		SELECT *
		FROM (
			SELECT DISTINCT ON (last_net) last_net,    -- choose at max 1 node from this IP or network
			id, type, address, free_bandwidth, free_disk, total_audit_count,
			audit_success_count, total_uptime_count, uptime_success_count,
			audit_reputation_alpha, audit_reputation_beta
			FROM nodes
			`+safeQuery+safeExcludeNodes+safeExcludeIPs+`
			AND last_net <> ''                         -- don't try to IP-filter nodes with no known IP yet
			ORDER BY last_net, RANDOM()                -- equal chance of choosing any qualified node at this IP or network
		) filteredcandidates
		ORDER BY RANDOM()                                  -- do the actual node selection from filtered pool
		LIMIT ?`), args...)

Bottom line; the code first picks one random node per /24 for all nodes with a known IP, and then picks as many nodes as needed for the transfer from that list. The result is no doubt that every /24 subnet will be picked at the same rate. I guess we got all excited over nothing.