Try to get an object from another SWISH server in the network. We
implement replication using the PUB/SUB protocol of Redis. This is
not ideal as this route of the synchronisation is only used if for
some reason this server lacks some object. This typically happens if
this node is new to the cluster or has been offline for a long time.
In a large cluster, most nodes will have the objects and each of
them will send the object around. A consumer group based solution is
not ideal either, as the message may be picked up by a node that
does not have this object, after which we need the failure recovery
protocol to get it right. This is particularly the case with two
nodes, where we have a fair chance to have be requested for the hash
we miss ourselves.
We could improve on this two ways: (1) put the hash published in a
short-lived key on Redis and make others check that. That is likely
to avoid many nodes sending the same object or (2) see how many
nodes are in the pool and switch to a consumer group based approach
if this number is high (and thus we are unlikely to be asked
ourselves for the missing hash).
- See also
- - publish_objects/2 for the incremental replication
True to get Hash if we do not have it locally. This initiates a
Redis discover request for the hash. The replies are picked up by
gitty_message/1 above.
The code may be subject to various race conditions, but fortunately
objects are immutable. It also seems possible that the Redis stream
gets lost. Not sure when and how. For now, we restart if we get no
reply, but nore more than once per minute.