Since the first versions of Ditto, devices always made multiple connections to other peers. For example, two iOS devices will try to connect simultaneously over WiFi, Apple Wireless Direct Link (AWDL), and Bluetooth Low Energy (BLE). We make multiple connections because each transport has different characteristics such as throughput and distance. For example, Bluetooth Low Energy works over long distances but has little bandwidth. AWDL has much more bandwidth but the devices need to be close together.

In the example below we see two iPhones syncing over AWDL and BLE. As one device gets further from the other, the AWDL connection will degrade and disconnect while the BLE connection sustains longer distances.

Pre-Version 1.0.0

Before version 1.0.0, Ditto created a unique replication session for every transport. This is the software component which tracks queries and data changes and ensures that every Ditto device stays in sync. These separate replication pathways can appear or disappear as connections come and go, without disrupting sibling sessions.

When there is new data to sync, a session packs that data into an update file. With multiple concurrent sessions, all of them would then race against each other to transmit that update file as quickly as possible, regardless of duplication between transports. Transactional locking on the internal database made sure that only one session at a time could modify the update file, which prevented race conditions. Since any session is able to maintain the update file, any individual session can fail and replication will always continue, providing a high level of reliability.

In this benchmark we see the consequences of each session sending data eagerly over every transport. In an attempt to send a document with about 2 megabytes of data, all five connected modes of transport aggressively sent data as fast as they could. We can see that the highly efficient AWDL transports could send all bytes first, and the remote peer quickly notified the slower connections that they could stop. However, the slower transports had already sent duplicate bytes. In the end, the peer had sent 5.4 megabytes even though the document size was about 2 megabytes. This was 2.6 times larger than the initial payload. Furthermore, this fully occupied the BLE radio, consuming bandwidth that could have been better used by a Bluetooth-only peer.

Introducing the Multiplexer in version 1.0.0

In version 1.0.0, we’ve introduced a completely new system for creating sync sessions between peers we call the multiplexer. Our first order of business was to reduce the duplication of sessions to the same peer over multiple transport types. We’ve introduced the concept of a virtual connection between two peers. No matter how many transport connections are active, there will only be one virtual connection and only one session. Now incoming data is buffered and intermediated from the transport layer to a single virtual connection.

This new architecture allows each virtual connection to intelligently send data over multiple physical transports with fine-grained control. For example, the multiplexer can switch active transports on the fly without unnecessary duplication.

In this example:

1. The multiplexer on the left device deemed that TCP (WiFi) was the best transport to start sending data.
2. Suddenly, the infastructure WiFi goes out, and the multiplexer switches to AWDL.
3. As the device moves away from its peer, AWDL is lost and the multiplexer switches to BLE.
4. The devices move closer together and the multiplexer finishes the rest of the transmission over AWDL.

Now the total bytes sent is equal to the size of the update file.

Going Forward

The introduction of the multiplexer is a gigantic step forward for Ditto's networking capabilities. Today, it focuses on using one transport at a time but this new foundation allows us to build even more powerful, dynamic and flexible replication techniques such as using multiple transports at a time over unreliable connections, streaming use cases, and decentralized data sync techniques reminiscent of BitTorrent.