Local Synchronization Feature Added to Increase Speed of Upload and Download

Large file collaboration often hits a practical limit when every transfer must traverse the internet, even when team members share the same local network. To address this, we have implemented a local synchronization capability that allows clients to discover and transfer data directly within the LAN.

LightUpOn.Cloud sync internals — LAN peer discovery with differential block transfer

How Local Sync Works

The feature leverages the Kademlia peer-to-peer protocol for distributed hash table functionality. Upon starting the application, the client sends UDP multicast messages to bootstrap participation in a local Kad network. Once connected, nodes exchange information about available files within shared groups. This allows clients on the same network to source data from each other rather than pulling everything from a central server over the internet.

In practice, this means large files can transfer at local network speeds — often orders of magnitude faster than internet-constrained paths.

Differential Synchronization

Complementing local discovery, we introduced differential synchronization. Instead of uploading entire files when only small portions have changed, the client identifies and transmits only the missing or modified blocks. This reduces bandwidth usage and speeds up both upload and download operations, particularly beneficial for large binary files that undergo frequent minor edits.

Dotted Version Vector conflict detection across three concurrent clients

Conflict Detection with Dotted Version Vectors

Reliable synchronization in distributed environments requires robust conflict detection. LightUpOn.Cloud uses Dotted Version Vectors (DVV), a sophisticated causality tracking mechanism based on academic research. Unlike traditional version vectors that can produce false conflicts, DVV provides precise tracking of causal relationships between updates, allowing the system to correctly identify concurrent modifications while efficiently resolving those that are causally related.

This approach maintains strong consistency guarantees without sacrificing the performance benefits of local and differential transfers.

Practical Impact

For teams working with large design files, video assets, or datasets, these improvements translate into noticeably faster workflows. Files that previously took minutes to sync across the office can now complete in seconds when devices are on the same LAN. Differential updates further reduce waiting time on incremental changes, while the underlying DVV mechanism ensures conflicts are handled accurately even under concurrent access.

6 July, 2021