High-performance multi-directional file synchronization and replication
IBM® Aspera® Sync is purpose-built by Aspera for high-performance, scalable, multi-directional asynchronous file replication, and synchronization. Designed to overcome the performance and scalability short-comings of conventional synchronization tools, Aspera Sync can scale up and out for
Welcome to Aspera Sync, a software application that provides high-speed and highly scalable multi-directional, file-based replication and synchronization. Aspera Sync is designed to fill the performance gap of uni-directional file synchronization tools like sync, which are often slow for synchronizing large files and large sets of files over the WAN. Additionally, Aspera Sync extends the capability of uni-directional synchronization tools with full support for synchronization that is bi-directional and multi-directional. Unlike traditional synchronization tools, Aspera Sync offers the following key capabilities:
- Utilizes high-speed Aspera fasp transport for moving data at maximum speed over the WAN, while traditional synchronization tools are built on TCP. Aspera Sync transfers new data between remote hosts at full bandwidth capacity, regardless of round-trip delay and packet loss, and does not degrade in performance for large file sizes.
- Compares against a local snapshot, thereby avoiding the process of making a comparison against the remote file system over the WAN (which can be slow). Most traditional tools, such as sync, determine what to transfer by comparing the local and remote file system over the WAN.
- Recognizes file system changes (such as moves and renames) and propagates these changes to peers. Traditional tools treat these operations as deletion of old data and then recreate or retransfer the new data, which can lead to costly data copying over the WAN.
- Supports bi-directional and multi-directional synchronization topologies, where files are changing on multiple nodes. For a bi-directional synchronization, Aspera Sync runs with a bi-directional option. For a multi-directional synchronization, one session is run for each peer to remain synced. Any topology that has an acyclic graph topology between peers is supported.
- Uses file system notifications for change notification, when available.
- Monitors file contents and waits for files to be stable (no longer changing in md5sum) before transferring. The wait period is configurable and is designed to avoid transferring only partially complete files.
Aspera Sync is a command-line program called sync that, like rsync, uses an SSH connection to establish connectivity with its remote peers and is spawned as an SSH subsystem binary on the remote system. The program can be run one time or periodically (through a cron tab scheduled job) on file systems that do not provide asynchronous change notification, or in a continuous mode on file systems that do support asynchronous change notification. Aspera Sync is designed to process files and transfer new data in a continuous pipeline for maximum speed, even when running in scan-only mode (when no file system change notification is available).
