JKad - a Java implementation of the Kademlia protocol, designed for academic research
JKad is an implementation of the Kademlia protocol, which is used for building Distributed Hash Tables (DHTs) within BitTorrent and other file sharing networks. A DHT is a Peer-to-Peer (P2P) network that provides distributed lookup and storage of key/value pairs, similar to a hash table. There are multiple Kademlia-based but mutually incompatible DHTs. JKad is fully compatible with the DHT of the BitTorrent Mainline client, also known as BitTorrent Mainline DHT. With several millions of concurrently participating peers, the BitTorrent Mainline DHT today constitutes as one of the most widely used P2P networks.
JKad was completely written in Java and was designed with focus on academic research. JKad is meant to be included as a library into other software projects, but for testing purposes a simple command line client is included. Compared to other common Kademlia libraries, JKad tries to offer better support in the following aspects:
- Scalability: Other than typical Kademlia libraries primarily used for file sharing purposes, JKad was designed to be able to execute a high number of lookups per second over a longer time. Hence, JKad suits this use case especially well in respect of memory management as well as thread safety. During measurement studies performed at our research group, JKad has proven to be able to continuously cope with high load for weeks.
- Synchronous and asynchronous transactions: For easing adaptation, JKad offers to start any DHT transactions (such as DHT lookups, storage of key/value pairs, or Kademlia PINGs) both synchronously (as blocking function call) and asynchronously (by providing a callback function). This makes it easier to include JKad in other projects and prevents typical threading issues.
- Object-oriented logging: JKad comes with an integrated, event-based logging engine that enables the user to extend JKad by so called Loggers. Loggers can be used to access any JKad-internal information, as for instance the current state of the routing table, in case user-definable events occur. This way, it is possible to react to arbitrarily events and analyze the Kademlia protocol in detail.
- Extensibility: JKad was designed to be easily extensible. In particular, it is easily possible to add new transactions (e.g., alternative lookup implementations).
- Framework architecture: JKad was implemented as a framework that could be extended to support other Kademlia derivatives as well. However, right now only the BitTorrent Mainline DHT is supported.
During our studies, two more advanced features were integrated that extend the Kademlia protocol. In the following, we briefly present an overview about these features. A more detailed description will be made available after the publication of the corresponding paper.
- Optimization of the lookup algorithm: the lookup algorithm was optimized to allow for shorter lookup durations and lower overhead. If for example compared to the open source client Transmission (v. 2.12), the average lookup duration was reduced by a factor of more than 25 while the average number of packets sent per lookup was reduced by 15% at the same time.
- Self-Optimization of lookups at run-time: JKad is able to automatically choose and adjust parameters used by a lookup algorithm at run-time to optimally adapt to the current state of the DHT. For doing so, JKad simulates DHT lookups at run-time based on a just-in-time collected model. The approach is not limited to the default lookup implementation but works with any alternative implementation, too.
JKad's source code is available on request under GPL (version 3).