KaZaA P2P FastTrack File Formats - 2003-10-12

Experiment with the KazaaNet v1.1 Supernode Hopping Tool. Try to make your KaZaA connect to other Continents.

Active Downloads (.DAT), Local Meta Database (.DBB), SuperNode List

KaZaA 2.[0,1] FastTrack v3, Whadde Hek...

A. FastTrack download*.DAT Appendix Format

B. FastTrack data*.DBB (Local Shared Objects Meta Database) File Format

C. Some known FastTrack Kazaa Meta Tags

D. KazaaNet Local SuperNode List Cache

Kazaa FastTrack files are organised as Sequences of Items. Items may be simple types or subsequences like Lists, tables, strings. Some items have a fixed length. Some items are prefixed with a count. Some items are terminated by a special constant value (asciiz/ansiz strings, chunklist).

All Numbers are serialised in Intel little endian byte order (least significant byte first). All Texts are strings of single byte characters (from the current user's MS Windows 'ANSI' codepage).

A. KaZaA FastTrack download*.DAT Appendix Format

A .DAT file is a staging file with a filename following a locally unique pattern like 'download103988081426526625.dat', that is produced by the FastTrack Download Manager in the client application's download directory. It extends in size as more data retrieval ranges at the end of the file are assigned to one or more Download Sources. Received data chunks (from the responses sent by peer nodes on HTTP GET range requests) are written directly into the .DAT file at the corresponding file offset. While the download is busy, a FastTrack Appendix with download job state information is present at the end of the file and contineously maintained. When all chunks of the file are received and written completely, that Appendix is removed and the .DAT file is renamed to the final filename that was selected for download in the application's search list.

Table 1: Format of FastTrack .DAT Appendix sequence:

B. KaZaA FastTrack data*.DBB (Local Shared Objects Meta Database) File Format

The FastTrack application's Local Content Database files contain meta information on the files that are shared/published/uploaded/advertised/contributed by the local machine on the FastTrack P2P network. These index files are maintained by Kazaa's LocalContentManager. An active Kazaa process permanently monitors the local folders that are shared, any File Add, Change and Delete event is reflected in the meta database files.

Meta info items are filename, foldername, filesize, and the filetime. In addition there are a variable number of file tags that describe the contents of the file. FastTrack Kazaa tags include mappings for most .MP3, .DOC, .PDF, .AVI and other popular Media file content properties. This file meta info is used in the FastTrack distributed network File Search protocol. An essential tag is the FastTrack ContentHash. The value represents a 'fingerprint' of the content, worldwide unique for each specific dataset. In Kazaa 2.[0,1] it is the (only) tag used in the Download URL (a HTTP GET Download request like http://peerIP:peerPort/.hash=ab56..83fe).

When a FastTrack client node (re)connects to (a selected and available supernode in) the virtual P2P network, the client node sends his database files to the supernode. This index replication probably isn't incremental. The supernode will make the list of shared files available for searching by other supernodes. This way the meta gets published in the distributed FastTrack file Search Cache.

KaZaA 2.[0,1] will recreate the meta database files at each new start, should they not exist already/anymore in the configured Database Directory. The pathname of that folder is kept in the Windows Registry at HKLM\Software\KaZaA\LocalContent\DatabaseDir.

Executing URL http://localhost:<KazaaPort>/ makes the Kazaa 2.[0,1] application generate a HTML-page with a table of contents showing the entries currently present in the LocalContent Database files [replace <KazaaPort> with the configured TCP port for the Kazaa download service, the default value is 1214]. This URL works with any HTTP Browser that runs on the same machine as the one Kazaa is running on. The Kazaa 2.[0,1] built-in HTTP-server responds with a "404 Not Found" HTTP-error when such a HTTP GET root request originates from a remote machine. Earlier versions of KaZaA reportedly do respond to such remote requests.

.DBB files contain arrays of fixed length slots.

• File .\data256.dbb contains slots for file meta entries that fit in ((2*4)+256) bytes.
• File .\data1024.dbb contains slots for file meta entries that fit in ((2*4)+1024) bytes.
• File .\data2048.dbb contains slots for file meta entries that fit in ((2*4)+2048) bytes.

Table 2: Format of FastTrack .DBB Meta Database file slot:

C. Some known Kazaa FastTrack Meta Tags

Table 3: Some Kazaa FastTrack Meta Tag Values:

[To Do:] Next tag candidates might still need to be assigned here:
size, signer, filename, Developer, Date, Author
truecolor?
audiochannels?
File format, Number of pages, Compilation, Description
Bandwidth?

D. KazaaNet Local SuperNode List Cache

FastTrack nodes provide several services. Nodes with stable bandwidths and enough upstream capacity, can - when allowed by the user - become supernodes. They then provide File Search and Network Management services to less-equipped client nodes. Client nodes connect to the FastTrack network with a single TCP-connection to a single supernode. When a FastTrack node needs to reconnect to this virtual P2P network, the availability and reachability of subsets of supernodes from a locally cached list of supernodes - typically with 200 entries - is probed. Reachable supernodes respond with updates of this Supernode List.

For probing, initially FastTrack protocol packets over UDP/IP are used. When no one responded to such an UDP-probe, connections over TCP/IP are tried next.

Candidate FastTrack "connection point" supernodes are selected from a dynamic SuperNode list that survives application terminations. The local supernode list cache is persisted in the Windows Local Machine Registry at HKLM\Software\KaZaA\ConnectionInfo\KazaaNet. Each time a new connection to the P2P network is (re)established - and sometimes (mostly after a Search Request) in between - some supernode entries in the list are dynamically replaced to reflect the more recent state. Supernodes send partial updates to clients and exchange updates between themselves. A (former) supernode may respond to a network connection request of a client with the advise to try to TCP-connect to another supernode ("redirect" the client connection request).

I did not see the Kazaa 2.[0,1] application in client mode try to connect to a supernode using any hardcoded IP-address. At each client-mode application start, the attempt to connect to the virtual P2P network fails when all probes to all supernodes cached in the registered supernode list fail. The user interface does not show such an unable to connect state though, it keeps saying "Connecting...".
Starting up in (forced full) supernode mode needs more research however. Joltid Ltd. - supplier of the FastTrack protocol stack code - (claims to) operate(s) so-called kaasupernode-s at the top level of the 3-layer network. These top nodes effectively control the network of peers that are allowed to be SuperNodes. The Kazaa application also seems to contain code to support an Exclusion list.

Normal FastTrack nodes (clients) tend to be only known in a local "small world" (within their ISP-network, country, continent). Their FastTrack network "connection point" is a nearest supernode, most recently registered by the network, having the lowest workload percentage. Their FastTrack world gets smaller each time they (re)connect and search because of supernodelist replication based on "distance" timings. The distances appear to get occasionally measured using "firewall testing" protocol handshakes, that are performed on the request of (other) supernodes. Search Requests might be completely satifistied by the near supernodes.

This works great to limit FastTrack traffic on the global Internet, but your Search Results tend to be localised too. Only a small percentage of supernodes in other small worlds seems to get (randomly ?) injected in the supernode list by the automated FastTrack P2P network management. Joltid Ltd sells PeerCache server software for ISP-s with which they are able to streamline the FastTrack-traffic in their subnets. The KaZaA 2.[0,1] application supports such CacheHost-s with a custom Cache Discovery protocol (/.p2pcache).

Any initial standard "traceroute" ICMP pinging is only used to determine up- and downstream bandwidths and probably to optimize distance measurements too. The bandwith measurement traceroutes are always directed to four - pretty randomly chosen, non-FastTrack - IP-addresses in four hardcoded IP-subnets, likely assumed to be in four different continents of the global Internet:

With the traceroute pings, the "common hop" from the local FastTrack machine to the world is determined. This way the node's relative location is detected and the selection of supernode probe candidates from the locally cached supernodelist can be optimised against a distance parameter too.

Brilliant Digital Altnet's TopSearch add-on does contain a hardcoded IP-address (media.altnet.com). On startup, it makes the Kazaa 2.[0,1] client retrieve a TopSearchIndex file with this URL:

http://217.116.227.249:80/topsearch/tsi2.cab

This MS Authenticode signed CABinet file (with encrypted contents) likely has nothing to do with FastTrack network management, and probably only contains predefined Search Hits to payed content.

The KazaaNet list is a sequence of fixed length SuperNode items. The list is prefixed with a single byte. The number of items is variable. In a Kazaa 2.[0,1] client typically 200 supernodes are and/or will get registered. If the list is empty or no supernode was reachable (no one responded to an UDP-probe), the application's TCP-connection attempt to the FastTrack P2P network (any single supernode) will fail.

Table 4: KazaaNet Local Supernode List Cache [Persisted in Windows Registry]:

I could not have created this page without the hacking performed by: Random Nut, idleloop, vlaibb, Sanderson, mospum, Scott J., Hasnain S., and everyone else I forgot to mention here. Thank you all. Keep sharing your information.