In this paper, we introduce an on-going effort within the WIDE project to collect a set of free tools to build a traffic data repository containing detailed information of our backbone traffic. The WIDE project makes the resulting data sets publicly accessible so that this project is not only on freely-redistributable software but also on freely-redistributable traffic data sets.
The WIDE project is a research consortium in Japan established in 1987. The members of the project include network researchers, engineers and students of universities, industries and government. The focus of the project is empirical study on a live large-scale internet. Thus, WIDE runs its own internet testbed carrying both commodity traffic and research experiments. WIDE is also responsible for various Internet operations including the M-root name server, NSPIXP(Network Service Provider Internet eXchange Point), AI3(Asian Internet Interconnection Initiatives), and 6Bone in Japan.
The goals of our traffic repository are to promote traffic analysis research as well as to promote development of tools. Traffic characteristics in a backbone network are considerably different from those in a local area network but few people have access to traffic traces from backbone networks. Obtaining details of backbone traffic is getting harder as more backbone networks are shifting to commercial ISPs, which motivate us to build a traffic repository [KMKA99]. Traffic traces from the 6Bone is also made available to promote development of IPv6-ready tools.
Traffic traces are collected at several points within the WIDE backbone. Traces are in the tcpdump raw format so that all header information is available and can be used for detailed analysis.
We use commodity hardware and the existing freely-available tools for building our traffic repository so that it has nothing technically fancy. Our focus is rather continuity in making the latest traces available. At this writing, daily traces at one sample point have added up to the record of more than a year.