Scientific communities and projects world wide, such as the Sloan Digital Sky Survey (SDSS), are gathering large, multi-TeraByte datasets that are beincoming increasingly difficult to distribute and manage. Necho combines Peer-to-Peer (P2P), volunteer computing and social networks to provide a way to distribute, contribute to and manage large datasets.
Current transfer methods for large datasets, typically involve point-to-point transfers using standard protocols, or the postage of physical disks. This transfer process is normally repeated for each party or institute interested in a particular dataset. P2P technologies, such as BitTorrent, have been emerging in recent years and provide a means where multiple parties can help each other download a copy of the same data. Volunteer computing, such as SETI@Home, is another technology, in which unused resources are contributed to a task for a particular common purpose. Social networks, such as Virtual Research Environments (VREs) and Facebook, are helping people collaborate on tasks across the internet.
The goal of this project is to combine P2P, volunteer computing and social networks to provide a way to distribute, contribute to and manage very large datasets. P2P technologies provide a means to transfer the data across a network. Peers within the network contribute resources, such as storage and bandwidth to facilitate the data transfer. Portal technologies present a standard interface for communities to collaborate with the management of these datasets.
Our aim is to develop a system and make it available for other projects to use in the 'real world'. Necho's design is unique in combining the social aspect of a large dataset transfer with the workings of a P2P network, along with our Web browser-based mini-peers. It is hoped that the Portal-based P2P network will find itself a useful contribution to projects with large and difficult to manage datasets, such as SDSS.
Valid XHTML | Copyright | Last Modified: 19/Sep/2008 |