Janet end-to-end performance initiative

Background

There is an increasing desire within Janet's research and education communities to use demanding networking applications such as real-time media or data streaming, low latency haptic applications, or tools to perform bulk transfers of research data in the order of terabytes or more.

The Janet end-to-end performance initiative sought to explore the varying requirements from our members and communities on network performance, with a view to establishing, sharing and promoting best practices.

What we did

The project engaged with our members to identify existing and emerging communities that ran or were looking to run data-intensive or real-time applications, including those that may have been unaware of the capabilities of the Janet Network.

We organised a number of events and workshops, undertook member site visits, and attended a variety of community meetings, from which an initial focus on support for large scale data transfers was prioritised. We engaged with over 100 sites, projects and other initiatives during the course of the project.

The resulting discussions explored existing approaches, tools and best practices, in particular ESnet's "Science DMZ" model, which describes an approach for differentiated treatment of regular campus and science traffic, in so doing delivering a 'friction-free' path for large-scale research data transfers.

The Science DMZ model makes recommendations for campus network architectures, tuning of data transfer nodes, careful section of data transfer tools, efficient application of security policies, and persistent monitoring of network characteristics.

The resulting E2EPI project outputs included:

• A summary Jisc advice and guidance page on how a site can best support large-scale data transfers on and beyond Janet, which includes Science DMZ considerations
• An example case study carried out at the University of Southampton to demonstrate how the network could be used to carry several TB of data that was previously moved by physical disk
• The establishment of a new Jisc Research Network Engineering Community to allow members to discuss their requirements and share best practice solutions
• Deployment of network test facilities in Slough (up to 10Gbit/s) and London (up to 100Gbit/s) to allow members to test their network and application performance

This project's work continues through the above activities for the benefit of our members.

Why this matters

There is an ever-growing need for efficient large-scale data movement, whether at the point of data capture, for remote processing, sharing, visualisation, correlation or archiving and backup. The data network enabling these transfers needs to meet the requirements of the users, researchers and communities.

Researchers are often unaware of best practices in data movement, and can frequently have a bad first experience by trying sub-optimal tools and then resort to using physical media which can significantly slow down and hamper their workflows.

For the best outcomes it is important that the researchers and those providing the research infrastructure are tightly coupled. This is made easier when best practices are widely agreed and implemented and their availability can then be disseminated appropriately.

How this benefits you

• Advice and guidance on establishing a network infrastructure for more effective collaboration and optimised workflows
• An established community where you can discuss your requirements and potential solutions with peers on Janet
• Regular research networking events with topical subjects presented and discussed
• Access to network test facilities on the Janet backbone against which you can benchmark network and application performance
• Support for deploying and using perfSONAR to obtain, store and visualise persistent network characteristic measurements, from which you can better understand and troubleshoot performance problems
• News and information on new and emerging technologies that may enhance your ability to support demanding network applications
• Access to assistance from the Janet Network performance team for any issues, whatever the network capacity involved