In planning for its coverage of the FIFA Women’s World Cup Canada 2015™ event originating in Canada, FOX Sports implemented a workflow system that would provide a very high level of immediacy and flexibility while at the same time trim production costs by utilizing existing post-production facilities located more than 1,000 miles from the sports venue. The system also had to accommodate multiple codecs and resolutions, and operate via Internet connectivity that had a maximum of one Gbps bandwidth. In addition, the Canadian portion of the system had to be implemented and completely operational in less than two weeks prior to the start of the games. Despite these challenges, the overall coverage aired with very few hitches, affording consumers an unprecedented viewing experience and keeping production costs to a minimum. Key to this successful deployment was the development of a flexible ecosystem for transparently moving content between the distant facilities and reliable storage systems.
The month-long Women’s World Cup marked Canada’s first hosting of the event and provided global viewing audiences with the opportunity to closely follow the action both on and off the playing field via high-definition television. FOX Sports was committed to providing unprecedented close-up coverage but also wished to control production costs. The network wanted to use existing post-production operations located in California and eliminate the need to construct a large and elaborate facility at the venue and send a large contingent of operations personnel on site for an extended period of time. Instead, FOX Sports elected to transport high-resolution mezzanine content for storage at its “Pico” facility in Los Angeles and also to a separate “overflow” facility, “JKam,” that was located in Burbank. Producers at these facilities could view content being sent to them in real time and create clip pointers for assembling game highlight packages. In an additional effort to contain costs, this “streamed” content was sent in proxy resolution, with full resolution recording taking place simultaneously at the event venue in Vancouver, British Columbia. As soon as the proxy-resolution packages were assembled, the pointers created could be used to access the selected portions of the high-definition recording; this would then be transported to the California post sites for assembly into the desired packages. Once these highlights packages were assembled, they could be transported back to Vancouver for integration into the live coverage.
Pronology was tasked with developing the ecosystem that made all this happen in such a way that these operations were completely transparent to production personnel, even though they were separated by more than 1,000 miles. In addition to the physical separation, a number of other challenges had to be overcome to enable such a workflow. These included an Internet connection between the Canadian and U.S. sites that was limited to one Gbps; access to the IBC (International Broadcast Center) in Vancouver for setting up and testing the remote equipment package, restricted to two weeks prior to the start of the games; the requirement for accommodating and distributing content being generated by a multiplicity of codecs in a variety of resolutions across several sites; and the enormous task of interfacing and connecting disparate systems and equipment from a number of suppliers.
Multiple Recording Systems Involved
Avid ISIS shared storage systems were used for high-res recoding at both the Vancouver and Burbank facilities; however, the Vancouver ISIS was really the core component of the FOX Sports World Cup highlights creation operation, as it served as the ingest point for all content. Due to the multitude of codecs and resolutions (RAW, mezzanine, 4K 1080, etc. that had to be accommodated, a Pronology transcode engine was operated in tandem with the ISIS to achieve a ”common denominator” prior to ingest. Pronology’s multi-resolution mRes™ encoding system was also deployed at the IBC for creating and storing low-resolution proxy versions of content from multiple switcher feeds originating from the FIFA game site control rooms. FIFA switching operations yielded three feeds for each of the four games per day: a clean, dirty and itinerant video feed. Also part of the mix was an EVS playout server located off-site from the main Vancouver facility that was used to cache and play out the highlights packages assembled in Los Angeles and Burbank. Adding to the overall complexity was a DMZ storage repository used at FOX’s California post facilities for filtering all out-of-house content, including that from Vancouver, before it could be placed on shared storage there.
The step-by-step workflow developed for the FOX Sports highlights package coverage is described as follows: (1) Content arriving from the field production control room recorded via Pronology’s mRes™, independent “ENG-type” shooters’ camcorder memory cards, and the World Cup’s FIFA MAX clip service. All content was ingested and transcoded, if needed, into the Vancouver ISIS system, with Pronology’s content management system indexing this material. Pronology’s mRes™ simultaneously created mezzanine and low-res proxy versions for viewing at the California ‘Pico’ and ‘JKam’ post facilities. (2) The highlights package producers screened these low-res proxies to select clips and using the Pronology storyboard tool, created sequences that were then sent to Adobe® Premiere® Pro CC. (3) The sequences were then available in Premiere via the embedded Pronology panel. From the storyboard sequences in Pronology a “subclip” delivery command was executed for plucking the corresponding high-res frames from the Vancouver ISIS and delivering them to the requesting California post facility. Transfer of this full-bandwidth content was facilitated via WAN acceleration, provided by FileCatalyst. By using this acceleration technology, it was usually possible to transfer small amounts (typically three minutes or less) of high-res content faster than real time, even with the one Gbps data pipe limitation. (4) As soon as the California facilities assembled the high-resolution highlights packages they were transferred to Vancouver where they were ingested into the ISIS system. The highlights packages were then transferred as files to the off-site EVS server where they were made available for use by the FOX field production control room.
Location Intelligence
Due to number of facilities involved in the highlights creation process, their wide geographic separation, and the very finite connectivity bandwidth available, this workflow would likely not have been possible without creating and adding some form of metadata to the content to ensure its prompt delivery at the intended destination (Pico, JKam, or Vancouver ISIS). Such location awareness, or intelligence, greatly facilitated operations and directly resulted from the Pronology ecosystem’s ability to select and migrate only the exact content required for production at various sites (Pico/World Cup, JKam World Cup, etc.). Because of this, any manual intervention or communication between production personnel was kept to an absolute minimum, thus speeding up the process and helping FOX Sports achieve a sense of unity across the widely separated operations.
Lessons Learned
Despite the physical separation of facilities and bandwidth limitations, the highlights creation process was termed a complete success by the network. Viewers of the World Cup event were completely unaware of the fact that portions of the programming were making a 2,000 mile cross-border round-trip, as content flowed smoothly and transparently. The Vancouver ISIS ingest and storage system is worthy of special mention, as it served as the core component of the whole operation and performed flawlessly throughout the event.
Initially there was a bit of an operator “learning curve,” as personnel at the California post facilities were used to working with media flows virtually unrestricted by system bandwidth and having entire assets available when the editing process began. However, the operators adapted very quickly to this new environment and modified their workflows accordingly, realizing that they could work quicker and more efficiently by pulling smaller pre-selected portions of content than waiting for lengthy complete game sequences to be transferred.
The operational efficiency and reduction in production costs sought by the network were achieved, and disruptions to normal operations were kept to a minimum as there was no need to temporarily relocate a sizable number of personnel to a remote location. It is anticipated that the success of this project will serve as a model for future special event coverage.