Workshop Sessions

This year’s PCS program includes four workshops.

Delivering Media for Billions

Facebook, through its variety of surfaces including Facebook, Instagram, Messenger, and WhatsApp, as well as hardware devices such as Portal and Oculus, processes and delivers media to billions of users around the world everyday. Making sure every one of those videos is delivered at the best quality with as little buffering as possible, means optimizing not only when and how our video codecs compress and decompress videos for viewing, but also which codecs are used for which videos. In addition, it is increasingly challenging to measure and optimize the experiences when most users interact with media on mobile devices over highly variable network conditions. In our session, we will show how Facebbok deals with its high demand for encoding high-quality video content by combining a benefit-cost model with a machine learning (ML) model that lets us prioritize advanced encoding for highly watched videos, how Facebook leverages visual quality metrics to optimize mobile playback experiences, and discuss the operational (audio, video, network, performance, etc.) issues related to real-time calling on hardware devices, focusing in our experience with Facebook Portal.

Schedule: View the Agenda

Speakers

Niklas Enbom:  Niklas is an engineering manager at Facebook Portal, leading the team responsible for call quality and performance. Niklas has worked with realtime communications over IP for over 20 years. At Global IP Solutions he built embedded software that powered the consumer VoIP boom with customers like Skype and Google Talk. In 2011 he joined Google as one of the founders of the WebRTC platform and lead the team that added WebRTC support for Chrome Web browser. Niklas has M.S. degree in Computer Science from Luleå Technical University.

Chema Gonzalez: Chema is a software engineer at Facebook Portal, working as the tech lead for the team responsible for call quality and performance from pre-launch. Chema has a PhD in Computer Science from UC Berkeley.

Taein Kim:  Taein Kim is an engineer at Facebook video encoding system team. He specializes in encoding benefit and cost model which maximizes overall users’ video experience by efficiently allocating compute resource to most important encoding jobs. Prior to joining Facebook, he spent 2 years at Amazon AWS Elastic MapReduce helping the service become GDPR compliant and making the system more reliable, 10 years at Microsoft working on improving developer experiences in Visual Studio IDE, internal engineering system and Visual Studio Team Services. He received his B.S. in Computer Science from University of Michigan, Ann Arbor in 2008.

Denise Noyes:  Denise is an engineer at Facebook leading improvements in video playback Quality of Experience and playback performance. She’s spent the last seven years at Facebook working on video playback in Facebook’s family of mobile apps spanning both live and video on demand on iOS and Android. Prior to joining the video team, she built audio calling for the Messenger app. Before Facebook, she spent seven years at Amazon working on everything from delivery estimates to online groceries.

Nick Wu: Nick is a software engineer at Facebook working on building key video infrastructure pieces to serve the highest quality videos at scale. He received his Ph.D. in Electrical Engineering from University of Southern California in 2010. After that, he worked on video content analysis and its application on indexing, ranking, and recommendations in a video search engine. He then spent several years in the cloud gaming space, working on interactive video streaming, graphics, and virtualization. Prior to joining Facebook, he was at Google working on Google App Streaming and Android virtual device. His research interests lie in video coding, adaptive streaming, and video content analysis.

Challenges in the Video Compression World

As one of the world leaders in video streaming on-demand, Netflix provides entertainment and joy to millions of members worldwide from an extensive catalog of content from soapy romance shows to action-packed movies. The encoding team at Netflix is responsible for the processing pipeline that generates all of the streams served to our members. Our goal is to provide reliable and engaging video and audio quality, irrespective of the device and network conditions at the receiving end. With this in mind, our team has developed several innovative and industry-leading encoding strategies throughout the years. However, there is still so much more to be done! In this workshop, we showcase some of the recent challenges from an industry point-of-view and highlight recent internal developments in video compression, encoding, and video quality assessment. Moreover, as members of the Alliance for Open Media (AOM), we are committed to the success of open and royalty-free video codecs. This session also provides an update on the AV1 rollout within our encoding pipeline and the development of compression technologies beyond AV1.

Schedule: View the Agenda

Speakers

  • Mariana Afonso (Senior Research Scientist – Video Encoding)
  • Kyle Swanson (Senior Software Engineer – Video Algorithms)

Multimedia Technologies and Solutions in Tencent

Tencent Media Lab is committed to research and development of emerging multimedia technologies that can be used for both existing and envisaged applications. Some of the Lab’s notable work focuses on advanced compression of audio, video and volumetric media content; media network transmission and real-time communication; processing, analysis, understanding and quality assessment of multimedia content based on signal processing and deep learning; and system design of end-to-end solutions for immersive media (XR, volumetric, and holographic). In addition, the Lab contributes to the development of international technology standards in fields such as multimedia data compression, network transmission protocols, multimedia systems, and open-source platforms. In this workshop, researchers from Tencent Media Lab will present their multimedia technologies and solutions which have been adopted in various international standards and used by hundreds of millions of users worldwide.

Schedule: View the Agenda

Speakers

  • Dr. Shan Liu, Distinguished Scientist and General Manager of Tencent Media Lab
  • Dr. Stephan Wenger, Principal Researcher and Sr. Director of Standards
  • Dr. Songnan Li, Principal Researcher and Director of Video Technology
  • Dr. Bing Jian, Principal Researcher and Tech Lead of Immersive Media

Versatile Video Coding – Open Optimized Implementations

The tutorial discusses an open and freely available encoder implementation VVenC of the latest video coding standard VVC (Versatile Video Coding) jointly developed by ITU-T and ISO/IEC. VVC has been designed to achieve significantly improved compression capability compared to previous standards such as HEVC, and at the same time to be highly versatile for effective use in a broadened range of applications.

Some key application areas for the use of VVC particularly include ultra-high-definition video (e.g. 4K or 8K resolution), video with a high dynamic range and wide color gamut (e.g., with transfer characteristics specified in Rec. ITU-R BT.2100), and video for immersive media applications such as 360° omnidirectional video, in addition to the applications that have commonly been addressed by prior video coding standards. Important design criteria for VVC have been low computational complexity on the decoder side and friendliness for parallelization on various algorithmic levels. VVC has been finalized in July 2020 and in September 2020. Fraunhofer HHI has made an optimized VVC software encoder (VVenC) and a VVC software decoder (VVdeC) implementations publicly available on GitHub.

The tutorial details the open encoder implementation VVenC with a specific focus on the challenges and opportunities in implementing the myriad of new coding tools. This includes algorithmic optimizations for specific coding tools such as block partitioning, motion estimation as well as implementation specific optimization such as SIMD vectorization and parallelization approaches. In addition to runtime optimizations, subjective quality measures and methods to increase the perceived quality by local QP adaptation are presented and discussed.

Workshop Agenda

  • The Versatile Video Coding Standard
    • Introduction to VVC and short overview of tools
    • VVC reference software (VTM) – general coding efficiency and runtime performance
    • VVenC – general coding efficiency and runtime performance
  • VVenC – Implementation and Algorithm Optimization
    • VVC encoding complexity analysis
    • Preset comparison and discussion
    • Quad-tree plus binary- and ternary-tree partitioning complexity
    • Tool complexity and configuration space analysis
  • VVenC – Subjective Quality Optimizations
    • Perceptually motivated block-wise distortion measures
    • Perceptual QP adaptation based on PSNR and XPSNR
    • Visual quality assessment (VQA) of image or video codecs via (MS-)SSIM, VMAF, and XPSNR
  • Summary and outlook (5min)

Schedule: View the Agenda

Speakers

  • Benjamin Bross – Group Head (Video Coding Solutions)
  • Dr.-Ing. Christian Helmrich – Research Associate
  • Adam Wieckowski – Research Assistant