“UK Start-up Stornaway is creating tools and workflows that allow content creators to quickly create stories that take advantage of the existing branching narrative capabilities of YouTube, browsers and games engines such as Unity to create and distribute branching narratives. Its tools are also compatible with the open source Object Based Media tools provided by the BBC. Stornaway reports a very strong and growing level of interest in creating simple stories that can be used in education and commerce as well as entertainment.”
Ofcom Report on Object Based Media, September 2021
This week, the UK government’s media industry regulator Ofcom featured Stornaway.io in an important report – the first of its kind – about the emerging era of Object Based Media.
The report introduces industry readers to Object Based Media (OBM), with illustrations of state of the art content and production tools, notably highlighting Stornaway as a leader.
“The UK is also home to many innovative and successful start-ups and scaleups working in object-based media, Endemol Shine and Stornaway have led in branching narrative objects”
What is Object-Based Media?
OBM can be used in different ways, including the way the Stornaway.io lets you create branching interactive video experiences by joining up “Story Islands”.
Best to hear a definition from BBC R&D’s Michael Armstrong – who first came up with the ideas behind Object Based Media back in the 90s while developing digital playout systems at BBC Radio Bristol and developed them further when investigating the issue of TV speech audibility. Michael (oddly not credited or quoted in the report) has described OBM at a high level as:
“A way of making media content that can be assembled in different ways to meet differing requirements for users and their devices.”
BBC R&D, who have done great pioneering work on developing workflows for OBM, have a section on their site where they explain what objects are in Object-Based Media:
“The ‘objects’ refer to the different assets that are used to make a piece of content. These could be large objects: the audio and video used for a scene in a drama – or small objects, like an individual frame of video, a caption, or a signer.
By breaking down a piece of media into separate objects, attaching meaning to them, and describing how they can be rearranged, a programme can change to reflect the context of an individual viewer.”
BBC R&D subsequently worked on developing StoryFormer (and other tools in their Makerbox) as a tool for BBC producers to create OBM interactive tv, interactive videos for the web and interactive radio stories for the web and Alexa. StoryFormer shares a lot of similarities with Stornaway.io – they were developed independently around the same time – and we have been working on interoperability.
“YouTube, Netflix and many other app and browser-based platforms now support these branching narrative features and tools such as the BBC’s StoryFormer and UK start-up Stornaway are making it easier to create, export and play-back the content.”
What has Ofcom said about Object-Based Media in the new report?
The report starts with some illustrations of how OBM can be used in practice to make interactive tv, video and radio.
To do so, it breaks the objects down into three common types which it calls chunks, layers and transmedia:
Object type 1: Chunks
Chunks are media objects that make up a part of a programme – the scenes, sequences, sections of a story or package, which we at Stornaway more poetically call Story Islands. In the Stornaway world, these are often video files, but can also be images and soon audio and 360/VR video files.
The report highlights the work that Stornaway has done in creating authoring tools and workflows for Object-Based Media stories, and making these available to anyone. Stornaway lets you quickly create stories which link together Islands as Objects to create interactive films and branching story games. You can then distribute them to the web, YouTube, Unity and other tools, for business and education as well as entertainment. You can see some latest staff picks of a range of projects on our examples page.
Alongside Stornaway projects like Pins and Needles immersive theatre film Select A Quest, the report also gives examples of the work that Netflix, BBC and YouTube have done, creating Object-based branching video stories like the Emmy-winning Bandersnatch, Bear Grylls: You Vs Wild, Click 1000 and A Heist with Markiplier.
Object type 2: Layers
Layers are objects designed to be consumed together or switched with other objects – for example subtitles, commentaries, graphics, languages and accessibility features.
The Ofcom report gives the example of a football match with a video object and 3 audio objects:
“a viewer watching a football match might choose to hear commentary from a pundit who is a supporter of their team, instead of the standard broadcast or the other side. By producing three audio objects, one for each team and one neutral commentator alongside one video object, choice can be offered to the viewer. The viewer can interact by making explicit choices through user controls (e.g. selecting a particular commentary from the list of options on screen), or implicitly through data provided by the user or their device.
We’re already very familiar with these from our TV and VOD platforms – turning on and off subtitles and languages. IMF packages for distribution of shows to VOD platforms, and DCP packages for films already contain a lot of these features – but Object-Based Media takes this a step further to allow more flexibility, control and automation.
Perhaps the best example of this is Dr Lauren Ward‘s work (also oddly not credited in the report) on audibility at BBC R&D in collaboration with the hit BBC programme Casualty to make a special episode focusing on Jade, a series regular, who is deaf and wears hearing aids. Through scenes from Jade’s audio perspective, they introduced viewers to her unique way of experiencing the world – termed ‘Jade’s World’.
Instead of separating sounds into “speech”, “background” and “music” types of object layers, which would ignore the narrative importance of non-speech sounds, Lauren interviewed and surveyed content producers to develop a system which balanced narrative comprehension and creativity with the needs of hearing impaired listeners. In this system, sounds in a particular mix are ranked by the producer based on their ‘narrative importance’ (how crucial they are to telling the story).
A simple slider lets the audience choose between 4 levels of hierarchy. They tested with focus groups of hard of hearing listeners to try out the control. The results were positive and participants noted that the approach improved the intelligibility whilst still maintaining the ‘depth’ and ‘colour’ of the content.
Object type 3: Transmedia
As the report summarises:
“Transmedia objects orbit the content itself and are not directly part of the consumption process. They can be consumed separately to add to the overall experience.
Examples of these objects include bumpers, trailers, making-of documentaries, outtakes Wikipedia/IMDB entities, cast and crew interviews, location information, reviews, social media pages, blog posts etc. They can even extend to a related radio drama version, toys, merchandise, magazines and spin-offs. Sky’s Immersive VR app with augmented content is a good example, as is Amazon’s X-ray feature.
All of these objects exist today but are very rarely arranged, ordered, tagged and searched as a true object database.
It is however very feasible to create such a rich object-based database and metadata which allows this orbiting cloud of objects to be linked to the content in ways which benefit both consumer and content provider.”
It also gives the example of Netflix’s HyperPersonalised UI, which displays stills that Netflix believes is closest to your programme tastes, increasing the likelihood that you select that programme. Another tool in the VOD platforms’ battle against churn.
What is the report’s conclusion about the next steps for Object Based Media?
It’s a wide ranging report, covering technology, consumer and commercial trends, accessibility, compliance, competition and interoperability. If you’re involved professionally in the tech and creative future of tv, radio, cinema and media, it’s worth a read.
We were delighted that the report’s key recommendation is for the industry to come together to develop open standards and interoperability.
“In order for OBM to leverage greater efficiencies, interoperable standards are essential.”
But it also highlights the lack of leadership in the industry, in the UK or internationally.
“Where is the home of object standardisation? A common thread in the study is the need for the development of standards and tools that can be used across the industry, not merely home-grown separately by each broadcaster. There are several organizations working on these problems but none are dedicated to the topic.
DVB and HbbTV have standardised TA, subtitling and AD, SMPTE have standardised the mastering formats such as IMF, and the DPP in the UK and EBU have developed other useful production standards in this area. Other parts come from the internet standardization bodies and industry fora.
This causes a patchwork quilt of standards that pertain to OBM with no overall coordination.”
Stornaway’s view
We have based Stornaway.io on interoperability (including competitive interoperability) and the development of our json data structure in a way that lets interactive video projects be exchanged between platforms – authored in Stornaway and/or compatible tools, delivered to YouTube, VOD, gaming platforms via Unity, etc.
And it’s remarkable how much innovation in this area comes out of the UK – from BBC R&D and us, obviously, but also how many of the world’s most successful interactive film, tv and audio experiences are British: Bandersnatch, Bear Grylls, Sam Barlow’s Her Story, The Complex, Late Shift, The Inspection Chamber, Erica…
This country is well positioned to lead the world in software and standards.
We’re DPP Members because we have been impressed with the way that the DPP rallied the industry to develop cooperative standards for IMF and AS-11 delivery formats – our CEO Ru was involved with testing and rolling these out for production at the BBC from the earliest stages. (He also spent years dealing with the format chaos of the digital transition before that.)
But across the media and entertainment industry, CTOs and CIOs have spent the last few years focusing on moving their ‘media supply chains’ to the cloud and finally engaging with metadata and SaaS, so standardisation work has mostly had to be around these business needs.
As a result, many of them might still not recognise that this OFCOM report on OBM is a harbinger of the media and entertainment industry’s next great value transition: unlocking and alchemizing the relationships between all their decentralised and interconnected media objects.
Most broadcast tech people still use the terms ‘library’ and ‘archive’ to talk about their content, with connotations of dusty old silos and tapes on shelves. A video or audio programme is still thought of as a self-contained black box of finished content, even if it’s a digital asset with metadata. That’s a conceptual blocker we’ll inevitably move past, but we need to start working on standards for this new Object-based world before the tech gets unfixably silo’d by vendors, broadcasters and Silicon Valley.
So we’re really grateful for Ofcom’s recognition of Stornaway’s work opening up Object-Based Media production tools and workflows, and we hope that its call for co-ordination around open standards in OBM reaches some smart ears who want to get ahead of this to join the dots. If that’s you, thanks for reading this far – please get in touch!
About the Ofcom Report on Object Based Media
The report was written by Elfed Howells of Macroblock and David Jackson from the School of Digital Arts (SODA) at Manchester Metropolitan University, researching from February to April 2021, with input from 31 contributors. It was published on September 6 2021 and you can download it from Ofcom’s Object Based Media page – and here’s a direct link to the PDF.
Attend Ofcom’s Workshop on 13 October 2021
If you’re reading this soon after release, Ofcom will be hosting a stakeholders’ workshop at its Riverside House Office in London on the 13th of October 2021 to discuss the report, the issues faced by industry and the potential opportunity for an Interest Group to drive further development of the technology in the UK. Visit their website to register interest.