Towards Future 3D Media Internet
Theodore Zahariadis1, Petros Daras2, Isidro Laso-Ballesteros3*
1
Synelixis, Chalkida, Greece; CERTH/ITI, Thessalonica, Greece; European Commission, Brussels, Belgium
E-mail: 1zahariad@[Link], 2daras@[Link], [Link]@[Link]
Abstract: The Internet is incontrovertibly a great success that has changed our social and economic world. However, Internet was designed for purposes that bear little resemblance to today's usage scenarios and related traffic patterns. In the longer term, the exponential increase of the user generated multimedia content and the number of mobile users will raise many new challenges. Based on work from the Future Internet Assembly and the Networked Media Task Force, we present some of the major issues and barriers towards the Future 3D Media Internet. Keywords: Future 3D Media Internet, Content aware networks, 3D networks, visual sensor networks overlays are expected to generate new models of interaction and cooperation, and be able to support new innovative applications on the move, like virtual collaboration environments, personalised services/ media, virtual sport groups, on-line gaming, edutainment. In this context, the interaction with content combined with interactive/multimedia search capabilities across distributed repositories, opportunistic P2P networks and the dynamic adaptation to the characteristics of diverse mobile terminals are expected to contribute towards such a vision. On the other hand, advances in scalable video coding and 3D video processing, dynamically adapted to the network conditions will give rise to innovative applications such as massive multiplayer mobile games, digital cinema and in virtual worlds, placing new types of traffic demands and constraints on mobile network architectures. Based on work that has taken place in the Future Internet Assembly and the Networked Media Task Force, in this paper, we present some issues that according to our opinion are the major challenges and barriers towards the Future 3D Media Internet.
INTRODUCTION
The Internet is incontrovertibly a great success that has changed our social and economic world. Today, over one billion of users access the Internet on regular basis, more than 100 million users have downloaded at least one (multi)media file and over 47 millions of them do so regularly, searching in more than 160 Exabytes 1 of content [1]. In the near future these numbers are expected to exponentially rise. It is expected that the Internet content will be increased by at least a factor of 6, rising to more than 990 Exabytes before 2012, fuelled mainly by the users themselves. Moreover, it is envisaged that in a near- to mid-term future, the Internet will provide the means to share and distribute (new) multimedia content and services with superior quality and striking flexibility, in a trusted and personalized way, improving citizens quality of life, working conditions, edutainment and safety. However, Internet was designed for purposes that bear little resemblance to today's usage scenarios and related traffic patterns. In the longer term, the exponential increase of the user generated multimedia content and the number of mobile users will raise many new challenges. In this respect, Future Media Internet will not simply be a faster way to go online. It will be designed to overcome current limitations and to address emerging trends including: network architecture, content and service mobility, diffusion of heterogeneous nodes and devices, mass digitisation, new forms of (3D) user centric/user generated content provisioning, emergence of software as a service and interaction with improved security, trustworthiness and privacy. In this evolving environment, machine-to-machine communication (including RFIDs), rich 3D content as well as community networks and the use of peer-to-peer (P2P)
1
NEAR FUTURE STREAMING MEDIA NETWORK ARCHITECTURE
Exabyte = 260 Bytes
Widespread and affordable broadband access opens up opportunities for delivery of new streaming services. However, what is expected to fundamentally change the way that people use the network is the ability to produce, and seamlessly deliver and share their own multimedia content. We believe that in a few years everyone will be a multimedia content producer (by publishing digital pictures, video recordings, remote e-health services, home surveillance, etc.), multimedia content mediator (by storing/forwarding streaming content) and multimedia content consumer (digital television, video on demand, mobile broadcasting and alike). Towards this age, new experience of seamless video delivery, maintaining the integrity, and wherever applicable, adapting and enriching the quality of the media across the whole distribution chain, is required. As shown in Figure 1, we believe that the user should be placed at the centre of a multimedia streaming, content aware network architecture, acting as content consumer, content mediator and content producer. The content aware network may consist of four types of content delivery networks: a) Broadcasting networks e.g. terrestrial (DVB-T), satellite (DVB-S/S2), cable (DVB-C), b) Interactive/on demand Bidirectional networks e.g. xDSL, WiMAX, c) Mobile networks e.g. 3G/4G, GERAN, UTRAN, DVB-H
* This paper does not necessarily represent the official position of the European Commission. Corresponding author: Th. Zahariadis, Synelixis, Farmakidou 10, Chalkida, Greece, Tel:+30 2221061309, zahariad@[Link]
d) Mesh P2P logical overlay topologies.
Broadcasting Networks
Bidirectional Networks
P2P Networks
Mobile Networks
Figure 1: Near Future Streaming Media Network Architecture
content, among different routes and reserve resources without user or application level signalling. In this respect, new open architectures and technologies for converged and scalable, seamless streaming services are required. Finally, existing networks cross-layer control (CLC) and adaptation provides significant improvements in the PQoE under specific networking and transmission conditions. However, further research is required especially in the case of P2P topologies, where the physical infrastructure may be an arbitrary, timely varying combination of links belonging to different networks. Moreover, CLC schemes are required to face the network and terminal heterogeneity and take advantage of new (3D) advanced coding and delivery schemes by proposing network abstraction mechanisms, able to model the underlined end-to-end paths, describe the functional dependencies and determine the optimum adaptation of the multimedia resources.
Though this categorization is not a formal one, it demonstrates the heterogeneous physical and logical architecture, over which personalised, scalable, seamless and trusted multimedia content delivery will be provided, while protecting content from unauthorized access.
2.2 3D Media Internet
Although the 3D media technology exists for a couple of years, only recently advances in capturing, processing, displaying and networking have turned it into a reality for the large majority of users. For years, one of the major barriers to the 3D media has been 3D scanning and modelling. Today, there are many techniques for creating 3D models, but depending on the geometry and the material characteristics of the object or scene, one technique may be much better suited than another. Once 3D data have been acquired, further processing is needed. For example, improvements in automatic decimation, solving large 3D puzzles automatically, exploiting shapes in combination with texture information, level-of-detail (LoD) processing. All these can also be expected to greatly benefit from a semantic understanding of the data [2]. After 3D content preparation, open architectures and technologies will be required for searching, streaming, caching, filtering, aggregation and presentation of 3D content with optimised PQoE and in-network content adaptation.
2.1 Content Aware Networks
There is a rapidly growing momentum behind worldwide broadband deployment and the emerging "triple-play" convergence of voice, video and data services. It is important to note, however, that streaming multimedia content in the Future Media Internet is very different from traditional cable/satellite TV services and will place significant new demands on telecom network infrastructure. Users will have a much wider channel selection and will ask to watch whatever they want whenever they want it. One of the greatest obstacle in the successful implementation of Future Media Internet is the transport network. As currently constructed, the transport network supports Internet traffic, VoIP and video over the Internet, but the perceived quality-ofexperience (PQoE) for each of these services is limited. For IPTV to succeed, carriers and service providers must offer a PQoE equal to or better than that offered by today's cable/satellite TV. The key to achieving this goal will be the development of content-aware networks or at least contentaware edge devices, which are capable of tracking, managing and prioritizing the multiple signal streams flowing through the edge of the network. Intelligent content-aware service and content discovery and content aware routing are additional key issue of the Future Media Internet. Today, the dominant routing protocols in the Internet such as Open Shortest Path First (OSPF) and Border Gateway Protocol (BGP) are capable of routing packets based on their IP addresses. However, these protocols have no knowledge of which server is suitable for a particular content or how to best route the media content in order to achieve significant improvements in the PQoS. Foreseen content delivery networks will be able to route different types of
2.3 Wireless Visual Sensor Networks
Recent advantages in microelectronics, tiny operating systems, wireless sensor networking and Audio-Visual digital signal processing have progressed significantly the last couple of years. The Wireless Visual Sensor Networks (WVSN) are positioning at the intersection of these areas (Figure 2). A wireless visual sensor is a miniaturized, light-weighted, low-cost, battery powered, intelligent device, able to sense the environment, capture events and audio/ images/ video streams utilizing an embedded microphone and/or micro-camera, and communicate utilizing a wireless transceiver. A Wireless Visual Sensor Network (WVSN) or visual sensor web consists of a large number of interconnected wireless visual sensors, densely deployed in a QoS aware, mesh-networking topology. A unique feature is that the sensors within a WVSN cooperate efficiently, and instead of sending raw data, they may carry
out locally simple pre-processing of the AV content via customized, embedded microprocessors and transmit only the valuable information. It is foreseen that WVSN will play an important role in the future media Internet as they are expected to be utilized in a large variety of application areas [3]. Foreseen content-rich WVSN will target application areas, such as: 1) Surveillance. Protection of facilities (e.g. airports, stations, stadiums, plants, buildings, highways, homes) requires a large number of automatic sensing devices, able to detect and track intruders over large areas. A WVSN may capture abnormal events, perform minimal AV analysis, and forward video streams of interest to an operator control center. 2) Monitoring & Discovery. There are many cases, where spread, inaccessible areas (e.g. toxic locations, disaster sites, power grids, waste disposal) should be visually monitored to acquire environmental data over long periods of time. 3) Remote Control. Static or moving sensors providing surveillance and/ or monitoring activities, combined with interactive bi-directional communication. Examples include industrial sensing & control (e.g. automatic tracing, industrial robots), Unmanned (Aerial) Vehicles etc. 4) Ambient Assisting Living and Intelligent Interactive Environment. Examples include virtual reality shows, virtual meetings, home surveillance, support for elderly or physically disabled people at home, tele-teaching etc.
challenging tasks, particularly when node mobility is considered.
EXPECTED TECHNOLOGICAL ACHIEVEMENTS
Wireless Ad-hoc Networking
Audiovisual Processing
Sensors
Wireless Visual Sensor Networking
Figure 2: Wireless Visual Sensor Networking (WVSN) positioning
Before they become widely deployed however, WVSN have to overcome a significant number of open issues coming from different research areas. Route optimization and autoconfiguration are complicated, application-specific topics, while self-organization and network coverage are also applied. The WVSN problem becomes more complex due to the large number of the nodes, the limited processing power and the battery fuelled nature of the peer nodes, which are prone to failures and can join or leave the WVSN at any time. On the other hand, multimedia streaming and content management are quite resource demanding tasks. Designing of decentralized, yet highly efficient, embedded multimedia processing and transmission systems, while guaranteeing continuous end-to-end streaming of A/V content are quite
The Networked Media sector depends on the continuous advancement of key technologies such as Information Technology, Networking, Electronic Equipment and Content. The continual innovation and adoption of these technologies in enterprises, the extended home and/or on the move, clearly illustrate the need to continue developing and enhancing these technologies, anticipating users needs and exploiting new venues for research and development. Main axes related to technological achievements in Networked Media sector include (but are not limited to): - the challenge of the true broadband, where Gigabits (or even higher) per second connectivity will be offered on the home, while broadband, pervasive and affordable mobile networking services will be a common practice, - the challenge of personalised intelligent media, elaborating on prominent key research directions such as real-time adaptation, interactivity and user inclusion - the challenge of distributed control, where neither the infrastructure nor the service is controlled by a single entity. In this evolving environment, user-generated/user-centric rich content as well as community networks and the use of peer-topeer (P2P) overlay systems are expected to generate new models of interaction and cooperation and be able to support new innovative applications, like virtual collaboration environments, personalised services/media, virtual sport groups, edutainment. In this context, the interaction with content combined with interactive/multimedia search capabilities across distributed repositories and P2P (also mobile) networks and the dynamic adaptation to the characteristics of diverse terminals are expected to contribute towards such a vision. On the other hand, advances in 3D processing will give rise to innovative applications such as massive multiplayer mobile games and in virtual worlds placing new types of traffic demands and constraints on network architectures. These environments coupled with their usage rules hold the promise of a "3D Media Internet" forming the basis of future networked and collaborative platforms in the residential and professional domains (including creation, delivery and rendering), in virtual/ gaming applications, and in digital and electronic cinema. The Future Internet of Media will not only radically change the entertainment industry, but also is expected to stimulate and enhance creativity, professional productivity and community relations.
3.1 Deployment Scenarios
In order to meet the challenges of Future Media Internet, traditional IT, telecom, mobile service providers, media
companies, suppliers of consumer electronics, (multimedia) search engine companies and other powerful players will have to overcome a series of deployment barriers, including: Open network architectures and technologies for seamless, converged and scalable multimedia content delivery, while maintaining the integrity and the quality of the media. Multi-layered/Multi-viewed content coding, considering the evolving H.264 AVC/SVC/MVC and their emerging successors (e.g. H.265 ?), as the major foreseen A/V coding technologies for content distribution over heterogeneous networks/terminals. Augmented virtual 3D worlds, 3D collaborative platforms and moving holograms create new requirements in terms of information representation, authoring environments, filtering, aggregation and networking. New 3D, self-aware, self-adaptive content formats from efficient mixing of real 3D captured content with Computer Generated Graphics (CGG) able to offer new visual sensations to the users. Increasing demand towards more sophisticated (multimedia) search tools, including tools for media professionals, P2P overlay and context aware networks. User controlled identity management, ownership and trading of virtual digital objects, right of use, and personalised advertisements. Media-to-(mobile) network cross-layer dynamic adaptation and use of network aware video coding techniques improving video quality beyond High Definition TV (HDTV) towards 3D and Ultra HDTV. Content distribution, distributed control, distributed caching and P2P multi-source/multi-network content streaming offering balanced network requirements. New viewing methods (and displays) and in general consumption of both professional and entertainment multimedia content depends on the availability of high quality content and ubiquitous network access. Reducing start-up/modification/adjustment delays and increasing interactivity to support real-time multi-party network sessions, supporting virtual 3D worlds for professional as well as community and gaming applications. Increasing demand for personalisation and aggregation of services supported by high throughput multimedia streams and sessions including data from smart objects.
Force, we have presented some of the major issues and barriers towards the Future 3D Media Internet. However, innovations and break-through achievements in a single domain may create only limited impact. Addressing emerging trends can be achieved via a cross-domain revolutionary, rather than evolutionary approach. Cross domain areas related to networked media include: Intelligent Services, dynamically optimised with the policies taking into account the content and the adaptation needs, Security and Trustworthiness: Identification of vulnerabilities and threats, content usage rights beyond DRM, content sharing (identity management), ownership and trading of personal and professional virtual (3D) digital objects, security through out the full lifecycle of the content. The Internet of 3D Media will be characterised by an increasing demand for personalisation through the aggregation of services. The Internet of 3D Media will need to be supported by multiple multimedia streams and sessions including data from smart objects and visual sensor nodes (i.e. "Internet of Things").
Acknowledges
This paper presents the authors opinion and does not necessarily represent the official position of the European Commission. Part of this paper is based on work performed in the framework of the Media Future Internet Workgroup, the Networked Media Task Force, the Media Delivery Platforms cluster, the User Centric Media cluster and the project SEA IST-214063, which is partially funded by the European Community.
References
[1] [2] IST SAPIR Large Scale Multimedia Search & P2P, [Link] L. Van Gool, B. Leibe, P. Mller, M. Vergauwen, Th. Weise1, 3D Challenges and a Non-In-Depth Overview of Recent Progress, 6th International Conference on 3-D Digital Imaging and Modeling (3DIM 2007), Montreal, Quebec, Canada Th. Zahariadis, S. Voliotis, Wireless Visual Sensor Networking, IWSSIP 2007, 27-30 June 2007, Maribor, Slovenia The Bled Declaration on the Future Internet, 31 March 2008, [Link]
[3] [4]
CONCLUSIONS CROSS DOMAIN PERSPECTIVES
The Internet has changed our social and economic world. However, in the longer term, the exponential increase of the user generated multimedia content and the number of mobile users will raise many new challenges. Based on work from the Future Internet Assembly and the Networked Media Task
