InterDigital CE Patent Holdings Patente

Adaptive prediction selection is disclosed. A video coding device, which may be, for example, an encoder device or a decoder device, may comprise at least one processor configured to derive, for a block, a set of block vector-based intra prediction modes. The set of block vector-based intra prediction modes may be associated with at least of DIMD or OBIC. The video coding device may derive a set of angular prediction modes based on a histogram associated with a plurality of neighboring samples of the block. The video coding device may obtain a fused prediction of the block based on at least one derived block vector-based intra prediction mode, at least one derived angular prediction mode, and a planar mode. The video coding device may encode or decode the block based on the fused prediction.

A method and device propose, for a signal handled using feature-based INR methods, to partition the signal and specify a highest level of latent grid to be used for each partition. As a result, higher levels may not be encoded nor decoded, saving bitrate and decoding operations. This allows to decrease the bit rate and the decoding complexity of low frequency part of a signal using feature-based INR methods. Such approach has a far lower computational complexity than end-to-end neural compression approaches.

Some embodiments of a method may include: obtaining a hairstyle description corresponding to a hairstyle model, wherein the hairstyle description comprises three-dimensional (3D) orientation information and occupancy information; building a 3D array of grid cells using grid size and cell counts; filling the 3D array of grid cells with orientation and occupancy data; obtaining a target avatar head mesh registered to the hairstyle model and a hair growth region; and performing a hair growth process for each candidate strand of a full hairstyle.

Methods, apparatus and data stream are provided for managing avatars in a 3D scene. Avatar skeleton comprising joint landmarks are encoded in a data stream. A set of joints landmarks for a given function. This function may be relative to a location of the joint landmarks of the set, like the positioning of the avatar in the 3D scene. The function may also be relative to a weighted sum of a location of the joint landmarks of the set, like the determining of a mass center.

In an example point cloud attribute coding method, a plurality of micropatches are obtained. Each micropatch has a plurality of pixels, and at least one pixel of each micropatch is associated with a respective point in a point cloud. Each pixel that is associated with a respective point has a pixel feature value. A first point in the point cloud is associated with respective pixels in each of a first plurality of the micropatches. These associated pixels in the first plurality of the micropatches have a first plurality of pixel feature values. A point feature value is determined for the first point by calculating a weighted sum of the first plurality of pixel feature values.

Examples are provided herein of combining measurements and estimates for carbon emission reporting. In an example, a content provider transmits a video segment to a first set of display devices and a second set of display devices. The content provider receives energy consumption reports from the first set of display devices, each energy consumption report including energy consumption for processing and display of the video segment by a respective display device of the first set of display devices. Further, the content provider aggregates the energy consumption reports for the first set of display devices to obtain a first total energy consumption. Also, the content provider estimates a second total energy consumption for the second set of display devices. Moreover, the content provider combines the first total energy consumption and the second total energy consumption into a final total energy consumption.

Systems, methods, and instrumentalities may be configured for mesh subdivision prediction. In examples, a coding device (e.g., an encoding and/or decoding device) may be configured to determine whether a mesh edge of a mesh sequence wherein a vertex is to be inserted is a non-boundary edge. The device may, based on the determination that the mesh edge of the mesh sequence is a non-boundary edge, select one of an arithmetic mean, a geometric mean, or a harmonic mean to predict a position of the vertex based on comparing an absolute value of a dot product of normal vectors with at least one of a first threshold value or a second threshold value.

Systems, methods, and instrumentalities are disclosed herein for dictionary learning for implicit neural compression (INR). In examples, a video decoder may decode an implicit neural representation (INR) parameter. An INR may be reconstructed based on the INR parameter. The decoder may compute a dictionary based on the reconstructed INR. A dictionary-based INR may be generated based on the computed dictionary. Data may be reconstructed based on the dictionary-based INR. in examples, the decoder may decoder a training procedure used for the computation of the dictionary. The dictionary may be computed based (e.g., further based) on the training procedure. In examples, a video encoder may determine an INR parameter. The dictionary may be computed based on the INR parameter. The encoder may determine a dictionary-based INR parameter based on the computed dictionary. The INR parameter and the dictionary-based INR parameter may be encoded.

In an example haptic rendering method, an implicit neural representation model of haptic data is obtained. An interaction input to the model, and an interaction output is obtained from the model, where the interaction output representing a value of at least one haptic parameter.

Methods and apparatus for associating energy use with a data-stream passing through a device, or system or service center are provided, enabling reporting and control of energy from data centers or content delivery networks. In one embodiment, information is transmitted from a data center, comprising a number of customers associated with the data center, corresponding data rates for each of the number of customers and an energy for each of the number of customers. A total energy can be determined for all customers and for individual users. In another embodiment, an energy corresponding to an individual user within a set time interval can be determined and transmitted using the determined energy corresponding to the data center.

Apparatuses and methods are described including techniques for assessing carbon emissions caused by the distribution and the consumption of media content. Techniques include determining information indicative of a carbon intensity associated with a device displaying a video segment of a content received from a content provider, recording information indicative of energy spent by the device to display the video segment, and then, transmitting an emission report generated based on the determined information and the recorded information. Techniques further include transmitting content to devices; receiving emission reports, from those devices, corresponding to respective video segments of the content displayed by respective devices; and then, aggregating information provided by the emission reports to obtain the emission amount caused by displaying, by the devices, at least a portion of the transmitted content.

In one implementation, methods and apparatuses for video decoding and video encoding are provided. For a component of a current block of the video, a single transform pair is selected from a plurality of transform pairs, wherein said selecting is based on a look up table associated with the component, and wherein the look up table indicates a mapping between (1) an intra prediction mode and a transform unit size associated with the component of the current block and (2) said single transform pair, transform coefficients of said component of the current block are inverse transformed based on said single transform pair, residuals are obtained for said component of the current block from said inverse transformed coefficients; and said component of the current block is reconstructed based on said residuals.

Systems, methods, and instrumentalities are disclosed for determining a plurality of candidate modes for encoding a video block. A first distortion associated with a first candidate mode of the plurality of candidate modes may be determined. The first candidate mode may be an inter-layer mode. A second distortion associated with a second candidate mode of the plurality of candidate modes may be determined. Whether the first distortion is less than the second distortion may be determined. A candidate mode may be selected from the plurality of candidate modes based on the determination of whether the first distortion is less than the second distortion. The video block may be encoded based on the selected candidate mode.

Systems, methods, and instrumentalities are disclosed for different filter sizes for decoder side intra mode derivation (DIMD). In examples, an apparatus video decoding may determine that a DIMD mode is used to decode a block. In examples, an apparatus for video encoding may determine that a DIMD mode is used to encode a block. Based on the determination, the apparatus may determine a number of pixels associated with the block. Based on the determined number of pixels, the apparatus may select a size of set of filters for building a histogram of gradients (HoG) associated with the DIMD mode. In examples, the apparatus (e.g., for video decoding) may decode the block based on the selected size of the set of filters. In examples, the apparatus (e.g., for video encoding) may encode the block based on the selected size of the set of filters.

A method of decoding is disclosed. It is determined that a current block is an intra sub-partition (ISP) coded block. At least one transform pair is selected for the current block from a plurality of transform pairs for the current block responsive to the determining. The current block is then decoded based on the selected at least one transform pair.

A decoding method is disclosed. A current filtering classifier is obtained, for a group of reconstructed samples, in a set of filtering classifiers. For each reconstructed sample of the group, a correction is determined responsive to a classification with the current filtering classifier.For the group of reconstructed samples, a scale classifier is obtained in the set of filtering classifiers. Each reconstructed sample of the group is classified with the scale classifier if different from the current filtering classifier.The correction of each reconstructed sample of the group is modified responsive to the classification. The modified correction is added to the corresponding reconstructed sample.

A device (e.g., a video encoding and/or decoding device) may obtain block vector candidates for a video block. Each block vector candidate may be associated with a refinement region. The device may identify, among the block vector candidates, a cluster of block vector candidates having overlapping refinement regions. The device may calculate a combined block vector candidate based on the cluster of block vector candidates. For example, a mean block vector of the cluster of block vector candidates may be calculated and used as the combined block vector. The device may determine an updated refinement region for the combined block vector candidate. The device may process (e.g., decode and/or encode) the video block based on the updated refinement region and the combined block vector candidate.

Neural network in-loop filter. One method for filtering a reconstructed video frame includes generating the reconstructed video frame and applying one or more in-loop filters to the reconstructed video frame to generate a filtered reconstructed video frame, the one or more in-loop filters including a neural network in-loop filter that receives an input including an alternative base quantization parameter that is distinct from a base quantization parameter.

Systems and methods directed to a rate-distortion optimization process. One example is directed to a video encoding device including an electronic processor. The electronic processor is configured to obtain motion information associated with a video block and determine, as a function of the motion information, a subset of coding tools from a plurality of coding tools for rate-distortion performance evaluation of the video block.

A method comprising: obtaining (601) input video data in an input dynamic range and interoperability metadata; converting (602) the input video data in output video data in an output dynamic range using the interoperability metadata; wherein, the interoperability metadata comprise at least an information representing an HDR reference level value, an information representing an SDR reference level value.

Systems, methods, and instrumentalities for constructing an energy aware bitrate ladder. An example device may obtain an indication of a quality metric that represents a trade-off between content features and a parameter associated with energy consumption. The device may determine, based on the quality metric, a plurality of quality metric values associated with a plurality of resolutions. The device may determine, based on the plurality of quality metric values, an acceptable set of resolutions from the plurality of resolutions. The device may determine, based on quality metric values associated with the acceptable set of resolutions, a plurality of energy reduction rates associated with the acceptable set of resolutions. The device may select, from the acceptable set of resolutions, a smallest resolution with an energy reduction rate that is less than a maximum energy reduction rate and assign that resolution to a bitrate.

A video decoding device may obtain available information associated with a coding block, CB, receive transform syntax element(s) associated with the CB, derive mapping information based on the transform syntax element(s) associated with the CB, convert the available information into transform indices using the mapping information, identify a transform kernel from a set of transform kernels using the transform indices, and decode the CB using the identified transform kernel. The available information associated with the CB may comprise an index of an intra prediction mode selected to predict the CB, a number of non-zero transform coefficients associated with the CB, a shape of the CB, and/or a number of non-zero coefficients. The device may (e.g., further) apply the identified transform kernel to a set of dequantized coefficients or to a set of reconstructed coefficients.

A method for deriving SAO control information from a previous decoded picture is provided wherein it is determined whether Sample Adaptive Offset (SAO) control information relating to one or more component of at least one block of a current picture of a video is obtained from SAO control information of a previous decoded picture of the video, SAO control information for the at least one block are obtained based on the determining, and the one or more component of the at least one block are filtered based on the obtained SAO control information.

A method comprising: obtaining (401) a compressed binary mask; broadcasting (402) the compressed mask on each element of a first vector of data; obtaining (403) a fixed binary mask, each element of the fixed mask having a single non-zero bit, the single non-zero bit of the rightmost element of the fixed mask being located at the rightmost position of the element, the single non-zero bit of a first element of the fixed mask being right shifted of one unit with respect to a position of the non-zero bit of a second element of the mask on its right when present; combining (404) the first vector with the fixed mask using a bitwise AND operator; and, obtaining (405) the binary mask from a comparison of the second vector with an all zero vector.

Some embodiments of a method may include: obtaining a three-dimensional (3D) model of an eye region of a user, wherein the 3D model is embedded with semantic features corresponding to at least one of gaze, eyelid aperture, and identity of the user; obtaining a first controller setting, wherein the first controller setting corresponds to one of gaze, eyelid aperture, and identity of the user; morphing the 3D model of the eye region using the first controller setting; and displaying the morphed 3D model of the eye region.

A method of encoding a surface of a non-rigid articulated object. The method includes signaling identity vector information in an encoded bitstream, the identity vector information including one of an identity vector or an instruction to use a pre-existing identity vector, the identity vector and the pre-existing identity vector are individually configured to condition a skinning network, an occupancy network, or both.

A method comprising: applying a correction process for correcting an input SDR picture into an output SDR picture intended to be transformed in a HDR picture using metadata, the correction process using a first LUT for transforming a luma component of the input SDR picture in a luma component of the output SDR picture and a second LUT representing a color correction function, the correction process comprising when applied to a current sample of the SDR input picture. The correction process comprises applying (801) a single LUT combining the first LUT and the second LUT to use less hardware resources and reduce the computation complexity.

Systems, methods, and apparatuses for encoding and/or decoding video are provided in various implementations. An encoder generates and/or receives a plurality of discrete input views. The encoder generates a volumetric data representation based on the plurality of discrete input views. The encoder further generates a plurality of pseudo views and/or a plurality of error pseudo views. The encoder transmits a bitstream to a decoder. The bitstream comprises an encoded volumetric data representation with a plurality of encoded pseudo views and/or a plurality of encoded error pseudo views. The decoder uses the plurality of pseudo views and/or a plurality of error pseudo views to optimize the volumetric data representation and generate an improved volumetric data representation. The decoder generates a plurality of new views based on the improved volumetric data representation. The decoder renders the plurality of new views, generates a final video data, and outputs the final video data.

Some embodiments of a method may include obtaining information for a three-dimensional (3D) Gaussian model corresponding to an avatar, wherein the information comprises a set of attributes of the 3D Gaussian model; parsing the information for a first attribute of the set of attributes, wherein the first attribute corresponds to an anchor type of the 3D Gaussian model; parsing the information for a second attribute of the set of attributes, wherein the second attribute corresponds to anchor index data of the 3D Gaussian model; and rendering the avatar using the parsed attributes of the 3D Gaussian model.

Some embodiments of a method may include: obtaining a description of services provided by a Content Delivery Network (CDN) according to a Media Presentation Description (MPD) file; obtaining a selection of a first service selected from the description of services provided by the CDN; connecting to a session for the first service; obtaining a plurality of content segments corresponding to a first representation for the first service; providing, to an application for playback, one or more of the plurality of content segments corresponding to the first representation; providing, to the CDN, a report of client energy-related information; obtaining an updated description of services provided by the CDN according to an updated MPD file, wherein the updated description of services provided by the CDN was updated based on the report of client energy-related information; and switching to a second representation for the first service.

Systems and methods for point cloud attribute encoding and decoding. An encoding method includes selecting an orientation of a first micropatch centered on a first point of a first point cloud. For each of a plurality of pixels in the first micropatch, a respective point of the first point cloud is assigned to the respective pixel. For each of the pixels having an assigned point, attribute information of the assigned point is associated with the respective pixel. The attribute information of the pixels of at least the first micropatch is encoded.

Systems, methods, and instrumentalities may be configured for an artificial intelligence (Al) based energy aware bitrate ladder. In examples, a computing device may determine tuples associated with a video, wherein each tuple includes a respective image resolution and a respective bitrate associated with the video. The computing device may predict, using an energy prediction machine learning (ML) model, a respective energy consumption associated with providing the video to a receiving device at the image resolution and the bitrate included in each of the tuples. The computing device may select one or more tuples from the tuples based at least on the respective energy consumptions predicted for the tuples. The computing device may convey information regarding the one or more selected tuples to a video encoder.

Improved storage of neural network models is provided by not storing zeros and non-zero model data is retrieved using various methods. In one embodiment, signal sparsity information is used to retrieve the position of the coefficients. In another embodiment, if a model tensor is sparse, packing information is signaled. In another embodiment, data is read according to sparsity information and packing information. In another embodiment, signaling is used at a higher level whether sparsity information is present, or inferred to be true or false.

A method and device for decoding an encoded data is based on a hybrid implicit neural representation (INR) network comprising a plurality of INR networks arranged in a plurality of hierarchic levels and latent variables being arranged in a plurality of hierarchic levels. The ground truth images are decomposed using Gaussian pyramid to generate the output images at different resolutions, making it possible to learn one synthesis network per resolution for example. The reconstruction process may be performed in a coarse to fine manner, i.e. from the lowest to the highest resolution.

A method of encoding a texture segmentation in a three-dimensional mesh topology. The method comprising signaling the three-dimensional mesh topology, a texture mapping, and a label mapping in an encoded bitstream, the label mapping including a mapping of pixel coordinates to one or more labels.

Some embodiments of a method may include: obtaining a description of services provided by a Content Delivery Network (CDN) according to a Media Presentation Description (MPD) file; obtaining a selection of a first service selected from the description of services provided by the CDN; connecting to a session for the first service; obtaining a plurality of content segments corresponding to a first representation for the first service; providing, to an application for playback, one or more of the plurality of content segments corresponding to the first representation; providing, to the CDN, a report of client energy-related information; obtaining an updated description of services provided by the CDN according to an updated MPD file, wherein the updated description of services provided by the CDN was updated based on the report of client energy-related information; and switching to a second representation for the first service.

Some embodiments of a method may include: obtaining Avatar JSON Interchange File (AJIF) data; decoding the AJIF data to generate avatar data and associated neural radiance field properties; obtaining user movement data corresponding to a movement of a user; generating updated avatar data based on the user movement data by using a neural networked configured with the neural radiance field properties; and sending the updated avatar data to a client device. Another method may include: generating AJIF data corresponding to an avatar; sending, to an application server, the AJIF data; obtaining user movement data corresponding to a movement of a user; sending, to the application server; receiving, from the application server, scene update data, wherein the scene update data includes updated avatar data corresponding to the movement of the user; and rendering a scene update based on the scene update data.

Systems and methods are described for encoding and decoding of 3D volumetric data. Volumetric data is encoded and/or decoded using a syntax that allows for different codec types, such as a point-cloud-based codec type (e.g.. V-PCC), a mesh-based codec type (e.g. V-DMC), and a depth-image-based codec type (e.g. MIV). Example encoding and decoding techniques improve bitrates by allowing for syntax functions that are specific to different codec types, such as syntax functions related to atlas coding, to be tailored to signal only parameters that are relevant to the corresponding codec type that is is used for the volumetric coding.

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …