InterDigital CE Patent Holdings Patente

A device (e.g., a decoder) may obtain a plurality of reference sample arrays associated with a video block. The device may determine an intra-prediction mode associated with the video block. The device may determine, based on a direction of the intra-prediction mode, a reference sample support of the intra-prediction mode, for example, from the plurality of reference sample arrays associated with the video block. The device may perform a matrix-based intra-prediction (MIP) of the video block using the reference sample support of the intra-prediction mode. The device may decode the video block based on the MIP of the video block. In examples, a reference sample support may include a plurality of primary reference samples associated with the intra-prediction mode. The device may determine an MIP matrix associated with the reference sample support and/or perform the MIP of the video block further based on the MIP matrix.

Some embodiments of a method may include sending a request to start a V-DMC media stream; receiving a first offer message to start the V-DMC media stream, wherein the first offer message comprises one or more parameters related to level of detail; determining compatibility with the receiver device of the one or more parameters related to level of detail; sending a first answer message based on the determined compatibility; receiving a media stream configured according to a first set of parameters related to level of detail determined to be compatible with the receiver device; sending a second offer message, wherein the second offer message indicates a second set of parameters related to level of detail to be updated; receiving a second answer message indicating which updated parameters were applied; and receiving an adjusted media stream adjusted according to parameters of the second set that were applied.

Some embodiments of a method may include: initializing a rendering session between a client and a server; obtaining scene adjustment information regarding an immersive scene; instructing the server to perform rendering of a first portion of the immersive scene using the scene adjustment information; rendering a second portion of the immersive scene using the scene adjustment information; obtaining one or more buffer frames from the server; processing the one or more buffer frames to generate rendering information; and instructing an XR runtime device to compose and render one or more frames using the rendering information.

A decoding method is presented. A current template area in a set of a plurality of template areas is obtained for a current block coded in inter mode. A current motion vector predictor is obtained, for the current block, based on the current template area. The current block is decoded based on the current motion vector predictor.

Disclosed herein are systems, methods, and instrumentalities associated with latent implicit neural representation. According to embodiments of the disclosure, a video decoding device may be configured to determine, using a first neural network, a probability distribution associated with a current latent code. The video decoding device may decode the current latent code from a video bitstream based on the determined probability distribution and may determine, based on the latent code, parameters of a second neural network. The video decoding device may use the second neural network to decode video data.

Systems, methods, and instrumentalities for dual merge with regression-based geometric partitioning mode partitioning. An example device may determine a first dual-merge candidate and a second dual-merge candidate. The device may determine a first prediction of a current block based on the first dual-merge candidate, and a second prediction of the current block based on the second dual-merge candidate. The first prediction may include a first set of samples. The second prediction may include a second set of samples. The device may apply a first weight to the first prediction and apply a second weight to the second prediction. The first weight may vary based on the position of samples in the first set of samples. The second weight may vary based on the position of samples in the second set of samples. The device may decode the current block based on the weighted first prediction and the weighted second prediction.

The present disclosure proposes a neural-network post-filter NNPF that, in addition to the decoded pictures, uses as input an indication of a task (i.e. control data) which enables generating a different output for the same input decoded pictures, the output picture being specified/guided for the task or control data. For instance, the NNPF may be named a generative NNPF, as the neural-network post-filter may generate a picture specified by a particular hypothesis or control input. For instance, encoding and decoding method and/or apparatus that comprises signaling one or more syntax structures for neural-network post-filter characteristics adapted to the present principles are disclosed.

Video coding using reduced-rank matrices for matrix intra prediction (MIP). In a video coding method, information is obtained indicating a matrix rank, k, and a matrix intra prediction mode for a current block. The current block has a block shape, and neighboring reference samples. Based on the intra prediction mode and the current block shape, a first and second matrix are selected from a stored set of intra prediction matrices. Each of the stored matrices is associated with a respective block shape and intra prediction mode. A first rank-k matrix is obtained from the first matrix, and a second rank-k matrix is obtained from the second matrix. The current block is then predicted using a product of the first rank-k matrix, the second rank-k matrix, and the reference samples.

Video coding may be performed by controlling chroma sample location, for example, in Low complexity enhancement video coding (LCEVC). For example, a device (e.g., a decoder for video decoding) may determine, for a dimension, a first phase associated with a luma upscaler. The device may set a location of an upsampled luma grid based on the determined first phase. The device may determine, for the dimension, a second phase associated with a chroma upscaler. The device may set a location of an upsampled chroma grid based on the determined second phase and an additional third phase. The device may, based on at least the location of the upsampled luma grid and the location of the upsampled chroma grid, reconstruct an upscaled high-resolution picture from a lower resolution base picture. The device may decode a high-resolution picture, for example, by adding residuals to the upscaled high-resolution picture.

Systems, methods, and instrumentalities are disclosed for entropy coding of on the border predicted probabilities. A context adaptive binary arithmetic coding (CABAC) probability associated with a bit may be determined. A range of a least probable symbol (LPS) may be determined based on the CABAC probability and a probability prediction threshold. The bit may be entropy decoded or encoded based on the range of the LPS. The range of the LPS may be determined to be a minimum range value based on the CABAC probability being less than or equal to the probability prediction threshold. The probability prediction threshold may be a first probability prediction threshold. A second probability prediction threshold may be determined that is greater than the first probability prediction threshold.

In various implementations, method and devices are disclosed that adapts the selection of transform set for both regular and regression-based Spatial Geometric Partitioning Mode SGPM modes. For instance, for a regular SGPM block, a virtual intra prediction mode VIPM corresponding to a largest magnitude of values in a histogram of gradients (HoG) computed on a neighboring template is used to select the transform set instead of the split direction of the SGPM. For instance, for a regression-based SGPM block, a pre-defined scheme of transform set may be used for that block independently of the VIPM. According to various embodiments, the transform selection criterion may adaptively switch between using the VIPM versus using the split direction for SGPM. The transform selection may depend on various factors such as size of each partition, an intra predictor type (directional vs non-directional), a size of a maximum bin in HoG, a variance of HoG.

Adaptive prediction selection is disclosed. A video coding device, which may be, for example, an encoder device or a decoder device, may comprise at least one processor configured to derive, for a block, a set of block vector-based intra prediction modes. The set of block vector-based intra prediction modes may be associated with at least of DIMD or OBIC. The video coding device may derive a set of angular prediction modes based on a histogram associated with a plurality of neighboring samples of the block. The video coding device may obtain a fused prediction of the block based on at least one derived block vector-based intra prediction mode, at least one derived angular prediction mode, and a planar mode. The video coding device may encode or decode the block based on the fused prediction.

In an example haptic rendering method, an implicit neural representation model of haptic data is obtained. An interaction input to the model, and an interaction output is obtained from the model, where the interaction output representing a value of at least one haptic parameter.

A method and device propose, for a signal handled using feature-based INR methods, to partition the signal and specify a highest level of latent grid to be used for each partition. As a result, higher levels may not be encoded nor decoded, saving bitrate and decoding operations. This allows to decrease the bit rate and the decoding complexity of low frequency part of a signal using feature-based INR methods. Such approach has a far lower computational complexity than end-to-end neural compression approaches.

Some embodiments of a method may include: obtaining a hairstyle description corresponding to a hairstyle model, wherein the hairstyle description comprises three-dimensional (3D) orientation information and occupancy information; building a 3D array of grid cells using grid size and cell counts; filling the 3D array of grid cells with orientation and occupancy data; obtaining a target avatar head mesh registered to the hairstyle model and a hair growth region; and performing a hair growth process for each candidate strand of a full hairstyle.

Methods, apparatus and data stream are provided for managing avatars in a 3D scene. Avatar skeleton comprising joint landmarks are encoded in a data stream. A set of joints landmarks for a given function. This function may be relative to a location of the joint landmarks of the set, like the positioning of the avatar in the 3D scene. The function may also be relative to a weighted sum of a location of the joint landmarks of the set, like the determining of a mass center.

In an example point cloud attribute coding method, a plurality of micropatches are obtained. Each micropatch has a plurality of pixels, and at least one pixel of each micropatch is associated with a respective point in a point cloud. Each pixel that is associated with a respective point has a pixel feature value. A first point in the point cloud is associated with respective pixels in each of a first plurality of the micropatches. These associated pixels in the first plurality of the micropatches have a first plurality of pixel feature values. A point feature value is determined for the first point by calculating a weighted sum of the first plurality of pixel feature values.

Examples are provided herein of combining measurements and estimates for carbon emission reporting. In an example, a content provider transmits a video segment to a first set of display devices and a second set of display devices. The content provider receives energy consumption reports from the first set of display devices, each energy consumption report including energy consumption for processing and display of the video segment by a respective display device of the first set of display devices. Further, the content provider aggregates the energy consumption reports for the first set of display devices to obtain a first total energy consumption. Also, the content provider estimates a second total energy consumption for the second set of display devices. Moreover, the content provider combines the first total energy consumption and the second total energy consumption into a final total energy consumption.

Systems, methods, and instrumentalities may be configured for mesh subdivision prediction. In examples, a coding device (e.g., an encoding and/or decoding device) may be configured to determine whether a mesh edge of a mesh sequence wherein a vertex is to be inserted is a non-boundary edge. The device may, based on the determination that the mesh edge of the mesh sequence is a non-boundary edge, select one of an arithmetic mean, a geometric mean, or a harmonic mean to predict a position of the vertex based on comparing an absolute value of a dot product of normal vectors with at least one of a first threshold value or a second threshold value.

Systems, methods, and instrumentalities are disclosed herein for dictionary learning for implicit neural compression (INR). In examples, a video decoder may decode an implicit neural representation (INR) parameter. An INR may be reconstructed based on the INR parameter. The decoder may compute a dictionary based on the reconstructed INR. A dictionary-based INR may be generated based on the computed dictionary. Data may be reconstructed based on the dictionary-based INR. in examples, the decoder may decoder a training procedure used for the computation of the dictionary. The dictionary may be computed based (e.g., further based) on the training procedure. In examples, a video encoder may determine an INR parameter. The dictionary may be computed based on the INR parameter. The encoder may determine a dictionary-based INR parameter based on the computed dictionary. The INR parameter and the dictionary-based INR parameter may be encoded.

In one implementation, a video decoder obtains a base block vector for a block of a picture in an intra template matching prediction mode; decode a block vector difference for the block; obtains a block vector for the block based on the base block vector and the block vector difference for the block; obtains a prediction block for the block based on the block vector for the block; and decodes the block based on the prediction block for the block. On the encoder side, a video encoder obtains the base block vector for the block in an intra template matching prediction mode; encodes the block vector difference; obtains the block vector for the block based on the base block vector and the block vector difference; obtains the prediction block for the block based on the block vector; and encodes the block based on the prediction block.

Neural network in-loop filter. One method for filtering a reconstructed video frame includes generating the reconstructed video frame and applying one or more in-loop filters to the reconstructed video frame to generate a filtered reconstructed video frame, the one or more in-loop filters including a neural network in-loop filter that receives an input including an alternative base quantization parameter that is distinct from a base quantization parameter.

A device (e.g., a video encoding and/or decoding device) may obtain block vector candidates for a video block. Each block vector candidate may be associated with a refinement region. The device may identify, among the block vector candidates, a cluster of block vector candidates having overlapping refinement regions. The device may calculate a combined block vector candidate based on the cluster of block vector candidates. For example, a mean block vector of the cluster of block vector candidates may be calculated and used as the combined block vector. The device may determine an updated refinement region for the combined block vector candidate. The device may process (e.g., decode and/or encode) the video block based on the updated refinement region and the combined block vector candidate.

A decoding method is disclosed. A current filtering classifier is obtained, for a group of reconstructed samples, in a set of filtering classifiers. For each reconstructed sample of the group, a correction is determined responsive to a classification with the current filtering classifier.For the group of reconstructed samples, a scale classifier is obtained in the set of filtering classifiers. Each reconstructed sample of the group is classified with the scale classifier if different from the current filtering classifier.The correction of each reconstructed sample of the group is modified responsive to the classification. The modified correction is added to the corresponding reconstructed sample.

A method of decoding is disclosed. It is determined that a current block is an intra sub-partition (ISP) coded block. At least one transform pair is selected for the current block from a plurality of transform pairs for the current block responsive to the determining. The current block is then decoded based on the selected at least one transform pair.

Systems, methods, and instrumentalities are disclosed for different filter sizes for decoder side intra mode derivation (DIMD). In examples, an apparatus video decoding may determine that a DIMD mode is used to decode a block. In examples, an apparatus for video encoding may determine that a DIMD mode is used to encode a block. Based on the determination, the apparatus may determine a number of pixels associated with the block. Based on the determined number of pixels, the apparatus may select a size of set of filters for building a histogram of gradients (HoG) associated with the DIMD mode. In examples, the apparatus (e.g., for video decoding) may decode the block based on the selected size of the set of filters. In examples, the apparatus (e.g., for video encoding) may encode the block based on the selected size of the set of filters.

Systems, methods, and instrumentalities are disclosed for determining a plurality of candidate modes for encoding a video block. A first distortion associated with a first candidate mode of the plurality of candidate modes may be determined. The first candidate mode may be an inter-layer mode. A second distortion associated with a second candidate mode of the plurality of candidate modes may be determined. Whether the first distortion is less than the second distortion may be determined. A candidate mode may be selected from the plurality of candidate modes based on the determination of whether the first distortion is less than the second distortion. The video block may be encoded based on the selected candidate mode.

In one implementation, methods and apparatuses for video decoding and video encoding are provided. For a component of a current block of the video, a single transform pair is selected from a plurality of transform pairs, wherein said selecting is based on a look up table associated with the component, and wherein the look up table indicates a mapping between (1) an intra prediction mode and a transform unit size associated with the component of the current block and (2) said single transform pair, transform coefficients of said component of the current block are inverse transformed based on said single transform pair, residuals are obtained for said component of the current block from said inverse transformed coefficients; and said component of the current block is reconstructed based on said residuals.

Apparatuses and methods are described including techniques for assessing carbon emissions caused by the distribution and the consumption of media content. Techniques include determining information indicative of a carbon intensity associated with a device displaying a video segment of a content received from a content provider, recording information indicative of energy spent by the device to display the video segment, and then, transmitting an emission report generated based on the determined information and the recorded information. Techniques further include transmitting content to devices; receiving emission reports, from those devices, corresponding to respective video segments of the content displayed by respective devices; and then, aggregating information provided by the emission reports to obtain the emission amount caused by displaying, by the devices, at least a portion of the transmitted content.

Methods and apparatus for associating energy use with a data-stream passing through a device, or system or service center are provided, enabling reporting and control of energy from data centers or content delivery networks. In one embodiment, information is transmitted from a data center, comprising a number of customers associated with the data center, corresponding data rates for each of the number of customers and an energy for each of the number of customers. A total energy can be determined for all customers and for individual users. In another embodiment, an energy corresponding to an individual user within a set time interval can be determined and transmitted using the determined energy corresponding to the data center.

Systems and methods directed to a rate-distortion optimization process. One example is directed to a video encoding device including an electronic processor. The electronic processor is configured to obtain motion information associated with a video block and determine, as a function of the motion information, a subset of coding tools from a plurality of coding tools for rate-distortion performance evaluation of the video block.

A method of encoding a surface of a non-rigid articulated object. The method includes signaling identity vector information in an encoded bitstream, the identity vector information including one of an identity vector or an instruction to use a pre-existing identity vector, the identity vector and the pre-existing identity vector are individually configured to condition a skinning network, an occupancy network, or both.

Some embodiments of a method may include: obtaining a three-dimensional (3D) model of an eye region of a user, wherein the 3D model is embedded with semantic features corresponding to at least one of gaze, eyelid aperture, and identity of the user; obtaining a first controller setting, wherein the first controller setting corresponds to one of gaze, eyelid aperture, and identity of the user; morphing the 3D model of the eye region using the first controller setting; and displaying the morphed 3D model of the eye region.

A method for deriving SAO control information from a previous decoded picture is provided wherein it is determined whether Sample Adaptive Offset (SAO) control information relating to one or more component of at least one block of a current picture of a video is obtained from SAO control information of a previous decoded picture of the video, SAO control information for the at least one block are obtained based on the determining, and the one or more component of the at least one block are filtered based on the obtained SAO control information.

A method comprising: obtaining (601) input video data in an input dynamic range and interoperability metadata; converting (602) the input video data in output video data in an output dynamic range using the interoperability metadata; wherein, the interoperability metadata comprise at least an information representing an HDR reference level value, an information representing an SDR reference level value.

Systems, methods, and instrumentalities for constructing an energy aware bitrate ladder. An example device may obtain an indication of a quality metric that represents a trade-off between content features and a parameter associated with energy consumption. The device may determine, based on the quality metric, a plurality of quality metric values associated with a plurality of resolutions. The device may determine, based on the plurality of quality metric values, an acceptable set of resolutions from the plurality of resolutions. The device may determine, based on quality metric values associated with the acceptable set of resolutions, a plurality of energy reduction rates associated with the acceptable set of resolutions. The device may select, from the acceptable set of resolutions, a smallest resolution with an energy reduction rate that is less than a maximum energy reduction rate and assign that resolution to a bitrate.

A video decoding device may obtain available information associated with a coding block, CB, receive transform syntax element(s) associated with the CB, derive mapping information based on the transform syntax element(s) associated with the CB, convert the available information into transform indices using the mapping information, identify a transform kernel from a set of transform kernels using the transform indices, and decode the CB using the identified transform kernel. The available information associated with the CB may comprise an index of an intra prediction mode selected to predict the CB, a number of non-zero transform coefficients associated with the CB, a shape of the CB, and/or a number of non-zero coefficients. The device may (e.g., further) apply the identified transform kernel to a set of dequantized coefficients or to a set of reconstructed coefficients.

A method comprising: obtaining (401) a compressed binary mask; broadcasting (402) the compressed mask on each element of a first vector of data; obtaining (403) a fixed binary mask, each element of the fixed mask having a single non-zero bit, the single non-zero bit of the rightmost element of the fixed mask being located at the rightmost position of the element, the single non-zero bit of a first element of the fixed mask being right shifted of one unit with respect to a position of the non-zero bit of a second element of the mask on its right when present; combining (404) the first vector with the fixed mask using a bitwise AND operator; and, obtaining (405) the binary mask from a comparison of the second vector with an all zero vector.

Systems, methods, and instrumentalities may be configured for an artificial intelligence (Al) based energy aware bitrate ladder. In examples, a computing device may determine tuples associated with a video, wherein each tuple includes a respective image resolution and a respective bitrate associated with the video. The computing device may predict, using an energy prediction machine learning (ML) model, a respective energy consumption associated with providing the video to a receiving device at the image resolution and the bitrate included in each of the tuples. The computing device may select one or more tuples from the tuples based at least on the respective energy consumptions predicted for the tuples. The computing device may convey information regarding the one or more selected tuples to a video encoder.

Improved storage of neural network models is provided by not storing zeros and non-zero model data is retrieved using various methods. In one embodiment, signal sparsity information is used to retrieve the position of the coefficients. In another embodiment, if a model tensor is sparse, packing information is signaled. In another embodiment, data is read according to sparsity information and packing information. In another embodiment, signaling is used at a higher level whether sparsity information is present, or inferred to be true or false.