InterDigital CE Patent Holdings Patente

A method comprising: obtaining (800) a current block of a picture; deriving (801) a first list of a first number of intra prediction modes; generating (802) a prediction of a template of samples in a neighborhood of the current block for each intra prediction mode of the first list; for each predictor, obtaining (803) a prediction error between the predictor and the template; obtaining (804) a second list of reference lines in a neighborhood of the current block and forming a TMRL list of a second number of template based multiple reference lines (TMRL) candidates by browsing the intra prediction modes of the first list in ascending order of the prediction error and browsing the reference lines of the second list from a nearest reference line to a furthest reference line, each TMRL candidate of the TMRL list comprising an intra prediction mode extracted from the first list during the browsing the first list and a reference lines extracted from the second list extracted during the browsing of the second list; and, determining (805) a TMRL predictor for the current block from the TMRL candidates of the TMRL list.

Methods and apparatuses for encoding/decoding a video are provided wherein at least two prediction blocks for a current block of an image of the video are obtained and wherein at least one of the at least two prediction blocks is obtained using a block vector-based prediction. The at least two prediction blocks are blended to provide a predicted block, wherein the blending uses blending weights determined from a template of the current block and the current block is encoded or decoded based on the predicted block.

Systems and method directed to point cloud deep learning inpainting of projections. One example is directed to a computer-implemented method for video encoding or decoding including deriving non-projected points from decoded geometry of a three-dimensional -3D-point cloud, predicting attributes for the non-projected points using a convolutional neural network, and encoding or decoding the 3D point cloud based on the attributes.

Systems, methods, and instrumentalities for signal-specific and universal dictionary based implicit neural representation. An example video decoding device may receive a coding block and video data associated with the coding block. The video data may indicate a signal-specific implicit neural representation (INR) dictionary, INR tail parameters, and a coefficient matrix. The device may determine a combined INR dictionary based on the signal-specific INR dictionary and a universal INR dictionary. The device may construct an INR based on the combined INR dictionary, the INR tail parameters, and the coefficient matrix. The device may reconstruct the coding block based on the INR.

In various implementations, methods and devices are disclosed that involves quantization aware training for dictionary-driven INR (Implicit Neural Representation). For instance, one or more quantization parameters for an INR network corresponding to an image region are obtained, wherein the INR network is reconstructed at both encoding and decoding by combining tail layers to head layers, the tail layers being defined by the one or more inverse quantized network parameters and the head layers being a weighted combination of one or more INR basis functions by the one or more inverse quantized coefficients, respectively. Then, the image region is reconstructed based on the reconstructed INR network. The one or more quantization parameters for the INR network, the one or more quantized coefficients of the head layers and the one or more quantized network parameters of the tail layers are encoded into or decoded from the bitstream.

Systems, methods, and instrumentalities are disclosed for entropy coding of on the border predicted probabilities. A device for video decoding or encoding may include obtaining an interval range represented by a number of bits that are greater than nine. The device may obtain a context adaptive binary arithmetic coding (CABAC) probability associated with a CABAC bin. The device may determine a range of a least probable symbol (LPS) based on the CABAC probability and the interval range. The device may entropy decode or encode the CABAC bin based on the range of the LPS. The device may determine the number of bits based on a range adjustment parameter. The range adjustment parameter may be any integer value of one to six.

In one implementation, a method of video decoding is presented, comprising: receiving at least a syntax element associated with energy adaptation; determining that one or more visual dimensions of a plurality of visual dimensions of a video are to be adjusted based on the at least a syntax element; and adapting a neural network based filter to adjust the one or more visual dimensions. At the encoder side, a method of video encoding is presented, comprising: determining that one or more visual dimensions of a plurality of visual dimensions of a video are to be adjusted for energy adaptation; and encoding at least a syntax element to indicate that the one or more visual dimensions are to be adjusted by a neural network based filter.

Systems, methods, and instrumentalities are disclosed for pruning latent variables for a hybrid implicit neural representation (INR). A device for video encoding may include a processor. The device may obtain a plurality of latent variables associated with an image. The device may identify a first portion of the plurality of latent variables and/or a second portion of the plurality of latent variables. The device may determine an INR network parameter based on the second portion of the plurality of latent variables. The device may encode the image based on the second portion of the plurality of latent variables and/or the INR network parameter. The device may generate a latent variable mask by analyzing the image. The device may apply the latent variable mask to the plurality of latent variables to identify the first portion of the plurality of latent variables and/or the second portion of the plurality of latent variables.

A method, e.g., a decoding method, is disclosed. A set of latent variables representative of a current picture, parameters of a set of at least one first neural network, and at least one parameter of a downsampling chroma network are first obtained. The current picture is reconstructed based on the set of latent variables, the parameters of the set of at least one first neural network and the parameters of the downsampling chroma network.

A method (e.g., a decoding method) is disclosed. A set of latent variables representative of a current picture and neural network parameters are obtained. The current picture is reconstructed using the set of latent variables as input of a set of at least one first neural network parametrized by the neural network parameters. To this aim, an output of the first neural network is used as input of a forward pass of an invertible downsampling chroma neural network to obtain the reconstructed current picture.

A device for video decoding may determine a range of values for a coordinate associated with a non-zero coefficient of a transform block (TB). The coordinate may indicate a sample location associated with the non-zero coefficient in the TB. The device may determine a context-adaptive binary arithmetic coding (CABAC) context for entropy decoding a coordinate indication indicating whether a coordinate suffix associated with the range of values may be received. The device may determine the coordinate based on the coordinate indication. The device may decode the TB based on the coordinate associated with the non-zero coefficient. The device may entropy decode the coordinate indication associated with the coordinate based on the CABAC context. The device may determine that the coordinate suffix is not to be received and/or that the coordinate is to be determined based on a subset of the range values and/or the size of the TB.

Systems, methods, and instrumentalities are disclosed for transform set selection based on template cost analysis. For example, a device (e.g., a video encoding device and/or a video decoding device) may obtain at least one prediction sample. The device may obtain a histogram of gradient (HoG). The device may determine whether to perform a template cost analysis. Based on a determination to perform the template cost analysis, the device may obtain at least one intra mode candidate based on the HoG. The device may determine a respective template cost associated with at least one intra mode candidate. The device may reorder at least one intra mode candidate. The device may obtain at least one virtual intra prediction mode (VIPM). The device may obtain a transform set and obtain a transform kernel. The video decoding device may decode the current block, and the video encoding device may encode the current block.

Systems, methods, and instrumentalities are disclosed for a video decoding and encoding device. A device may include a processor configured to determine that intra template matching prediction (intraTMP) is enabled for a video block. The processor may be configured to determine that intraTMP is used for decoding a first adjacent block of the video block. The processor may be configured to determine, based on a local illumination compensation (LIC) indication of the first adjacent block, a context-adaptive binary arithmetic coding (CABAC) context associated with entropy decoding a LIC indication associated with the video block. The processor may be configured to entropy decode the LIC indication associated with the video block based on the CABAC context. The processor may be configured to decode the video block based on the LIC indication.

Systems, methods, and instrumentalities are disclosed for pruning latent variables for a hybrid implicit neural representation (INR). In examples, a decoding/encoding device may include a processor. The device may be configured to extract a set of features from an image. The processor may select, from a plurality of sets of predefined weights, a set of weights based on the extracted set of features. The processor may determine, based on the selected set of predefined weights, a probability distribution associated with a plurality of latent variables. The processor may entropy encode the plurality of latent variables based on the probability distribution.

Video coding (e.g., encoding and/or decoding) may be performed by optimizing a block vector search process. In examples, a device (e.g., a decoder for video decoding) may be configured to obtain a plurality of candidate block vectors (BVs) for a current video block. The device may determine a refinement area associated with a current BV from the plurality of candidate BVs. The refinement area may define BV refinement candidates for the current BV. The device may determine a refined BV, for example, based on a cost analysis. The cost analysis may be applied to (e.g., only) a subset of the BV refinement candidates defined by the refinement area. The device may add the refined BV to a candidate BV list for the current video block.

A method and device use a processing algorithm to perform energy control of a content. The impact of the energy control may be determined, in relation with health, economics, pollution, raw material, social recognition or biodiversity. An indicator representing the impact of energy control may be displayed, allowing the user to evaluate values of these indicators in relation to the quality level obtained on the energy-controlled content.

Apparatuses and methods are described including techniques for processing color data of a video. Techniques include obtaining an input color segment containing colors of collocated video pixels. In a first color space, a set of color segments is searched to find segments that consume less power than that consumed by the input color segment. If such a set of color segments is found, to preserve energy, the input color segment is replaced with the closest color segment from that set. If such a set of color segments is not found, in a second color space, a color segment is determined that minimizes power consumption and the distance to the input color segment. Then, to preserve energy, the input color segment is replaced with the determined color segment providing that it consumes less power than that consumed by the input color segment.

Some embodiments of a method may include: obtaining mesh data and images of an avatar head captured by input cameras; initializing parameters of a plurality of three-dimensional (3D) Gaussians from the mesh data; modifying a local density of the 3D Gaussians adaptively to values of their respective parameters; processing the plurality of 3D Gaussians using one or more Gaussian update processes; obtaining camera viewpoints for one or more of the input cameras; generating a plurality of 2D Gaussians by performing a two-dimensional (2D) projection of the plurality of processed 3D Gaussians to an image plane for each of the one or more input cameras using the respective camera viewpoints; generating an output image by performing a differentiable rasterization of the plurality of 2D Gaussians; and updating the parameters of the 3D Gaussians and parameters of the one or more Gaussian update processes.

A method comprising obtaining (700) two consecutive pictures of a video sequence; computing (701) a first histogram of a first of the two consecutive pictures and a second histogram of a second of the two consecutive pictures; obtaining (702) a plurality of weighting curves, each weighting curve weighting a value of each bin of an histogram in function of a bin number; for each weighting curve of the plurality: applying (704) the weighting curve to the first and the second histograms to obtain weighted first and second histograms; summing (705) values of bins of the weighted first histogram to obtain a first sum and summing values of bins of the weighted second histogram to obtain a second sum; obtaining (706) a ratio of sums from the first sum and the second sum; and, calculating (707) an evidence value from the first sum, the second sum and the ratio; and, determining (708) that a scene cut occurs between the first and the second pictures based on at least one of the evidence values.

Some embodiments of a method may include: obtaining avatar mesh data and avatar images captured from different viewpoints associated with an avatar; obtaining a plurality of processed 3D Gaussians fitted to the avatar mesh data and avatar images; determining geometric parameters associated with the plurality of processed 3D Gaussians; determining appearance parameters associated with the plurality of processed 3D Gaussians; populating an avatar data structure associated with the geometric and appearance parameters; encoding the avatar data structure to generate Avatar JSON Interchange File (AJIF) data; and transmitting the encoded AJIF data to a device.

Methods, apparatus and data stream are provided for encoding and decoding a three-dimensional (3D) point cloud. The 3D point cloud is processed to obtain a coded geometry, a reconstructed geometry obtained by decoding the coded geometry and adapted attributes. The adapted attributes are obtained by tracking indices of points of the point cloud or by transferring attributes to the reconstructed geometry. The 3D point cloud is divided in a regular grid of blocks, projecting adapted attributes of points of each non-empty block on inner faces of the blocks and selecting one of the inner faces to obtain patches. The patches are packed in an atlas image according to a scanning process and the atlas image is encoded in a data stream with metadata indicating selected inner faces.

The systems, methods and devices are configured to use the notion of haptic illusions and convert haptics to at least one of audio, image and video. Generally, the conversions occur when an application does not have the appropriate haptic device, or even does not have any haptic device to play haptics. Instead of not playing the haptic tracks, a conversion is performed to at least one of audio and video to enable the system to provide the illusion of haptics feedback. These systems, methods and devices may be applicable in media applications, such as gaming, movies, music, XR, communication, for example.

Systems, methods, and instrumentalities are disclosed for using a neural network (NN)-based film grain (FG) analysis. For example, a device may include a processor that is configured to perform one or more of the following. The device (e.g., a video processing device) may obtain a grainy video. The grainy video may be a video source with film grain. The device may feed the grainy video to a film grain analysis neural network (FGA-NN) module. The FGA-NN module may include a feature extraction network (e.g., a backbone) and at least one task-specific head. At least one task-specific head may be configured for a prediction task. For example, the prediction task may include a classification task and/or a regression task. The device may obtain at least one film grain characteristic (FGC) parameter based on an output of the FGA-NN module.

Systems and method directed to spanning intra prediction mode information using reconstructed coding unis. One example is directed to a computer-implemented method for video encoding or decoding including deriving intra prediction mode -IPM- information from one or more reconstructed samples of a coding unit, storing the IPM information in an IPM buffer corresponding to the coding unit, and encoding or decoding a picture including the coding unit based on the IPM buffer.

Methods and apparatus are provided for signaling of a bound offset used in the quantization correction and defined from an integer bound offset parameter O_i and an integer shift parameter S_i, as the ratio oi2Si. The proposed methods apply the same way for the signaling of either a minimum bound offset o_min (i.e., i=0) or a maximum bound offset o_max (i.e., i=1). In one embodiment the integer bound offset parameter is signaled using its full bitdepth N_i. In a second embodiment, the integer bound offset parameter is signaled using a reduced bitdepth M_i and associated mapping parameters. In a second embodiment, signaling or inferring a flag indicates that a default value of the integer bound offset parameter is used.

Systems, methods, and instrumentalities are disclosed for film grain synthesis frequency shaping. As described herein, a device may include a processor that is configured to perform one or more of the following. The device may obtain a model parameter set. The model parameter set may be, or may include, a cutoff frequency. Based on the model parameter set (e.g., the cutoff frequency), the device may generate a soft frequency window. The soft frequency window may be at least one of a 2D low-pass window, a separable 2D low-pass window, or a low-pass 20 frequency window. The device may generate a synthetic grain based on the soft frequency window. For example, the device may generate the synthetic grain based on (e.g., applying) an inverse transform of the soft frequency window. The inverse transform may be associated with a discrete cosine transform (DCT).

Methods and apparatus are provided for performing the quantization constrained (QC) correction of a given transform coefficient cf associated to a quantization level cq when the minimum and maximum bound offsets are individually enabled or disabled. In one embodiment, signaling methods are provided to save signaling cost of bound offset parameters if the corresponding bound offset is disabled. Another embodiment provides performing QC correction of a transform coefficient using individually enabled or disabled bound offsets. In another embodiment, computing both the minimum and the maximum bound offsets is performed using a single bound offset signaled in the bitstream.

Some embodiments of a method may include obtaining a bitstream of immersive video content captured by a combination of at least a first camera and a second camera; obtaining a first set of camera parameters corresponding to the first camera; obtaining a second set of camera parameters corresponding to the second camera; adapting one or more displacement sensitivity parameters associated with the immersive video content based on the first and second set of camera parameters; and rendering the immersive video content using the adapted one or more displacement sensitivity parameters.

Encoding and decoding information for generation of dynamic three-dimensional (3D) clothing. For example, a method includes receiving, with a decoder, a plurality of parameters indicating a body motion, a body shape, and a characteristic of a clothing material and generating, with the decoder, a dynamic three-dimensional clothing based on the plurality of parameters, wherein the dynamic three-dimensional clothing is dynamic with respect to the body motion and the body shape.

A decoding method is disclosed. Gradients for a current block are first obtained by filtering samples in a neighboring template of the current block, wherein obtaining gradients for a current block comprises substituting at least one of a Sobel filter or an edge detection filter by a generic filter. A histogram of oriented gradients (HoG) is then obtained for the current block based on the gradients. An intra mode is selected for the current block based on the HoG. The current block is then decoded using the selected intra mode.

In an example of a split inferencing method, a first endpoint obtains information describing a graph structure of a machine learning, ML, model with a plurality of nodes whose outputs are tensors, each tensor having an original branch name. The first endpoint obtains information indicating a split of the model into first and second sub-models. For at least one of the tensors that is an output of a node in the first sub-model, the first endpoint sends, to a second endpoint, information associating the original branch name of the first tensor with a replacement branch name. The first endpoint runs the first sub-model to obtain the first tensor, and it sends the first tensor to the second endpoint, the first tensor being identified by the replacement branch name. The second node may run the second sub-model using the first tensor as an input.

In an example of a split-inferencing method, a first endpoint obtains information describing at least a first sub-model of a split-inferencing machine learning model. In communication with a second endpoint, a tensor sorting algorithm is selected from among a plurality of available tensor sorting algorithms. The first endpoint runs the first sub-model to obtain a plurality of tensors, and it sends the tensors to the second endpoint in an order determined by the selected tensor sorting algorithm. The tensors may be sent without tensor names. The second endpoint may interpret the tensors based on the selected tensor sorting algorithm and run the second sub-model.

In an example method for use in a split inferencing architecture, information is obtained at a first endpoint describing at least a first sub-model of a split-inferencing machine learning model. In communication with a second endpoint, a tensor indexing algorithm is selected. The first sub-model is executed to obtain at least one tensor. The tensor is sent to the second endpoint, each tensor being identified by a respective reference determined according to the selected tensor indexing algorithm.

Method and apparatus are described wherein a filtered reconstructed image block from a prediction block and a reconstructed prediction residual is obtained and a Quantization Constrained correction is applied to the filtered reconstructed image block, wherein the Quantization Constrained correction applied depends on whether at least one part of transformed coefficients decoded for the image block have been zeroed out during encoding of the image block. In some embodiments, QC correction is skipped for these transformed coefficients. In some variants, scanning of the transformed coefficients is adapted for the image block.

Systems, methods, and/or instrumentalities are disclosed herein for a device that may be related to adaptive loop filter residual scaling with optimal initialization. In examples, a device may include a processor. The device may determine a correction of a reconstructed picture sample. The device may determine a classification associated with the reconstructed picture sample. The device may, based on the classification, determine a first scaling parameter. The device may determine a second scaling parameter based on the first scaling parameter. The device may generate a scaled correction based on the second scaling parameter and the correction. The device may apply the scaled correction to the reconstructed picture sample.

A method comprising: obtaining (900) YUV data resulting from a conversion of a YUV signal with a first transfer function and a first color gamut to a YUV signal with a second transfer function and a second color gamut; and applying a correction to a component of the YUV data to limit effects of quantization errors.

Some embodiments of a method may include: obtaining a MPEG Avatar Representation Format (MARF)-based file; parsing the MARF-based file into a data structure; selecting an asset from the data structure; verifying the asset comprises a controller set property; obtaining controller set data corresponding to the controller set property; obtaining container data using the controller set data; determining a mime type using the controller set data; decoding the container data using the mime type; obtaining a mesh and a skeleton using the container data; and animating an avatar corresponding to the mesh and the skeleton.

Systems, methods, and instrumentalities are disclosed related to external data in avatar representation formats. In examples, a decoding/encoding device may include a processor. The device may be configured to determine that a data file comprises a plurality of data components. The device may determine that a data component of the plurality of data components is associated with a first data format that an application does not support. The device may obtain a link to access an external data content associated with a second data format that the application supports. The device may obtain external data of the external data content using the link. The device may decode the external data.

Systems, methods, and instrumentalities are configured for transform derivation using decoder-side intra mode derivation (DIMD) for one or more of template-based intra mode derivation (TIMD) mode or DIMD mode. In examples, a video coding device may be configured to determine that a template-based intra mode derivation (TIMD) mode is enabled for a current block. The device may apply a decoder side intra mode derivation (DIMD) like process to derive a virtual intra prediction mode (VIPM) from a neighboring reconstructed template of a current block or a prediction block associated with the current block. The device may, based on the derived VIPM, determine a transform kernel set for predicted residual coding.

A system, codec and method of optimizing an MPEG haptics synthesizer are disclosed. The method includes determining a minimum sampling rate, generating a haptic signal with the determined minim sampling rate and upsampling the generated haptic signal to synthesize a final haptic signal. The optimizing may include determining a minimum sampling rate by an end user, by an automatic computation. The optimizing may include upsampling of the generated haptic signal based on a target sampling rate. The optimizing may include the defined by content creator causes a value of the minimum sampling rate to be stored with haptic data. The optimizing may include, in a context of the MPEG haptic standard, the minimum sampling rate being part of the normative part of the specification, the minimum sampling rate being stored as an optional value of a haptic experience level, or the minimum sampling rate being stored for each band.