InterDigital CE Patent Holdings Patente

Some embodiments of a method may include: obtaining a MPEG Avatar Representation Format (MARF)-based file; parsing the MARF-based file into a data structure; selecting an asset from the data structure; verifying the asset comprises a controller set property; obtaining controller set data corresponding to the controller set property; obtaining container data using the controller set data; determining a mime type using the controller set data; decoding the container data using the mime type; obtaining a mesh and a skeleton using the container data; and animating an avatar corresponding to the mesh and the skeleton.

A method comprising: obtaining (900) YUV data resulting from a conversion of a YUV signal with a first transfer function and a first color gamut to a YUV signal with a second transfer function and a second color gamut; and applying a correction to a component of the YUV data to limit effects of quantization errors.

Systems, methods, and instrumentalities are disclosed for avatar keypoints and landmarks representation. A file (e.g., an MPEG avatar representation format (MARF) file) may be received. The file may include a plurality of assets. The file may indicate keypoint and landmark component information. An asset may be selected from the plurality of assets in the file based on parsing the file. A container may be determined based on the selected asset and the keypoint and landmark component information. At least one of a keypoint or a landmark may be determined based on the determined container. At least one of a keypoint or a landmark may animate an avatar based on the at least one of the keypoint or landmark.

Systems, methods, and instrumentalities for energy saving mode in processing order. An example device may obtain encoded video data and processing order metadata that comprises an energy savings indication associated with performing a post-processing series in a processing order. The device may determine, based on the energy savings indication, whether to perform the post-processing series. The device may process the video data based on the determining.

A point cloud with attributes is obtained. Its geometry is encoded and decoded. Holes are detected in the decoded geometry and metadata representative of coordinates of the holes are generated. In an embodiment, points are added at these locations to obtain a reconstructed geometry. The attributes of the point cloud are transferred on the decoded or on the reconstructed geometry and, the decoded geometry, the transferred attributes, and the metadata are encoded in a data stream. At the decoding side, a data stream representative of a point cloud is obtained. The geometry is decoded. In an embodiment, metadata are used to add point to the decoded geometry. The attributes of the point cloud are decoded the decoded or reconstructed geometry. In the first case, metadata are used to add points to the point cloud and their attributes are estimated according to the decoded attributes.

Some embodiments of a method may include: obtaining information describing a haptic noise function; generating a noise data structure corresponding to the haptic noise function, wherein the noise data structure complies with a generic noise model, and wherein the generic noise model comprises identification of the haptic noise function and a set of one or more parameters associated with the haptic noise function; and storing the noise data structure in a memory location.

Methods, apparatus and data stream are provided for processing of triggers related to avatar interactions in a 3D scene, in particular pointing triggers. A scene description file is received. A user equipment or an application server parses the file and iterates on each described behavior. If triggers in the scene description are related to avatar interactions the device starts tracking the avatars and the events (expressions, gestures, gaze and/or pointing an object) referenced in these triggers. Each time an event occurs and is detected the device starts the one or more actions specified in the corresponding behavior.

A point cloud with attributes is obtained. Its geometry is encoded and decoded. Holes are detected in the decoded geometry and points are added at these locations to obtain a reconstructed geometry. The attributes of the point cloud are transferred on this reconstructed geometry and, the decoded geometry and the transferred attributes are encoded in a data stream. At the decoding side, a data stream representative of a point cloud is obtained. The geometry is decoded, and holes are detected to obtain a reconstructed geometry. The attributes of the point cloud are transferred on this reconstructed geometry to obtain a point cloud with attributes.

Neural network in-loop filter. One method for filtering a reconstructed video frame includes generating the reconstructed video frame, and applying one or more in-loop filters to the reconstructed video frame to generate a filtered reconstructed video frame, the one or more in-loop filters including a neural network in-loop filter that receives an input including a quantization parameter, the quantization parameter received in place of a default value associated with the reconstructed video frame.

Systems, methods, and instrumentalities are disclosed for energy saving in video coding. A supplemental enhancement information (SEI) message may include a first syntax element that indicates whether energy information related to a neural network post-processing filter (NNPF) is present in the SEI message. The SEI message may further include a second syntax element that indicates whether information related to a complexity or a property of the NNPF is present in the SEI message. The video data may be encoded or decoded based on the SEI message.

Some embodiments of a method may include: obtaining information describing a probabilistic space; generating a probabilistic haptic data structure corresponding to the probabilistic space, wherein the probabilistic haptic data structure comprises identification of a distribution type associated with the probabilistic space and a set of one or more parameters associated with the probabilistic space; and storing the probabilistic haptic data structure in a memory location.

Video coding may be performed by associating energy information with a neural network post filter group (NNPFG) supplemental enhancement information (SEI) message. For example, a device (e.g., a decoder for video decoding) may receive a bitstream. The device may determine an SEI message from the bitstream. The SEI message may indicate characteristics of an NNPFG and energy information associated with the NNPFG. The device may, e.g., on a condition that the energy information satisfies criteria, decode an image by applying one of more of the NNPF in the NNPFG to one or more image pixel values from the bitstream. The device may send the decoded image for use, for example to a display.

An extended reality system is proposed comprising a presentation engine and a scene understanding module configured to manage representation of the real part of the scene including real light in a generic way. The presentation engine initializes the scene understanding module with the scene description that comprises data describing parameters to integrate the real scene including real light and the virtual part of the extended reality scene. The scene understanding module manages a scene understanding representation thanks to data obtained from sensors or databases and warns the presentation engine of updates of the scene understanding representation. A generic API is defined for interactions between the two modules.

A haptics decoding method according to an example embodiment includes obtaining a file representing a haptic experience. The file includes at least a first affective model data structure identifying an affective model. The file further includes at least a first haptic effect data structure including a first physical description of a first haptic effect. The first haptic effect data structure further includes a first affective response data structure defining a first affective response according to the identified affective model. A rendered haptic effect is provided to a user according to the first haptic effect data structure.

Systems, methods, and instrumentalities for an energy saving mode indicator. An example device may receive a bitstream that includes encoded video data and metadata that indicates an energy saving indication. The device may determine, based on the energy saving indication, whether the encoded video data has been pre-processed for energy savings. The device may process the encoded video data based on the determining.

Video coding may be performed by removing wasteful computations in neural networks with sparse weights. For example, a device (e.g., a decoder for video decoding) may determine a neural network (NN). The NN may be associated with sparse weights. The NN may comprise an architecture and a set of parameters. The device may determine a set of sparse matrices of weights inside the NN that satisfies a criterion. The device may determine a modified set of matrices of weights, for example, by processing each sparse matrix of weights in the set of sparse matrices of weights, such that one or more non-zero rows and/or one or more non-zero columns from the corresponding sparse matrix may be transferred to the modified matrix of weights.

A device may transmit a request for a portion of an avatar three-dimensional (3D) representation, the request based on a representation format that defines access to a predefined range of data of the avatar three-dimensional (3D) representation. A device may receive the portion of the avatar 3D representation or access to the portion of the avatar 3D representation.

Procedures, methods, architectures, apparatuses, systems, devices, and computer program products for energy-aware content distribution. A client device receives, from a content server, information corresponding to a manifest file for a content item, the manifest file including energy consumption information for provision of the content item for at least one of a plurality of versions of the content item, selects a version of the content item whose corresponding energy consumption information matches a device profile of the client device and satisfies at least one energy control criterion, and downloads units of the selected version of the content item.

In various implementations, methods and devices are disclosed that improve the diversity of the predictors used in the blending process of DIMD. For instance, predictors may be obtained by applying derived intra modes to different reference lines instead of the reference line closest to the current block. For instance, predictors may be obtained by computing independently a Histogram of Gradients per reference line. For instance, predictors obtained by computing independently a Histogram of Gradients per reference line may be used in template intra mode derivation TIMD.

Systems, methods, and instrumentalities are disclosed for entropy coding of on the border predicted probabilities. A context adaptive binary arithmetic coding (CABAC) probability associated with a bit may be determined. A range of a least probable symbol (LPS) may be determined based on the CABAC probability and a probability prediction threshold. The bit may be entropy decoded or encoded based on the range of the LPS. The range of the LPS may be determined to be a minimum range value based on the CABAC probability being less than or equal to the probability prediction threshold. The probability prediction threshold may be a first probability prediction threshold. A second probability prediction threshold may be determined that is greater than the first probability prediction threshold.

Video coding may be performed by selecting tone mapping metadata. For example, a device may receive a bitstream. The device may determine a first metadata from the bitstream. The first metadata may indicate a first change to one or more image pixel values from the bitstream. The device may determine a second metadata from the bitstream. The second metadata may indicate a second change to the one or more image pixel values. The second metadata may indicate neural network post filter characteristics (NNPFC). The device may select the second change based on the indication of the NNPFC, for example, if it is determined that the first change conflicts with the second change. The device may decode an image by applying the selected second change to the one or more image pixel values.

Systems, methods, and instrumentalities for dual merge with regression-based geometric partitioning mode partitioning. An example device may determine a first dual-merge candidate and a second dual-merge candidate. The device may determine a first prediction of a current block based on the first dual-merge candidate, and a second prediction of the current block based on the second dual-merge candidate. The first prediction may include a first set of samples. The second prediction may include a second set of samples. The device may apply a first weight to the first prediction and apply a second weight to the second prediction. The first weight may vary based on the position of samples in the first set of samples. The second weight may vary based on the position of samples in the second set of samples. The device may decode the current block based on the weighted first prediction and the weighted second prediction.

A decoding method is presented. A current template area in a set of a plurality of template areas is obtained for a current block coded in inter mode. A current motion vector predictor is obtained, for the current block, based on the current template area. The current block is decoded based on the current motion vector predictor.

Some embodiments of a method may include: initializing a rendering session between a client and a server; obtaining scene adjustment information regarding an immersive scene; instructing the server to perform rendering of a first portion of the immersive scene using the scene adjustment information; rendering a second portion of the immersive scene using the scene adjustment information; obtaining one or more buffer frames from the server; processing the one or more buffer frames to generate rendering information; and instructing an XR runtime device to compose and render one or more frames using the rendering information.

Some embodiments of a method may include sending a request to start a V-DMC media stream; receiving a first offer message to start the V-DMC media stream, wherein the first offer message comprises one or more parameters related to level of detail; determining compatibility with the receiver device of the one or more parameters related to level of detail; sending a first answer message based on the determined compatibility; receiving a media stream configured according to a first set of parameters related to level of detail determined to be compatible with the receiver device; sending a second offer message, wherein the second offer message indicates a second set of parameters related to level of detail to be updated; receiving a second answer message indicating which updated parameters were applied; and receiving an adjusted media stream adjusted according to parameters of the second set that were applied.

A device (e.g., a decoder) may obtain a plurality of reference sample arrays associated with a video block. The device may determine an intra-prediction mode associated with the video block. The device may determine, based on a direction of the intra-prediction mode, a reference sample support of the intra-prediction mode, for example, from the plurality of reference sample arrays associated with the video block. The device may perform a matrix-based intra-prediction (MIP) of the video block using the reference sample support of the intra-prediction mode. The device may decode the video block based on the MIP of the video block. In examples, a reference sample support may include a plurality of primary reference samples associated with the intra-prediction mode. The device may determine an MIP matrix associated with the reference sample support and/or perform the MIP of the video block further based on the MIP matrix.

In one implementation, a video decoder obtains a base block vector for a block of a picture in an intra template matching prediction mode; decode a block vector difference for the block; obtains a block vector for the block based on the base block vector and the block vector difference for the block; obtains a prediction block for the block based on the block vector for the block; and decodes the block based on the prediction block for the block. On the encoder side, a video encoder obtains the base block vector for the block in an intra template matching prediction mode; encodes the block vector difference; obtains the block vector for the block based on the base block vector and the block vector difference; obtains the prediction block for the block based on the block vector; and encodes the block based on the prediction block.

Systems, methods, and instrumentalities are disclosed for bilateral filtering on intra prediction. In examples, a device (e.g., a video encoding and/or decoding device) may be configured to determine that a decoder side intra mode derivation (DIMD) mode is used to decode a block. The device may, based on the determination that the DIMD mode is used to decode the block, obtain DIMD histogram of gradients (HoG) information. The device may, based at least on the DIMD HoG information, determine whether to enable intra bilateral filtering (BIF) on the block. The device may, based on a determination to enable the intra BIF, apply the intra BIF to process (e.g., encode and/or decode) the block.

Systems, methods, and instrumentalities for regression-based geometric partition mode (RGPM) fallback procedures. An example device may determine that RGPM is enabled. The device may determine whether to use RGPM-derived blending weights for a prediction. The RGPM-derived blending weights may vary based on the position of samples in the prediction. Based on a determination to not use the RGPM-derived blending weights, the device may determine a fallback procedure to determine unequal blending weights. The device may determine the unequal blending weights based on the fallback procedure. The device may decode a current block based on the unequal blending weights.

Disclosed herein are systems, methods, and instrumentalities associated with in-loop filtering. According to embodiments of the disclosure, a video decoding device may be configured to obtain, from a video bitstream, a plurality of coefficients of a filter to be applied to a video block. The video decoding device may be further configured to determine one or more inputs to the filter, wherein the one or more inputs may include a first set of samples generated for the video block using a bilaterial filter. The video decoding device may then apply the filter to the video block based on the one or more inputs, wherein the filter may be applied using the plurality of coefficients obtained from the video bitstream.

A method and an apparatus for loop filter control information derivation is provided that can be used in video encoding and decoding systems. A reconstructed version of a block of a video is obtained and loop filter control information for the block is derived from loop filter control information used in a reference frame of the video. One or more loop filters are applied to the reconstructed version of the block using the derived loop filter control information. The loop filter control information can be derived using motion data of the block or from collocated block in the reference frame.

Video coding using reduced-rank matrices for matrix intra prediction (MIP). In a video coding method, information is obtained indicating a matrix rank, k, and a matrix intra prediction mode for a current block. The current block has a block shape, and neighboring reference samples. Based on the intra prediction mode and the current block shape, a first and second matrix are selected from a stored set of intra prediction matrices. Each of the stored matrices is associated with a respective block shape and intra prediction mode. A first rank-k matrix is obtained from the first matrix, and a second rank-k matrix is obtained from the second matrix. The current block is then predicted using a product of the first rank-k matrix, the second rank-k matrix, and the reference samples.

In various implementations, method and devices are disclosed that adapts the selection of transform set for both regular and regression-based Spatial Geometric Partitioning Mode SGPM modes. For instance, for a regular SGPM block, a virtual intra prediction mode VIPM corresponding to a largest magnitude of values in a histogram of gradients (HoG) computed on a neighboring template is used to select the transform set instead of the split direction of the SGPM. For instance, for a regression-based SGPM block, a pre-defined scheme of transform set may be used for that block independently of the VIPM. According to various embodiments, the transform selection criterion may adaptively switch between using the VIPM versus using the split direction for SGPM. The transform selection may depend on various factors such as size of each partition, an intra predictor type (directional vs non-directional), a size of a maximum bin in HoG, a variance of HoG.

Disclosed herein are systems, methods, and instrumentalities associated with latent implicit neural representation. According to embodiments of the disclosure, a video decoding device may be configured to determine, using a first neural network, a probability distribution associated with a current latent code. The video decoding device may decode the current latent code from a video bitstream based on the determined probability distribution and may determine, based on the latent code, parameters of a second neural network. The video decoding device may use the second neural network to decode video data.

Video coding may be performed by controlling chroma sample location, for example, in Low complexity enhancement video coding (LCEVC). For example, a device (e.g., a decoder for video decoding) may determine, for a dimension, a first phase associated with a luma upscaler. The device may set a location of an upsampled luma grid based on the determined first phase. The device may determine, for the dimension, a second phase associated with a chroma upscaler. The device may set a location of an upsampled chroma grid based on the determined second phase and an additional third phase. The device may, based on at least the location of the upsampled luma grid and the location of the upsampled chroma grid, reconstruct an upscaled high-resolution picture from a lower resolution base picture. The device may decode a high-resolution picture, for example, by adding residuals to the upscaled high-resolution picture.

Systems, methods, and instrumentalities are disclosed for determining a mean absolute difference (MAD) of a plurality of coding block samples. The MAD of the plurality of coding block samples may be determined to be used as a bilateral filtering (BIF) parameter for intra BIF. An intra-predicted sample of a coding block may be generated based on the BIF parameter. The coding block may be decoded based on the intra-predicted sample of the coding block. The BIF parameter may be a scale factor. The scale factor may not be determined using a look up table (LUT). The scale factor may not be determined using a look up table (LUT) that includes one or more coefficients associated with the coding block.

A method comprising obtaining video data representing an inter block in merge mode; applying a parsing process to the video data, wherein the parsing process comprises parsing a sub-block-based merge flag indicating that a sub-block merge mode applies to the inter block responsive to a condition is fulfilled (1804), the condition being independent of a size of the inter block.

The present disclosure proposes a neural-network post-filter NNPF that, in addition to the decoded pictures, uses as input an indication of a task (i.e. control data) which enables generating a different output for the same input decoded pictures, the output picture being specified/guided for the task or control data. For instance, the NNPF may be named a generative NNPF, as the neural-network post-filter may generate a picture specified by a particular hypothesis or control input. For instance, encoding and decoding method and/or apparatus that comprises signaling one or more syntax structures for neural-network post-filter characteristics adapted to the present principles are disclosed.

The present disclosure proposes an NNPF, signaled by an NNPFC SEI, that generates more than one version of a decoded picture at a given time stamp by one single inference of the NNPF. For instance, the NNPF may be referred to as a multi-hypothesis NNPF, as the neural-network post-filter generates at least two filtered pictures being packed in a large output picture, each of the filtered pictures corresponding to a particular hypothesis or control characteristic. For instance, encoding and decoding method and/or apparatus that comprises signaling one or more syntax structures for neural-network post-filter characteristics adapted to the present principles are disclosed.

Adaptive prediction selection is disclosed. A video coding device, which may be, for example, an encoder device or a decoder device, may comprise at least one processor configured to derive, for a block, a set of block vector-based intra prediction modes. The set of block vector-based intra prediction modes may be associated with at least of DIMD or OBIC. The video coding device may derive a set of angular prediction modes based on a histogram associated with a plurality of neighboring samples of the block. The video coding device may obtain a fused prediction of the block based on at least one derived block vector-based intra prediction mode, at least one derived angular prediction mode, and a planar mode. The video coding device may encode or decode the block based on the fused prediction.