InterDigital CE Patent Holdings Patente

Method and apparatus are described wherein a filtered reconstructed image block from a prediction block and a reconstructed prediction residual is obtained and a Quantization Constrained correction is applied to the filtered reconstructed image block, wherein the Quantization Constrained correction applied depends on whether at least one part of transformed coefficients decoded for the image block have been zeroed out during encoding of the image block. In some embodiments, QC correction is skipped for these transformed coefficients. In some variants, scanning of the transformed coefficients is adapted for the image block.

In an example of a split-inferencing method, a first endpoint obtains information describing at least a first sub-model of a split-inferencing machine learning model. In communication with a second endpoint, a tensor sorting algorithm is selected from among a plurality of available tensor sorting algorithms. The first endpoint runs the first sub-model to obtain a plurality of tensors, and it sends the tensors to the second endpoint in an order determined by the selected tensor sorting algorithm. The tensors may be sent without tensor names. The second endpoint may interpret the tensors based on the selected tensor sorting algorithm and run the second sub-model.

In an example of a split inferencing method, a first endpoint obtains information describing a graph structure of a machine learning, ML, model with a plurality of nodes whose outputs are tensors, each tensor having an original branch name. The first endpoint obtains information indicating a split of the model into first and second sub-models. For at least one of the tensors that is an output of a node in the first sub-model, the first endpoint sends, to a second endpoint, information associating the original branch name of the first tensor with a replacement branch name. The first endpoint runs the first sub-model to obtain the first tensor, and it sends the first tensor to the second endpoint, the first tensor being identified by the replacement branch name. The second node may run the second sub-model using the first tensor as an input.

In an example method for use in a split inferencing architecture, information is obtained at a first endpoint describing at least a first sub-model of a split-inferencing machine learning model. In communication with a second endpoint, a tensor indexing algorithm is selected. The first sub-model is executed to obtain at least one tensor. The tensor is sent to the second endpoint, each tensor being identified by a respective reference determined according to the selected tensor indexing algorithm.

A method comprising: obtaining (900) YUV data resulting from a conversion of a YUV signal with a first transfer function and a first color gamut to a YUV signal with a second transfer function and a second color gamut; and applying a correction to a component of the YUV data to limit effects of quantization errors.

Some embodiments of a method may include: obtaining a MPEG Avatar Representation Format (MARF)-based file; parsing the MARF-based file into a data structure; selecting an asset from the data structure; verifying the asset comprises a controller set property; obtaining controller set data corresponding to the controller set property; obtaining container data using the controller set data; determining a mime type using the controller set data; decoding the container data using the mime type; obtaining a mesh and a skeleton using the container data; and animating an avatar corresponding to the mesh and the skeleton.

Systems, methods, and instrumentalities are disclosed related to external data in avatar representation formats. In examples, a decoding/encoding device may include a processor. The device may be configured to determine that a data file comprises a plurality of data components. The device may determine that a data component of the plurality of data components is associated with a first data format that an application does not support. The device may obtain a link to access an external data content associated with a second data format that the application supports. The device may obtain external data of the external data content using the link. The device may decode the external data.

Systems, methods, and instrumentalities are configured for transform derivation using decoder-side intra mode derivation (DIMD) for one or more of template-based intra mode derivation (TIMD) mode or DIMD mode. In examples, a video coding device may be configured to determine that a template-based intra mode derivation (TIMD) mode is enabled for a current block. The device may apply a decoder side intra mode derivation (DIMD) like process to derive a virtual intra prediction mode (VIPM) from a neighboring reconstructed template of a current block or a prediction block associated with the current block. The device may, based on the derived VIPM, determine a transform kernel set for predicted residual coding.

Systems, methods, and/or instrumentalities are disclosed herein for a device that may be related to adaptive loop filter residual scaling with optimal initialization. In examples, a device may include a processor. The device may determine a correction of a reconstructed picture sample. The device may determine a classification associated with the reconstructed picture sample. The device may, based on the classification, determine a first scaling parameter. The device may determine a second scaling parameter based on the first scaling parameter. The device may generate a scaled correction based on the second scaling parameter and the correction. The device may apply the scaled correction to the reconstructed picture sample.

A system, codec and method of optimizing an MPEG haptics synthesizer are disclosed. The method includes determining a minimum sampling rate, generating a haptic signal with the determined minim sampling rate and upsampling the generated haptic signal to synthesize a final haptic signal. The optimizing may include determining a minimum sampling rate by an end user, by an automatic computation. The optimizing may include upsampling of the generated haptic signal based on a target sampling rate. The optimizing may include the defined by content creator causes a value of the minimum sampling rate to be stored with haptic data. The optimizing may include, in a context of the MPEG haptic standard, the minimum sampling rate being part of the normative part of the specification, the minimum sampling rate being stored as an optional value of a haptic experience level, or the minimum sampling rate being stored for each band.

Systems, methods, and instrumentalities are disclosed for avatar keypoints and landmarks representation. A file (e.g., an MPEG avatar representation format (MARF) file) may be received. The file may include a plurality of assets. The file may indicate keypoint and landmark component information. An asset may be selected from the plurality of assets in the file based on parsing the file. A container may be determined based on the selected asset and the keypoint and landmark component information. At least one of a keypoint or a landmark may be determined based on the determined container. At least one of a keypoint or a landmark may animate an avatar based on the at least one of the keypoint or landmark.

A device (e.g., video encoder or video decoder) may determine transform sets for MTSS. For example, an MTSS indication (e.g., MTSS flag) may be obtained. The MTSS flag may be used to determine a transform set. A histogram of gradients (HoG) may be obtained. A VIPM (e.g., first VIPM) may be obtained, for example, based on the HoG. Whether the MTSS indication is enabled may be determined. Based on a determination that the MTSS indication is not enabled, a mode-specific transform set corresponding to VIPM may be obtained. Inverse transform may be performed based on the obtained transform set.

A system, codec and method for encoding parametric haptic data are disclosed. This includes providing a property designating an effect of a parametric model including a type of signal, providing in the effect of a parametric model a designation of a symmetry of the signal, indicating properties in the parametric model to define a pulse including pulse width and interphase delay and providing properties in a plurality of haptic effect patterns including effect type, number of pulses, cycle offset, and number of cycles. The property may be monopolar or bipolar. The designation may be symmetric or asymmetric. The interphase delay may indicate symmetric or asymmetric. The effect type may indicate the pattern via a set of keyframes. The number of pulses may indicate pulses in one primitive. The cycle offset may indicate a time between two cycles. The number of cycles may indicate cycles in the pattern effect.

In various implementations, methods and devices are disclosed that improve the diversity of the predictors used in the blending process of DIMD. For instance, predictors may be obtained by applying derived intra modes to different reference lines instead of the reference line closest to the current block. For instance, predictors may be obtained by computing independently a Histogram of Gradients per reference line. For instance, predictors obtained by computing independently a Histogram of Gradients per reference line may be used in template intra mode derivation TIMD.

Procedures, methods, architectures, apparatuses, systems, devices, and computer program products for energy-aware content distribution. A client device receives, from a content server, information corresponding to a manifest file for a content item, the manifest file including energy consumption information for provision of the content item for at least one of a plurality of versions of the content item, selects a version of the content item whose corresponding energy consumption information matches a device profile of the client device and satisfies at least one energy control criterion, and downloads units of the selected version of the content item.

A device may transmit a request for a portion of an avatar three-dimensional (3D) representation, the request based on a representation format that defines access to a predefined range of data of the avatar three-dimensional (3D) representation. A device may receive the portion of the avatar 3D representation or access to the portion of the avatar 3D representation.

Video coding may be performed by removing wasteful computations in neural networks with sparse weights. For example, a device (e.g., a decoder for video decoding) may determine a neural network (NN). The NN may be associated with sparse weights. The NN may comprise an architecture and a set of parameters. The device may determine a set of sparse matrices of weights inside the NN that satisfies a criterion. The device may determine a modified set of matrices of weights, for example, by processing each sparse matrix of weights in the set of sparse matrices of weights, such that one or more non-zero rows and/or one or more non-zero columns from the corresponding sparse matrix may be transferred to the modified matrix of weights.

Systems, methods, and instrumentalities for an energy saving mode indicator. An example device may receive a bitstream that includes encoded video data and metadata that indicates an energy saving indication. The device may determine, based on the energy saving indication, whether the encoded video data has been pre-processed for energy savings. The device may process the encoded video data based on the determining.

A haptics decoding method according to an example embodiment includes obtaining a file representing a haptic experience. The file includes at least a first affective model data structure identifying an affective model. The file further includes at least a first haptic effect data structure including a first physical description of a first haptic effect. The first haptic effect data structure further includes a first affective response data structure defining a first affective response according to the identified affective model. A rendered haptic effect is provided to a user according to the first haptic effect data structure.

An extended reality system is proposed comprising a presentation engine and a scene understanding module configured to manage representation of the real part of the scene including real light in a generic way. The presentation engine initializes the scene understanding module with the scene description that comprises data describing parameters to integrate the real scene including real light and the virtual part of the extended reality scene. The scene understanding module manages a scene understanding representation thanks to data obtained from sensors or databases and warns the presentation engine of updates of the scene understanding representation. A generic API is defined for interactions between the two modules.

Video coding may be performed by associating energy information with a neural network post filter group (NNPFG) supplemental enhancement information (SEI) message. For example, a device (e.g., a decoder for video decoding) may receive a bitstream. The device may determine an SEI message from the bitstream. The SEI message may indicate characteristics of an NNPFG and energy information associated with the NNPFG. The device may, e.g., on a condition that the energy information satisfies criteria, decode an image by applying one of more of the NNPF in the NNPFG to one or more image pixel values from the bitstream. The device may send the decoded image for use, for example to a display.

Some embodiments of a method may include: obtaining information describing a probabilistic space; generating a probabilistic haptic data structure corresponding to the probabilistic space, wherein the probabilistic haptic data structure comprises identification of a distribution type associated with the probabilistic space and a set of one or more parameters associated with the probabilistic space; and storing the probabilistic haptic data structure in a memory location.

Systems, methods, and instrumentalities are disclosed for energy saving in video coding. A supplemental enhancement information (SEI) message may include a first syntax element that indicates whether energy information related to a neural network post-processing filter (NNPF) is present in the SEI message. The SEI message may further include a second syntax element that indicates whether information related to a complexity or a property of the NNPF is present in the SEI message. The video data may be encoded or decoded based on the SEI message.

Neural network in-loop filter. One method for filtering a reconstructed video frame includes generating the reconstructed video frame, and applying one or more in-loop filters to the reconstructed video frame to generate a filtered reconstructed video frame, the one or more in-loop filters including a neural network in-loop filter that receives an input including a quantization parameter, the quantization parameter received in place of a default value associated with the reconstructed video frame.

A point cloud with attributes is obtained. Its geometry is encoded and decoded. Holes are detected in the decoded geometry and points are added at these locations to obtain a reconstructed geometry. The attributes of the point cloud are transferred on this reconstructed geometry and, the decoded geometry and the transferred attributes are encoded in a data stream. At the decoding side, a data stream representative of a point cloud is obtained. The geometry is decoded, and holes are detected to obtain a reconstructed geometry. The attributes of the point cloud are transferred on this reconstructed geometry to obtain a point cloud with attributes.

Methods, apparatus and data stream are provided for processing of triggers related to avatar interactions in a 3D scene, in particular pointing triggers. A scene description file is received. A user equipment or an application server parses the file and iterates on each described behavior. If triggers in the scene description are related to avatar interactions the device starts tracking the avatars and the events (expressions, gestures, gaze and/or pointing an object) referenced in these triggers. Each time an event occurs and is detected the device starts the one or more actions specified in the corresponding behavior.

Some embodiments of a method may include: obtaining information describing a haptic noise function; generating a noise data structure corresponding to the haptic noise function, wherein the noise data structure complies with a generic noise model, and wherein the generic noise model comprises identification of the haptic noise function and a set of one or more parameters associated with the haptic noise function; and storing the noise data structure in a memory location.

A point cloud with attributes is obtained. Its geometry is encoded and decoded. Holes are detected in the decoded geometry and metadata representative of coordinates of the holes are generated. In an embodiment, points are added at these locations to obtain a reconstructed geometry. The attributes of the point cloud are transferred on the decoded or on the reconstructed geometry and, the decoded geometry, the transferred attributes, and the metadata are encoded in a data stream. At the decoding side, a data stream representative of a point cloud is obtained. The geometry is decoded. In an embodiment, metadata are used to add point to the decoded geometry. The attributes of the point cloud are decoded the decoded or reconstructed geometry. In the first case, metadata are used to add points to the point cloud and their attributes are estimated according to the decoded attributes.

Systems, methods, and instrumentalities for energy saving mode in processing order. An example device may obtain encoded video data and processing order metadata that comprises an energy savings indication associated with performing a post-processing series in a processing order. The device may determine, based on the energy savings indication, whether to perform the post-processing series. The device may process the video data based on the determining.

The present disclosure proposes a neural-network post-filter NNPF that, in addition to the decoded pictures, uses as input an indication of a task (i.e. control data) which enables generating a different output for the same input decoded pictures, the output picture being specified/guided for the task or control data. For instance, the NNPF may be named a generative NNPF, as the neural-network post-filter may generate a picture specified by a particular hypothesis or control input. For instance, encoding and decoding method and/or apparatus that comprises signaling one or more syntax structures for neural-network post-filter characteristics adapted to the present principles are disclosed.

Some embodiments of a method may include: obtaining a hairstyle description corresponding to a hairstyle model, wherein the hairstyle description comprises three-dimensional (3D) orientation information and occupancy information; building a 3D array of grid cells using grid size and cell counts; filling the 3D array of grid cells with orientation and occupancy data; obtaining a target avatar head mesh registered to the hairstyle model and a hair growth region; and performing a hair growth process for each candidate strand of a full hairstyle.

Systems, methods, and instrumentalities are disclosed for bilateral filtering on intra prediction. In examples, a device (e.g., a video encoding and/or decoding device) may be configured to determine that a decoder side intra mode derivation (DIMD) mode is used to decode a block. The device may, based on the determination that the DIMD mode is used to decode the block, obtain DIMD histogram of gradients (HoG) information. The device may, based at least on the DIMD HoG information, determine whether to enable intra bilateral filtering (BIF) on the block. The device may, based on a determination to enable the intra BIF, apply the intra BIF to process (e.g., encode and/or decode) the block.

In one implementation, a video decoder obtains a base block vector for a block of a picture in an intra template matching prediction mode; decode a block vector difference for the block; obtains a block vector for the block based on the base block vector and the block vector difference for the block; obtains a prediction block for the block based on the block vector for the block; and decodes the block based on the prediction block for the block. On the encoder side, a video encoder obtains the base block vector for the block in an intra template matching prediction mode; encodes the block vector difference; obtains the block vector for the block based on the base block vector and the block vector difference; obtains the prediction block for the block based on the block vector; and encodes the block based on the prediction block.

A device (e.g., a decoder) may obtain a plurality of reference sample arrays associated with a video block. The device may determine an intra-prediction mode associated with the video block. The device may determine, based on a direction of the intra-prediction mode, a reference sample support of the intra-prediction mode, for example, from the plurality of reference sample arrays associated with the video block. The device may perform a matrix-based intra-prediction (MIP) of the video block using the reference sample support of the intra-prediction mode. The device may decode the video block based on the MIP of the video block. In examples, a reference sample support may include a plurality of primary reference samples associated with the intra-prediction mode. The device may determine an MIP matrix associated with the reference sample support and/or perform the MIP of the video block further based on the MIP matrix.

In an example haptic rendering method, an implicit neural representation model of haptic data is obtained. An interaction input to the model, and an interaction output is obtained from the model, where the interaction output representing a value of at least one haptic parameter.

Some embodiments of a method may include sending a request to start a V-DMC media stream; receiving a first offer message to start the V-DMC media stream, wherein the first offer message comprises one or more parameters related to level of detail; determining compatibility with the receiver device of the one or more parameters related to level of detail; sending a first answer message based on the determined compatibility; receiving a media stream configured according to a first set of parameters related to level of detail determined to be compatible with the receiver device; sending a second offer message, wherein the second offer message indicates a second set of parameters related to level of detail to be updated; receiving a second answer message indicating which updated parameters were applied; and receiving an adjusted media stream adjusted according to parameters of the second set that were applied.

A method and device propose, for a signal handled using feature-based INR methods, to partition the signal and specify a highest level of latent grid to be used for each partition. As a result, higher levels may not be encoded nor decoded, saving bitrate and decoding operations. This allows to decrease the bit rate and the decoding complexity of low frequency part of a signal using feature-based INR methods. Such approach has a far lower computational complexity than end-to-end neural compression approaches.

Methods, apparatus and data stream are provided for managing avatars in a 3D scene. Avatar skeleton comprising joint landmarks are encoded in a data stream. A set of joints landmarks for a given function. This function may be relative to a location of the joint landmarks of the set, like the positioning of the avatar in the 3D scene. The function may also be relative to a weighted sum of a location of the joint landmarks of the set, like the determining of a mass center.

Examples are provided herein of combining measurements and estimates for carbon emission reporting. In an example, a content provider transmits a video segment to a first set of display devices and a second set of display devices. The content provider receives energy consumption reports from the first set of display devices, each energy consumption report including energy consumption for processing and display of the video segment by a respective display device of the first set of display devices. Further, the content provider aggregates the energy consumption reports for the first set of display devices to obtain a first total energy consumption. Also, the content provider estimates a second total energy consumption for the second set of display devices. Moreover, the content provider combines the first total energy consumption and the second total energy consumption into a final total energy consumption.

In an example point cloud attribute coding method, a plurality of micropatches are obtained. Each micropatch has a plurality of pixels, and at least one pixel of each micropatch is associated with a respective point in a point cloud. Each pixel that is associated with a respective point has a pixel feature value. A first point in the point cloud is associated with respective pixels in each of a first plurality of the micropatches. These associated pixels in the first plurality of the micropatches have a first plurality of pixel feature values. A point feature value is determined for the first point by calculating a weighted sum of the first plurality of pixel feature values.