InterDigital CE Patent Holdings Patente

Encoding and decoding information for generation of dynamic three-dimensional (3D) clothing. For example, a method includes receiving, with a decoder, a plurality of parameters indicating a body motion, a body shape, and a characteristic of a clothing material and generating, with the decoder, a dynamic three-dimensional clothing based on the plurality of parameters, wherein the dynamic three-dimensional clothing is dynamic with respect to the body motion and the body shape.

A decoding method is disclosed. Gradients for a current block are first obtained by filtering samples in a neighboring template of the current block, wherein obtaining gradients for a current block comprises substituting at least one of a Sobel filter or an edge detection filter by a generic filter. A histogram of oriented gradients (HoG) is then obtained for the current block based on the gradients. An intra mode is selected for the current block based on the HoG. The current block is then decoded using the selected intra mode.

In an example method for use in a split inferencing architecture, information is obtained at a first endpoint describing at least a first sub-model of a split-inferencing machine learning model. In communication with a second endpoint, a tensor indexing algorithm is selected. The first sub-model is executed to obtain at least one tensor. The tensor is sent to the second endpoint, each tensor being identified by a respective reference determined according to the selected tensor indexing algorithm.

In an example of a split inferencing method, a first endpoint obtains information describing a graph structure of a machine learning, ML, model with a plurality of nodes whose outputs are tensors, each tensor having an original branch name. The first endpoint obtains information indicating a split of the model into first and second sub-models. For at least one of the tensors that is an output of a node in the first sub-model, the first endpoint sends, to a second endpoint, information associating the original branch name of the first tensor with a replacement branch name. The first endpoint runs the first sub-model to obtain the first tensor, and it sends the first tensor to the second endpoint, the first tensor being identified by the replacement branch name. The second node may run the second sub-model using the first tensor as an input.

In an example of a split-inferencing method, a first endpoint obtains information describing at least a first sub-model of a split-inferencing machine learning model. In communication with a second endpoint, a tensor sorting algorithm is selected from among a plurality of available tensor sorting algorithms. The first endpoint runs the first sub-model to obtain a plurality of tensors, and it sends the tensors to the second endpoint in an order determined by the selected tensor sorting algorithm. The tensors may be sent without tensor names. The second endpoint may interpret the tensors based on the selected tensor sorting algorithm and run the second sub-model.

Method and apparatus are described wherein a filtered reconstructed image block from a prediction block and a reconstructed prediction residual is obtained and a Quantization Constrained correction is applied to the filtered reconstructed image block, wherein the Quantization Constrained correction applied depends on whether at least one part of transformed coefficients decoded for the image block have been zeroed out during encoding of the image block. In some embodiments, QC correction is skipped for these transformed coefficients. In some variants, scanning of the transformed coefficients is adapted for the image block.

A method comprising: obtaining (900) YUV data resulting from a conversion of a YUV signal with a first transfer function and a first color gamut to a YUV signal with a second transfer function and a second color gamut; and applying a correction to a component of the YUV data to limit effects of quantization errors.

Some embodiments of a method may include: obtaining a MPEG Avatar Representation Format (MARF)-based file; parsing the MARF-based file into a data structure; selecting an asset from the data structure; verifying the asset comprises a controller set property; obtaining controller set data corresponding to the controller set property; obtaining container data using the controller set data; determining a mime type using the controller set data; decoding the container data using the mime type; obtaining a mesh and a skeleton using the container data; and animating an avatar corresponding to the mesh and the skeleton.

A system, codec and method for encoding parametric haptic data are disclosed. This includes providing a property designating an effect of a parametric model including a type of signal, providing in the effect of a parametric model a designation of a symmetry of the signal, indicating properties in the parametric model to define a pulse including pulse width and interphase delay and providing properties in a plurality of haptic effect patterns including effect type, number of pulses, cycle offset, and number of cycles. The property may be monopolar or bipolar. The designation may be symmetric or asymmetric. The interphase delay may indicate symmetric or asymmetric. The effect type may indicate the pattern via a set of keyframes. The number of pulses may indicate pulses in one primitive. The cycle offset may indicate a time between two cycles. The number of cycles may indicate cycles in the pattern effect.

A device (e.g., video encoder or video decoder) may determine transform sets for MTSS. For example, an MTSS indication (e.g., MTSS flag) may be obtained. The MTSS flag may be used to determine a transform set. A histogram of gradients (HoG) may be obtained. A VIPM (e.g., first VIPM) may be obtained, for example, based on the HoG. Whether the MTSS indication is enabled may be determined. Based on a determination that the MTSS indication is not enabled, a mode-specific transform set corresponding to VIPM may be obtained. Inverse transform may be performed based on the obtained transform set.

Systems, methods, and instrumentalities are disclosed for avatar keypoints and landmarks representation. A file (e.g., an MPEG avatar representation format (MARF) file) may be received. The file may include a plurality of assets. The file may indicate keypoint and landmark component information. An asset may be selected from the plurality of assets in the file based on parsing the file. A container may be determined based on the selected asset and the keypoint and landmark component information. At least one of a keypoint or a landmark may be determined based on the determined container. At least one of a keypoint or a landmark may animate an avatar based on the at least one of the keypoint or landmark.

A system, codec and method of optimizing an MPEG haptics synthesizer are disclosed. The method includes determining a minimum sampling rate, generating a haptic signal with the determined minim sampling rate and upsampling the generated haptic signal to synthesize a final haptic signal. The optimizing may include determining a minimum sampling rate by an end user, by an automatic computation. The optimizing may include upsampling of the generated haptic signal based on a target sampling rate. The optimizing may include the defined by content creator causes a value of the minimum sampling rate to be stored with haptic data. The optimizing may include, in a context of the MPEG haptic standard, the minimum sampling rate being part of the normative part of the specification, the minimum sampling rate being stored as an optional value of a haptic experience level, or the minimum sampling rate being stored for each band.

Systems, methods, and/or instrumentalities are disclosed herein for a device that may be related to adaptive loop filter residual scaling with optimal initialization. In examples, a device may include a processor. The device may determine a correction of a reconstructed picture sample. The device may determine a classification associated with the reconstructed picture sample. The device may, based on the classification, determine a first scaling parameter. The device may determine a second scaling parameter based on the first scaling parameter. The device may generate a scaled correction based on the second scaling parameter and the correction. The device may apply the scaled correction to the reconstructed picture sample.

Systems, methods, and instrumentalities are configured for transform derivation using decoder-side intra mode derivation (DIMD) for one or more of template-based intra mode derivation (TIMD) mode or DIMD mode. In examples, a video coding device may be configured to determine that a template-based intra mode derivation (TIMD) mode is enabled for a current block. The device may apply a decoder side intra mode derivation (DIMD) like process to derive a virtual intra prediction mode (VIPM) from a neighboring reconstructed template of a current block or a prediction block associated with the current block. The device may, based on the derived VIPM, determine a transform kernel set for predicted residual coding.

Systems, methods, and instrumentalities are disclosed related to external data in avatar representation formats. In examples, a decoding/encoding device may include a processor. The device may be configured to determine that a data file comprises a plurality of data components. The device may determine that a data component of the plurality of data components is associated with a first data format that an application does not support. The device may obtain a link to access an external data content associated with a second data format that the application supports. The device may obtain external data of the external data content using the link. The device may decode the external data.

Systems, methods, and instrumentalities for an energy saving mode indicator. An example device may receive a bitstream that includes encoded video data and metadata that indicates an energy saving indication. The device may determine, based on the energy saving indication, whether the encoded video data has been pre-processed for energy savings. The device may process the encoded video data based on the determining.

A haptics decoding method according to an example embodiment includes obtaining a file representing a haptic experience. The file includes at least a first affective model data structure identifying an affective model. The file further includes at least a first haptic effect data structure including a first physical description of a first haptic effect. The first haptic effect data structure further includes a first affective response data structure defining a first affective response according to the identified affective model. A rendered haptic effect is provided to a user according to the first haptic effect data structure.

An extended reality system is proposed comprising a presentation engine and a scene understanding module configured to manage representation of the real part of the scene including real light in a generic way. The presentation engine initializes the scene understanding module with the scene description that comprises data describing parameters to integrate the real scene including real light and the virtual part of the extended reality scene. The scene understanding module manages a scene understanding representation thanks to data obtained from sensors or databases and warns the presentation engine of updates of the scene understanding representation. A generic API is defined for interactions between the two modules.

Video coding may be performed by associating energy information with a neural network post filter group (NNPFG) supplemental enhancement information (SEI) message. For example, a device (e.g., a decoder for video decoding) may receive a bitstream. The device may determine an SEI message from the bitstream. The SEI message may indicate characteristics of an NNPFG and energy information associated with the NNPFG. The device may, e.g., on a condition that the energy information satisfies criteria, decode an image by applying one of more of the NNPF in the NNPFG to one or more image pixel values from the bitstream. The device may send the decoded image for use, for example to a display.

Some embodiments of a method may include: obtaining information describing a probabilistic space; generating a probabilistic haptic data structure corresponding to the probabilistic space, wherein the probabilistic haptic data structure comprises identification of a distribution type associated with the probabilistic space and a set of one or more parameters associated with the probabilistic space; and storing the probabilistic haptic data structure in a memory location.

Systems, methods, and instrumentalities are disclosed for energy saving in video coding. A supplemental enhancement information (SEI) message may include a first syntax element that indicates whether energy information related to a neural network post-processing filter (NNPF) is present in the SEI message. The SEI message may further include a second syntax element that indicates whether information related to a complexity or a property of the NNPF is present in the SEI message. The video data may be encoded or decoded based on the SEI message.

Video coding may be performed by removing wasteful computations in neural networks with sparse weights. For example, a device (e.g., a decoder for video decoding) may determine a neural network (NN). The NN may be associated with sparse weights. The NN may comprise an architecture and a set of parameters. The device may determine a set of sparse matrices of weights inside the NN that satisfies a criterion. The device may determine a modified set of matrices of weights, for example, by processing each sparse matrix of weights in the set of sparse matrices of weights, such that one or more non-zero rows and/or one or more non-zero columns from the corresponding sparse matrix may be transferred to the modified matrix of weights.

Neural network in-loop filter. One method for filtering a reconstructed video frame includes generating the reconstructed video frame, and applying one or more in-loop filters to the reconstructed video frame to generate a filtered reconstructed video frame, the one or more in-loop filters including a neural network in-loop filter that receives an input including a quantization parameter, the quantization parameter received in place of a default value associated with the reconstructed video frame.

A device may transmit a request for a portion of an avatar three-dimensional (3D) representation, the request based on a representation format that defines access to a predefined range of data of the avatar three-dimensional (3D) representation. A device may receive the portion of the avatar 3D representation or access to the portion of the avatar 3D representation.

A point cloud with attributes is obtained. Its geometry is encoded and decoded. Holes are detected in the decoded geometry and points are added at these locations to obtain a reconstructed geometry. The attributes of the point cloud are transferred on this reconstructed geometry and, the decoded geometry and the transferred attributes are encoded in a data stream. At the decoding side, a data stream representative of a point cloud is obtained. The geometry is decoded, and holes are detected to obtain a reconstructed geometry. The attributes of the point cloud are transferred on this reconstructed geometry to obtain a point cloud with attributes.

Methods, apparatus and data stream are provided for processing of triggers related to avatar interactions in a 3D scene, in particular pointing triggers. A scene description file is received. A user equipment or an application server parses the file and iterates on each described behavior. If triggers in the scene description are related to avatar interactions the device starts tracking the avatars and the events (expressions, gestures, gaze and/or pointing an object) referenced in these triggers. Each time an event occurs and is detected the device starts the one or more actions specified in the corresponding behavior.

Some embodiments of a method may include: obtaining information describing a haptic noise function; generating a noise data structure corresponding to the haptic noise function, wherein the noise data structure complies with a generic noise model, and wherein the generic noise model comprises identification of the haptic noise function and a set of one or more parameters associated with the haptic noise function; and storing the noise data structure in a memory location.

Systems, methods, and instrumentalities for energy saving mode in processing order. An example device may obtain encoded video data and processing order metadata that comprises an energy savings indication associated with performing a post-processing series in a processing order. The device may determine, based on the energy savings indication, whether to perform the post-processing series. The device may process the video data based on the determining.

A point cloud with attributes is obtained. Its geometry is encoded and decoded. Holes are detected in the decoded geometry and metadata representative of coordinates of the holes are generated. In an embodiment, points are added at these locations to obtain a reconstructed geometry. The attributes of the point cloud are transferred on the decoded or on the reconstructed geometry and, the decoded geometry, the transferred attributes, and the metadata are encoded in a data stream. At the decoding side, a data stream representative of a point cloud is obtained. The geometry is decoded. In an embodiment, metadata are used to add point to the decoded geometry. The attributes of the point cloud are decoded the decoded or reconstructed geometry. In the first case, metadata are used to add points to the point cloud and their attributes are estimated according to the decoded attributes.

Procedures, methods, architectures, apparatuses, systems, devices, and computer program products for energy-aware content distribution. A client device receives, from a content server, information corresponding to a manifest file for a content item, the manifest file including energy consumption information for provision of the content item for at least one of a plurality of versions of the content item, selects a version of the content item whose corresponding energy consumption information matches a device profile of the client device and satisfies at least one energy control criterion, and downloads units of the selected version of the content item.

In various implementations, methods and devices are disclosed that improve the diversity of the predictors used in the blending process of DIMD. For instance, predictors may be obtained by applying derived intra modes to different reference lines instead of the reference line closest to the current block. For instance, predictors may be obtained by computing independently a Histogram of Gradients per reference line. For instance, predictors obtained by computing independently a Histogram of Gradients per reference line may be used in template intra mode derivation TIMD.

Systems, methods, and instrumentalities may be configured for mesh subdivision prediction. In examples, a coding device (e.g., an encoding and/or decoding device) may be configured to determine whether a mesh edge of a mesh sequence wherein a vertex is to be inserted is a non-boundary edge. The device may, based on the determination that the mesh edge of the mesh sequence is a non-boundary edge, select one of an arithmetic mean, a geometric mean, or a harmonic mean to predict a position of the vertex based on comparing an absolute value of a dot product of normal vectors with at least one of a first threshold value or a second threshold value.

Systems, methods, and instrumentalities are disclosed for bilateral filtering on intra prediction. In examples, a device (e.g., a video encoding and/or decoding device) may be configured to determine that a decoder side intra mode derivation (DIMD) mode is used to decode a block. The device may, based on the determination that the DIMD mode is used to decode the block, obtain DIMD histogram of gradients (HoG) information. The device may, based at least on the DIMD HoG information, determine whether to enable intra bilateral filtering (BIF) on the block. The device may, based on a determination to enable the intra BIF, apply the intra BIF to process (e.g., encode and/or decode) the block.

In one implementation, a video decoder obtains a base block vector for a block of a picture in an intra template matching prediction mode; decode a block vector difference for the block; obtains a block vector for the block based on the base block vector and the block vector difference for the block; obtains a prediction block for the block based on the block vector for the block; and decodes the block based on the prediction block for the block. On the encoder side, a video encoder obtains the base block vector for the block in an intra template matching prediction mode; encodes the block vector difference; obtains the block vector for the block based on the base block vector and the block vector difference; obtains the prediction block for the block based on the block vector; and encodes the block based on the prediction block.

Systems, methods, and instrumentalities are disclosed herein for dictionary learning for implicit neural compression (INR). In examples, a video decoder may decode an implicit neural representation (INR) parameter. An INR may be reconstructed based on the INR parameter. The decoder may compute a dictionary based on the reconstructed INR. A dictionary-based INR may be generated based on the computed dictionary. Data may be reconstructed based on the dictionary-based INR. in examples, the decoder may decoder a training procedure used for the computation of the dictionary. The dictionary may be computed based (e.g., further based) on the training procedure. In examples, a video encoder may determine an INR parameter. The dictionary may be computed based on the INR parameter. The encoder may determine a dictionary-based INR parameter based on the computed dictionary. The INR parameter and the dictionary-based INR parameter may be encoded.

Methods, apparatus and data stream are provided for managing avatars in a 3D scene. Avatar skeleton comprising joint landmarks are encoded in a data stream. A set of joints landmarks for a given function. This function may be relative to a location of the joint landmarks of the set, like the positioning of the avatar in the 3D scene. The function may also be relative to a weighted sum of a location of the joint landmarks of the set, like the determining of a mass center.

A method and device propose, for a signal handled using feature-based INR methods, to partition the signal and specify a highest level of latent grid to be used for each partition. As a result, higher levels may not be encoded nor decoded, saving bitrate and decoding operations. This allows to decrease the bit rate and the decoding complexity of low frequency part of a signal using feature-based INR methods. Such approach has a far lower computational complexity than end-to-end neural compression approaches.

Some embodiments of a method may include: obtaining a hairstyle description corresponding to a hairstyle model, wherein the hairstyle description comprises three-dimensional (3D) orientation information and occupancy information; building a 3D array of grid cells using grid size and cell counts; filling the 3D array of grid cells with orientation and occupancy data; obtaining a target avatar head mesh registered to the hairstyle model and a hair growth region; and performing a hair growth process for each candidate strand of a full hairstyle.

In an example haptic rendering method, an implicit neural representation model of haptic data is obtained. An interaction input to the model, and an interaction output is obtained from the model, where the interaction output representing a value of at least one haptic parameter.

In an example point cloud attribute coding method, a plurality of micropatches are obtained. Each micropatch has a plurality of pixels, and at least one pixel of each micropatch is associated with a respective point in a point cloud. Each pixel that is associated with a respective point has a pixel feature value. A first point in the point cloud is associated with respective pixels in each of a first plurality of the micropatches. These associated pixels in the first plurality of the micropatches have a first plurality of pixel feature values. A point feature value is determined for the first point by calculating a weighted sum of the first plurality of pixel feature values.