InterDigital CE Patent Holdings Patente

Systems, methods, and instrumentalities associated with fixed-filters with adaptive loop filter for deblocking filtering. The device may include a processor configured to obtain a plurality of reconstructed samples. The device may obtain a plurality of fixed adaptive loop filtered reconstructed samples by applying a fixed adaptive loop filter (ALF) on the plurality of reconstructed samples. The device may apply a deblock filter (DBF) on the plurality of fixed adaptive loop filtered reconstructed samples to obtain a plurality of deblock filtered reconstructed samples. The device may obtain a reconstructed picture based on the plurality of deblock filtered reconstructed samples.

Features described herein are associated with decoding using external pictures and related high-level syntax (HLS). In examples, a device (e.g., a video decoding device) may determine that a triggering criterion for obtaining an external picture has been met. The video decoding device may obtain the external picture from an external decoding source. The video decoding device may store the external picture in a decoded picture buffer (DPB) associated with the video content. In examples, a device (e.g., a video encoding device) may obtain an external picture from an external decoding source. The video encoding device may store the external picture in a reference picture buffer associated with the video content. The video encoding device may send an indication associated with a triggering criterion for obtaining the external picture.

In some implementations, a method is presented, comprising: assigning a first importance indication to a first data set for an Extended Reality scene, the first data set corresponding to first one or more media contents in the Extended Reality scene; and assigning a second importance indication to a second data set for the Extended Reality scene, the second data set corresponding to second one or more media contents in the Extended Reality scene, wherein the first importance indication is different from the second importance indication, and wherein the first importance indication and the second importance indication are based on at least one of a viewing distance, a viewing frustum, a screen coverage, rendering quality, and quality of experience.

A method of video decoding is disclosed. For a zone of a frame, at least one budget of use of at least one coding tool is obtained. A current block of the zone of the frame is decoded using the at least one coding tool. A value is added to at least one counter associated with the at least one coding tool. The coding tool is deactivated for decoding a rest of the zone of the frame responsive to the counter being higher than the at least one budget of use.

A first prediction mode of lowest index and a second prediction mode of highest index are obtained. A first prediction error is determined on a template for the first prediction mode and a second prediction error is determined for the second prediction mode. Responsive to the first prediction error being lower than the second prediction error, the second prediction mode is updated (a) as being equal to an average of the lowest index and the highest index and an updated second prediction error is determined. Responsive to the second prediction error being lower than the first prediction error, the first prediction mode is updated (b) as being equal to an average of the lowest index and the highest index and an updated first prediction error is determined. Steps a) and b) are repeated until an end criterion. The intra prediction mode with smallest prediction error is selected.

Systems, methods, and instrumentalities are disclosed for film grain scaling function estimation improvements. For example, a video processing device may obtain current film grain characteristics data associated with a current picture. The video processing device may obtain aggregated film grain characteristics data associated with a plurality of previous pictures. A film grain model may be determined for the current picture based on the current film grain characteristics data associated with a current picture and the aggregated film grain characteristics data associated with a plurality of previous pictures. The video processing device may include an indication of the determined film grain model for the current picture. In examples, the video processing device may combine the aggregated film grain characteristics data with the current film grain characteristics data. The film grain analysis may be performed based on the combined film grain characteristics data.

Systems, methods, and instrumentalities are disclosed for enabling partial access to an avatar representation format (ARF) file. A client device may determine a manifest (e.g., a manifest file) associated with an avatar, wherein the avatar may include a first subset of components and a second subset of components, and wherein the manifest may indicate a request for data associated with the first subset of components. The client device may transmit the manifest to a server device and receive an ARF (e.g., a partial or reduced ARF) file from the server device. The ARF file may include the data associated with the first subset of components of the avatar, and the client device may generate a partial rendition of the avatar based on the ARF file, wherein the partial rendition may include the first subset of components of the avatar.

Some embodiments of a method may include: obtaining a MPEG Avatar Representation Format (ARF)-based file, wherein the MPEG ARF-based file comprises a list identifying one or more controller sets of an avatar, wherein each of the one or more controller sets comprise one or more controllers, wherein each of the one or more controllers comprise a first list of one or more channels, and wherein at least one of the one or more channels comprises a texture, a UV mapping, or a shader property; verifying the MPEG ARF-based file comprises information corresponding to the one or more channels; parsing the information corresponding to the one or more channels into one or more respective channel data structures; and controlling a component of the avatar using at least one of the one or more channel data structures.

A method and apparatus for updating metadata of NNPF are provided. A neural-network post-filter activation (NNPFA) message associated to at least one picture of a video is received wherein the NNPFA message indicates that at least one neural-network post-filter (NNPF) is to be applied to the at least one picture. From the NNPFA, an indication is obtained that indicates whether at least one purpose of the at least one NNPF is updated. An update value of at least one parameter of the at least one NNPF is obtained and a value of the at least one parameter of the at least one NNPF is updated with the update value. The at least one NNPF is applied for the updated purpose using the at least one parameter set to the update value.

Some embodiments of a method may include: receiving a bitstream associated with a haptic signal; decoding the bitstream, wherein the decoded bitstream comprises properties used to support haptic spatialization and further comprises a list of haptic mediums at a file level, a list of haptic sources at the file level, and a list of haptic listeners at the file level; and generating the haptic signal using the decoded bitstream.

Systems, methods, and instrumentalities associated with chroma film grain processing. The device may include a processor configured to obtain a picture comprising a luma component and a chroma component. The device may perform luma film grain analysis based on the luma component of the picture. The device may determine chroma film grain data associated with the picture based on the luma film grain analysis. The device may include, in video data, an indication of the chroma film grain data associated with the picture.

Some embodiments of a method may include: obtaining a set of avatar blendshapes data; obtaining a set of avatar anatomy section names respectively corresponding to the set of avatar blendshapes; obtaining a set of metric weights respectively corresponding to the set of avatar anatomy section names; and determining a set of metric values using the set of metric weights, wherein the set of metric values correspond respectively to the set of avatar blendshapes, and wherein the set of metric values indicate a level of human perception corresponding to changes in the avatar blendshapes data.

Some embodiments of a method may include: obtaining avatar animation information, wherein the avatar animation information comprises a UV animation stream format, animation data, and a reference to a UV set; determining whether the UV animation stream format is supported; determining whether the referenced UV set supports the UV animation stream format; and animating an avatar using the animation data and the referenced UV set

A system, device and method are disclosed. The method includes obtaining an avatar animation unit (AAU) comprising a header and a payload, the obtained AAU including a gaze animation, determining if a flag indicating a renderer is positive, and rendering the gaze animation based on the determined indicated renderer.

Transform parameters of at least one UV map are obtained from a bitstream. The UV map is a list of 2D coordinates. A transform is applied on the at least one UV map based on the transform parameters to obtain a transformed UV map. A target avatar may finally be rendered based on the transformed UV map.

Systems, methods, and instrumentalities are associated with history-based template-based TIMD info for temporal TIMD merge mode. In examples, a device (e.g., a device for video encoding and/or decoding) may be configured to store template-based intra mode derivation (TIMD) information associated within a first frame as a history-based TIMD information. The first frame may be an intra frame. On a condition that an I-slice exists in a second frame that is temporally subsequent to the first frame and at least one video block of the second frame is coded in a TIMD mode or a TIMD merge mode, the device may use at least the history-based TIMD information to predict a video block of the second frame.

Systems, methods, and instrumentalities disclosed herein may be related to a carriage of avatar animation streams in ISO base media file format (ISOBMFF) containers. In examples, a device may include a processor. The device may be configured to determine an animation parameter associated with avatar information. The device may obtain an avatar animation track that comprises an animation field indicator. The animation field indicator may indicates the animation parameter associated with the avatar information. The device may send, in a media container, the avatar animation track. In examples, a device may obtain a media container. The device may determine an avatar animation track based on the obtained media container. The device may determine an animation parameter associated with avatar information based on an animation field indicator of the avatar animation track. The device may decode the avatar information based on the animation field indicator.

A method, in a receiver, is disclosed. An avatar animation bitstream is received from a transmitter. The avatar animation bitstream comprises at least one avatar animation unit (AAU). A header of the at least one avatar animation unit (AAU) is decoded. It is then determined from the header that the AAU is a silent AAU, a silent AAU indicating to the receiver that the transmitter is not sending any animation data. Responsive to the determining, a first syntax element indicating a number of ticks per second and a second syntax element indicating a duration of the silent AAU are decoded. A target avatar is rendered taking into account the first and second syntax elements.

Systems, methods, and instrumentalities associated with film model estimation. The device may include a processor configured to determine a dispersion value associated with a block in a picture. The device may determine whether to use the block for film grain analysis based on the dispersion value of the block. The device may, based on the determination to use the block for film grain analysis, add the block to a list of blocks for film grain analysis. The device may derive a film grain model for the picture based on the list of blocks for film grain analysis.

Systems, methods, and devices disclosed herein are related to haptic primitives for haptic object representation(s). A device may include a processor. The device may be configured to identify a haptic property associated with a virtual object of a virtual reality (VR) scene. The device may determine, based on at least the haptic property, at least a first primitive shape associated with the virtual object. The device may establish a propagation path through the virtual object based on at least the first primitive shape. The device may determine, based on at least the haptic property, to partition the virtual object into at least the first primitive shape and a second primitive shape. The propagation path may be established through the virtual object based at least on the first primitive shape and/or the second primitive shape. The device may determine a third primitive shape.

A system, device and method used for mapping in an animation link are disclosed. The method includes obtaining an Avatar Representation Format (ARF) comprising a preamble, the preamble including at least one supported animation with at least one animation link, parsing the at least one animation link, parsing a type and a mapping array of the parsed at least one animation link, and parsing at least one index in the parsed mapping array.

A method comprising: obtaining (70) a set of representations, each representation representing one subpart of a plurality of subparts representative of a 3D scene and being of a first type or a second type, the first type being a 3D gaussian based on a covariance matrix, the second type being a simplified 3D gaussian in which the covariance matrix is replaced by an element depending on a scalar value; obtaining (70) a set of representations, each representation representing one subpart of a plurality of subparts representative of a 3D scene and being of a first type or a second type, the first type being a 3D gaussian based on a covariance matrix, the second type being a simplified 3D gaussian in which the covariance matrix is replaced by an element depending on a scalar value; signaling (71) the set of representations in image data, the image data being organized such that all representations of a same type are signaled consecutively; and, signaling (72) at least one first information representative of a number of representations of one type among the first and the second type in the image data.

Systems, methods, and instrumentalities are disclosed for adding network architecture-related information to metadata messages associated with neural network post filters. For example, a device (e.g., video decoding device), may obtain metadata information associated with a network architecture of a neural network. The device may obtain the metadata information within a message (e.g., supplemental enhancement information (SEI) message). The metadata information may include additional information that may enable an estimation of energy needed to run a neural-network post filter (NNPF). The device may estimate an amount of energy associated with running the NNPF from the additional information. The device may apply the NNPF based on the estimated amount of energy associated with running the NNPF.

In one implementation, a method of video encoding is provided, comprising: obtaining a first parameter indicative of a quantization step size corresponding to a quantization parameter of a block, said first parameter is an adjusted version of a second parameter indicative a different quantization step size corresponding to the quantization parameter of the block, wherein the second parameter is scaled to obtain the first parameter; quantizing transform coefficients based on the first parameter to form quantized transform coefficients; and entropy coding the quantized transform coefficients for the block.

A system for haptic spatialization introduces the notion of a haptic source, haptic medium and haptic listener. A haptic source is a virtual transmitter that is able to provide haptic information. A haptic medium is an object with haptic properties based on physical characteristics, to modulate the propagation of a haptic signal through the object. A haptic listener is a virtual sensor that captures haptic information. A haptic engine handles the workflow and interactions between these different elements of the system. More particularly, it determines how a haptic signal is propagated according to the spatial localization and the properties of the elements. In addition, the propagation depends on haptic layers that correspond to different contexts. Syntax elements describing the elements composing such system allow to create information representative of a scene comprising spatialized haptic effects. Such information can be provided to a rendering device in order to be played back accordingly.

Features described herein are associated with multilayer with neural network coded external pictures and related high-level syntax (HLS). In examples, a device (e.g., a video decoding device may) obtain video data associated with layers. The device may obtain a first picture associated with a first layer of the layers. The first picture may be coded using a first codec that is based on a neural network (NN). The device may obtain a second picture associated with a second layer of the layers. The second picture may be coded using a second codec that is different from the first codec. The device may decode the second picture using the first picture and the second codec.

Systems, methods, and devices disclosed herein are related to a transform unit (TU) split interaction with template-based tools. An encoding/decoding device may include a processor. The device may be configured to receive an indication that a template-based tool is enabled for a coding unit. The device may split the coding unit into a first TU and/or a second TU. The device may determine a first prediction of the first TU based on a first template associated with the first TU. The device may determine a second prediction of the second TU based on a second template associated with the second TU. The device may encode/decode the coding unit based on the first prediction and/or the second prediction. The device may include, in video data, the encoded coding unit and/or an indication that a template-based tool is enabled for the coding unit.

A decoding method is disclosed. A first intra prediction mode is decoded for a block splitted into a plurality of subpartitions. For at least one subpartition of the block, a second intra prediction mode is selected from a set of intra prediction modes surrounding the first intra prediction mode based on template analysis. The block is decoded wherein the at least one subpartition of the block is decoded using the second intra prediction mode.

A method comprising: obtaining (30,31) a 3D Gaussian representative of a part of a 3D scene, the 3D Gaussian being based on a covariance matrix; obtaining (33) a scalar value from the 3D Gaussian; obtaining (34) a simplified representation of the 3D Gaussian wherein the covariance matrix is replaced by an element depending on the scalar value; and, signalling (35) information representative of the simplified representation of the 3D gaussian.

Apparatuses and methods are disclosed for encoding and decoding data frames. The encoding comprises training a hybrid INR network to produce a data region of a frame from latent variables. The training determines the parameters of the hybrid INR network based on a coding cost being a function of the latent variables modified by corresponding elements of a mask. The encoding further comprises selecting, using the mask, a subset of the latent variables, and coding into the bitstream the selected subset of the latent variables as well as the parameters of the hybrid INR network and the mask. The decoding comprises decoding from the bitstream the parameters of the hybrid INR network and the mask that indicates the subset of the latent variables. The latent variables are then obtained, where a latent variable belonging to the subset is decoded from the bitstream based on its distribution and a latent variable not belonging to the subset is derived based on its distribution. The data region of the frame is produced by the hybrid INR network from the obtained latent variables using the decoded parameters of the hybrid INR network.

Methods and apparatus are provided for in-loop filtering. In one embodiment, the filter coefficients are derived using templates of samples around a sample. In another embodiment, multiple templates are used around a block containing a current sample. The coefficients providing a minimized error between filtered and non-filtered templates can be used for filtering. The embodiments are suitable for in-loop filtering and coefficients can be derived by a decoder using reconstructed samples.

A method and apparatus for video encoding/decoding are provided. For a current block of an image of a video encoded using a compound intra prediction mode, a first intra predictor is obtained from among a first set of intra predictors wherein the first set of intra predictors comprising at least one intra predictor being a neural-network-based intra predictor. At least one second intra predictor is obtained from among a second set of intra predictors. A prediction is obtained for the current block by combining the first intra predictor and the at least one second intra predictor and the current block is encoded or decoded based on the prediction. In a variant, a planar mode is used as a representative of the neural-network-based intra predictor for cost calculation.

Systems, methods, and instrumentalities associated with intra template matching prediction with persistent template characteristics are provided. The device may include a processor configured obtain template matching type information associated with a plurality of positions in a reconstructed area of a picture. The device may predict a current block based on the template matching type information. The device may decode or encode the current block based on the prediction of the current block. In examples, the device may perform a first intra template matching prediction (ITMP) based on a first template matching type to identify a first position. The device may associate the first position with the first template matching type. The device may perform a second ITMP based on a second template matching type to identify a second position. The device may associate the second position with the second template matching type.

A video coding device such as a video decoder or a video encoder may determine a block vector (BV) for a current pixel of a current video block and obtain, based on the BV, one or more reference samples for the current pixel. The coding device may reconstruct the current pixel of the current video block to obtain a reconstructed current pixel and apply an adaptive loop filter (ALF) to the reconstructed current pixel, wherein an input to the ALF may be determined based on the one or more reference samples obtained using the BV.

In an example method, a scene description file representing an avatar is obtained, the scene description file including a texture sets component. The texture sets component includes at least one texture set. Each of the texture sets identifies at least one texture. At least a first one of the texture sets identifies a first texture and a second texture. A stream of texture weights is obtained including first weights associated with the first texture and second weights associated with the second texture. A parametric texture is determined for the avatar using a combination of the textures in the first texture set, including the first texture weighted by the first weights and the second texture weighted by the second weights. The avatar is animated using the parametric texture.

Apparatuses and methods are disclosed including techniques for encoding and decoding video content. Techniques disclosed include reconstructing samples of a video frame of the video data to be encoded or decoded. The video frame is divided into full-regions and partial-regions according to a grid, where the partial-regions are located along the boundary of the video frame. Then, in-loop filtering is applied to the reconstructed samples, where for a full-region adjacent to a partial-region, the in-loop filtering comprises generating an extra processing region from reconstructed samples of the partial-region, and filtering the full-region using the extra processing region.

In various implementations, methods and devices are disclosed that process one or more syntax elements for in-loop filtering a reconstructed image region using a neural network loop filter NNLF. The one or more syntax elements may comprise an identifier of a NNLF model type for the NNLF among a reduced list of five NNLF model types; an indication specifying a size, in log 2 minus 2, in width and height of the image region processed by the NNLF; an indication specifying a size, in log 2 minus 2, in width and height of an extension of the image region processed by the NNLF or an indication specifying a maximum number minus 1 of conditional scaling parameter candidates of the image region processed by the NNLF.

Gaze signaling in an avatar representation format. A method including: receiving, with a decoder, an eye animation stream format and animation data, determining, with the decoder, whether the eye animation stream format is supported by the decoder, responsive to determining that the eye animation stream format is supported by the decoder, determining, with the decoder, whether the animation data includes an eye set that is supported by the eye animation stream format, responsive to determining that the animation data includes the eye set that is supported by the eye animation stream format, determining, with the decoder, whether the animation data includes gaze properties associated with the eye set, and ,responsive to determining that the animation data includes the gaze properties associated with the eye set, initiating, with the decoder, an animation of an avatar with the animation data and the gaze properties.

Systems, methods, and instrumentalities may be associated with sub-block based spatial block vector derivation. In examples, a device (e.g., a device for video decoding) may obtain a block vector (BV) associated with a first subblock of a video block. The video block may be adjacent to a current video block. The device may obtain a subblock merge candidate based on the BV. The device may generate a merge candidate list including the subblock merge candidate. The device may predict a subblock of the current video block based on the merge candidate list. The device may decode the current video block based on the predicted subblock.

In an example method, a set of functions to be performed in an extended reality, XR, session is determined, set including an inpainting function, a masking function, and/or a shadow removal function. A user equipment, UE, sends a capabilities message to an application server. The capabilities message indicates, for each of the functions, whether the UE has a capability to perform the respective function. The server sends to the UE an assignment message indicating, for each function, whether the UE or the application server is assigned to perform the respective function. A set of objects to be removed from a scene is determined, and an XR session is provided in which the determined set of objects are removed from the scene. In providing the session, the functions of the set of functions are performed according to the assignment message.