InterDigital CE Patent Holdings Patente

Methods and apparatus are provided for adaptive quantization parameter selection based on a cost using a neural network. In one embodiment, the neural network can be trained from a training database comprising numerous coding units or blocks. In another embodiment, a rate distortion cost is determined for combinations of quantization parameter and split configuration. The quantization parameter with a minimal rate distortion cost for a particular split is used for encoding. In another embodiment, rate and distortion are separately approximated, resulting in lowest rate cost and lowest distortion cost for each possible type of split.

In various implementations, methods and devices are disclosed that process one or more syntax elements for reconstructing at least one part of an picture end-to-end coded using a neural network. The one or more syntax elements comprise at least one of an indication (ph_nn_allowed_flag) enabling end-to-end neural network coding for at least a part of the picture; an indication specifying (ph_only_nn_slice) that all parts of the picture are end-to-end neural network coded; or an indication (sh_nn_flag) enabling end-to-end neural network coding at a given level of the picture, the given level comprises a slice level, a coding tree level, a block level.

Methods and devices enable an adaptive haptic experience that endows users with adjustments to their profile or to their physiological reactions. In such context, data representative of a haptic experience not only contains haptic data but also affective data. This additional data indicates the desired affective response of the user during the haptic experience. Two embodiments for adapting a haptic signal to provide consistency in users perceived affect to a haptic stimulus are proposed herein. In a first embodiment, the adaptation is based on an affective haptic profile that describes the user's preferred settings regarding haptic content and affect and comprises parameters allowing to adapt a haptic signal accordingly. In a second embodiment, the adaptation takes into account real time data extracted from users' physiological reactions, thus delivering a physiological closed loop control system.

In example methods and apparatus, during an extended reality, XR, spatial computing session between the UE and a server, at least a first sensor data stream is provided from the UE to the server using at least a first sensor data setting. Information indicating an updated first sensor data setting is communicated between the UE and the server. In response to the communication between the UE and the server, the first sensor data setting is updated to the updated first sensor data setting. The UE then continues to provide the first sensor data stream from the UE to the server using the updated first sensor data setting.

A method comprising:obtaining (40) at least two video sequences;determining (41) a first reference level value for each video sequence, each first reference level value being representative of a luminance level of a same area of interest in each video sequence;obtaining (43) a second reference level value; and,modifying (44) at least one of the video sequences so that, for each modified video sequence, the luminance level of the same area of interest in the video sequence is equal to the second reference level value using a conversion curve depending on the first reference level value of the video sequence and on the second reference level value.

Some embodiments of a method may include: obtaining avatar format data associated with an avatar; parsing the avatar format data for a textures structure; and performing a loop for each texture element of the textures structure, wherein the loop includes: determining a U-Dimensional (UDIM)-based extension exists within the textures structure; parsing data associated with the UDIM-based extension; determining a u-coordinate (uValue) attribute exists within the texture element; parsing the uValue attribute; determining a v-coordinate (vValue) attribute exists within the texture element; parsing the vValue attribute; determining a source attribute exists within the texture element; and parsing the source attribute.

Method and apparatus for encoding and decoding an input signal, for example an image, a video or a 3D object, using a hybrid Implicit Neural Representation network are provided. The method comprises obtaining one or more syntax elements providing for reconstructing at least one part of the input signal using at least one hybrid Implicit Neural Representation network, the at least one hybrid Implicit Neural Representation network comprising: at least one mapping transformation that maps input coordinates of the image or 3D object to at least one grid of features, and an INR network that takes as input, features of the at least one grid of features and outputs values of the image or 3D object at the input coordinates, wherein the one or more syntax elements comprises a first syntax element indicating whether a first grid of features of the at least one grid of features has same dimensions as dimensions of the input coordinates, and signaling the one or more syntax elements in a bitstream.

Method and apparatus are provided wherein one or more syntax elements are decoded providing for reconstructing at least one part of an image or a 3D object using at least one hybrid Implicit Neural Representation network, the at least one hybrid Implicit Neural Representation network comprising at least one mapping transformation that maps input coordinates of the image or 3D object to at least two successive grids of features, and an INR network that takes as input, features of the at least one grid of features and outputs values of the image or 3D object at the input coordinates, wherein dimensions of a current grid of the at least two successive grids of features are derived from a first grid of the at least two successive grids of features based on the one or more syntax elements. An extension of at least one dimension of a first grid of the at least two successive grids of features is determined providing an extended dimension of the first grid of the at least two successive grids of features such that dimensions of grids of the at least two successive grids of features are deteremined using the extended dimension of the first grid and input coordinates are mapped to grid positions of the grids of the at least two successive grids of features to reconstruct the at least one part of the image or 3D object.

In one general aspect, a method is provided that includes receiving from a first user device, an animation data stream for animating an avatar associated with the first user device, the avatar being part of a shared scene of the multi-user avatar-based session between the first user device and at least one second user device; determining for the at least one second user device, whether the at least one second user device has capabilities to animate the avatar associated with the first user device using the animation data stream. If the at least one second user device has capabilities to animate the avatar, the animation data stream is forwarded to the at least one second user device, otherwise the animation data stream is converted based on at least one second user device's capability before being sent to the at least one second user device.

Systems, methods, and instrumentalities for an energy savings credit based on display dimming. An example video decoding device may determine that a user of a display has caused the display to dim. The device may calculate an amount of energy saved by dimming the display. The device may determine an energy savings credit based on the amount of energy saved by dimming the display. The device may store the energy savings credit in an account associated with the user.

Systems, methods, and instrumentalities are disclosed to save watts from a device idleness. An energy saving device may be configured to reduce carbon emissions of a user. The device may include a processor. The device may determine that a user of the device has caused the device to idle or switch off. The device may calculate an amount of energy saved by idling the device. The device may determine an energy savings credit based on the amount of energy saved by idling the device. The device may store the energy savings credit in an account associated with the user. In examples, the device may determine that the user of the device has caused the device to idle comprises the processor being configured to determine that the user has idled or switched off a display of the device. The device may monitor user activity associated with idleness of the device.

In one implementation, a method of decoding a signal comprises: decoding information indicating partitioning of the signal into a plurality of partitions; for each partition of the plurality of partitions: decoding information indicating association of the partition to a respective processing model of a plurality of processing models; decoding latent variables; processing the latent variables based on the respective processing models to form processed latent variables; and reconstructing the signal from the processed latent variables. At the encoder side, a method comprises: encoding information indicating partitioning of the signal into a plurality of partitions; for each partition of the plurality of partitions: encoding information indicating association of the partition to a respective processing model of a plurality of processing models; obtaining latent variables associated with coordinates of samples of the signal; encoding the latent variables; and encoding model parameters for the plurality of processing models.

A device may provide media content as a digital reward. The device may determine a saved balance value associated with a quantity of saved watts associated with an operator of the device. The device may determine one or more digital rewards based on the saved balance value by querying a database. The device may display an indication of the one or more digital rewards. Each of the digital rewards may be displayed with an associated cost. The device may receive input selecting one of the one or more digital rewards and may update the saved balance value to reflect the selected one of digital rewards. The device may communicate an indication of the selected one of the one or more digital rewards and may receive digital content associated with the selected one of the one or more digital rewards. The device may display the received digital content.

An energy saving device may be configured to reduce carbon emissions of a user. The device may include a processor. The device may obtain media content. The device may estimate a first energy consumption value based on the media content. The device may receive a user input indicating a request to adjust the first energy consumption value or adjust a quality level associated with the media content. The device may, based on the user input: apply a video processing algorithm to the media content to obtain a modified media content, estimate a second energy consumption value based on the modified media content, and/or compare the first and the second energy consumption values to generate an estimated energy savings. The device may accumulate and/or store the estimated energy savings in an energy savings account.

A first device may include a processor configured to perform one or more of the following. The first device may receive replacement configuration information. The replacement configuration information may include replaceable content information and/or replacement mode information. The replaceable content information may indicate at least a first replaceable content associated with a first replacement value. The replacement mode information may indicate at least a first replacement mode. The first device may display a user interface that indicates one or more of the following: a saved balance value, at least a portion of the replaceable content information, and/or at least a portion of the replacement mode information. The first device may receive a user input indicating a selected replacement mode. The first device may determine an updated saved balance value based on the user input. The first device may generate replacement content based on the user input. The first device may display the generated replacement content.

A method and apparatus for Quantization constrained correction with dependent quantization are provided. In some embodiments, a reconstructed image block is determined from a prediction block and a reconstructed prediction residual, and a filtered version of the reconstructed image block is obtained. A quantization index of the dependent quantization is obtained for at least one transform coefficient of a transform block determined for a residual obtained from the filtered reconstructed image block. At least one correction bound is determined based on the quantization index, a bound offset and a quantization step size of the dependent quantization, and the at least one transform coefficient is corrected as a function of the at least one correction bound providing at least one corrected transform coefficient.

A method for converting a first signal in a HDR domain in a second signal in a SDR domain comprising:determining (602) a first tone mapping curve from an analysis of the first signal;modifying (80) the first tone mapping curve to obtain a second tone mapping curve passing through at least one reference level, a reference level representing a couple of a HDR luminance value and a SDR luminance value; and,converting (603) a luminance component of the first signal into a luminance component of the second signal using the second tone mapping curve.

Some embodiments of a method may include: obtaining a MPEG Avatar Representation Format (ARF)-based file, wherein the MPEG ARF-based file comprises information corresponding to a set of blendshapes to animate an avatar; populating a blendshapes data structure with the information corresponding to a description of the set of blendshapes, wherein the blendshapes data structure comprises an animation framework type, an animation framework index, and a semantic mapping; and animating the avatar using the populated blendshapes data structure.

In an embodiment, a method, implemented in a network node, comprises receiving, from a media application provider, a first message comprising first information indicating an energy exposing policy template including an energy exposing specifications; determining whether the energy exposing specifications are supported by policy resources of an application function (AF); on condition that the energy exposing specifications are supported by the policy resources of the AF, determining a policy template resource of the AF including the energy exposing specifications; and providing one or more media delivery sessions to a wireless transmit/receive unit (WTRU) based on the determined policy template resource of the AF.

A wireless transmit and receive unit (WTRU) and method for energy data collection reporting are disclosed. The WTRU includes a processor and a transceiver communicatively coupled to the processor. The processor and transceiver operating to monitor an energy consumption and/or a carbon emission of a provisioning session. The monitoring may occur at one of a plurality of levels including at a provisioning level, a media delivery session level, and a component level of a media delivery session. The processor and transceiver may further operate to receive reports on energy related information for at least one of a PDU session component, a PDU session and a slice. The received reports may be at least one level of granularity selected from a WTRU level, a RAN level, a CN level during downlink or uplink media delivery sessions. The WTRU may further include a media session handler configured to perform collecting and reporting of energy consumption. The WTRU may further include a media player configured to receive and play video content within the WTRU. The WTRU may further include a display configured to receive a stream from the media player to depict the stream to a user. The reporting may occur via an EoS metrics reporting configuration. The EoS metrics reporting configuration may define rules for EoS metrics collection. The WTRU may request the AF to retrieve energy metrics for the media player level. The energy metrics may include at least one of carbon intensity, energy consumption, and power consumption. The reporting may occur via a EoS metric scheme. The reporting may be capable of accounting for privacy issues associated with client data.

In an embodiment, a method implemented in a network node comprises receiving, from a media application provider, a first message comprising first information indicating an energy controlling policy template including an energy controlling specifications; determining whether the energy controlling specifications are supported by policy resources of an application function (AF); on condition that the energy controlling specifications are supported by the policy resources of the AF, determining an energy policy template resource of the AF based on the energy controlling specifications; and providing, to a wireless transmit/receive unit (WTRU), the energy policy template resource of the AF for one or more media delivery sessions with the WTRU.

Some embodiments of a method may include: obtaining a set of blendshape weights; determining a streaming format is supported; determining rescaling of one or more of the set of blendshape weights is to be done for the streaming format; rescaling one or more of the blendshape weights in the set of blendshape weights for the streaming format; and animating an avatar using the rescaled set of blendshape weights.

Methods and apparatus are provided for adding new data types to the Avatar Representation Format data property. The type property is a string providing the mime types of the data. At least one embodiment refers to different signaling ways for the geometry objects using one or more data types. The embodiments define one single signaling way of the Mesh object and add the geometry properties as a Data property. The embodiments then switch between the different signaling ways of the geometry objects by changing the value of type in mesh data. In this case the definition of the Geometry containers is external to the ARF description.

Some embodiments of a method may include: obtaining a MPEG Avatar Representation Format (ARF)-based file, wherein the MPEG ARF-based file comprises information corresponding to one or more assets of an avatar, and wherein the information corresponding to the one or more assets comprises a respective identifier for each of the one or more assets; verifying each of the respective identifiers is unique; and populating a data structure with the information corresponding to the one or more assets.

Systems, methods, and instrumentalities are disclosed for multiple transform set selection (MTSS) for spatial geographic partition mode (SGPM). A device for video decoding/encoding may include a processor. The device may be configured to obtain a plurality of candidate intra prediction modes for a video block. The device may determine whether a partition direction associated with a SGPM or an intra prediction mode of the plurality of candidate intra prediction modes is to be used to obtain a first transform set associated with the video block. The device may obtain a second transform set associated with the video block based on the determination of whether the partition direction or the intra prediction mode is to be used to obtain the first transform set. The device may decode/encode the video block using the first and second transform sets.

In a method according to some embodiments, a plurality of potential viewpoints of a 3D Gaussian model are selected. For a first subset of the plurality of potential viewpoints, a first determination is made of a number of 3D Gaussians contributing to rendering of a synthetic view at the respective viewpoint. For a second subset of the plurality of potential viewpoints, a second determination is made of whether any incompletely filled pixels are present in a rendering of a synthetic view at the respective viewpoint. A a first region of potential captured viewpoints is selected based at least in part on the first determination and the second determination.

In an embodiment, a method for video decoding is provided, the method comprising: decoding a current picture of a video providing a decoded picture, obtaining a first reference picture from the decoded picture, the first reference picture being a reconstructed picture output by an in-loop filtering process, decoding an indication indicating that at least one part of at least one second reference picture has to be determined from the decoded picture, the at least one second reference picture being distinct from the first reference picture, determining at least one part of the at least one second reference picture using a filtering of the decoded picture, decoding at least one block of a next picture of the video using the at least one part of the at least one second reference picture.

A system for haptic spatialization introduces the notion of a haptic source, haptic object and haptic listener. A haptic source is a virtual transmitter that is able to provide haptic information. A haptic object is an object with haptic properties based on physical characteristics, to modulate the propagation of a haptic signal through the object. A haptic listener is a virtual sensor that captures haptic information. A haptic engine handles the workflow and interactions between these different elements of the system. More particularly, it determines how a haptic signal is propagated according to the spatial localization and the properties of the elements. Syntax elements describing the elements composing such system allow to create information representative of a scene comprising spatialized haptic effects. Such information can be provided to a rendering device in order to be played back accordingly.

Some embodiments of a method may include obtaining a MPEG Avatar Representation Format (ARF)-based file, wherein the MPEG ARF-based file comprises a list identifying one or more controller sets of an avatar, wherein each of the one or more controller sets comprise one or more controllers, and wherein each of the one or more controllers comprise a list of one or more channels; verifying the MPEG ARF-based file comprises information corresponding to the one or more channels; parsing the information corresponding to the one or more channels into one or more respective channel data structures; and controlling a component of an avatar using at least one of the one or more channel data structures.

An encoding method is disclosed. An avatar animation unit (AAU) is obtained that comprises a header and a payload. From the header, it is determined whether the AAU is a keyframe AAU or a non-keyframe AAU. The AAU is finally encoded responsive to the determining.

Some embodiments of a method may include: obtaining a scene description file corresponding to a shared scene environment; parsing the scene description file into one or more data structures; identifying elements of the shared scene environment that describe user interactions; determining which elements are related to only one user and which elements are related to multiple users; tracking, for each element related to multiple users, events referenced by the respective user interaction; detecting satisfaction of a multi-user condition corresponding to occurrence of the respective event by one or more users; and responsive to detecting the satisfaction of the multi-user condition, executing an action related to the identified element of the shared scene environment.

Some embodiments of a method may include initializing a three-dimensional (3D) Gaussian (3DG) generation process, wherein initializing the 3DG generation process is based on two or more temporally consecutive frames with views of a 3D environment; and performing the 3DG generation process to generate a representation of the 3D environment.

Specific syntax elements define a neural-network post-filter characteristics (NNPFC) supplemental enhancement information (SEI) message carrying information related to a tone mapping operation that can be applied as a post filter to the decoded image. This NNPFC SEI message may comprise information representative of the colour volume resulting from the tone mapping operation, information representative of characteristics of a target mastering display and information representative of adaptation parameters, related to the adaptation of the luminance level, quality level and the energy adaptation level that result from the tone mapping operation are provided. The information representative of the colour volume resulting from the tone mapping operation may also be carried using a second SEI message. An encoding method, a decoding method, an encoding apparatus and a decoding apparatus are described.

The present principles generally relate to the storage and transmission of avatar texture animation data.

Specific syntax elements define a neural-network post-filter characteristics (NNPFC) supplemental enhancement information (SEI) message carrying information related to a tone mapping operation that can be applied as a post filter to the decoded image. This NNPFC SEI message may comprise information representative of the colour volume resulting from the tone mapping operation and information representative of adaptation parameters, related to the adaptation of the luminance level, quality level and the energy adaptation level that result from the tone mapping operation are provided. The information representative of the colour volume resulting from the tone mapping operation may also be carried using a second SEI message. An encoding method, a decoding method, an encoding apparatus and a decoding apparatus are described.

A method comprising: obtaining (800) a current block of a picture; deriving (801) a first list of a first number of intra prediction modes; generating (802) a prediction of a template of samples in a neighborhood of the current block for each intra prediction mode of the first list; for each predictor, obtaining (803) a prediction error between the predictor and the template; obtaining (804) a second list of reference lines in a neighborhood of the current block and forming a TMRL list of a second number of template based multiple reference lines (TMRL) candidates by browsing the intra prediction modes of the first list in ascending order of the prediction error and browsing the reference lines of the second list from a nearest reference line to a furthest reference line, each TMRL candidate of the TMRL list comprising an intra prediction mode extracted from the first list during the browsing the first list and a reference lines extracted from the second list extracted during the browsing of the second list; and, determining (805) a TMRL predictor for the current block from the TMRL candidates of the TMRL list.

Specific syntax elements define a target colour volume resulting from some processing on pictures in a video coding system. An encoded video bitstream carries such syntax elements from an encoding device to a decoding device using a Target Colour Volume Information Supplemental Enhancement Information message or a Content Colour Volume Supplemental Enhancement Information message, thus allowing the encoding device to provide to the decoding device the target colour volume and thus adapt the processing accordingly. An encoding method, a decoding method, an encoding apparatus and a decoding apparatus are described.

A first indicator indicating that a prediction mode of a current block belongs to a list of most probable modes is decoded. A second indicator is then obtained that indicates that perspective prediction is to be applied to the current block. An index is also obtained that identifies a prediction mode in a list of most probable perspective modes derived from the list of most probable modes. The current block is reconstructed based on the prediction mode.

Methods and apparatuses for encoding/decoding a video are provided wherein at least two prediction blocks for a current block of an image of the video are obtained and wherein at least one of the at least two prediction blocks is obtained using a block vector-based prediction. The at least two prediction blocks are blended to provide a predicted block, wherein the blending uses blending weights determined from a template of the current block and the current block is encoded or decoded based on the predicted block.

Systems, methods, and instrumentalities are configured for intra template matching prediction fusion. In examples, a device (e.g., a video encoding and/or decoding device) may be configured to obtain, for a video block, a block vector (BV) candidate of a first fusion set. The device may compare a template matching (TM) cost of the BV candidate to a threshold associated with a first fusion set. The device may, based on the comparison, determine to bypass the BV candidate from the first fusion set. The device may determine a fused prediction block based on the first fusion set, a second fusion set, and the bypassed BV candidate. The device may decode the video block based on the fused prediction block.