Sony Interactive Entertainment Europe

There is provided an image streaming method for streaming image content from a remote computing device to a client device. The method comprises receiving, from a remote computing device, one or more masked images, each masked image comprising a first image with one or more portions of the first image being masked, reconstructing the one or more first images from the one or more masked images using a Masked Auto Encoder, MAE, model, the MAE model being trained to reconstruct an image from a masked version of the image, and outputting the one or more reconstructed first images for display.

A method of generating an intermediate frame of video, comprising the steps of: receiving a plurality of frames of video, wherein the frames are temporally consecutive; determining, from the received plurality of frames, at least one vector flow field representing a required pixel displacement between a received frame and the intermediate frame; determining at least one motion vector field representing the motion of an object shown in the received frames; and generating the intermediate frame from the received plurality of frames using the determined vector flow field and the determined motion vector field.

A method for rendering an image of a three-dimensional scene using path tracing. Scene feature data for a pixel of the image to be rendered using path tracing is obtained, indicating visual features of a location of the three-dimensional scene for depiction by the pixel in the image, is obtained. The scene feature data is processed at an artificial neural network, ANN, to determine a budget allocation parameter for rendering the pixel using path tracing. The budget allocation parameter is indicative of an amount of computing resources to be used for rendering the pixel using path tracing. The ANN is trained to determine, from the scene feature data, the budget allocation parameter based on the visual features indicated by the scene feature data. The determined budget allocation parameter is output from the ANN to control a rendering of the pixel using path tracing.

A method for rendering an image of a three-dimensional scene using path tracing. For a pixel of the image to be rendered using path tracing, a budget allocation parameter for rendering the pixel using path tracing is determined. The budget allocation parameter is indicative of an amount of computing resources to be used for rendering the pixel using path tracing. The budget allocation parameter is determined to optimise the entropy of the image of the three-dimensional scene generated using the rendered pixel. The determined budget allocation parameter is output from the ANN to control a rendering of the pixel using path tracing.

A method for rendering an image of a three-dimensional scene using path tracing. For a current pixel of the image to be rendered using path tracing, an associated pixel is identified, associated with the current pixel, wherein at least one parameter for rendering the associated pixel using path tracing has been previously determined. The at least one parameter for rendering the associated pixel is obtained, and used to determine at least one parameter for rendering the current pixel. The at least one determined parameter is output to control a rendering of the current pixel using path tracing.

A method for encoding video data, comprising: receiving, at a machine learning model, encoding statistics derived from an encoding, performed by an external encoder, of at least one frame of a first video scene; processing the received encoding statistics using the machine learning model to determine one or more encoder settings for the external encoder; and outputting, from the machine learning model, the determined one or more encoder settings for use by the external encoder to encode a second video scene. The machine learning model is trained to predict, using encoding statistics input into the machine learning model, encoder settings which, when used by the external encoder to encode video data, optimise a data rate and/or video quality associated with the encoded video data.

A computer-implemented method for image upscaling at a client device is provided. The method comprising: receiving, from a server device, an image which is one of a plurality of images forming an image stream, wherein the image comprises a plurality of image portions; determining a first group of one or more image portions from the plurality of image portions to apply a first image upscaling process to from a plurality of available image upscaling processes; selecting the first group of image portions based on the determination; and applying the first image upscaling process to the first group of image portions. The upscaling process may be an image upscaling process such as super resolution.

Zusammenfassung wird geladen …

An image generation system operable to generate updated views of an object in a scene, the system comprising an input obtaining unit configured to obtain one or more user inputs for controlling the pose of the object, an object pose modification unit configured to determine an updated pose of the object in accordance with the obtained user inputs, a hand pose determination unit configured to determine a pose for each of the hands associated with the object in accordance with the obtained user inputs, wherein the determining comprises generating a corresponding representation of the hands with the respective determined poses based upon two or more reference views of the hands, the reference views corresponding to different respective poses of the hands, a grab pose determination unit configured to determine a grab pose for each of the hands, the grab pose comprising a pose for fingers of each of the hands determined in dependence upon the updated object pose and a corresponding hand pose, and a rendering unit configured to generate one or more images of the object and associated hands in dependence upon the determined poses of the object, hands, and fingers.

Zusammenfassung wird geladen …