Sony Interactive Entertainment

Methods and systems are provided for verifying an input provided at a controller including detecting a finger gesture on a surface of the controller. Responsive to detecting the finger gesture, multi-modal data is collected from a plurality of sensors and components tracking the finger gesture. The multi-modal data is used to generate an ensemble model using machine learning algorithm. The ensemble model is trained in accordance to training rules defined for different finger gestures. An output is identified from the ensemble model for the finger gesture. The output is interpreted to define an input for an interactive application selected for interaction.

Methods, systems, and apparatus for executing, by a local processing device, a first instance of the application associated with a first user of the local processing device, receiving data output by a second instance executed by the remote processing device, the second instance of the application being associated with a second user of the local processing device, identifying a first synchronization point within the first instance, identifying a second synchronization point associated with the second instance, determining a temporal offset between the first instance and the second instance using the first synchronization point and the second synchronization point, and implementing one or more modifications to one or more of the first instance and the second instance to reduce the temporal offset.

Methods, systems, and apparatus for executing the first instance responsive to first input received from a first input device associated with the local processing device, the second instance being responsive to second input received from a second input device associated with the local processing device, determining a first degree of similarity between a first video output of the first instance and a second video output of the second instance, and determining that the first degree of similarity meets or exceeds a threshold degree of similarity and, and at least partially in response, implementing one or more modifications, a first modification including ceasing, by the local processing device, display of one of the first video output and the second video output, and displaying, by the local processing device, another of the first video output and the second video output.

A streaming method for interactive content comprises the steps of starting generation of a first frame of content that reflects a current state of an application, the generation comprising a multi-stage process; receiving via a network from a client device input data that affects the state of the application, during the generation of the first frame; evaluating progress of the generation against a first criterion; responsive to the evaluations against the first criterion, stopping generation of the first frame of content; updating the current state of the application responsive to the input data; and generating a second frame of content that reflects an updated state of the application; and outputting the generated second frame and not the stopped first frame.

A system enhances existing audio-visual content with audio describing the setting of the visual content. A scene annotation module classifies scene elements from an image frame received from a host system and generates a caption describing the scene elements. A text to speech synthesis module may then convert the caption to synthesized speech data describing the scene elements within the image frame

A method of image streaming, comprises receiving at least a first stream comprising video packet data and separate supplementary packet data, upon failure to receive video packet data, providing as inputs to a generative model one or more last video images, and data based on currently received supplementary packet data, receiving from the generative model a predicted image, being an estimate of a current missing video image based on these inputs, and outputting the predicted image for display.

A computer-implemented method for modifying video game audio output using a video gaming system. The method comprises: obtaining a hearing loss profile associated with a user, the hearing loss profile defining a hear loss value varying with frequency; obtaining audio data to be output on a video gaming system, the audio data corresponding to an audio asset, the audio data comprising a plurality of frequency bands; determining, for each frequency band of the plurality of frequency bands, a respective loudness level; selecting, for each frequency band of the plurality of frequency bands of the audio data, a respective modulation profile based on the frequency band and the respective loudness levels, the modulation profile defining a gain value to be applied dependent on a hearing loss value provided by the hearing loss profile; and applying, to each frequency band of the audio data, the respective modulation profiles using the hearing loss profile to generate modified video game audio.

A computer-implemented method for adjusting the audio output of a video gaming system to compensate for a user's hearing loss profile. The method comprises: determining an optimal boost curve to be applied to an output audio signal, the optimal boost curve defining a frequency dependent gain value for compensating for a user's hearing loss profile; selecting a plurality of signal processing filters from a filter bank stored in a memory such that, when combined, the plurality of signal processing filters approximate the optimal boost curve; combining the plurality of filters to form an approximate boost curve and applying the approximate boost curve to the output audio signal to compensate for a user's hearing loss profile.

A computer-implemented method for modifying video game audio output using a video gaming system. The method comprises: obtaining a hearing loss profile associated with a user, the hearing loss profile defining a hear loss value varying with frequency; obtaining audio data to be output on a video gaming system, the audio data corresponding to an audio segment spanning a period of time, the audio data comprising a plurality of frequency bands; determining, for each frequency band of the plurality of frequency bands, a respective loudness level; selecting, for each frequency band of the plurality of frequency bands of the audio data, a respective modulation profile based on the frequency band and the respective loudness levels, the modulation profile defining a gain value to be applied dependent on a hearing loss value provided by the hearing loss profile; and applying, to each frequency band of the audio data, the respective modulation profiles using the hearing loss profile to generate modified video game audio.

There is provided a computer-implemented method of generating a point cloud, the method comprising: receiving first data representing a 3D model, the 3D model comprising a plurality of surfaces; performing ray tracing from a camera view, wherein the ray tracing comprises defining one or more rays passing through a location of the camera view; determining one or more intersection locations, corresponding to the location of an intersection of a ray with one or more surfaces of the 3D model; and generating a point cloud representation of the 3D model from the one or more intersection locations.