GDM

This specification relates to processing data using machine learning models. A computer implemented method is provided comprising: receiving an input query; and generating an output. The generating comprises, for a plurality of iterations: generating, using a machine-learning model, a data element for the iteration based on the input query, wherein the data element comprises: (i) an output data element indicative of an output of the machine-learning model; and/or (ii) an interrupt data element indicative of one or more interrupt options. For one or more of the plurality of iterations, the method comprises: in response to determining that the data element for the iteration and/or a previous iteration is an output data element, causing the output indicated by the output data element to be output; and in response to determining that the generative data element for the iteration and/or a previous iteration is an interrupt data element, causing the one or more interrupt options indicated by the interrupt data to be exposed via an interface. The method further comprises, during generation of the output: receiving, via the interface, data indicative of a selection of at least one interrupt option of the one or more interrupt options; and adapting the generation of the output based on the at least one selected interrupt option.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for updating a local operating state of a first user device. In one aspect, a method includes identifying a context of the first user device; identifying a second user device within a proximity of the first user device; determining relevant aspects of a local operating state of the first user device that are relevant to the context; performing an automated data exchange session between a first agent executing on the first user device and a second agent executing on the second user device to determine one or more actions that each define an update to the local operating state; and performing the one or more actions by the first user device to update the local operating state of the first user device.

The systems, methods and apparatus described herein are directed towards at least the use of a two-stage augmentation process for vocal audio samples to generate a training dataset for a machine-learning embedding model, the training of machine-learning embedding models on such training data, and the use of such a trained machine-learning embedding models to generate embedding of audio snippets for downstream tasks. The two-stage augmentation process has a first augmentation stage, in which strong augmentations are applied at an audio sample level, and a second augmentation stage, in which weak augmentations are applied at an audio snippet level, where an audio snippet is a section of audio extracted from an audio sample after the first augmentations are applied.

Computer implemented method comprising: obtaining an audio stem sample, wherein the audio stem sample comprises audio originating from one or more sources of a particular type in an audio sample; and generating, using a machine-learning embedding model, an embedding of the audio stem sample in an embedding space. Computer implemented method comprising: obtaining an embedding of an audio stem sample, wherein the audio stem sample comprises audio originating from one or more sources of a particular type in an audio sample, inputting the embedding of the audio stem sample into a machine-learning generative model; and generating, by the machine-learning generative model and conditioned on the embedding of the audio stem sample, output audio data comprising an output audio stem, wherein the output audio stem has one or more audio properties in common with the audio stem sample.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for performing a machine learning task on a network input to generate a network output. In one aspect, one of the systems includes a plurality of host computer systems configured to receive the network input for a neural network. The network input includes an input tokens, and the neural network includes multiple layers that include a mixture of experts (MoE) layer that includes (i) a router, (ii) multiple expert neural networks, and (iii) a consolidation layer. The system processes the network input using the neural network, and the system generates a layer output token for the input token from the outputs of the assigned expert neural networks based on outputting one or more failure tokens that indicate a failure occurred during the processing.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating sequences of network outputs using a generative machine learning model following a learned generation order. In one aspect, a method comprises receiving a network input and generating a sequence of network outputs using a generative model, the generating comprising, at each of a plurality of iterations: identifying a set of candidate positions for the iteration; determining uncertainty scores for each of the set of candidate positions using an uncertainty scoring neural network; selecting a position of the sequence of network outputs from the set of candidate positions using the uncertainty scores; and generating the network output at the selected position of the sequence of network outputs using the generative model.

Methods, systems, and computer storage media are provided for training generative neural network systems, such as Large Language Models (LLMs) or Vision-Language Models (VLMs), using multiple reward models. The process involves generating one or more output sequences from a training input sequence and processing these examples using a plurality of different reward models to generate respective reward values. These reward values are combined to obtain an aggregated reward, which may be calculated as a product of the values, a weighted geometric mean, or a Nash score. The generative neural network is trained using an objective function determined using the aggregated reward, such as a cross-entropy loss or a contrastive objective function, often utilizing log score differences between the system and a reference model. These techniques allow for multi-objective alignment and are adapted for implementation on parallel processing computer systems.

Methods and systems for fine-tuning a machine-learning model using a fusion-frame representation are provided. Model weights associated with a layer of a machine-learning model are updated using a projection of a block diagonal matrix into a fusion frame, with each block of the block diagonal matrix projected onto a respective subspace of the fusion frame. This reduces the number of parameters that need to be handled during fine-tuning, reducing demand on processor and memory resources, as it is not necessary to fine-tune all the weights, only the elements of the block diagonal matrix. The use of fusion frames also enables targeting of particular parameters or parameter sets using the subspaces of the fusion frame. Furthermore, embodiments described herein show how the fusion frame implementation lends itself to parallelisation of calculations associated with each respective subspace.

Implementations enable scalable generation of high-quality, diverse training data for vision-language model(s) (VLM(s)). Processor(s) of a system can configure a dialog between at least a first VLM and a second VLM, cause the dialog to be conducted, and generate training instance(s) based on the dialog. In configuring the dialog, the first VLM is provided by a target image and a first set of instructions for the dialog, and the second VLM is provided with an ordered set of candidate images (e.g., the target image and additional image(s)) and a second set of instructions for the dialog. In causing the dialog to be conducted, the second VLM generates question(s) (e.g., using the second set of instructions) to ask the first VLM in furtherance of identifying the target image, in the ordered set of candidate images, and the first VLM generates response(s) (e.g., using the first set of instructions) to the question(s).

Implementations enable efficient identification of model vulnerabilities and subsequent model improvement without extensive real-world testing, leading to more robust and reliable models. Some implementations are directed to causing a training image to be processed to generate a synthetic image that includes one or more variations to the training image. The training image is one based on which a model has been trained. The variations in the synthetic image can include various changes, such as adding a new object, changing lighting, modifying the background, or altering the position of an object. The synthetic image is processed, using the model, to generate output. This output is then processed to determine whether the synthetic image is an outlier for the model. In response to determining that the synthetic image is an outlier for the model, remediating action(s) are performed, such as causing the model to be further trained based on the synthetic image.