GDM Patente

Implementations enable scalable generation of high-quality, diverse training data for vision-language model(s) (VLM(s)). Processor(s) of a system can configure a dialog between at least a first VLM and a second VLM, cause the dialog to be conducted, and generate training instance(s) based on the dialog. In configuring the dialog, the first VLM is provided by a target image and a first set of instructions for the dialog, and the second VLM is provided with an ordered set of candidate images (e.g., the target image and additional image(s)) and a second set of instructions for the dialog. In causing the dialog to be conducted, the second VLM generates question(s) (e.g., using the second set of instructions) to ask the first VLM in furtherance of identifying the target image, in the ordered set of candidate images, and the first VLM generates response(s) (e.g., using the first set of instructions) to the question(s).

Methods and systems for fine-tuning a machine-learning model using a fusion-frame representation are provided. Model weights associated with a layer of a machine-learning model are updated using a projection of a block diagonal matrix into a fusion frame, with each block of the block diagonal matrix projected onto a respective subspace of the fusion frame. This reduces the number of parameters that need to be handled during fine-tuning, reducing demand on processor and memory resources, as it is not necessary to fine-tune all the weights, only the elements of the block diagonal matrix. The use of fusion frames also enables targeting of particular parameters or parameter sets using the subspaces of the fusion frame. Furthermore, embodiments described herein show how the fusion frame implementation lends itself to parallelisation of calculations associated with each respective subspace.

Methods, systems, and computer storage media are provided for training generative neural network systems, such as Large Language Models (LLMs) or Vision-Language Models (VLMs), using multiple reward models. The process involves generating one or more output sequences from a training input sequence and processing these examples using a plurality of different reward models to generate respective reward values. These reward values are combined to obtain an aggregated reward, which may be calculated as a product of the values, a weighted geometric mean, or a Nash score. The generative neural network is trained using an objective function determined using the aggregated reward, such as a cross-entropy loss or a contrastive objective function, often utilizing log score differences between the system and a reference model. These techniques allow for multi-objective alignment and are adapted for implementation on parallel processing computer systems.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating sequences of network outputs using a generative machine learning model following a learned generation order. In one aspect, a method comprises receiving a network input and generating a sequence of network outputs using a generative model, the generating comprising, at each of a plurality of iterations: identifying a set of candidate positions for the iteration; determining uncertainty scores for each of the set of candidate positions using an uncertainty scoring neural network; selecting a position of the sequence of network outputs from the set of candidate positions using the uncertainty scores; and generating the network output at the selected position of the sequence of network outputs using the generative model.

Implementations enable efficient identification of model vulnerabilities and subsequent model improvement without extensive real-world testing, leading to more robust and reliable models. Some implementations are directed to causing a training image to be processed to generate a synthetic image that includes one or more variations to the training image. The training image is one based on which a model has been trained. The variations in the synthetic image can include various changes, such as adding a new object, changing lighting, modifying the background, or altering the position of an object. The synthetic image is processed, using the model, to generate output. This output is then processed to determine whether the synthetic image is an outlier for the model. In response to determining that the synthetic image is an outlier for the model, remediating action(s) are performed, such as causing the model to be further trained based on the synthetic image.

The present technology is related to transistor synthesis optimization during the physical design stage of EDA. Aspects of the present technology introduces new ways to restructure transistor-level netlists, such as by reducing transistor count, for digital circuit designs based on the enumeration of "don't cares" of internal nets. One aspect of the present technology includes connecting internal nets with the same truth table up to don't cares without changing the functionality of the entire circuit. With an additional connection in the netlist, optimization opportunities can be identified that may remove transistors and/or improve placement, routing, and/or timing metrics. Furthermore, optimization opportunities may be further increased by (pre)computing satisfiability don't cares at the inputs of the netlist and observability don't cares at the outputs of the netlist from the environment.

The present technology includes a resynthesis engine for improving digital circuits' power, performance, and area (PPA). The resynthesis engine operates directly on a mapped netlist space. The resynthesis engine replaces subnetworks with low-cost structures from a database, which can be updated dynamically during optimization. The resynthesis engine can efficiently optimize for various objectives, including area, delay, and dynamic power. The resynthesis engine is also capable of significantly reducing glitching, which contributes to power consumption in arithmetic circuits.

A computer-implemented method for token generation. The method comprises obtaining as an input a set of tokens. One or more positions of the input set of tokens are masked for prediction. The method comprises, for each of a plurality of iterations, generating a predicted token for each masked position by inputting the set of tokens into a token prediction neural network, selecting one or more predicted tokens for inclusion into the set of tokens, and updating the set of tokens so that, for each selected predicted token, the selected predicted token becomes an unmasked token at the corresponding masked position. The plurality of iterations comprise a plurality of groups of one or more iterations. A different token prediction neural network is used for each group, and a size of the token prediction neural network increases with each subsequent group.

This specification describes a method performed by one or more data processing apparatus. The method comprises obtaining a user prompt comprising instructions for generating data and generating, using a data generation system, output data. Generating the output data comprises: processing, using a generative machine learning model, the user prompt to generate a modified prompt concurrently with performing a first data generation phase using the data generation system conditioned on the user prompt to generate intermediate data; and performing a second data generation phase using the data generation system conditioned on the modified prompt to process the intermediate data to generate the output data.

Implementations for robot control are provided. A method involves, based on vision data depicting an environment of a robot and a natural language instruction for the robot, determining an affordance plan for performing a task. The affordance plan comprises a sequence of intermediate representations of the robot in visual space, such as end effector poses. An action input prompt is assembled with data indicative of the vision data, the natural language instruction, and the affordance plan. The action input prompt is processed using one or more generative models to generate action output indicative of one or more actions to be performed by the robot. Subsequently, a robot control signal is generated based on the one or more actions. This provides a spatially precise and dimensionally concise form of guidance for robot manipulation tasks, which can improve performance and generalization.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing a machine learning task. One of the methods includes generating an output sequence by, at each of a plurality of output time steps: generating a current input sequence from at least the tokens at output time steps that precede the output time step in the output sequence; generating a respective embedding for each input in the current input sequence; and processing the respective embeddings for the inputs in the current input sequence through one or more layer blocks in the sequence of layer blocks until a termination criterion is satisfied.

According to one aspect, there is provided a computer-implemented method comprising: obtaining a first solution generated by a candidate model using a first query; generating a performance metric, using an evaluation model and conditioned on the first query, first solution and privileged information, wherein the performance metric comprises an evaluation output representing a performance of the candidate model; wherein the privileged information was not available to the candidate model when generating the first solution.

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating corrected simulations using video generation models. One of the methods includes obtaining an input video including a sequence of frames depicting a state transition of an environment that does not meet a state transition criterion; generating, based on a frame from a sequence of frames that depicts an incorrect end state of the state transition, a synthetic ending frame depicting a corrected end state of the state transition; and processing an input including one or more key frames from sequence of frames of the input video and the synthetic ending frame depicting the corrected end state of the state transition using a video generation model to generate an output video depicting a synthetic state transition of the environment that meets the state transition criterion.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a set of training data for training a speech processing model. One of the methods may include receiving a plurality of source audio signals that each represent speech; generating, for each source audio signal, a respective semantic representation of the source audio signal; obtaining, for each of a plurality of speakers, a respective speaker prompt embedding characterizing speech of the speaker; generating, for each source audio signal, one or more synthetic audio signals; and generating a set of training data for training a speech processing model, wherein the set of training data comprises a plurality of paired training examples.

A computer-implemented method of training a neural network. The method comprises processing a first transformed view of a training data item, e.g. an image, with a target neural network to generate a target output, processing a second transformed view of the training data item, e.g. image, with an online neural network to generate a prediction of the target output, updating parameters of the online neural network to minimize an error between the prediction of the target output and the target output, and updating parameters of the target neural network based on the parameters of the online neural network. The method can effectively train an encoder neural network without using labelled training data items, and without using a contrastive loss, i.e. without needing "negative examples" which comprise transformed views of different data items.

Implementations are provided for an interactive machine learning methodology that allows non-expert users to use natural language to teach new skills, particularly to robots, through language grounding and understanding. In various implementations, a plurality of natural language summaries may be retrieved. Each of the natural language summaries may describe details of robotic performance of a task, and may include, or be usable to retrieve, a corresponding set of reference modulation values. A set of modulation values corresponding to a natural language request may be generated based on the plurality of natural language summaries. The natural language request may specify one or more constraints on robotic performance of the task. A robot control signal may be generated based on the generated set of modulation values.

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …

Zusammenfassung wird geladen …