AWS Bedrock updates with model teaching and hallucination detector

AWS announced further updates for Bedrock aimed at spotting hallucinations and building smaller models faster as companies want more customization and accuracy from models.

AWS announced at re:Invent 2024 Amazon Bedrock Model Distillation and Automated Reasoning Checks as a preview for enterprise customers interested in training smaller models and capturing hallucinations.

Amazon Bedrock Model Distillation will allow users to use a larger AI model to train a smaller model and give businesses access to a model they believe would work best with their workload.

Larger models, like Call 3.1 405Bthey have more knowledge but are slow and unwieldy. A smaller model responds faster but most often has limited knowledge.

AWS said Bedrock Model Distillation would make the process of transferring knowledge from a larger model to a smaller one without sacrificing response time.

Users can select the heaviest model they want and find a small model within the same family, such as Llama or Claude, which have a range of model sizes in the same family, and write sample suggestions. Bedrock will generate responses and fine-tune the smaller model and continue to create more sample data to complete the distillation of the larger model’s knowledge.

At this time, the model distillation works with AnthropicAmazon e Half models. Bedrock Model Distillation is currently in preview.

Why businesses are interested in model distillation

For companies that want a faster response model, such as being able to respond quickly to customer questions, there needs to be a balance between knowing a lot and responding quickly.

While they may choose to use a smaller version of a large template, AWS is seeing that more and more companies want more customization in the types of templates, both larger and smaller, that they want to use.

AWS, which offers a choice of models in Bedrock’s model garden, hopes that companies will want to choose any model family and train a smaller model for their needs.

Many organizations, especially model vendors, use model distillation to train smaller models. However, AWS said the process usually requires a lot of machine learning expertise and manual tuning. They used template providers like Meta distillation model to bring broader knowledge based on a smaller model. Nvidia took advantage of distillation and pruning techniques to make Blade 3.1-Minitron 4Ba small language model performs better than similarly sized models.

Model distillation is nothing new for Amazon, it has been working on model distillation methods from 2020.

Spot factual errors faster

Hallucinations remain a problem for AI models, although companies have created workarounds such as fine-tuning and limiting what the models will respond to. However, even the most refined model that only performs augmented generation (RAG) retrieval tasks with a dataset can still make errors.

The AWS solution is automatic reasoning checking on Bedrock, which uses mathematical validation to prove that an answer is correct.

“Automated reasoning checks are the first and only generative AI safeguard that helps prevent factual errors due to hallucinations by using logically accurate and verifiable reasoning,” AWS said. “By increasing the trust customers can place in the model’s answers, automatic reasoning checks open generative AI to new use cases where accuracy is critical.”

Customers can access automated reasoning checks from Amazon Bedrock Guardrails, the product that brings responsible AI and model tuning. Researchers and developers often use automated reasoning to handle precise answers to complex math problems.

Users need to upload their data and Bedrock will develop the rules for the model to follow and guide customers to ensure the model is tuned to them. Once selected, the automatic reasoning checks on Bedrock will verify the model’s responses. If it returns something incorrect, Bedrock will suggest a new answer.

AWS CEO Matt Garman said during his keynote that automated controls ensure a company’s data remains its differentiator, and their AI models accurately reflect this.