|
Organizations face a tough trade-off when adapting AI models to their specific business needs: settle for generic models that deliver mediocre results, or deal with the complexity and cost of advanced model customization. Traditional approaches force a choice between poor performance on smaller models or the high cost of deploying larger model variants and managing complex infrastructure. Boost fine-tuning is an advanced technique that trains models using feedback instead of massive labeled data sets, but its implementation typically requires specialized ML expertise, complicated infrastructure, and significant investment—with no guarantee of achieving the accuracy needed for specific use cases.
Today, we’re announcing amplification tuning in Amazon Bedrock, a new model customization capability that creates smarter, more cost-effective models that learn from feedback and deliver higher-quality outputs for specific business needs. Gain fine-tuning uses a feedback-based approach where models are iteratively improved based on reward signals, yielding an average 66% increase in accuracy over baseline models.
Amazon Bedrock automates the workflow of fine-tuning amplification and makes this advanced model fitting technique accessible to everyday developers without requiring deep machine learning (ML) knowledge or large labeled datasets.
How reinforcement fine-tuning works
Reinforcement fine-tuning is based on reinforcement learning principles to tackle a common problem: getting models to consistently produce outputs that align with business requirements and user preferences.
While traditional fine-tuning requires large, labeled data sets and expensive human annotations, gain fine-tuning takes a different approach. Instead of learning from fixed examples, it uses reward functions to evaluate and judge which answers are considered good for specific business use cases. This teaches models to understand what creates a quality response without the need for massive amounts of pre-labeled training data, making advanced model fitting in Amazon Bedrock more accessible and cost-effective.
Here are the benefits of using gain fine-tuning in Amazon Bedrock:
- Easy to use – Amazon Bedrock automates much of the complexity, making amplification tuning more accessible to developers building AI applications. Models can be trained using existing Amazon Bedrock APIs or by uploading datasets as training data, eliminating the need for labeled datasets or infrastructure setup.
- Better model performance – Fine-tuning the gain improves model accuracy by an average of 66% over baseline models, enabling cost and performance optimization by training smaller, faster, and more efficient model variants. It works with the Amazon Nova 2 Lite model, improving quality and price performance for specific business needs, with support for other models coming soon.
- Security – Data remains in a secure AWS environment throughout the customization process, mitigating security and compliance concerns.
This feature supports two complementary approaches that provide flexibility for optimizing models:
- Reinforcement learning with verifiable rewards (RLVR) uses rule-based comparators for objective tasks such as code generation or mathematical reasoning.
- Reinforcement Learning from AI Feedback (RLAIF) employees make judgments based on artificial intelligence for subjective tasks such as following guidelines or moderating content.
We start fine-tuning the boost in Amazon Bedrock
Let’s walk through creating a gain fine-tuning task.
First, I access the Amazon Bedrock console. I then navigate to Custom models page. i choose Create and then choose Fine tuning of reinforcement.
I’ll start by entering a name for this customization job and then selecting my base model. It supports Amazon Nova 2 Lite gain fine-tuning at launch, and support for other models will be available soon.
Next, I need to provide training data. I can use my saved recall logs directly, eliminating the need to upload separate datasets. I can also upload new JSONL files or select existing datasets from Amazon Simple Storage Service (Amazon S3). Fine-tuning the gain automatically validates my training dataset and supports the OpenAI Chat Completions data format. If I provide invoke logs in invoke or converse format to Amazon Bedrock, Amazon Bedrock automatically converts them to Chat Completions format.
The reward function setup is where I define what constitutes a good response. I have two options here. For objective tasks, I can choose Custom code and write custom Python code that runs through AWS Lambda functions. For a more subjective evaluation, I can choose A model as a judge use foundational models (FM) as referees by providing evaluation guidelines.
I choose here Custom codeand I create a new Lambda function or use an existing function as a reward. I can start with one of the templates provided and adapt it to my specific needs.
I can optionally modify default hyperparameters such as learning rate, batch size, and epochs.
For better security, I can configure virtual private cloud (VPC) settings and AWS Key Management Service (AWS KMS) encryption to meet my organization’s compliance requirements. Then I will choose Create to start the model fitting task.
During the training process, I can monitor real-time metrics to understand how the model is learning. The training metrics dashboard displays key performance indicators including reward scores, loss curves and accuracy improvement over time. These metrics help me understand if the model is converging correctly and if the reward function is effectively driving the learning process.
When the Gain Fine Tuning job is complete, I see the final job status at Model details page.
Once the job is done, I can deploy the model with a single click. i choose Set conclusionthen select Deployment on demand.
Here are some details about my model.
After deployment, I can quickly evaluate the model’s performance using the Amazon Bedrock playground. This helps me test the tuned model with sample challenges and compare its responses to the base model to verify improvements. i choose Field test.
The playground provides an intuitive interface for rapid testing and iteration, helping me confirm that the model meets my quality requirements before integrating it into production applications.
Interactive demo
Learn more in an interactive demo of fine-tuning Amazon Bedrock amplification in action.
Other things you should know
Here are the key points to note:
- Templates — There are seven ready-to-use reward function templates that cover common use cases for both objective and subjective tasks.
- Prices — For more pricing information, visit the Amazon Bedrock pricing page.
- Security — Training data and custom models remain private and are not used to improve FM for public use. Supports VPC and AWS KMS encryption for better security.
Get started with rebar fine-tuning by visiting the rebar fine-tuning documentation and accessing the Amazon Bedrock console.
Happy building!
— Donnie