Fine tuning GPU options
Finetune jobs can be run with different GPUs- affecting speed, price and quality.
Sample Comparison
Overview:
In this exercise, we compare the performance and cost-effectiveness of three different model training pipelines: A100 (Balanced), H100 (Fast), and L40S (Low Cost). The training was conducted using the following parameters:
Dataset: 10MB zip file
Steps: 500
Training Configuration
{
"resolution": 1024,
"repeats": 100,
"learning_rate": 0.0004,
"lr_scheduler": "constant",
"optimizer_type": "Adafactor",
"num_train_epochs": 500,
"steps": 500,
"gradient_accumulation_steps": 2,
"center_crop": false,
"lora_rank": 32,
"noise_offset": 0,
"max_grad_norm": 0,
"bucket_steps": 64,
"weight_decay": 0.01,
"relative_step": false,
"auto_caption": false,
"content_or_style": "balanced",
"batch_size": 2,
"linear": 16,
"linear_alpha": 16,
"prompt": "driving a F1 car",
"iterations": 300,
"captioning": true,
"priority": "QUALITY"
}
Comparison Table:
Pipeline Name
Characteristics
Charge Rate (/s)
# of Seconds
Total Cost
A100
Balanced
$0.002
1868.67
$3.74
H100
Fast
$0.0043
1085.85
$4.67
L40S
Low Cost
$0.0014
2532.14
$3.54
Insights:
A100 (Balanced):
Offers a balanced performance with moderate cost and time.
Suitable for scenarios where a trade-off between speed and cost is acceptable.
H100 (Fast):
Significantly faster but at a higher charge rate.
Ideal for time-sensitive tasks where speed is prioritized over cost.
L40S (Low Cost):
Lowest cost option but takes the longest time to complete.
Best suited for non-urgent tasks where cost efficiency is more important.
Last updated