Finegrain/refiners
1
0
Fork 0
mirror of https://github.com/finegrain-ai/refiners.git synced 2024-11-22 14:18:46 +00:00
Code Issues Actions Packages Projects Releases Wiki Activity

Use bias correction on Prodigy

Browse source
This commit is contained in:
Doryan Kaced 2023-09-11 23:14:56 +02:00
parent 9364c0ea1c
commit 2f2510a9b1
1 changed files with 1 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

1
src/refiners/training_utils/config.py
View file

@ -165,6 +165,7 @@ class OptimizerConfig(BaseModel):
betas=self.betas, betas=self.betas,
weight_decay=self.weight_decay, # type: ignore weight_decay=self.weight_decay, # type: ignore
safeguard_warmup=True, safeguard_warmup=True,
use_bias_correction=True, # recommended for diffusion models
) )