Finegrain/refiners
1
0
Fork 0
mirror of https://github.com/finegrain-ai/refiners.git synced 2024-09-19 10:55:27 +00:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity

Change weight decay for Optimizer to normal PyTorch default

Browse source
This commit is contained in:
limiteinductive 2024-03-12 14:09:34 +00:00 committed by Benjamin Trom
parent 46612a5138
commit ff5341c85c
1 changed files with 1 additions and 1 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
src/refiners/training_utils/config.py
View file

@ -90,7 +90,7 @@ class OptimizerConfig(BaseModel):
learning_rate: float = 1e-4
betas: tuple[float, float] = (0.9, 0.999)
eps: float = 1e-8
weight_decay: float = 0.0
weight_decay: float = 1e-2
model_config = ConfigDict(extra="forbid")