Finegrain/refiners: A microframework on top of PyTorch with first-class citizen APIs for foundation model adaptation

mirror of https://github.com/finegrain-ai/refiners.git synced 2024-11-21 13:48:46 +00:00

A microframework on top of PyTorch with first-class citizen APIs for foundation model adaptation https://refine.rs/

background-generation background-removal controlnet diffusion-models dinov2 image-generation ip-adapter lcm lcm-lora lora sam sdxl segment-anything segment-anything-model shadow-generation stable-diffusion t2i-adapter text-to-image textual-inversion

Find a file

Laurent 614c150123 use torch.optim.optimizer.ParamsT in training_utils		2024-10-15 13:39:46 +00:00
.github/workflows	add comfyui registry github action	2024-09-05 12:05:55 +02:00
assets	README: upgrade hello world	2023-10-20 18:28:31 +02:00
docs	deprecate LatentDiffusionAutoencoder's decode_latents	2024-10-15 15:19:36 +02:00
src	use torch.optim.optimizer.ParamsT in training_utils	2024-10-15 13:39:46 +00:00
tests	deprecate LatentDiffusionAutoencoder's decode_latents	2024-10-15 15:19:36 +02:00
.gitignore	add DINOv2-FD metric	2024-04-03 16:45:00 +02:00
CONTRIBUTING.md	update CONTRIBUTING.md	2024-10-14 15:12:59 +02:00
LICENSE	Update LICENSE	2024-02-02 14:08:09 +01:00
mkdocs.yml	write Training 101 guide	2024-02-26 14:44:02 +01:00
pyproject.toml	use torch.optim.optimizer.ParamsT in training_utils	2024-10-15 13:39:46 +00:00
README.md	README.md: fix typo	2024-09-24 14:46:44 +02:00
requirements.docs.txt	make requirements.docs.txt follow the [doc] optional deps from pyproject.toml	2024-08-23 13:44:32 +02:00
requirements.lock	remove gdown dependency	2024-10-14 15:12:59 +02:00

README.md

The simplest way to train and run adapters on top of foundation models

Manifesto | Docs | Guides | Discussions | Discord

Latest News 🔥

Added ELLA for better prompts handling (contributed by @ily-R)
Added the Box Segmenter all-in-one solution (model, HF Space)
Added MVANet for high resolution segmentation
Added IC-Light to manipulate the illumination of images
Added Multi Upscaler for high-resolution image generation, inspired from Clarity Upscaler (HF Space)
Added HQ-SAM for high quality mask prediction with Segment Anything
...see past releases

Installation

The current recommended way to install Refiners is from source using Rye:

git clone "git@github.com:finegrain-ai/refiners.git"
cd refiners
rye sync --all-features

Documentation

Refiners comes with a MkDocs-based documentation website available at https://refine.rs. You will find there a quick start guide, a description of the key concepts, as well as in-depth foundation model adaptation guides.

Projects using Refiners

Finegrain Editor: use state-of-the-art visual AI skills to edit product photos
Visoid: AI-powered architectural visualization
brycedrennan/imaginAIry: Pythonic AI generation of images and videos
chloedia/layerdiffuse: an implementation of LayerDiffuse (foreground generation only)

Awesome Adaptation Papers

If you're interested in understanding the diversity of use cases for foundation model adaptation (potentially beyond the specific adapters supported by Refiners), we suggest you take a look at these outstanding papers:

Credits

We took inspiration from these great projects:

tinygrad - For something between PyTorch and karpathy/micrograd
Composer - A PyTorch Library for Efficient Neural Network Training
Keras - Deep Learning for humans

Citation

@misc{the-finegrain-team-2023-refiners,
  author = {Benjamin Trom and Pierre Chapuis and Cédric Deltheil},
  title = {Refiners: The simplest way to train and run adapters on top of foundation models},
  year = {2023},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/finegrain-ai/refiners}}
}