Hi guys! As some of you know, I've been involved in creating AI art for almost two years. When I'm not building my own models, I use a lot of different Google Colab notebooks in my art. Even though there are a lot of really excellent notebooks available, tracking them down can be a challenge. I haven't seen a really comprehensive list of the ones used by the nascent AI art community, so I figured I'd put one together and give more people the freedom to create. This post will continue to grow as I find new notebooks, and as new methods are created. If you find any I've missed or have created any, let me know and I'll add them!
Table of Contents
- The Big Sleep
- VQGAN + Clip
2a. VQGAN+CLIP with Video Features - Clip-Guided Diffusion
- Styledream Notebook
- Pixray PixelDraw
- Real-ESRGAN Practical Image Restoration
- ruDALLE [Added Nov 2, 2021; Updated Nov 3, 2021]
- Looking Glass AI [Added Dec. 28 2021]
- JAX Clip-Guided Diffusion *[Added Mar. 11, 2022]
- CLIP Guided Deep Image Prior *[Added Mar. 11, 2022]
- Disco Diffusion [Added Mar. 11, 2022]
- Disco Diffusion Turbo [Added Mar. 11, 2022]
1. The Big Sleep
A Beautiful House by BigSleep
Description: The original Clip-guided AI Colab tool; uses Clip to guide BigGAN. Fairly outdated, but absolutely foundational to the entire art field.
Author: Ryan Murdock (advadnoun on Twitter)
Notebook Link
2. VQGAN + Clip
Gordian Gallery by KaliYuga, 2021
Description:
By far the most widely-used of the Clip-guided methods[Edit April 2022: very few people still seem to use this tool, but it's a foundational and very important one]; uses Clip to guide VQGAN.
Author: Katherine Crowson (RiversHaveWings on twitter)
Notebook Link (The notebook is in Spanish, but is easily translated to English using Chrome).
2a. VQGAN+CLIP with Video Features
Description: What it says on the tin! VQGAN + Clip with added video features and also some really good demystifications of parameters.
Author: @jotakrevs
Notebook Link
3. Clip-Guided Diffusion
Psychic Sermon in the Temple of the Mind by KaliYuga, 2021
Description: Produces absolutely unbelievable results, but very slow and resource-heavy. Uses Clip to guide a Diffusion model.
Author: Katherine Crowson (RiversHaveWings on twitter)
Notebook Link
4. Styledream Notebook
Portraits of DC heroes in the style of Alex Ross *by .hack//HAMBRE
Description: Portraits only! Clip-guided StyleGAN model.
Author: Ariel Ekgren (ArYoMo on twitter)
Notebook Link
5. Pixray PixelDraw
A synthwave great white shark by Viktor Alm
Description: A fork of NerdyRodent's fork of RiversHaveWings'VQGAN + Clip Notebook; uses VQGAN + Clip to produce pretty impressive pixel art.
Author:Dribnet
Notebook Link
Extra! You can also start with an initial image in a different version of Pixray; if you use an image originally created by pixray, the results can be pretty trippy! For example:
Via the Colab Notebook's demo params
Other bits and pieces of this project can be found here.
6. Real-ESRGAN Practical Image Restoration
Examples via the Colab Notebook
Description: A demo of Real-ESRGAN,which repairs many types of image degradation and also upscales the image 4x.
Authors: Xintao Wang, Liangbin Xie, Chao Dong, Ying Shan
Notebook Link
7. ruDALLE
Description: A Fully-Trained, Accessible, Multi-Billion-Parameter Answer to OpenAI's DALL-E Model (in Russian)
Authors: Sber AI and Sber Devices
Notebook Link
Bonus: A faster Colab notebook for free users
8. Looking Glass AI
"Untitled" by KaliYuga, 2021
Description: An Accessible One-Shot Finetuner For RuDalle
Authors: AI_Curio and Others
Notebook Link
9. JAX Clip-Guided Diffusion
By KaliYuga, February 2022
Description: [Via Gituhub] "The models are denoising diffusion probabilistic models (https://arxiv.org/abs/2006.11239), which are trained to reverse a gradual noising process, allowing the models to generate samples from the learned data distributions starting from random noise."
Authors: Katherine Crowson and John David Pressman
Notebook Link
10. CLIP Guided Deep Image Prior
By KaliYuga, February 2022
Description: Creates soft, print-like images quite unlike what you generally get out of other CLIP-guided diffusion models
Authors: Daniel Russell, based on work by Katherine Crowson and Ryan Murdock
Notebook Link
11. Disco Diffusion
By KaliYuga, March 2022
Description: My daily drive; Very high quality image results very quickly; comes fully loaded with 3D animation capabilities and the latest Clip models.
Authors: Somnai and Adam Letts
Notebook Link
12. Disco Diffusion Turbo
~~~ embed:1502057904693723137?s=20&t=MLWq2-K9-Rc7Bu5IN9wZWg twitter metadata:S2FsaVl1Z2FfYWl8fGh0dHBzOi8vdHdpdHRlci5jb20vS2FsaVl1Z2FfYWkvc3RhdHVzLzE1MDIwNTc5MDQ2OTM3MjMxMzd8 ~~~
Description: A fork of Disco which speeds up and cleans up 3D animation significantly. I also use this all the time!
Author: Chris Allen
Notebook Link
Bonus by Chris Allen Notebook: Wiggle Standalone Keyframe Generator; lets you generate keyframe instructions that give you fine control over Turbo's motion.
By the way, I think @juecoree should see this, and join our boat (:
Thanks for the heads up! I'll definitely check out and try this list of tools.
Awesome!
This post was perfectly timed. I was just about to experiment and play around with some AI art and came to look up a post for links and found this! Thanks for the in depth post and all the invaluable resources! It's much appreciated! =) Awesome art outcomes too! Keep up the great work. =)
Hey! Glad you found it helpful--make sure to post what you come up with :)
I have been slacking on posting. I have a folder for each day's worth of work for about 20 or more days. So, i have a lot of generated content... i just haven't posted or shared yet. I need to get busy and just put some out there. I hate the part of me that wants to be a perfectionist... hahaha it doesn't suit me. Thanks for the encouragement and interest! Cheers. =)
Just what I was looking for and was in fact planning to include whatever notebooks I knew of in my next post. Will bookmark this post, thank you for the comprehensive list :)
Oh this is awesome! You've here some interesting resources.
By the way, thanks for the attribution in the "Video Features" notebook, but I just fixed some outdated URLs and a bad typo. These people are who deserve the attribution, not me (:
Oh, gotcha—I’ll update that :)
Cool (:
Hello,
You may want to add the IC-GAN https://colab.research.google.com/github/facebookresearch/ic_gan/blob/main/inference/icgan_colab.ipynb
This is excellent. Using VQGAN + Clip
Thanks for creating this list; in terms of 11 and 12 in your inventory (Disco Diffusion), these have now been merged, and additional forks added to the main notebook to form 5.4 - https://colab.research.google.com/github/alembics/disco-diffusion/blob/main/Disco_Diffusion.ipynb