Stable Diffusion Webgpu

Freemium

Stable Diffusion: Web Image Generation

Introduction:

Are you looking for a powerful AI tool that can generate stunning web images? Introducing Stable Diffusion WebGPU, a web-based application that utilizes the create-react-app framework to create captivating visuals.

With Stable Diffusion WebGPU, users can effortlessly generate images by following a simple process. However, to access this cutting-edge application, make sure you have JavaScript enabled and the latest version of Chrome, with the “Experimental WebAssembly” and “Experimental WebAssembly JavaScript Promise Integration (JSPI)” flags enabled.

This application employs a series of inference steps, taking approximately 1 minute per step, along with an additional 10 seconds for the VAE decoder to generate the image. Keep in mind that having DevTools open may slow down the process. The UNET model responsible for image generation runs exclusively on the CPU, ensuring optimal performance and accurate results.

To achieve satisfactory outcomes, we recommend a minimum of 20 steps, although for demonstration purposes, 3 steps will suffice. The model files are cached, eliminating the need for repeated downloads. With a user-friendly interface, you can effortlessly load the model, initiate the image generation process, and view the final result. Troubleshooting guidance is also available in the FAQ section.

While the webgpu implementation in onnxruntime is still in its early stages, causing some operations to be incomplete, the developer is actively working on addressing these issues through proposed spec changes and engine patches. The source code for Stable Diffusion WebGPU demo is available on GitHub, allowing users to run it locally. Additionally, a patched version of onnxruntime is provided, enabling the use of large language models with transformers.js, although its reliability in all scenarios is not guaranteed. Stay tuned as the developer plans to submit a pull request to the onnxruntime repository.

Overview:

Stable Diffusion WebGPU is an AI tool that allows users to generate images using the create-react-app framework. It is a web-based application that requires JavaScript enabled and the latest version of Chrome with specific flags enabled. The application performs a series of inference steps to generate an image, with each step taking approximately 1 minute plus additional time for the VAE decoder. The UNET model responsible for image generation runs on the CPU for better performance and accuracy. The minimum recommended number of steps for acceptable results is 20, but for demonstration purposes, 3 steps are sufficient. The model files are cached to avoid repeated downloads. The application provides a user-friendly interface with options to load the model, run the image generation process, and view the result. An FAQ section is available for troubleshooting guidance. However, the webgpu implementation in onnxruntime is still in its early stage, causing incomplete operations and impacting performance due to continuous data transfer between the CPU and GPU. Multi-threading is not supported, and limitations in WebAssembly prevent the creation of 64-bit memory with SharedArrayBuffer. The developer plans to address these issues through proposed spec changes and engine patches. The source code for Stable Diffusion WebGPU is available on GitHub for local usage, and a patched version of onnxruntime is provided for the use of large language models with transformers.js, although its reliability in all scenarios is not guaranteed. The developer also plans to submit a pull request to the onnxruntime repository.