Overview

We can generate image data utilizing Stable Diffusion models.

Manager

The manager handles the generation of media. It is responsible for loading the models, and generating media. It is automatically initialized when the server is started.

Setup:

StableDiffusionManager = stableDiffusionManager.StableDiffusionManager()

Generate Image

uaiBrainServer.routes.imageSDRoutes.GenerateImageSD()

Generate an image from a prompt and other parameters utilizing Stable Diffusion

Formatted as a JSON request

API Route: /image

API Methods: POST

Parameters:

Name	Type	Description	Default
model	str	The model to use for generation	required
name	str	The name of the image	required
prompt	str	The prompt to use for generation	required
neg_prompt	str	The negative prompt to use for generation. Attempts to remove things from the final image.	required
customSDBinWeights	str	The custom weights to use for fine tuning on a subject or theme	required
textualInversionWeights	str	The textual inversion weights to use	required
loraWeights	str	The lora weights to use	required
schedueler	str	The schedueler to use. Defaults to "EulerAncestralDiscreteScheduler"	required
device	str	The device to use. Defaults to "cuda"	required
steps	int	The number of steps to use. Higher the longer the genration takes, but possibly better quality. Defaults to 15	required
width	int	The width of the image. Defaults to 512	required
height	int	The height of the image. Defaults to 512	required
guidance_scale	float	Handles how much the prompt guides the findal output. Defaults to 7.5	required
seed	int	The seed to use for consistent generation or variations. Defaults to 42	required
imagesToGenerate	int	The number of images to generate. Defaults to 1	required
zip	bool	Whether or not to zip the images and return them as a zip file. Defaults to False	required

Returns:

Name	Type	Description
json	str	The generated as a MultipleMediaRequest Json string

uaiBrainServer.routes.textRoutes.fortuneImage()

Generate an image from a fortune that is generated from parameters utilizing Stable Diffusion and a fine tuned GPT3 model for fortune generation

API Route: /fortune/image

API Methods: POST

Parameters:

Name	Type	Description	Default
imageGenPrepend	str	The text to prepend to the generated fortune	required
imageGenAppend	str	The text to append to the end of the generated fortune	required
steps	int	The number of steps to use. Higher the longer the genration takes, but possibly better quality. Defaults to 15	required
model	str	The StableDiffusion model to use for generation. Can be any model from HuggingFace or a custom local model in a directory.	required
device	str	The device to use. Defaults to "cuda"	required
customSDBinWeights	str	The custom weights to use for fine tuning on a subject or theme	required
textualInversionWeights	str	The textual inversion weights to use	required
loraWeights	str	The lora weights to use	required
schedueler	str	The schedueler to use. Defaults to "EulerAncestralDiscreteScheduler"	required
device	str	The device to use. Defaults to "cuda"	required
steps	int	The number of steps to use. Higher the longer the genration takes, but possibly better quality. Defaults to 15	required
width	int	The width of the image. Defaults to 512	required
height	int	The height of the image. Defaults to 512	required
guidance_scale	float	Handles how much the prompt guides the findal output. Defaults to 7.5	required
seed	int	The seed to use for consistent generation or variations. Defaults to 42	required
imagesToGenerate	int	The number of images to generate. Defaults to 1	required

Returns:

Name	Type	Description
json	str	The generated image and prompt as a json string formatted as {"prompt": prompt, "image": uaiDiffusers.ImagesToBase64(pilImages[0]).decode()}

Augment Image

You can augment an image into another image with these functions.

Depth Image

routes.imageDepthRoutes.generateDepthBase()

Generate a depth map from an image

Formatted as a JSON request

API Route: /image/depth/base

API Methods: POST

Parameters:

Name	Type	Description	Default
input	str	The base64 encoded image to segment. RGB format	required
modelName	str	The model to use for generation. Defaults to "Intel/dpt-large"	required

Returns:

Name	Type	Description
jsonObject	json	Returns a JSON object with the following fields: {'image': uaiDiffusers.ImagesToBase64(compedImageFace), "depth": uaiDiffusers.ImagesToBase64(imagedepth), "alpha": uaiDiffusers.ImagesToBase64(compedImageAlpha), "composite": uaiDiffusers.ImagesToBase64(compedImage)}

Basic Pix2Pix

routes.imagePix2PixRoutes.pix2pixdiffusers()

Generate and image from another image from a prompt, and other parameters utilizing Pix2Pix

Formatted as a JSON request

API Route: /image/pix2pix/diffusers

API Methods: POST

Parameters:

Name	Type	Description	Default
image	str	The image to use for generation as a Base64 string. RGB Format NOT RGBA.	required
modelName	str	The model to use for generation	required
schedueler	str	The scheduler to use.	required
num_inference_steps	int	The number of inference steps to use. .	required
image_guidance_scale	int	The image guidance scale to use.	required
prompt	str	The prompt to use.	required
negative_prompt	str	The negative prompt to use.	required
device	str	The device to use.	required
seed	int	The seed to use.	required
progressSteps	int	How often to update progress. Default: 1	required

Returns:

Name	Type	Description
json	str	The generated as a MultipleMediaRequest Json string

Edge Based Pix2Pix

routes.imageControlNetRoutes.AugmentImageCanny()

Augment an image with canny edge detection

Formatted as a JSON request

API Route: /augment/image/canny

API Methods: POST

Parameters:

Name	Type	Description	Default
model	str	The model to use for generation	required
name	str	The name of the image	required
prompt	str	The prompt to use for generation	required
neg_prompt	str	The negative prompt to use for generation. Attempts to remove things from the final image.	required
customSDBinWeights	str	The custom weights to use for fine tuning on a subject or theme	required
textualInversionWeights	str	The textual inversion weights to use	required
loraWeights	str	The lora weights to use	required
schedueler	str	The schedueler to use. Defaults to "EulerAncestralDiscreteScheduler"	required
device	str	The device to use. Defaults to "cuda"	required
steps	int	The number of steps to use. Higher the longer the genration takes, but possibly better quality. Defaults to 15	required
width	int	The width of the image. Defaults to 512	required
height	int	The height of the image. Defaults to 512	required
guidance_scale	float	Handles how much the prompt guides the findal output. Defaults to 7.5	required
seed	int	The seed to use for consistent generation or variations. Defaults to 42	required
imagesToGenerate	int	The number of images to generate. Defaults to 1	required
zip	bool	Whether or not to zip the images and return them as a zip file. Defaults to False	required

Returns:

Name	Type	Description
json	str	The generated as a MultipleMediaRequest Json string

Segment Human

routes.imageSegmentRoutes.segmentHumanRotobot()

Segment a human from an image using Rotobot. Best for single person frames

Formatted as a JSON request

API Route: /image/segment/human/Rotobot

API Methods: POST

Parameters:

Name	Type	Description	Default
modelName	str	The model to use for generation	required
input	str	The base64 encoded image to segment. RGB format	required
outputdirectory	str	The directory to save the output image to	required
downsampleratio	float	The ratio to downsample the image by. HD uses 0.25, 4K uses 0.125	required
crop	bool	Whether to crop the image by the face. Default is True	required
padding	int	The padding to add to the face crop. 0 is no padding, 1 is 1 pixel of padding, etc.	required
sizeX	int	The X size of the output image	required
sizeY	int	The Y size of the output image	required
outputType	str	The type of output to return. Either "bytes" or "base64"	required

Returns:

Name	Type	Description
jsonObject	json	Returns a JSON object with the following fields: {'image': uaiDiffusers.ImagesToBase64(compedImageFace), "depth": uaiDiffusers.ImagesToBase64(imagedepth), "alpha": uaiDiffusers.ImagesToBase64(compedImageAlpha), "composite": uaiDiffusers.ImagesToBase64(compedImage)}

routes.imageSegmentRoutes.segmentHumanmp()

Segment a human from an image using MediaPipe. Works with small to medium groups and single person frames. REALLY FAST

Formatted as a JSON request

API Route: /image/segment/human/mp

API Methods: POST

Parameters:

Name	Type	Description	Default
modelName	str	The model to use for generation	required
input	str	The base64 encoded image to segment. RGB format	required
threshold	float	The threshold to use for the segmentation. Default is 0.5	required
depthThreshold	float	The threshold to use for the depth segmentation. Default is 0.8	required
depthMax	float	The maximum depth to use for the depth segmentation. Default is 1	required
maskFeather	int	The feathering to use for the mask. Default is 1	required
model_selection	int	The model to use for the segmentation. Default is 0	required
maxSize	int	The maximum width size of the image. Default is 0 for no resize.	required

Returns:

Name	Type	Description
jsonObject	json	Returns a JSON object with the following fields: {'image': uaiDiffusers.ImagesToBase64(compedImageFace), "depth": uaiDiffusers.ImagesToBase64(imagedepth), "alpha": uaiDiffusers.ImagesToBase64(compedImageAlpha), "composite": uaiDiffusers.ImagesToBase64(compedImage)}

Utility

uaiBrainServer.routes.imageSDRoutes.changeModelImage()

Change the Stable Diffusion model to a new model.

Formatted as a JSON request

API Route: /changeModel/image

API Methods: POST

Parameters:

Name	Type	Description	Default
modelName	str	The model to use for generation	required

Returns:

Name	Type	Description
string	str	"Changed model to " + modelName

uaiBrainServer.routes.imageSDRoutes.CleanPipeSD()

Clear the Stable Diffusion GPU Cache and purge the model. This is useful if you are running out of GPU memory.

API Route: /image/pipe/clear

API Methods: GET

Returns:

Name	Type	Description
string	str	Cache Cleared

uaiBrainServer.routes.utilityRoutes.shutdownServer()

Shutdown the server

API Route: /shutdown

API Methods: GET

Returns:

Name	Type	Description
json	str	{ "success": True, "message": "Server is shutting down..." }