API Endpoints
Start typing to search…

/generate/controlnet-ip-adapter

post
https://api.cloud.scenario.com/v1/generate/controlnet-ip-adapter

Trigger a new image generation in ControlNet + IpAdapter mode. The control image is used to guide the generation; it can be a pose, canny map, or similar. The second reference image is used as an IPAdapter to guide the generation process.

Query Params
string

If set to true, returns the original asset without transformation

string
Body Params
ipAdapterImageIds
array of strings

The IpAdapter images as an AssetId. Will be ignored if the ipAdapterImages parameter is provided

ipAdapterImageIds
number
0.1 to 1

Specifies how long the ControlNet guidance should be applied during the inference process.

Only available for Flux.1-dev based models.

The value represents the percentage of total inference steps where the ControlNet guidance is active. For example:

  • 1.0: ControlNet guidance is applied during all inference steps
  • 0.5: ControlNet guidance is only applied during the first half of inference steps

Default values:

  • 0.5 for Canny modality
  • 0.6 for all other modalities
string
enum
required

The modality associated with the control image used for the generation: it can either be an object with a combination of maximum

For models of SD1.5 family:

  • up to 3 modalities from canny, pose, depth, lines, seg, scribble, lineart, normal-map, illusion
  • or one of the following presets: character, landscape, city, interior.

For models of the SDXL family:

  • up to 3 modalities from canny, pose, depth, seg, illusion, scribble
  • or one of the following presets: character, landscape.

For models of the FLUX schnell or dev families:

  • one modality from: canny, tile, depth, blur, pose, gray, low-quality

Optionally, you can associate a value to these modalities or presets. The value must be within ]0.0, 1.0].

Examples:

  • canny
  • depth:0.5,pose:1.0
  • canny:0.5,depth:0.5,lines:0.3
  • landscape
  • character:0.5
  • illusion:1

Note: if you use a value that is not supported by the model family, this will result in an error.

string

Used to reproduce previous results. Default: randomly generated number.

string
required

The model id to use for the inference

string
enum
Defaults to style

The type of IP Adapter model to use. Must be one of [style, character], default to `style``

Allowed:
string

Deprecated for type txt2img-ip-adapter and img2img-ip-adapter, use ipAdapterImages instead. The IpAdapter image as a data url. Will be ignored if the ipAdapterImages parameter is provided.

string
enum

The scheduler to use to override the default configured for the model. See detailed documentation for more details.

boolean
Defaults to true

If false, the process uses the given image to detect the modality. If true (default), the process will not try to detect the modality of the given image.

For example: with pose modality and false value, the process will detect the pose of people in the given image with depth modality and false value, the process will detect the depth of the given image with scribble modality and truevalue, the process will use the given image as a scribble

⚠️ For models of the FLUX schnell or dev families, this parameter is ignored. The modality detection is always disabled. ⚠️

ipAdapterImages
array of strings

The IpAdapter images as a data url.

ipAdapterImages
string

Specifies the parent asset Id for the image when provided as a dataurl.

string

The base model to use for the inference. Only Flux LoRA models can use this parameter. Allowed values are available in the model's attribute: compliantModelIds

number
0 to 0.9

Specifies the starting point of the ControlNet guidance during the inference process.

Only available for Flux.1-dev based models.

The value represents the percentage of total inference steps where the ControlNet guidance starts. For example:

  • 0.0: ControlNet guidance starts at the beginning of the inference steps
  • 0.5: ControlNet guidance starts at the middle of the inference steps
number
64 to 2048

The height of the generated images, must be a 8 multiple (within [64, 2048], default: 512) If model.type is sd-xl, sd-xl-lora, sd-xl-composition the height must be within [512, 2048] If model.type is sd-1_5, the height must be within [64, 1024] If model.type is flux.1.1-pro-ultra, you can use the aspectRatio parameter instead

boolean
Defaults to false

Toggles the hidden status of the image when provided as a dataurl.

string

The controlnet input image as an AssetId. Will be ignored if the controlnet parameter is provided

string

The input image as a data URL (example: "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVQYV2NgYAAAAAMAAWgmWQ0AAAAASUVORK5CYII=") or the asset ID (example: "asset_GTrL3mq4SXWyMxkOHRxlpw")

string

Deprecated: The input image as an AssetId. Prefer to use image with the asset ID instead.

string

Deprecated for type txt2img-ip-adapter and img2img-ip-adapter, use ipAdapterImageIds instead. The IpAdapter image as an AssetId. Cannot be set if ipAdapterImage is provided. Will be ignored if the ipAdapterImageIds parameter is provided.

number
0 to 1

Deprecated for type txt2img-ip-adapter and img2img-ip-adapter, use ipAdapterScales instead. IpAdapter scale factor (within [0.0, 1.0], default: 0.9). Will be ignored if the ipAdapterScales parameter is provided

string

The epoch of the model to use for the inference. Only available for Flux Lora Trained models.

boolean
Defaults to false

If set, generated assets will be hidden and not returned in the list of images of the inference or when listing assets (default: false)

ipAdapterScales
array of numbers

IpAdapter scale factors (within [0.0, 1.0], default: 0.9).

ipAdapterScales
string

The prompt not to guide the image generation, ignored when guidance < 1 (example: "((ugly face))") For Flux based model (not Fast-Flux): requires negativePromptStrength > 0 and active only for inference types txt2img / img2img / controlnet.

boolean
Defaults to false

Enable or disable the intermediate images generation (default: false)

concepts
array of objects
concepts
string

The controlnet input image as a data URL (example: "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVQYV2NgYAAAAAMAAWgmWQ0AAAAASUVORK5CYII=")

number
0 to 20

Controls how closely the generated image follows the prompt. Higher values result in stronger adherence to the prompt. Default and allowed values depend on the model type:

  • For Flux dev models, the default is 3.5 and allowed values are within [0, 10]
  • For Flux pro models, the default is 3 and allowed values are within [2, 5]
  • For SDXL models, the default is 6 and allowed values are within [0, 20]
  • For SD1.5 models, the default is 7.5 and allowed values are within [0, 20]
number
1 to 150

The number of denoising steps for each image generation (within [1, 150], default: 30)

number
1 to 128

The number of images to generate (within [1, 128], default: 4)

number
64 to 2048

The width of the generated images, must be a 8 multiple (within [64, 2048], default: 512) If model.type is sd-xl, sd-xl-lora, sd-xl-composition the width must be within [512, 2048] If model.type is sd-1_5, the width must be within [64, 1024] If model.type is flux.1.1-pro-ultra, you can use the aspectRatio parameter instead

number
0 to 10

Only applicable for flux-dev based models for txt2img, img2img, and controlnet inference types.

Controls the influence of the negative prompt. Default 0 means the negative prompt has no effect. Higher values increase negative prompt influence. Must be > 0 if negativePrompt is provided.

string
required

Full text prompt including the model placeholder. (example: "an illustration of phoenix in a fantasy world, flying over a mountain, 8k, bokeh effect")

Responses

Updated 5 months ago

Language
Credentials
Header
Response 
Click Try It! to start a request and see the response here! Or choose an example:
application/json
Updated 5 months ago