Retrieve Training Dataset

models.scores.retrieve_training_dataset(, ) -> ScoreRetrieveTrainingDatasetResponse

GET/models/{modelId}/scores/training-dataset

Get the training dataset scores for the given modelId

ParametersExpand Collapse

model_id: str

project_id: Optional[str]

The projectId used for ownership resource management. Either to assert ownership or to set the owner of the resource(s)

ReturnsExpand Collapse

class ScoreRetrieveTrainingDatasetResponse: …

details: Details

Different components used to compute the final score

asset_couples_captions_distance: List[DetailsAssetCouplesCaptionsDistance]

For each couple of assets in the training dataset, the distance between the captions

couple: List[str]

The couple of AssetIds (example: [‘asset_GTrL3mq4SXWyMxkOHRxlpw’, ‘asset_6VrLs8nC1DHWCfKGaVne0gBn’])

distance: float

The Dice-Sørensen distance between the captions of the assetIds of the couple.

maximum1

minimum0

asset_couples_similarity: List[DetailsAssetCouplesSimilarity]

For each couple of assets in the training dataset, the similarity between the assets

couple: List[str]

The couple of AssetIds (example: [‘asset_GTrL3mq4SXWyMxkOHRxlpw’, ‘asset_6VrLs8nC1DHWCfKGaVne0gBn’])

similarity: float

The cosine similarity between the assetIds of the couple.

The value is a number between -1 and 1. The higher the value, the more similar the assets are to each other

maximum1

minimum-1

assets_embedding_similarity: List[DetailsAssetsEmbeddingSimilarity]

For each asset, the similarity with the model’s embedding (excluding the current asset)

asset_id: str

The AssetId associated with their similarity (example: “asset_GTrL3mq4SXWyMxkOHRxlpw”)

similarity: float

The cosine similarity between the assetId and the model’s embedding minus the asset’s embedding

The value is a number between -1 and 1. The higher the value, the more similar the asset is to the model’s embedding

maximum1

minimum-1

assets_similarity: DetailsAssetsSimilarity

The min, mean, median of the assetsEmbeddingSimilarity

mean: float

maximum1

minimum-1

median: float

maximum1

minimum-1

min: float

maximum1

minimum-1

couples_captions_distance: DetailsCouplesCaptionsDistance

The min, mean, median of the assetCouplesCaptionsDistance

mean: float

maximum1

minimum0

median: float

maximum1

minimum0

min: float

maximum1

minimum0

couples_similarity: DetailsCouplesSimilarity

The min, mean, median of the assetCouplesSimilarity

mean: float

maximum1

minimum-1

median: float

maximum1

minimum-1

min: float

maximum1

minimum-1

metadata_score: float

Takes into account all missing data and warnings to compute a metadata score

maximum1

minimum-1

model_id: str

The model ID (example: “model_eyVcnFJcR92BxBkz7N6g5w”)

score: float

The training dataset’s score for the input modelId

The score is a number between 0 and 1. The higher the score, the most chance the model has to train well

maximum1

minimum0

status: Literal["complete", "incomplete", "unknown"]

The status of the training dataset’s score

Details if all the necessary data is present to compute the score

One of the following:

"complete"

"incomplete"

"unknown"

asset_couples_warnings: Optional[List[AssetCouplesWarning]]

Present only if their is a warning between the assets of the couples

couple: List[str]

The couple of AssetIds (example: [‘asset_GTrL3mq4SXWyMxkOHRxlpw’, ‘asset_6VrLs8nC1DHWCfKGaVne0gBn’])

message: List[Literal["Captions are identical", "Captions are too close", "Empty caption", 6 more]]

An indicative message concerning the couple of assets

One of the following:

"Captions are identical"

"Captions are too close"

"Empty caption"

"Missing caption"

"Missing metadata"

"Same asset"

"Similarity is too high"

"Similarity with other assets is too low"

"Width and height must be larger than 1024"

assets_warnings: Optional[List[AssetsWarning]]

Present only if the training image assets of the modelId have one or more warnings

asset_id: str

The AssetId which has a warning (example: “asset_GTrL3mq4SXWyMxkOHRxlpw”)

message: List[Literal["Captions are identical", "Captions are too close", "Empty caption", 6 more]]

An indicative message concerning the asset

One of the following:

"Captions are identical"

"Captions are too close"

"Empty caption"

"Missing caption"

"Missing metadata"

"Same asset"

"Similarity is too high"

"Similarity with other assets is too low"

"Width and height must be larger than 1024"

Retrieve Training Dataset

import os
from scenario_sdk import Scenario

client = Scenario(
    api_key=os.environ.get("SCENARIO_SDK_API_KEY"),  # This is the default and can be omitted
    api_secret=os.environ.get("SCENARIO_SDK_API_SECRET"),  # This is the default and can be omitted
)
response = client.models.scores.retrieve_training_dataset(
    model_id="modelId",
)
print(response.details)

{
  "details": {
    "assetCouplesCaptionsDistance": [
      {
        "couple": [
          "string",
          "string"
        ],
        "distance": 0
      }
    ],
    "assetCouplesSimilarity": [
      {
        "couple": [
          "string",
          "string"
        ],
        "similarity": -1
      }
    ],
    "assetsEmbeddingSimilarity": [
      {
        "assetId": "assetId",
        "similarity": -1
      }
    ],
    "assetsSimilarity": {
      "mean": -1,
      "median": -1,
      "min": -1
    },
    "couplesCaptionsDistance": {
      "mean": 0,
      "median": 0,
      "min": 0
    },
    "couplesSimilarity": {
      "mean": -1,
      "median": -1,
      "min": -1
    },
    "metadataScore": -1
  },
  "modelId": "modelId",
  "score": 0,
  "status": "complete",
  "assetCouplesWarnings": [
    {
      "couple": [
        "string",
        "string"
      ],
      "message": [
        "Captions are identical"
      ]
    }
  ],
  "assetsWarnings": [
    {
      "assetId": "assetId",
      "message": [
        "Captions are identical"
      ]
    }
  ]
}

Returns Examples

{
  "details": {
    "assetCouplesCaptionsDistance": [
      {
        "couple": [
          "string",
          "string"
        ],
        "distance": 0
      }
    ],
    "assetCouplesSimilarity": [
      {
        "couple": [
          "string",
          "string"
        ],
        "similarity": -1
      }
    ],
    "assetsEmbeddingSimilarity": [
      {
        "assetId": "assetId",
        "similarity": -1
      }
    ],
    "assetsSimilarity": {
      "mean": -1,
      "median": -1,
      "min": -1
    },
    "couplesCaptionsDistance": {
      "mean": 0,
      "median": 0,
      "min": 0
    },
    "couplesSimilarity": {
      "mean": -1,
      "median": -1,
      "min": -1
    },
    "metadataScore": -1
  },
  "modelId": "modelId",
  "score": 0,
  "status": "complete",
  "assetCouplesWarnings": [
    {
      "couple": [
        "string",
        "string"
      ],
      "message": [
        "Captions are identical"
      ]
    }
  ],
  "assetsWarnings": [
    {
      "assetId": "assetId",
      "message": [
        "Captions are identical"
      ]
    }
  ]
}