Retrieve Training Dataset

GET/models/{modelId}/scores/training-dataset

Get the training dataset scores for the given modelId

Path ParametersExpand Collapse

modelId: string

Query ParametersExpand Collapse

projectId: optional string

The projectId used for ownership resource management. Either to assert ownership or to set the owner of the resource(s)

ReturnsExpand Collapse

details: object { assetCouplesCaptionsDistance, assetCouplesSimilarity, assetsEmbeddingSimilarity, 4 more }

Different components used to compute the final score

assetCouplesCaptionsDistance: array of object { couple, distance }

For each couple of assets in the training dataset, the distance between the captions

couple: array of string

The couple of AssetIds (example: [‘asset_GTrL3mq4SXWyMxkOHRxlpw’, ‘asset_6VrLs8nC1DHWCfKGaVne0gBn’])

distance: number

The Dice-Sørensen distance between the captions of the assetIds of the couple.

maximum1

minimum0

assetCouplesSimilarity: array of object { couple, similarity }

For each couple of assets in the training dataset, the similarity between the assets

couple: array of string

The couple of AssetIds (example: [‘asset_GTrL3mq4SXWyMxkOHRxlpw’, ‘asset_6VrLs8nC1DHWCfKGaVne0gBn’])

similarity: number

The cosine similarity between the assetIds of the couple.

The value is a number between -1 and 1. The higher the value, the more similar the assets are to each other

maximum1

minimum-1

assetsEmbeddingSimilarity: array of object { assetId, similarity }

For each asset, the similarity with the model’s embedding (excluding the current asset)

assetId: string

The AssetId associated with their similarity (example: “asset_GTrL3mq4SXWyMxkOHRxlpw”)

similarity: number

The cosine similarity between the assetId and the model’s embedding minus the asset’s embedding

The value is a number between -1 and 1. The higher the value, the more similar the asset is to the model’s embedding

maximum1

minimum-1

assetsSimilarity: object { mean, median, min }

The min, mean, median of the assetsEmbeddingSimilarity

mean: number

maximum1

minimum-1

median: number

maximum1

minimum-1

min: number

maximum1

minimum-1

couplesCaptionsDistance: object { mean, median, min }

The min, mean, median of the assetCouplesCaptionsDistance

mean: number

maximum1

minimum0

median: number

maximum1

minimum0

min: number

maximum1

minimum0

couplesSimilarity: object { mean, median, min }

The min, mean, median of the assetCouplesSimilarity

mean: number

maximum1

minimum-1

median: number

maximum1

minimum-1

min: number

maximum1

minimum-1

metadataScore: number

Takes into account all missing data and warnings to compute a metadata score

maximum1

minimum-1

modelId: string

The model ID (example: “model_eyVcnFJcR92BxBkz7N6g5w”)

score: number

The training dataset’s score for the input modelId

The score is a number between 0 and 1. The higher the score, the most chance the model has to train well

maximum1

minimum0

status: "complete" or "incomplete" or "unknown"

The status of the training dataset’s score

Details if all the necessary data is present to compute the score

One of the following:

"complete"

"incomplete"

"unknown"

assetCouplesWarnings: optional array of object { couple, message }

Present only if their is a warning between the assets of the couples

couple: array of string

The couple of AssetIds (example: [‘asset_GTrL3mq4SXWyMxkOHRxlpw’, ‘asset_6VrLs8nC1DHWCfKGaVne0gBn’])

message: array of "Captions are identical" or "Captions are too close" or "Empty caption" or 6 more

An indicative message concerning the couple of assets

One of the following:

"Captions are identical"

"Captions are too close"

"Empty caption"

"Missing caption"

"Missing metadata"

"Same asset"

"Similarity is too high"

"Similarity with other assets is too low"

"Width and height must be larger than 1024"

assetsWarnings: optional array of object { assetId, message }

Present only if the training image assets of the modelId have one or more warnings

assetId: string

The AssetId which has a warning (example: “asset_GTrL3mq4SXWyMxkOHRxlpw”)

message: array of "Captions are identical" or "Captions are too close" or "Empty caption" or 6 more

An indicative message concerning the asset

One of the following:

"Captions are identical"

"Captions are too close"

"Empty caption"

"Missing caption"

"Missing metadata"

"Same asset"

"Similarity is too high"

"Similarity with other assets is too low"

"Width and height must be larger than 1024"

Retrieve Training Dataset

curl https://api.cloud.scenario.com/v1/models/$MODEL_ID/scores/training-dataset \
    -u "$SCENARIO_SDK_API_KEY:SCENARIO_SDK_API_SECRET"

{
  "details": {
    "assetCouplesCaptionsDistance": [
      {
        "couple": [
          "string",
          "string"
        ],
        "distance": 0
      }
    ],
    "assetCouplesSimilarity": [
      {
        "couple": [
          "string",
          "string"
        ],
        "similarity": -1
      }
    ],
    "assetsEmbeddingSimilarity": [
      {
        "assetId": "assetId",
        "similarity": -1
      }
    ],
    "assetsSimilarity": {
      "mean": -1,
      "median": -1,
      "min": -1
    },
    "couplesCaptionsDistance": {
      "mean": 0,
      "median": 0,
      "min": 0
    },
    "couplesSimilarity": {
      "mean": -1,
      "median": -1,
      "min": -1
    },
    "metadataScore": -1
  },
  "modelId": "modelId",
  "score": 0,
  "status": "complete",
  "assetCouplesWarnings": [
    {
      "couple": [
        "string",
        "string"
      ],
      "message": [
        "Captions are identical"
      ]
    }
  ],
  "assetsWarnings": [
    {
      "assetId": "assetId",
      "message": [
        "Captions are identical"
      ]
    }
  ]
}

Returns Examples

{
  "details": {
    "assetCouplesCaptionsDistance": [
      {
        "couple": [
          "string",
          "string"
        ],
        "distance": 0
      }
    ],
    "assetCouplesSimilarity": [
      {
        "couple": [
          "string",
          "string"
        ],
        "similarity": -1
      }
    ],
    "assetsEmbeddingSimilarity": [
      {
        "assetId": "assetId",
        "similarity": -1
      }
    ],
    "assetsSimilarity": {
      "mean": -1,
      "median": -1,
      "min": -1
    },
    "couplesCaptionsDistance": {
      "mean": 0,
      "median": 0,
      "min": 0
    },
    "couplesSimilarity": {
      "mean": -1,
      "median": -1,
      "min": -1
    },
    "metadataScore": -1
  },
  "modelId": "modelId",
  "score": 0,
  "status": "complete",
  "assetCouplesWarnings": [
    {
      "couple": [
        "string",
        "string"
      ],
      "message": [
        "Captions are identical"
      ]
    }
  ],
  "assetsWarnings": [
    {
      "assetId": "assetId",
      "message": [
        "Captions are identical"
      ]
    }
  ]
}