Music Tag

Feature Introduction

Music tag capability automatically generates classification tags and probability values for each dimension through multi-dimensional analysis of music. Currently supported seven dimensions include:

• Vocal
• Mood
• Language
• Genre
• Scene
• Background Music
• Intensity

API Description

Request Method:

POST (HTTP)

Request URL:

https://api.mediax.tencent.com/job

Request Header:

Content-Type: application/json

Request Flow:

The API includes 'Create Task' and 'Query Task'. After creating a task, users can actively query the task to know the result, or input a callback address when creating the task, and the task will automatically callback to that address after completion

Other Requirements:

File format: Common audio formats are recommended, such as mp3, wav

Create Task

Parameter Description

Input

Note: Only one of the source and url fields can be present

Output

- vocal, 2 types of vocal tags
- language, 21 types of language tags
- mood, 11 types of mood tags
- genre, 37 types of genre tags
- scene, 36 types of scene tags
- bgm, 2 types of background music tags
- intensity, 2 types of intensity tags
- all, get all 7 dimensions of tags

Request Example:

{
  "action": "CreateJob",
  "secretId": "{secretId}",
  "secretKey": "{secretKey}",
  "createJobRequest": {
    "customId": "{customId}",
    "callback": "{callback}",
    "inputs": [
      {
        "url": "{url}"
      }
    ],
    "outputs": [
      {
        "inputSelectors": [0],
        "smartContentDescriptor": {
          "musicTagV2": {
            "tagType": ["mood", "vocal", "language", "scene", "genre", "bgm"]
          }
        }
      }
    ]
  }
}

Response Example:

{
  "requestId": "ac004192-110b-46e3-ade8-4e449df84d60",
  "createJobResponse": {
    "job": {
      "id": "13f342e4-6866-450e-b44e-3151431c578b",
      "state": 1, // See state description below
      "customId": "{customId}",
      "callback": "{callback}",
      "inputs": [
        {
          "url": "{url}"
        }
      ],
      "outputs": [
        {
          "inputSelectors": [0],
          "smartContentDescriptor": {
            "musicTagV2": {
              "tagType": ["mood", "vocal", "language", "scene", "genre", "bgm"]
            }
          }
        }
      ],
      "timing": {
        "createdAt": "1603432763000",
        "startedAt": "0",
        "completedAt": "0"
      }
    }
  }
}

State

Get Task Information

There are two ways to get information: active query and passive callback.

• Active query has two types of query interfaces based on id category: one is to query based on user-defined id, since the platform cannot guarantee the uniqueness of this id, it returns a Job array (see 1); the other is to query by the id in the response after creating the task (see 2)
• Passive callback requires filling in the callback field when creating the task, the platform will send the Job structure to the address specified by callback after the task enters the completion state (COMPLETED/ERROR) (see 3), the platform recommends using passive callback to get task results.

In the music tag capability, if the queried task is successful (state=3), the task's Output will carry a smartContentResult structure, in which the audioTagger field is a result object array, each result object has tag type (tagType), tag name (tag), probability (probility)

Active Query, Based on User-Defined customId Passed When Creating Task

Request Example:

{
  "action": "ListJobs",
  "secretId": "{secretId}",
  "secretKey": "{secretKey}",
  "listJobsRequest": {
    "customId": "{customId}"
  }
}

Note: jobs in the listJobs response is an array

Response Example:

{
  "requestId": "c9845a99-34e3-4b0f-80f5-f0a2a0ee8896",
  "listJobsResponse": {
    "jobs": [
      {
        "id": "a95e9d74-6602-4405-a3fc-6408a76bcc98",
        "state": 3,
        "customId": "{customId}",
        "callback": "{callback}",
        "timing": {
          "createdAt": "1610513575000",
          "startedAt": "1610513575000",
          "completedAt": "1610513618000"
        },
        "inputs": [
          {
            "url": "{url}"
          }
        ],
        "outputs": [
          {
            "inputSelectors": [0],
            "smartContentDescriptor": {
              "musicTagV2": {
                "tagType": ["mood", "vocal", "language", "scene", "genre", "bgm"]
              }
            },
            "smartContentResult": {
              "musigTagV2": {
                "musicTagV2": {
                  "vocal": {
                    "label": "Vocal",
                    "prob": "1.0000"
                  },
                  "language": {
                    "label": "Other",
                    "prob": "0.0817"
                  },
                  "mood": {
                    "label": "Exciting",
                    "prob": "0.2953"
                  },
                  "genre": {
                    "label": "Zhuang",
                    "prob": "0.7710"
                  },
                  "scene": {
                    "label": "Prayer",
                    "prob": "0.8886"
                  },
                  "intensity": {}
                }
              }
            }
          }
        ]
      }
    ],
    "total": 1
  }
}

Active Query, Based on id in Response After Creating Task

Request Example:

{
  "action": "GetJob",
  "secretId": "{secretId}",
  "secretKey": "{secretKey}",
  "getJobRequest": {
    "id": "{id}"
  }
}

Response Example:

{
  "requestId": "c9845a99-34e3-4b0f-80f5-f0a2a0ee8896",
  "getJobResponse": {
    "job": {
      "id": "a95e9d74-6602-4405-a3fc-6408a76bcc98",
      "state": 3,
      "customId": "{customId}",
      "callback": "{callback}",
      "timing": {
        "createdAt": "1610513575000",
        "startedAt": "1610513575000",
        "completedAt": "1610513618000"
      },
      "inputs": [
        {
          "url": "{url}"
        }
      ],
      "outputs": [
        {
          "inputSelectors": [0],
          "smartContentDescriptor": {
            "musicTagV2": {
              "tagType": ["mood", "vocal", "language", "scene", "genre", "bgm"]
            }
          },
          "smartContentResult": {
            "musigTagV2": {
              "musicTagV2": {
                "vocal": {
                  "label": "Vocal",
                  "prob": "1.0000"
                },
                "language": {
                  "label": "Other",
                  "prob": "0.0817"
                },
                "mood": {
                  "label": "Exciting",
                  "prob": "0.2953"
                },
                "genre": {
                  "label": "Zhuang",
                  "prob": "0.7710"
                },
                "scene": {
                  "label": "Prayer",
                  "prob": "0.8886"
                },
                "intensity": {}
              }
            }
          }
        }
      ]
    }
  }
}

Passive Callback

The entire Job structure of the task that enters the completion state (COMPLETED/ERROR) will be sent to the address corresponding to the callback field specified by the user when creating the task. See the active query example (under getJobResponse) for the Job structure

cURL Example

Create Task

curl --location --request POST 'https://api.mediax.tencent.com/test/job' \
--header 'Content-Type: application/json' \
--data-raw '{
    "action": "CreateJob",
    "secretId": "{secretId}",
    "secretKey": "{secretKey}",
    "createJobRequest": {
        "customId": "{customId}",
        "callback": "{callback}",
        "inputs": [{
                "url": "{url}"
            }],
        "outputs": [
            {
                "inputSelectors": [0],
                "smartContentDescriptor": {
                    "audioTagger": {
                        "tags": [1, 2, 3]
                    }
                }
            }]
    }
}'

Query Task

# Based on customId
curl --location --request POST 'https://api.mediax.tencent.com/test/job' \
--header 'Content-Type: application/json' \
--data-raw '{
    "action": "ListJobs",
    "secretId": "{secretId}",
    "secretKey": "{secretKey}",
    "listJobsRequest": {
        "customId": "{customId}"
    }
}'

# Based on id
curl --location --request POST 'https://api.mediax.tencent.com/test/job' \
--header 'Content-Type: application/json' \
--data-raw '{
    "action": "GetJob",
    "secretId": "{secretId}",
    "secretKey": "{secretKey}",
    "getJobRequest": {
        "id": "{id}"
    }
}'

Appendix: Currently Supported Tags