Music Structure Analysis

Feature Introduction

Music structure analysis capability can parse the structure of user-input pop music, locate intro, verse, chorus, interlude, bridge, outro and other sections. Input a music file, output the parsed structure results

API Description

Request Method: POST (HTTP)

Request URL: https://api.mediax.tencent.com/job

Request Header: Content-Type: application/json

Request Flow: The API includes 'Create Task' and 'Query Task'. After creating a task, users can actively query the task to know the result, or input a callback address when creating the task, and the task will automatically callback to that address after completion

Other Requirements: Common audio formats are recommended, such as mp3, wav; and the input file should be pop music

Create Task

Parameter Description

Parameter	Required	Type	Description
action	Yes	string	Common parameter, here is CreateJob
secretId	Yes	string	Common parameter, user SecretId
secretKey	Yes	string	Common parameter, user SecretKey
createJobRequest	Yes	object
- inputs	Yes	Array of Input	Input, input structure array
- outputs	Yes	Array of Output	Output, output structure array
- callback	No	string	Callback address, default: callback disabled
- customId	No	string	User-defined task ID, less than 64 characters
- timeout	No	int	Task timeout, in seconds. Task will be set to ERROR after timeout

Input

Parameter	Required	Type	Description
url	No	string	Source URL address, choose one with source field
source	No	object	Repository source settings, choose one with url field
- contentId	Yes	string	Repository ID
- path	Yes	string	Source path

Output

Parameter	Required	Type	Description
inputSelectors	Yes	Array of int	Input source for this output
smartContentDescriptor	Yes	SmartContentDescriptor	Smart capability description, default: empty
- musicStructureAnalysis	Yes	object	Music structure analysis
-- structureAnalysis	Yes	bool	Music structure analysis switch

Request Example:

{
  "action": "CreateJob",
  "secretId": "{secretId}",
  "secretKey": "{secretKey}",
  "createJobRequest": {
    "customId": "{customId}",
    "callback": "{callback}",
    "inputs": [
      {
        "url": "{url}"
      }
    ],
    "outputs": [
      {
        "inputSelectors": [0],
        "smartContentDescriptor": {
          "musicStructureAnalysis": {
            "structureAnalysis": true
          }
        }
      }
    ]
  }
}

Response Example:

{
  "requestId": "ac004192-110b-46e3-ade8-4e449df84d60",
  "createJobResponse": {
    "job": {
      "id": "13f342e4-6866-450e-b44e-3151431c578b",
      "state": 1,
      "customId": "{customId}",
      "callback": "{callback}",
      "inputs": [
        {
          "url": "{url}"
        }
      ],
      "outputs": [
        {
          "inputSelectors": [0],
          "smartContentDescriptor": {
            "musicStructureAnalysis": {
              "structureAnalysis": true
            }
          }
        }
      ],
      "timing": {
        "createdAt": "1603432763000",
        "startedAt": "0",
        "completedAt": "0"
      }
    }
  }
}

State

Value	Meaning
1	SUBMITTED
2	PROCESSING
3	COMPLETED
4	ERROR
5	CANCELED

Get Task Information

There are two ways to get information: active query and passive callback.

Active query has two types of query interfaces based on id category: one is to query based on user-defined id, since the platform cannot guarantee the uniqueness of this id, it returns a Job array (see 1); the other is to query by the id in the response after creating the task (see 2)
Passive callback requires filling in the callback field when creating the task, the platform will send the Job structure to the address specified by callback after the task enters the completion state (COMPLETED/ERROR) (see 3), the platform recommends using passive callback to get task results.

In the music structure analysis capability, if the queried task is successful (state=3), the task's Output will carry a smartContentResult structure, in which the musicStructureAnalysis field (MusicStructureAnalysisResult) stores the structure analysis results.

MusicStructureAnalysisResult Field Description

Parameter	Type	Description
chrous	Array of StartEndPair	Chorus section list
verse	Array of StartEndPair	Verse section list
intro	Array of StartEndPair	Intro section list
inter	Array of StartEndPair	Interlude section list
outro	Array of StartEndPair	Outro section list
bridge	Array of StartEndPair	Bridge section list

StartEndPair

Parameter	Type	Description
start	string	Start time
end	string	End time

1. Active Query, Based on User-Defined customId Passed When Creating Task Request Example:

{
  "action": "ListJobs",
  "secretId": "{secretId}",
  "secretKey": "{secretKey}",
  "listJobsRequest": {
    "customId": "{customId}"
  }
}

Response Example:

{
  "requestId": "c9845a99-34e3-4b0f-80f5-f0a2a0ee8896",
  "listJobsResponse": {
    "jobs": [
      {
        "id": "a95e9d74-6602-4405-a3fc-6408a76bcc98",
        "state": 3,
        "customId": "{customId}",
        "callback": "{callback}",
        "timing": {
          "createdAt": "1610513575000",
          "startedAt": "1610513575000",
          "completedAt": "1610513618000"
        },
        "inputs": [{ "url": "{url}" }],
        "outputs": [
          {
            "inputSelectors": [0],
            "smartContentDescriptor": {
              "musicStructureAnalysis": {
                "structureAnalysis": true
              }
            },
            "smartContentResult": {
              "musicStructureAnalysis": {
                "chrous": [{ "start": "60.0", "end": "90.0" }],
                "verse": [{ "start": "10.0", "end": "60.0" }],
                "intro": [{ "start": "0.0", "end": "10.0" }],
                "inter": [],
                "outro": [{ "start": "90.0", "end": "100.0" }],
                "bridge": []
              }
            }
          }
        ]
      }
    ],
    "total": 1
  }
}

2. Active Query, Based on id in Response After Creating Task Request Example:

{
  "action": "GetJob",
  "secretId": "{secretId}",
  "secretKey": "{secretKey}",
  "getJobRequest": {
    "id": "{id}"
  }
}

Response Example:

{
  "requestId": "c9845a99-34e3-4b0f-80f5-f0a2a0ee8896",
  "getJobResponse": {
    "job": {
      "id": "a95e9d74-6602-4405-a3fc-6408a76bcc98",
      "state": 3,
      "customId": "{customId}",
      "callback": "{callback}",
      "timing": {
        "createdAt": "1610513575000",
        "startedAt": "1610513575000",
        "completedAt": "1610513618000"
      },
      "inputs": [{ "url": "{url}" }],
      "outputs": [
        {
          "inputSelectors": [0],
          "smartContentDescriptor": {
            "musicStructureAnalysis": {
              "structureAnalysis": true
            }
          },
          "smartContentResult": {
            "musicStructureAnalysis": {
              "chrous": [{ "start": "60.0", "end": "90.0" }],
              "verse": [{ "start": "10.0", "end": "60.0" }],
              "intro": [{ "start": "0.0", "end": "10.0" }],
              "inter": [],
              "outro": [{ "start": "90.0", "end": "100.0" }],
              "bridge": []
            }
          }
        }
      ]
    }
  }
}

3. Passive Callback

The entire Job structure of the task that enters the completion state (COMPLETED/ERROR) will be sent to the address corresponding to the callback field specified by the user when creating the task. See the active query example (under getJobResponse) for the Job structure