Audjust API Overview

Analyze

Extracts a self-similarity matrix, beat(-ish) boundaries, duration, BPM, and optionally raw audio features from an audio file. You can cache these results and pass them to the resize API below or use them to build new functionality of your own. Useful for beat extraction, aligning animations or video to audio, and audio similarity search.

POST https://api.audjust.com/analyze

{
	"resultWebhook": "https://example.com/audjust-webhook",
	"sourceFileUrl": "https://audjust.com/sample.mp3",
	"features": ["raw_features"]
}

You will get back a response like this immediately:

{
	"success": true,
	"message": "Request queued.",
	"requestId": "z7yzictf6uct4vo9f70buboy97pgstms",
	"resultUrl": "https://results.api.audjust.com/z7yzictf6uct4vo9f70buboy97pgstms/analysis.json"
}

Then, later, your webhook will get an HTTP POST request with a JSON payload as follows:

{
	"requestId": "z7yzictf6uct4vo9f70buboy97pgstms",
	"sourceFileUrl": "https://audjust.com/sample.mp3",
	"result": {
		"duration": 175.255,
		"tempo": 99,
		"similarity": {
			"data": "...snip...",
			"min": 32,
			"max": 126
		},
		"boundaries": [
			0.0, 0.069, 0.65, ...snip... 172.57, 173.174, 175.24
		],
		"features": [
			[0.275, 0.202, 0.244, 0.296, 0.372, 0.359, 0.392, 1.0, 0.579, 0.328, 0.3, 0.288, 0.201],
			...snip...
			[0.578, 0.763, 1.0, 0.732, 0.529, 0.416, 0.375, 0.389, 0.593, 0.593, 0.544, 0.426, 0.0]
		]
	},
	"success": true,
	"creditCost": 15
}

You can (and should!) save this response for later API queries or to use in your application yourself.

Resize

Given extracted information from the analysis API and a target length, this API generates a list of segments that forms a new piece of audio close to the provided target duration.

POST https://api.audjust.com/resize

{
	"resultWebhook": "https://example.com/audjust-webhook", // optional
	"targetLengthSeconds": 12,
	"similarity": {
		"data": "...snip...",
		"min": 32,
		"max": 126
	},
	"boundaries": [
		0.0, 0.069, 0.65, ...snip... 172.57, 173.174, 175.24
	],
}

You will get back a response like this immediately if you provide the resultWebhook parameter:

{
	"success": true,
	"message": "Request queued.",
	"requestId": "v76xq87at9e2zbyyrrqs39jpwvev5uty",
	"resultUrl": "https://results.api.audjust.com/v76xq87at9e2zbyyrrqs39jpwvev5uty/resize.json"
}

If you do not provide the resultWebhook parameter, you will get back the response below. If you do, your webhook will later get an HTTP POST request with a JSON payload as follows:

{
	"requestId": "v76xq87at9e2zbyyrrqs39jpwvev5uty",
	"result": [
		{
			"dissimilarity": 0.798,
			"blendedScore": 0.229,
			"segmentsBeats": [
				[0, 2],
				[284, 291]
			],
			"segmentsSeconds": [
				[0, 1.137],
				[169.552, 175.24]
			],
			"durationSeconds": 7
		},
		...snip...
	],
	"success": true,
	"creditCost": 4
}

result will be an array of (generally 20) results. Each result is made up of segments that, pieced together, represent the final audio. Each result has:

In your application you can use blendedScore to sort the results or build your own metric (using features like closeness to the target length or number of cuts).

Export

Generates one or more audio files by stitching together a list of segments (from the resize API or whatever other source you might have).

POST https://api.audjust.com/export

{
	"sourceFileUrl": "https://audjust.com/sample.mp3",
	"resultWebhook": "https://webhook.site/56a06326-c557-487b-9aad-dccec6623a5e",
	"exportFormat": "mp3_128",
	"segmentsGroups": [
		[
			[0, 1.137],
			[169.552, 175.24]
		],
		[
			[0, 2.739],
			[170.155, 175.24]
		]
	]
}

You will get back a response like this immediately:

{
	"success": true,
	"message": "Request queued.",
	"requestId": "32pwfpqqogl1k284ww41c4xmacg0obwa",
	"resultUrls": [
		"https://results.api.audjust.com/32pwfpqqogl1k284ww41c4xmacg0obwa/export_1.mp3",
		"https://results.api.audjust.com/32pwfpqqogl1k284ww41c4xmacg0obwa/export_2.mp3"
	]
}

Then, later, your webhook will get an HTTP POST request with a JSON payload as follows:

{
	"requestId": "32pwfpqqogl1k284ww41c4xmacg0obwa",
	"sourceFileUrl": "https://audjust.com/sample.mp3",
	"resultUrls": [
		"https://results.api.audjust.com/32pwfpqqogl1k284ww41c4xmacg0obwa/export_1.mp3",
		"https://results.api.audjust.com/32pwfpqqogl1k284ww41c4xmacg0obwa/export_2.mp3"
	],
	"failures": {},
	"success": true,
	"creditCost": 1
}

You will need to grab the actual audio files from the URLs provided in the response. The URLs expire, so make sure to download them promptly.

Failed Requests

After queueing, requests might not succeed because of issues in Audjust service (like failing to retrieve the source file or internal issues). In that case, your webhook will be notified of the failure on a best-effort basis. Requests will not take longer than 7 minutes to complete. You can consider a request failed if it takes longer than that.

See recent requests on the API console to see the status of in-flight requests and view debugging information for failed requests.

Self-Similarity Matrix

The similarity object encodes a self-similarity matrix that represents how similar each beat is to another beat in the same audio.

You can use the self-similarity matrix to determine how similar some part of the song is to another part. The matrix is a compressed string that you can decode using the following functions:

function parseSimilarityMapString(similarityString: string, start: number, end: number) {
	return new Float32Array(
		[...similarityString].map((_, i) => (similarityString.charCodeAt(i) - start) / (end - start)),
	);
}

function getDiagIndex(row: number, col: number): number {
	const i = Math.max(row, col);
	const j = Math.min(row, col);
	return Math.floor(((i + 1) * i) / 2 + j) as number;
}

You can then use the matrix like this to get the similarity between two beats i and j:

const matrix = parseSimilarityMapString(similarity.data, similarity.min, similarity.max);
const similarity = matrix[getDiagIndex(i, j)];

similarity will be a number between 0 and 1 where 0 is completely dissimilar and 1 is identical. In practice, you can use a threshold to determine if two beats are similar enough.

Result Delivery

Some of the operations below are long-running and take some time to complete. To avoid having clients wait on responses to their HTTP requests (and potentially time out), they will instead be asynchronously notified upon completion of the request. This is done using a webhook.

You need to specify a webhook that will be notified when the processing is done for the analyze and export endpoints (they take a while). It is optional for the resize endpoint. When you make an API call:

1. Your request is immediately acknowledged, assigned a requestId, and put in a queue.
2. Audjust will work on processing your request in the background.
3. When finished, the results are POSTed to your webhook and the request is marked as finished.

For development, you can create a debug webhook on sites like Webhook.site.

Artifact Expiration

Generated artifacts (like analysis results and exported files) are stored for 30 days (subject to change). Make sure to download them promptly.

The analyze, resize, and export endpoints are billed in credits and generally increase with the length of the audio and when you process more data.

Durations are rounded up to the nearest minute and successful requests cost at least 1 credit.