HomeEducationReverse-Engineering YouTube: Revisited • Oleksii Holub Receive US

Reverse-Engineering YouTube: Revisited • Oleksii Holub Receive US

Again in 2017 I wrote an article through which I tried to elucidate how YouTube works underneath the hood, the way it serves streams to the consumer, and in addition how one can exploit that data to obtain movies from the positioning. The first purpose of that write-up was to share a few of the issues I discovered whereas engaged on YoutubeExplode — a .NET library that gives a structured abstraction layer over YouTube’s inside API.

There’s one factor that builders like greater than constructing issues — and that’s breaking issues constructed by different folks. So, naturally, my article attracted fairly a little bit of consideration and nonetheless stays one of the crucial standard posts on this weblog. In any case, I had a number of enjoyable doing the analysis, and I am glad that it was additionally helpful to different folks.

Nevertheless, many issues have modified within the 5 years because the article was printed: YouTube has advanced as a platform, went by a number of UI redesigns, and utterly overhauled its frontend codebase. Many of the inside endpoints that have been reverse-engineered within the early days have been step by step getting eliminated altogether. The truth is, practically all the things I wrote within the authentic submit has turn into out of date and now solely serves as a historic reference.

I do know that there is nonetheless plenty of curiosity round this matter, so I have been which means to revisit it and make a follow-up article with new and up to date data. Seeing as YouTube has as soon as once more entered a quiet part when it comes to change and innovation, I figured that now’s lastly a superb time to do it.

On this article, I am going to cowl the present state of YouTube’s inside API, spotlight an important adjustments, and clarify how all the things works as we speak. Similar to earlier than, I’ll deal with the video playback side of the platform, outlining all the things it is advisable do to be able to resolve video streams and obtain them.

Retrieving the metadata

For those who’ve labored with YouTube up to now, you may in all probability keep in mind /get_video_info. This inside API controller was used all through YouTube’s consumer code to retrieve video metadata, obtainable streams, and all the things else the participant wanted to render it. The origin of this endpoint traces again to the Flash Participant days of YouTube, and it was nonetheless accessible till as late as July 2021, earlier than it was lastly eliminated.

In addition to /get_video_info, YouTube has additionally dropped many different endpoints, resembling /get_video_metadata (in November 2017) and /list_ajax (in February 2021), as half of a bigger effort to ascertain a extra organized API construction. Now, as a substitute of getting a bunch of randomly scattered endpoints with unpredictable codecs and utilization patterns, YouTube’s inside API is comprised out of a coherent set of routes nested beneath the /youtubei/ path.

Particularly, a lot of the information beforehand obtainable from /get_video_info can now be pulled utilizing the /youtubei/v1/participant route. In contrast to its predecessor, this endpoint expects a POST request — and the payload seems to be like this:

// POST 

  "videoId": "e_S9VvJM1PI",
      "clientName": "ANDROID",
      "clientVersion": "17.10.35",
      "androidSdkVersion": 30

Very first thing you may discover is that this endpoint requires an API key, which is handed by the key parameter within the URL. Every YouTube consumer has its personal key assigned to it, however I discovered that the endpoint would not really care which one you employ, so long as it is legitimate. As a result of the keys do not rotate both, it is secure to choose one and arduous code it as a part of the URL.

The request physique itself is a JSON object with two top-level properties: videoId and context. The previous is the 11-character ID of the video you wish to retrieve the metadata for, whereas the latter comprises numerous data that YouTube makes use of to tailor the response to the consumer’s preferences and capabilities.

Particularly, relying on the consumer you select to impersonate utilizing the clientName and clientVersion properties, the response could include barely totally different information, or simply fail to resolve altogether for sure movies. Whereas there are a lot of obtainable purchasers, only some of them present a measurable benefit over the others — which is why I used ANDROID within the instance above, because it’s the best to work with.

After you obtain the response, you need to discover a JSON object that comprises the video metadata, stream descriptors, closed captions, exercise monitoring URLs, advert placements, post-playback display screen components — mainly all the things that the consumer wants to be able to present the video to the consumer. It is a huge blob of knowledge, so to make issues less complicated I’ve outlined solely essentially the most attention-grabbing components beneath:

    "videoId": "e_S9VvJM1PI",
    "title": "Icon For Rent - Make A Transfer",
    "lengthSeconds": "184",
    "key phrases": ["Icon", "For", "Hire", "Make", "Move", "Tooth", "Nail", "(TNN)", "Rock"],
    "channelId": "UCKvT-8xU_BTJGvsQ5lR23TQ",
    "isOwnerViewing": false,
    "shortDescription": "Music video by Icon For Rent performing Make A Transfer. (P) (C) 2011 Tooth & Nail Information. All rights reserved. Unauthorized replica is a violation of relevant legal guidelines.  Manufactured by Tooth & Nail,nn#IconForHire #MakeAMove #Vevo #Rock #VevoOfficial #OfficialMusicVideo",
    "isCrawlable": true,
      "thumbnails": [
          "url": "
          "width": 120,
          "height": 90
          "url": "
          "width": 320,
          "height": 180
          "url": "
          "width": 480,
          "height": 360
          "url": "
          "width": 640,
          "height": 480
    "allowRatings": true,
    "viewCount": "54284943",
    "writer": "IconForHireVEVO",
    "isPrivate": false,
    "isUnpluggedCorpus": false,
    "isLiveContent": false
    "standing": "OK",
    "playableInEmbed": true
    "expiresInSeconds": "21540",
    "codecs": [
      /* ... */
    "adaptiveFormats": [
      /* ... */
  /* ... omitted ~1800 traces of irrelevant information ... */

As you possibly can instantly see, the response comprises a spread of helpful data. From videoDetails, you possibly can extract the video title, period, writer, view rely, thumbnails, and different related metadata. This consists of a lot of the stuff one can find on the video web page, except for likes, dislikes, channel subscribers, and different bits that aren’t rendered immediately by the participant. If you wish to get that information as effectively, you’ll have to scrape the /watch web page individually.

Subsequent, the playabilityStatus object signifies whether or not the video is playable inside the context of the consumer that made the request. In case it isn’t, a motive property can be included with a human-readable message explaining why — for instance, as a result of the video is meant for mature audiences, or as a result of it isn’t accessible within the present area. When coping with unplayable movies, you may nonetheless be capable of get hold of their metadata, however you will not be capable of retrieve any streams.

Lastly, assuming the video is marked as playable, streamingData ought to include the record of streams that YouTube offered for the playback. These are divided into the codecs and adaptiveFormats arrays contained in the response, and correspond to the varied high quality choices obtainable within the participant.

The separation between codecs and adaptiveFormats is a bit complicated and I discovered that it would not refer a lot to the delivery method, however fairly to the way in which the streams are encoded. Particularly, the codecs array describes conventional video streams, the place each the audio and the video tracks are mixed right into a single container forward of time, whereas adaptiveFormats lists devoted audio-only and video-only streams, that are overlaid at run-time by the participant.

You may discover that a lot of the playback choices, particularly the higher-fidelity ones, are offered utilizing the latter method, as a result of it is extra versatile when it comes to bandwidth. By having the ability to swap the audio and video streams independently, the participant can adapt to various community situations, in addition to totally different playback contexts — for instance, by requesting solely the audio stream if the consumer is consuming content material from YouTube Music.

So far as the metadata is worried, each arrays are very comparable and include objects with the next construction:

  "itag": 18,
  "url": "
  "mimeType": "video/mp4; codecs="avc1.42001E, mp4a.40.2"",
  "lastModified": "1665725827618480",
  "approxDurationMs": "183994",
  "bitrate": 503351,
  "width": 640,
  "top": 360,
  "projectionType": "RECTANGULAR",
  "fps": 30,
  "high quality": "medium",
  "qualityLabel": "360p",
  "audioQuality": "AUDIO_QUALITY_LOW",
  "audioSampleRate": "22050",
  "audioChannels": 2

Many of the properties listed here are pretty self-explanatory as they element the format and general high quality of the stream. For instance, from the knowledge above you possibly can inform that it is a muxed (i.e. audio and video mixed) mp4 stream, encoded utilizing the H.264 video codec and AAC audio codec, with a decision of 640x360 pixels, 30 frames per second, and a bitrate of 503 kbps. If performed on YouTube, this stream can be obtainable because the 360p high quality choice.

Every stream can also be uniquely recognized by one thing referred to as an itag, which is a numeric code that refers back to the encoding preset used internally by YouTube to remodel the supply media right into a given illustration. Up to now, this worth was essentially the most dependable method to decide the precise encoding parameters of a selected stream, however the brand new response has sufficient metadata to make this method redundant.

After all, essentially the most attention-grabbing a part of all the object is the url property. That is the URL that you should utilize to fetch the precise binary stream information, both by sending a GET request or by opening it in a browser:

Be aware that should you attempt to open the URL from the JSON snippet I’ve proven above, you may get a 403 Forbidden error. That is as a result of YouTube stream URLs will not be static — they’re generated individually for every consumer and have a set expiration time. When you get hold of the stream manifest, the URLs inside it are solely legitimate for roughly 6 hours and can’t be accessed from an IP handle apart from the one which requested them.

You possibly can affirm this by wanting on the ip and expire question parameters within the URL, which include the consumer’s IP handle and the expiration timestamp respectively. Whereas it might be tempting, these values can’t be modified manually to raise these limitations, as a result of their integrity is protected by a particular parameter referred to as sig. Attempting to vary any of the parameters listed inside sparams, with out accurately updating the signature, will lead to a 403 Forbidden error as effectively.

In any case, the steps outlined to date must be sufficient to resolve and obtain streams for many YouTube movies. Nevertheless, some movies could require a bit of additional work, which is what I will cowl within the subsequent part.

Working round content material restrictions

YouTube has an in depth content material moderation system, so you could often encounter movies that can’t be performed and, thus, downloaded. The 2 commonest causes for which might be:

  • The video is blocked in your nation, as a result of it options content material that the uploader has not licensed to be used in your area
  • The video is age-gated, as a result of it options content material that’s not appropriate for minors, as decided by YouTube or the uploader themselves

The best way region-based restrictions work is pretty simple — YouTube identifies whether or not your IP handle maps to one of many blocked nations and prohibits entry to the video if that is the case. There’s not a lot that may be carried out about it, apart from utilizing a VPN to spoof your location.

For age-based restrictions, however, YouTube doesn’t infer any data from the consumer, however fairly depends on the consumer’s consent. To offer it, the consumer is required to check in to their account and make sure that they’re 18 years or older:

Age-restricted video

Whereas it’s attainable to simulate the identical circulate programmatically — by authenticating on the consumer’s behalf after which passing cookies to the /youtubei/v1/participant endpoint — the method may be very cumbersome and error-prone. Fortunately, there’s a method to bypass this restriction altogether.

As it turns out, there may be one obscure YouTube consumer that allows you to entry age-gated movies utterly unauthenticated, and that is the embedded participant used for Sensible TV browsers. Because of this should you impersonate this consumer within the preliminary request, you may get working stream manifests for age-restricted movies, with out worrying about cookies or consumer credentials. To do this, replace the request physique as follows:

// POST 

  "videoId": "e_S9VvJM1PI",
      "clientVersion": "2.0"
      "embedUrl": "

The principle distinction from the ANDROID consumer is that TVHTML5_SIMPLY_EMBEDDED_PLAYER additionally helps a thirdParty object that comprises the URL of the web page the place the video is supposedly embedded. Whereas it isn’t strictly required to incorporate this parameter, specifying permits the request to succeed even for movies that prohibit embedding on third-party web sites.

One vital downside of impersonating this consumer, nevertheless, is that it doesn’t signify a local app like ANDROID, however fairly a JavaScript-based participant that runs within the browser. Any such consumer is prone to a further safety measure utilized by YouTube, which obfuscates the URLs contained inside the stream metadata. Right here is how a person stream descriptor seems to be in that case:

  "itag": 18,
  "signatureCipher": "s=CCpercent3DQ8o2zpxwirVyNq_miGGr282CaNsFfzUBBPgQU-8sKj2BiANNbb7LJ8ukTNpercent3DNAn-PJD-m57czWRI1DsA6uqrtC0slMAhIQRw8JQ0qOTT&sp=sig&url=
  "mimeType": "video/mp4; codecs="avc1.42001E, mp4a.40.2"",
  "lastModified": "1665725827618480",
  "xtags": "Cg8KB2hlYXVkaW8SBHRydWU",
  "approxDurationMs": "183994",
  "bitrate": 503351,
  "width": 640,
  "top": 360,
  "projectionType": "RECTANGULAR",
  "fps": 30,
  "high quality": "medium",
  "qualityLabel": "360p",
  "audioQuality": "AUDIO_QUALITY_LOW",
  "audioSampleRate": "22050",
  "audioChannels": 2

Though the construction is generally an identical to the instance from earlier, one can find that the url property is absent from the metadata. As an alternative, it is changed by signatureCipher — a URL-encoded dictionary that comprises the next key-value pairs:


Right here, the offered url worth is the bottom a part of the stream URL, nevertheless it’s lacking an necessary ingredient — the signature string. With the intention to get hold of the proper hyperlink, it is advisable get well the signature from the s worth and append it again to the bottom URL as a question parameter recognized by sp. The problem, nevertheless, is that the signature is encoded with a particular cipher, which means that it is advisable decipher it earlier than you should utilize it.

Usually, when operating within the browser, the deciphering course of is carried out by the participant itself, utilizing the directions saved inside it. The precise set of operations and their order adjustments with every model, so the one method to reproduce this programmatically is by downloading the participant’s supply code and extracting the implementation from there.

To do this, it is advisable first establish the most recent model of the participant, which will be carried out by querying the /iframe_api endpoint. It is the identical endpoint that YouTube makes use of for embedding movies on third-party web sites, and it returns a script file that appears like this:

var scriptUrl = '/s/participant/4248d311/www-widgetapi.vflset/www-widgetapi.js';

/* ... omitted ~40 traces of irrelevant code ... */

Inside it, one can find a variable named scriptUrl that references one of many participant’s JavaScript property. Whereas this URL will not be significantly helpful by itself, it does embrace the participant model inside, which is 4248d311 on this case. Having obtained that, you possibly can obtain the participant’s supply code by substituting the model into the template beneath:


Regardless that the supply file is an enormous blob of minified, unreadable JavaScript code, finding the deciphering directions is pretty easy. All it is advisable do is seek for a=a.break up("");, which ought to convey you to the entry step of the deciphering course of. In my case, this lead me to the next block:

// Prettified for readability
fta = perform (a) 
  a = a.break up('');
  hD.mL(a, 79);
  hD.L5(a, 2);
  hD.mL(a, 24);
  hD.L5(a, 3);
  return a part of('');

As you possibly can see, the deciphering algorithm is applied because the fta perform that takes a single argument (the s worth from earlier) and passes it by a sequence of transforms. The transforms themselves are outlined as strategies on the hD object positioned in the identical scope:

// Prettified for readability
var hD = 
  // Swap remodel
  i1: perform (a, b) 
    var c = a[0];
    a[0] = a[b % a.length];
    a[b % a.length] = c;
  // Splice remodel
  L5: perform (a, b) 
    a.splice(0, b);
  // Reverse remodel
  mL: perform (a) 

Relying on the model of the participant, the names of the above objects and capabilities can be totally different, however the deciphering algorithm will all the time be applied as a randomized sequence of the next operations:

  • Swap, which replaces the primary character within the string with the character on the specified index
  • Splice, which removes the desired variety of characters from the start of the string
  • Reverse, which reverses the order of characters within the string

Trying again on the fta perform, we are able to conclude that this model of the algorithm solely depends on the final two operations, and combines them like so:

  1. hD.mL(a): reverses the string
  2. hD.L5(a, 2): removes the primary 2 characters
  3. hD.mL(a): reverses the string once more
  4. hD.L5(a, 3): removes the primary 3 characters

Earlier than you possibly can apply these steps on the stream signature, nevertheless, it is advisable resolve a manifest that is really synchronized with the present implementation of the cipher. That is as a result of there are a lot of variations of the participant in use on the similar time, so the manifest returned by the preliminary request is probably not suitable with the deciphering directions you have extracted.

To establish a selected implementation of the cipher, YouTube doesn’t depend on the model of the participant, however fairly on a particular worth referred to as signatureTimestamp. This worth is used as a random seed to generate the cipher algorithm, and to maintain it constant between the consumer and the server. You possibly can extract it from the participant’s supply code by looking for signatureTimestamp:

// Prettified for readability
var v = 
  splay: !1,
  lactMilliseconds: c.LACT.toString(),
  playerHeightPixels: Math.trunc(c.P_H),
  playerWidthPixels: Math.trunc(c.P_W),
  vis: Math.trunc(c.VIS),
  // The worth you are in search of:
  signatureTimestamp: 19369,
  // -----------------------------
  autonavState: MDa(a.participant.V())

Lastly, replace the unique request to the /youtubei/v1/participant endpoint to incorporate the retrieved worth contained in the JSON payload. Particularly, the timestamp must be handed as a part of a further top-level object referred to as playbackContext:

// POST 

  "videoId": "e_S9VvJM1PI",
      "clientVersion": "2.0"
      "embedUrl": "
      "signatureTimestamp": "19369"

With that, the returned stream descriptors ought to include suitable signature ciphers, permitting you to accurately reconstruct the URLs utilizing the deciphering directions extracted earlier.

Working round price limiting

One widespread difficulty that you will possible encounter is that sure streams may take an abnormally very long time to totally obtain. That is normally attributable to YouTube’s price limiting mechanism, which is designed to forestall extreme bandwidth utilization by capping the speed at which the streams are served to the consumer.

It is smart from a logical perspective — there isn’t a motive for YouTube to switch the video sooner than it’s being performed, particularly if the consumer could determine to not watch it right through. Nevertheless, when the purpose is to obtain the content material as rapidly as attainable, it will possibly turn into a serious impediment.

All YouTube streams are rate-limited by default, however relying on their kind and the consumer you are impersonating, you could discover some that aren’t topic to this measure. With the intention to establish whether or not a selected stream is rate-limited, you possibly can verify for the ratebypass question parameter within the URL — if it is current and set to sure, then the speed limiting is disabled for that stream, and you need to be capable of fetch it at full velocity:

  # Charge limiting is disabled:
  # --------------------------

Sadly, the ratebypass parameter will not be all the time current within the stream URL, and even when it’s, it isn’t assured to be set to sure. On high of that, as already talked about earlier than, you possibly can’t merely edit the URL so as to add the parameter manually, as that may invalidate the signature and render the hyperlink unusable.

Nevertheless, YouTube’s price limiting has one attention-grabbing side — it solely impacts streams whose content material size exceeds a sure threshold. Because of this if the stream is sufficiently small, the information can be served at most velocity, no matter whether or not the ratebypass parameter is ready or not. In my checks, I discovered that the precise cut-off level appears to be round 10 megabytes, with something bigger than that inflicting the throttling to kick in.

What makes this conduct extra helpful is that the brink would not really apply to the general dimension of the stream, however fairly to the scale of the requested content material. In different phrases, should you attempt fetching solely a portion of the information — utilizing the Range HTTP headerYouTube will serve the corresponding content material at full velocity, so long as the desired byte vary is smaller than 10 megabytes.

Consequently, you should utilize this method to bypass the speed limiting mechanism by dividing the stream into a number of chunks, downloading them individually, after which combining them collectively right into a single file. To do this, you have to to know the full dimension of the stream, which will be extracted both from the contentLength property within the metadata (if obtainable), or from the Content material-Size header within the preliminary response.

Beneath is an instance of how that total logic will be applied utilizing the curl command-line utility in a easy Bash script:

# Set the URL of the stream

# Get the full dimension of the stream
SIZE=$(curl -I $URL | grep -i Content material-Size | awk 'print $2')

# Fetch the stream in 10 MB chunks and append the information to the output
for ((i = 0; i < $SIZE; i += 10000000)); do
  curl -r $i-$((i + 9999999)) $URL >> 'output.mp4'
carried out

Muxing streams regionally

YouTube presents a choice of totally different codecs for every video, however one can find that the high-definition choices are served solely by adaptive audio-only and video-only streams. And whereas that works out effectively for playback — as you possibly can merely play each of them concurrently — it isn’t supreme when the intent is to obtain the video as a single file.

Ever since /get_video_info was eliminated, YouTube has been offering fewer muxed streams for many movies, normally limiting them to low-end choices resembling 144p and 360p. Which means if you wish to retrieve content material as near the unique high quality as attainable, you’ll undoubtedly must depend on adaptive streams and mux them your self.

Luckily, that is pretty straightforward to do utilizing FFmpeg, which is an open-source device for processing multimedia recordsdata. For instance, assuming you’ve gotten downloaded the 2 streams as audio.mp4 and video.webm, you possibly can mix them collectively in a file named with the next command:

$ ffmpeg -i 'audio.mp4' -i 'video.webm' ''

Understand that muxing generally is a computationally costly job, particularly if it includes transcoding between totally different codecs. At any time when attainable, it is advisable to make use of an output container that’s suitable with the desired enter streams, as that may remove the necessity to convert information, making the method a lot sooner.

Most YouTube streams are offered in webm and mp4 codecs, so should you stick with both of these containers for all inputs and outputs, you need to be capable of carry out muxing with out transcoding. To do this, add the -c copy flag to the command, instructing FFmpeg to repeat the enter streams on to the output file:

$ ffmpeg -i 'audio.mp4' -i 'video.mp4' -c copy 'output.mp4'

Nevertheless, should you plan to obtain YouTube movies for archival functions, you’ll in all probability wish to prioritize decreasing the output dimension over the execution time. In that case, you possibly can re-encode the information utilizing the H.265 codec, which ought to lead to a way more environment friendly compression price:

$ ffmpeg -i 'audio.mp4' -i 'video.mp4' -c:a aac -c:v libx265 'output.mp4'

Utilizing the above command, I used to be in a position to obtain and mux a 4K video, whereas slicing the file dimension by greater than 50% in comparison with the streams that YouTube offered. If you wish to enhance the compression even additional, you can too specify a slower encoding preset with the -preset choice, however observe that it’s going to make the conversion course of take considerably longer:

$ ffmpeg -i 'audio.mp4' -i 'video.mp4' -c:a aac -c:v libx265 -preset sluggish 'output.mp4'

Total, FFmpeg is a really highly effective device, and it isn’t restricted to only muxing — you should utilize it to trim or resize movies, add customized metadata, inject subtitles, and carry out a wide range of different operations that may be helpful when working with YouTube content material. As a command-line utility, it additionally lends itself extraordinarily effectively to automation, making it straightforward to combine as half of a bigger workflow.


Regardless that many issues have modified, downloading movies from YouTube remains to be attainable and, in some methods, simpler than earlier than. As an alternative of /get_video_info, now you can retrieve metadata and stream manifests utilizing the /youtubei/v1/participant endpoint, which is a part of YouTube’s new inside API.

The method of figuring out and resolving streams is generally the identical as earlier than, and workarounds resembling price bypassing are nonetheless related. Nevertheless, signature deciphering has turn into much less of a priority, as a result of the overwhelming majority of movies are actually playable with out it.

Generally, the required steps to obtain a YouTube video will be outlined as follows:

  1. Fetch the video metadata utilizing the /youtubei/v1/participant endpoint, impersonating the ANDROID consumer
  2. If the video is playable:
    1. Extract the stream descriptors from the response
    2. Establish essentially the most optimum stream and retrieve its URL
  3. If the video is age-restricted:
    1. Retrieve a sound participant model from /iframe_api
    2. Obtain the participant’s supply code
    3. Reverse-engineer the signature deciphering algorithm
    4. Extract the signature timestamp from the supply code
    5. Fetch the video metadata once more, this time impersonating the TVHTML5_SIMPLY_EMBEDDED_PLAYER consumer
    6. Extract the stream descriptors from the response
    7. Use the deciphering algorithm to get well the signatures and stream URLs
    8. Establish essentially the most optimum stream and retrieve its URL
  4. Obtain the stream in chunks utilizing the Vary HTTP header
  5. If wanted, use FFmpeg to mux a number of streams right into a single file

In case you have any questions or simply need a extra in-depth have a look at how all of the items match collectively, be at liberty to undergo YoutubeExplode’s source code on GitHub. It is pretty well-documented and must be an honest reference level for anybody fascinated about constructing their very own YouTube downloader.

#ReverseEngineering #YouTube #Revisited #Oleksii #Holub

Continue to the category


Please enter your comment!
Please enter your name here

- Advertisment -spot_img

Most Popular

Recent Comments