Okay, I'm recording again now, and we were discussing about resolutions that you have from a source format. When Hollywood produces a movie, they typically record in a very large screen format in order for them to render on something like an IMAX screen. That's what I will use as a proxy for a source. So, our source file video resolution will be something of the order of 1800p. And that translates to, let's actually cross check this number by doing a search on Google and saying IMAX resolution. And so, IMAX resolution is, do they publish that? IMAX estimates approximately 12,000 lines of horizontal resolution, that's 12k. That's the resolution expected for IMAX movie. So, that translates to something like a movie with a ratio, with a movie ratio is typically anywhere from say 16 by 9 ratio, and so if you have 12,000 lines here, then you will have something like 16 divided by 9 times 12,000 number of vertical lines and 12,000 horizontal lines to make it a IMAX movie. And you need to have about at least 25 frames per second and continue that for a movie of two hour long, so two hour long. And in addition to this video, you have to have audio. Audio can be multi-channel audio. You can have multiple speakers sometimes, IMAX here has multiple speakers all around you. So, it can be 8 or 10 channels sometimes, 12 channel audio. So, that makes it an extra file that you have to add on top of just to carry the audio format. Audio is relatively very small compared to the picture that you carry on a gigantic frame size like this and 25 pictures like that. So, the frame size is big in the picture file that you actually play in IMAX theater, which these days do not play with a film. There is no film in theaters these days. So, they're basically files that are in source format that they just play on a video player. That's what you see when you go to a theater. And so, that's the resolution that you typically expect to receive from your sources. So, you have a source file which is very large and typically raw. And this is actually very large, I would say, very large format. And your goal would be to convert this into 37 formats, like this is our fictitious number I just pick up, which are different, 37 different types of devices. Each one of them will be a different resolution, a different video codec, different audio codec, different sampling rate, and certain other characteristics. And you have to basically for every movie that you receive from the source, you have to do this operation for every target device and somehow magically have a place to store these movies in some place closer to the user, almost closer to the user. And when the user expects to play a video movie on that device, that you deliver that movie on that resolution that you support on that device with that codec, that audio, that frame per second, and render the experience for this user at that point in time for every movie in a very long tail. That's another challenge. You have a very long tail of movie selections, your file sources are really big, and your target devices are a variety of them, like at least these many, I just picked up a number randomly based on what I just saw. It may be more, it may be less, but that's the complexity that we are dealing with. And how do we solve that problem in order for what I'm trying to do is now go to the next level of breaking things down into simpler examples for us to comprehend the bigger picture and see what would Netflix do to accomplish this? How will they go about implementing a solution that actually addresses this problem and deliver the user experience that they are known for? So this action of converting a source file into a different format is called encoding. And when the encoded format plays on the device, the device is supposed to decode and play. That's what happens on the device level, so you don't have to worry about the device, but at least from a sending or streaming point of view, you have to worry about encoding appropriate to the device that the user selects. So that encoding is a major, major activity at Netflix. This encoding thing is typically done using this library. It's an open source library called FFmpeg. It is a cross-platform comprehensive solution to convert, record, and stream audio and video from one format to the other, and it is open source. The way to run this is simply this, FFmpeg, input file, output file, one format, another format. So simple example, illustration. We are not actually encoding anything, but I'm just telling you that this is how you run it. Now this is an extremely popular library available in Windows, Mac, PC, everywhere. You can download it. It's already made, and it also is on the command line in Terminal, by the way, if you like. So there is a command you can just execute, and that will convert one into the other. So here is a comprehensive example of how I was dealing with something earlier this morning. Let me just see where it is stored. So it's opening up, and here is how a typical video recording I will encode, pretty much like this. You see that command line number 11 through 20 is an encoding command to compress a video that I will record here in a big screen, compress it down to a small screen, and upload it to a location that you may have seen already. Another example, you may have seen this thing that pops up here somewhere. I'll just go to that location and quickly point out you may have seen it already, but there is a video that is self-hosted, and that's a good example to look at. So here is a video that you see here. You may have seen this video at least once when you downloaded the VM, and that video essentially is a downloadable, and it actually runs in this location, which is in the CloudFront website. So it's a content delivery network. That's where the screencast for setting up a workstation.mp4 file comes down from. So there is a folder called screencasts in a bucket somewhere that contains how to set up workstation videos stored, and the other things are the key associated to enable the download of that video from that bucket. So the point I'm trying to illustrate is that Netflix also has to do similar things for a larger screen format. So this is the input file, and here is the output file. And it includes copyright information, year of creation, some scaling factors, codec used, and things of that nature. So you have to specify a whole lot more detail. This is just one example I'm just illustrating to you is what Netflix will do across the board on every single scenario that they're targeting to construct from a source file. They want to create an output file that goes to device number D1, D2, device type number D1, type number D2, type number D37, and like that. You have to create files that match that device type. So say D1 is iPhone 4S, and D2 is iPhone 5, and D3 is iPhone 7, and D4 is 7+. So everybody has a different resolution. You have to address that resolution. So this number 37 is actually too small. It should be a bigger number. So the complexity that you are dealing with is actually more than just 37, I suspect because of every device will have a different resolution type to address, to worry about. The screen sizes are different. And now let us look at a picture that Amazon has produced for us for this discussion. I'm just using that picture. I'll go and find that, and by the way, this is a website called Amazon Architecture website. You should study that. I'll give you a link for that. And that has some interesting illustrations as to how Amazon recommends as best practices for use in a variety of cases. So that is the Architecture Center. Very similar to this is the Azure Architecture Center. So let's go see Azure Architecture is another website where Microsoft also draws illustrations and diagrams about how Microsoft thinks is the best practice for creation of a variety of business solutions using the Azure Cloud. And so those two links I just gave you, but I will use one of the examples that Amazon has provided, which applies very closely to the discussion we want to have right now. And that is what I'm trying to go to. And so if you look at, I think this is a good one. I think this is the right one I wanted to talk about, right? Yes. So this picture is what I would like to discuss. So I'll grab that link and I'll like you to read this and I'll explain this as we go along discussing item by item. So this is the link that I'm reading and I'm going to download it locally. So I have a local copy. And so I'm receiving it, the file is now available to me locally here. And I'll just double click and open it. So that's the file we are talking about. In the picture, you have access to it in the Slack chat. So how does this apply to the discussion we are having? I will now try to drive an example, the Netflix example, through this picture as to how it makes sense and what could Netflix do in order for Netflix to deliver the solution for the end user. So let's begin from the point of view of a Netflix employee who is now just received the La La Land movie for delivery through Netflix as a service, the movie as a service service that they have. So this employee essentially is somebody who is receiving a stack of, let me see if I can get a, do you see my mouse moving? Yes. So that's important because I'll point to a variety of things here. So this is the stack of movies that the employee receives. And that employee has to somehow make sure that these movies are processed in a fashion that become eventually available to the end user for them to be selecting from. So if I want to watch La La Land on Netflix, I should be, as an end user, should be able to select the movie and just click and play and it should play on my device. That's the outcome that they want to accomplish. But the Netflix employee will eventually receive La La Land from the Hollywood studios or somebody will give it to them. Exact procedures may differ. I don't know exactly how they receive it, but somebody who creates La La Land has to give it to them. Hey, here's my source file, the raw source file. That file comes in here and that needs to go and get processed. So the first thing that the employee will eventually do somehow, maybe it is automated, maybe it is not, but imagine it is not. Imagine a person who is actually receiving that file has to upload that file to some backend solution for processing. That person has to receive and send that up to the backend on the internal machines. And so here, this elastic load balancer you're looking at, this box is the load balancer that will interface with that particular employee trying to upload that source video into the cloud. So now it needs to get processed. So the application on the other end, which runs in the backside, are these upload web servers that you're looking at, will receive those black colored ones, these EC2 machines that you have, will receive that video from the user of this, which is the Netflix employee trying to upload. And this elastic load balancer will help divert that traffic to a multiple of upload web servers, whether it is one or more, and it will scale according to how many employees are currently or concurrently uploading a movie into the backend. Like for example, here, this is the backend portion, upload web servers, the movies are getting uploaded to the cloud. And if there are a large number of movies tagged up, this load balancer will automatically scale the number of upload web servers so that these movies get processed. So that's the concept we're beginning discussion with. Now once you begin finishing, sorry, finish uploading that movie into the Netflix backend as an employee, you send that movie out and the upload server receives it, a couple of things that need to happen is what this upload web server will do. And one of the first things it will do is to put the source file as is in a repository, which is the bucket here, the central bucket that you're looking at. And so it will put the movie right there, as is, as received, just place it. Now that we have the movie in the repository, we need to now begin processing. And let's say that this movie called La La Land, for example, and I don't even know whether La La Land is available on Netflix or not, I'm just using it as an example. But imagine it's a new movie. So it may not be released yet on Netflix according to what Hollywood desires. So they may have some time before it actually becomes available on Netflix. Because typically you will know this, that a movie that becomes, the movie created is probably does a premiere show somewhere. And then it comes on the big screen. And eventually it will come down to maybe Netflix or a DVD at some point, depending on whether this comes first or that comes first, these days it is different. But it used to be the case that Netflix used to be after, so late in the game, but these days it is changing. So the creator decides, the creator really decides what timeframe do you want to release a particular movie on which channel. They get to choose where that particular product that they create. So you create something, say La La Land, for example, and you want to premiere it in Los Angeles or New York, go on big screen after, maybe on the weekend Friday. And then two months down the road, you create a DVD and three months down the road, you let Netflix play, something like that. An example of this would be Adele comes with a song that she creates, the 25 album. And that album was just not available in any of the streaming services at all for so long that I think it was more than seven months until it came on Spotify and streaming services. So similar examples, the creator decides what they will choose as the time at which their product, La La Land, for example, becomes available on Netflix. They might give it to Netflix right away, but the release date they set according to what the creators desire. And so that boils down to the movie processing, which will need to happen in the processing pipeline right here on the top section. These boxes that you see here in the EC2 section are auto scaling boxes. And what these boxes are responsible for is running the processing part, which is encoding these movies into different form factors, different device types, different aspect ratio, different resolution, codecs, frame rates, and all the variety that we discussed. That needs to happen at the level of these boxes. Now how will they know that this is a thing to do? And so when this upload server actually uploads a source movie into the bucket, at the very same time, it also adds an item in the queue. This idea of a queue is essentially an implementation of another open source project called Queues, RabbitMQ, a variety of queues are available. But the idea is that you are receiving a source file and you put that in a bucket. At the same time, you will put an entry in a queue. This is basically a to-do list. In that list, say, process La La Land. And here is the La La Land movie sitting, the source file in a bucket. So the initial ingestion service will not only put the item in the bucket, but also create an entry in the to-do list, new entry, to-do item. Process that movie, please. Somebody has to do it. So they create that queue item in the to-do list and that queue entry is typically stored or rather Amazon recommends that you store it in their service called Amazon SQS, which is a queuing service. All it does is give you a frame where you can put items in a queue, in a line, in a list. Think of it that way. And so that is what SQS does is puts an entry somewhere that says process this, process that. It's a queue and it sits and does nothing. All it does is it's just a queue. That's the whole purpose in life for that thing, SQS. But now somebody has to process this. Somebody has to look at the queue, understand it, and do the next logical step for actually processing the queue. What does the queue processing say? Process la la land. So what you need to do is have a set of servers in EC2, which will automatically scale. So grow up and shrink. And you don't need a load balancer here because the movie is already inside a bucket. Nobody is actually uploading it to you. You already have the movie in your bucket. All you need to worry about is make sure that the queue is empty because you want to process it. And you can make the queue empty by marking the item done. And that is the whole point of dealing with a queue is that you basically queue is think of it as a stack of stuff, stack of to-do items. I have to do this thing. So I have a stack accumulated, piled up, I do this, I do that, I do this, like that. It's a thing to-do list that accumulates and grows over time as new movies come in. One of the items is process la la land. Maybe other items. We don't know. Any new movie pops up in the queue, things of to-do. And our purpose for these boxes, auto scaling, is to process those queue items. Basically, as the queue grows, our goal is to bring the queue down by marking them done, done, done. How do you mark them done? By doing stuff. What do you do in doing stuff? You encode. You run that FFmpeg encoding method. By the way, there are similar to that, another product called MENcoder is available, which is conceptually similar, probably uses the same FFmpeg library at the foundation. And like this, there is encoding.com as a service available that you can use if you like to encode. It's a software as a service. And what these guys do is you give them a video, they will process it for you and give it back. So you basically put that video in a bucket, hand them a control, they will process for you and give it back to you. They'll charge you money for that, but that's the idea behind this service. So back in what we are discussing is that we have the to-do list item. And this La La Land movie needs to be ready by three months down the road. So we have three months of time, so ample time, right? So we can process this thing at our leisure. The processing pipeline doesn't need to rush for La La Land as such. And what does that mean is that although the actual movie, it's not just the only movie we have, we might have more movies in the queue, so we might need to scale the number of machines that we have in our EC2 processing pipeline to grow out and shrink back in depending on how big the queue is. But there is one key thing that I would like to bring to your attention, which we never discussed before, and that is something is the nature of this processing pipeline, is that it can use a different type of instance in the Amazon Cloud. That instance type is called spot instances, spot. Do you know what spot is? Or if you have already heard of or known, please let me know. But the idea is spot instance is a type of instance that you have in Amazon. But without reading this, it is hard to understand, so I'll simplify it really, really easily for you. So in Amazon, there are actually three types of instances. One of them that you know already, those are called the standard instances or they have a name for it, or rather they call it the on-demand instances. That's what you have been using up until now. So when you click a button, you get an on-demand instance. You pay them 0.007 dollars per hour and you get a T2 micro, something like that. That's an on-demand instance. There's a second type and a third type. The second type instance is called a spot instance. And the third type is called a reserve instance. Now think about spot instance is like eBay auction. Think about reserved instance is like contract, long-term contract. So when you enter into a long-term contract, let's say three years and you promise to pay them this money for a T2 micro and you want to use it for three years nonstop, they will substantially reduce the price of this and make it 000 or something like that, some ridiculous number like this and ask you to charge that amount per month, sorry, per hour for a period of three years provided you promise to pay that for a period of three years. They will give you substantial discounts because you are entering into a long-term contract. So that is another type of, it's basically a pricing arrangement. It's not an instance type difference. It's more of a pricing relationship that you will have with a cloud provider that you will promise to enter into a long-term contract with them. And so they are giving you a reservation and they are promising that they will give you a T2 micro for three years and you give them a ridiculously low price, lower than the already ridiculous price. And so that's the contractual arrangement you will enter into. That's another device instance type they call the reserved instances and those can be purchased for a lower price than an on-demand instance. That you know, that you understand this. Now let us discuss this item, it's kind of difficult to immediately comprehend unless you compare it with what eBay auctions do. And so let us see a real live example. I will go to the Amazon cloud and attempt to buy a spot instance right now. And what's going on? So I'm logged in. I will go to EC2, try to create a machine and say that I want to use Ubuntu. And I will say, I want to get a T2 micro, but T2 micro is so cheap that it doesn't even give you a spot instance, by the way, just saying that it's not available. You have to go slightly larger, like an M3 large is a good one. So I want to get this size, M3 large. And for that, I will say, okay, cross it forward. And now you see that you have this option available that says purchasing option, request spot instances. So I'll click on that. And as soon as I click, I see something. I see prices and their prices are all over the map. Some are ridiculously high, like a $1.40, huh? Why is that so? And some are ridiculously cheap. I mean, all others are cheap, like 0.02. And this is for a medium instance, like M3 medium prices are not cheap, M3 medium, M3 medium pricing is not that cheap. But let's see what the pricing for an M3 medium is. On demand instances, pricing for M3 medium happens to be in M3 medium, where is M3, M3 medium is 0.67 per hour. So that's the pricing for M3 medium, 0.6, so 6.7 cents. Now if you go here, they're giving you a machine for a $1.40, what's up with them? And if you go to this, it is 2.8 cents and 2.7 cents and 2.7, 2.7, but 1.4. And so am I an idiot to go 1B? I am if I go there, but otherwise I'm not an idiot to actually go this direction and this direction. 1A, C, D, and E are interesting to me. So why not I bid my maximum price to something like 0.029. I can get a machine like that. If I bid higher than what they're asking for, I will get a machine. If I bid lower than what they're asking for, I will not receive a machine because the market prices is what dictates whether you actually get a machine allocated or not. That's what's going on. You will also see that this is a weird behavior, 1.4. There is something that Amazon does that I will describe to you what they're doing under the hood is to make this artificially high price, whereas others are low. If I bid lower than the low prices, I will not get allocated a machine. Even if I say I want so many instances, request spot instance, I want to get say 10 machines for a price of 0.026. I will not be allocated until I go above or at least match the price they're asking for and I will then get allocated, very likely. If I go higher, I'm very, very likely to get allocated a machine or 10 machines. However, let's understand what is going on under the hood. Oracle calls it non-meter, correct, yes. Various companies call it different names and you're right, absolutely. Under the hood, what's going on is why does Amazon do it like this and why does other cloud companies do it like the way they are doing? Why do they discount it so heavily when the actual price is 0.067 for a M3 medium? Why are they getting away with a price of 0.027? Why are they wasting money? Why do they want you and I and everybody to let this allocated at a cheap price like this? The same box, by the way, at a low price. This is more than 50% discount. Why? Do you know why? Why do they do that? And they're still profitable, by the way. Why do they do that? Anybody? Why would you do that? If it was you, would you drop your price randomly like this and say, oh, come on party, 50% off for no reason? Probably at non-peak intervals when the power, I mean, we can. Correct, exactly. So the idea that what Shiv is suggesting is the idea that I would like to explain with an example of airline seat ticket price. The airline ticket price is one of the most ridiculously dynamic item. It changes so rapidly and erratically is that you know that a person sitting right next to you in a plane may have paid something different than you paid, very likely. The prices fluctuate all over the map. There is a reason for that. There is a reason why these prices fluctuate is when you are in a flight that happens to be traveling from, let's say, three months down the road versus a week from now versus like a one hour from now or two hours from now, like that. So a plane that flies two hours from now, a plane that flies one week from now, and a plane that flies three months down the road, the price of a ticket will vary dramatically based on how much time you have left for the plane to go. A week from now, the price will be high. A three month down the road price will be probably reasonable. Do you know what this price may be? For example, if you have a plane that has 100 seats and it is going to fly in three hours from now and you have 97 seats occupied and three hours to go and you are at the airport with your luggage in hand and you don't have a ticket yet and this airline is about to fly in three hours, you have three seats vacant and you may want to actually go to the destination. So let's say they're going from Seattle to Maryland, Baltimore to meet Shivshankar there. So Baltimore, that's a flight. And you have this inclination to go to Baltimore for some reason. You want to go there to meet Shiv. And so the question is, and by the way, the ticket, the usual ticket that you have is $100. There are three seats vacant and you are the airline manager on the spot in Seattle airport and you see this customer, a potential customer, it doesn't even have a ticket yet. What do you think you will be able to willing to negotiate the price of this ticket with this customer if you win the choice? Even the choice, you have complete flexibility. As long as you make profit for the company, you're okay to discount. How much will you discount? Do you know the cost of flying one additional person on the 97 people already sitting three vacant seats and you have this guy sitting there waiting to go to Baltimore, he's negotiating with you for a price. The cost of flying one extra person. How much is that? Do you know? Add one more person to a flight which has three more vacant seats. How much does it cost? One bag of peanuts, right? Some bathroom cleaning, amortized or maybe spread out across, so bathroom cleaning and some more. That's pretty much it. There is virtually no extra gas needed, practically speaking, but maybe we'll give $10 for gas per person. Just allocate that. So estimate this to be say $25. That's the cost of adding one extra person. You typically charge $100 and the company wants to make a profit. You can make a profit if you charge $26, you would make $1 profit and that is what is going on here in Amazon. That is exactly what's going on is what Amazon is doing is that they calculate their cost of running the machines and let this person take the flight for $26 because they still make profit. The cost of filling that one extra seat is not that much. If they don't use that person, they don't give him the seat at all, they lose this dollar and they go vacant, three seats, which is a bad idea. If I were the manager there, I would probably discount say $50, come on board, let him sit and make $25 profit, extra profit that I would otherwise lose. So that is the underlying idea that you see in this situation, it's like 0.27. So it's 2.7 cents for a machine that is ordinarily priced at 0.67. They're giving away this machine because by the way, the hardware is already there. It's going to go waste, the price of this ticket after three hours is zero. Its value is zero. It decays so dramatically after that, after the plane takes off, the value of a ticket that already gone plane is zero. I mean, there's no value. How will you board the plane? It's not possible. It dies straight to zero after the plane flies and that you can compare with the idea here. When you have the machines there already in your cloud, and if somebody is not using it, it's wasted. So you want to fully fill up your cloud instances as much as you can and basically collect the last micro cent, these are million micro cents here. And so we are talking about sub cent numbers, but they are collecting that by giving away at a very low price because they're still making profit. Nilesh? Yes. So is it like a roping price where they charge 27 cents for one year and then you go to 67 or something else? No, the price is on a per hour basis. And the catch with spot instances is that they have the contract says that if they will yank the machine away from you anytime they feel like. So they work for an experimental machine, but not for a production instance. Yeah, exactly. It can be production machines also. That's what I'm trying to communicate is that it can be production machines as well. For example, for example, let's go here. This machine can actually be spot machines. You know why? Because La La Land processing can finish in any time between now and three months from now. It doesn't matter if the processing halts in the way. And if it does break, fine, get rid of the machine, get a new one and start processing again. You still have ample time, three months. So processing can eventually finish in cheap, cheap spot instances. Even though it is production, it is not time critical. It is batch process. Anytime you have a situation where you can do something in a batch process, which is not critical on time, which is not interfacing with the customer, it is okay to use spot instances. It may be okay to use spot instances. The reason is why waste money on a full instance, on a on-demand instance when you can get away with a cheap one. If you can, do it. You're typically dealing with thousands of machines on a typical situation, on a large scale situation like this. So the savings can be substantial, really big. So make use of those. That's the core idea is use spot instances so that it doesn't really matter because there is no customer interfacing with any of these services. You know, it's just a queue that is getting processed. The processing queue will go down. But it's okay if it doesn't go all the way down to zero right away, it can maybe happen tomorrow or maybe the day after or next week, not critical, not critical. So when it is not critical, use spot instances, save money. That's the key thing here to remember. Now with this processing pipeline, what they will do is actually create video files that you will do something kind of sort of like this. Not exactly. This is not how they do it. This is how I do it for my processing. But the idea that I want to show you is, let me finish this one. So 1.4. This is 1.4. What does that do for them? What they're really doing is that in East 1B, they're killing out all the cheapos and say, you know, make some room here. I want to get some room. I want to throw out all the cheapos by bumping the price. So when the price goes up like this, your willingness to pay that you have maximum price you will pay is zero to nine, you get yanked out of the system and they will arbitrarily bump the price high like this to basically clean the ground and to kill all your machines that are spot instances and make room so that other people who are on demand can come and play. So if the availability of machines is running low, they will yank the price up, jack the price up and kill all the spot instances. That's the trick that they apply here. And you will see this sudden bump of a price. And let's see spot instance, price history, we'll show you pictorially somewhere here. Where is that? The pricing history, where is the history available? So I'll find that link for you, but it basically looks like this. Over time, you will see that the pricing history of a machine is kind of as the red line shows, but suddenly it will bump like the yellow line. So I should say the yellow line, the spot pricing is low here, all the way low here. And then they suddenly bump it. And that bump basically kills all the spot machines. And you make room for the regular machines, on-demand machines or other other reserved pricing machines, create some room by bumping the price up. And eventually the price will go down again. When you have more available machines than there are customers, you will lower your price and adjust. So there is a dynamic chart that actually this is a screenshot of, I'll find that link for you. You can investigate pricing, historical spot pricing for a given machine type in a given availability zone for a given operating system type. So you have a variety of choices to study from. Having discussed this part, let's continue along in what we are trying to accomplish in that picture here is that processing pipeline. What it is doing is taking your source file from the bucket and creating target outputs for all the target devices, all the resolutions. And I'm just using the number 37 as an arbitrary number. So let's say we have a file called LaLa Land source file in its raw format. That's the file we have in our bucket to begin with. And we are beginning processing of this file. So we'll create then LaLa Land for iPhone 4s.mp4, LaLa Land dash iPad pro.mp4. I'm just randomly making stuff up as we discuss this. But the idea is to create these resulting smaller size file formats that address that particular device type and put that also in the same bucket so that we have a collection of smaller size files that meet appropriate target devices because you cannot play the source file that you receive from Hollywood directly on an iPhone, you cannot even send it there. You have to shrink it, compress it, encode it, and then send to the target device that asks for it. And that is what we will accumulate a variety of different file sizes, file size, resolution, codec, frame rate, and a bunch of other variations that you will see if you decide to go that direction to study it further is what you will see this picture is trying to create is those video files generate because they are actually doing the processing based on what the queue tells it to do and take the source file from the repository and put the output of the processing back in the same repository. Now, having done that part, they will also do another entry in the database and tell that my processing is actually done. So in the data store, they will have references about La La Land movie saying that the repository now contains encoded formats for all the target devices that we support. And they are these files in that bucket, in that location. So you keep track of that data so that data will remember which file in the repository contains La La Land movie for a particular target device. And that data will be stored in the data store. This is another database service that you may see. You may have seen it already, maybe somewhere here. Go back and there are a bunch of services that Amazon provides for database as a service. So there are these guys. RDS is the one that is very likely candidate for the one we are discussing. They have DynamoDB, which is a NoSQL database, Elastic Cache, and Redshift. Redshift is more like data warehousing. And that's what the data store will very likely be a relational data store, RDS, something like MySQL, Postgres SQL, or Microsoft SQL, or Oracle databases. Those will be this store. It will store the information about the movie processed. It will also store information about, let's say, the La La Land. So if you go to this location, it has the metadata about the movie itself. For example, is this La La Land or what? No. Yeah, this is La La Land. So the movie, the name of the movie, the date, year, PG-13 rating, duration, classification, the date of release, then who are the director, writer, stars, and scores, and all that detail that you expose, you need to store this data. And also the picture album cover and this trailer, all that detail is data about the content, about La La Land. And that detail has to also get stored in that database. And so that data store will store information about what the name of the movie is, what the other things that we saw IMDB about, it will store that detail, and also store references to the actual encoded result from the processing that is a link back to the repository. This repository will contain references to the target devices that we encoded for in the processing pipeline. So this data store will build as a consequence of processing. You have to have another method to actually put data about the data, which is metadata, which is information about Ryan Gosling or things like that. That is the actor in the movie. So you have to provide that information and expose it to the end user so that the user gets interested in the movie that you want him to watch or her to watch. And that data has to be also stored in the data store somewhere. This picture doesn't show you how you will gather the data information about Ryan Gosling or things like that, but this is focused more on the encoding pipeline part. Having stored the data about the movie and the movie itself encoded here in the repository, the encoding part finishes. Now let us change the role of the person who is going to watch movie, like you, for example. And for you, the idea would be slightly different. You will also... Let me get to the whiteboard again. So the first user we talked about is a Netflix employee trying to upload movies into the backend processing. So the processing happens and the resulting action becomes available in a data store and the resulting files become available in a bucket. That concludes the encoding part. Now from a user perspective, which is you and me and people like us who watch Netflix, what we really do is start some software in our hands, in our homes or wherever. And that software runs on some device. That device needs to be connected to the internet. I should actually say it usually is connected to the internet because these days Netflix also allows downloading of movies. You can store it locally and play it later on, even if you do not have internet connection. That just released a couple of months ago apparently. So that device does not need to be always connected to the internet, but typically is connected to the internet. And so that device, when it's connected to the internet, it will then have this software connect to the backend. And this backend is different than what you saw here. What you see in the black sections here, meaning this elastic load balancer and this upload servers and the job queue and all the black colored boxes here, they're not what the backend will constitute for the end user. The end user will look at the white colored boxes here, the web servers, the auto scaling, the elastic load balancer. And the user is typically people like you who will want to see what you see when you visit this site, what do you see is the web server that Netflix shows to the customer? This is the web server, the customer facing front end. Now, if you happen to have a login, that's a different front end than what you see here. The difference in the two are that this is a sales site. It entices you into buying their stuff by giving you a one month free option, for example. So this is the sales and marketing site. That's what you're looking at. As soon as you log in, so you sign in, you visit a different site on the backend. So just remember that there are two distinctly different services. One is focused on selling you stuff. And that is a sales site where you put your credit card number and you subscribe to something and all that. That's a different backend. You have another backend that you see when you log in. As a paying customer, you log in, you see something different. What you see is not this, sorry, not this interface, but something that looks different. I don't have a login to show you, but you probably know what I'm talking about. Whatever you see after you sign in, I don't have an account with Netflix, I don't use it. But whatever you see after you log in is a different backend than this. It's not this, it's different. In that backend, you see the long tail. You see recommendations and you see what you saw before, like a list of history. And you also see some things like action buttons. The action buttons are asking you, basically inviting you to, let me show you something similar where I have login, I can describe that to you. Amazon.com will have a method, a very similar looking interface. So I see the Amazon Prime, where is the Amazon video? It's very similar, Amazon Prime, watch, and it's conceptually similar. So I'm going to the interface that you probably see. So this is the interface that I'm looking at, the Amazon's long tail. I get to see what I have in my Prime membership and the selection of movies. I can watch trailers and I can click and play a movie that I like. That website is a different website than the pre-sales website. So this is the selling website, this is the servicing website. When you're servicing a customer, you're providing them a way to watch a movie by clicking the play button. That's a different site. That is what we're talking about here in this example, which is here. So those web servers, that will also auto scale, by the way, and they should not be your cheapo servers, should not be your spot instances, primarily because there is a paying customer like this guy who is visiting those web servers through an elastic load balancer. And that auto scales these web servers. And these web servers is what you see when you go, where is Amazon, here, the choice, the list of all the movies that are available that you can watch. And I can click on any one of these and watch if I want to. And so that list of movies is actually stored in a data store where one of the movies that we now have is La La Land, that will be stored in the data store. And correspondingly, all the corresponding files that are for that La La Land movie are stored in the repository. So this web servers, the application running on these web servers will actually look at the same data store that was populated by the processing pipeline and will present that data to the end user, this guy, because he or she is visiting the load balancer through an auto scaling group, looking at these web servers, which is painting the representation that is sort of similar to this, a choice of long tail that you can basically click and play. Now when I search for something, there is a search interface so I can actually see Prime Video. Do I have La La Land here? Maybe not, or maybe do. I don't know. We'll find out. So La La Land shows up, no, it doesn't, La La Land apparently is not there, it seems. They don't have it, La La Land on Amazon, whatever. But just an example. So they don't seem to have it, La La Land, which is fine. But if they did, that would show up in the result. And the web server will show me the result of La La Land with a link to click. So I can then play on the play button and then something happens next. Do you know what that next thing is? When you select a movie, like I want to see a list of movies that I searched for and then I find the one I want to watch, and I decide to see La La Land today, assuming it is available on Netflix, I have no idea. And so if it does, it will show up. And when you click the play button, there's something different happens and a different kind of pipeline comes into play. And it is right in your face in this picture. And it is not these web servers, by the way. It is not these web servers. It is not these black ones. It is something right in the middle. Is the content delivery network. The CDN will take over at that time and actually serve the video from CloudFront by looking up the correct selected movie that you choose, that you chose in here, will refer to the media repository for the correct file that you chose for La La Land for your device iPhone, for example, and pick up the right file and serve that exact file from that S3 bucket through the content delivery network on CloudFront to you without going through the web servers. This is exactly what I also do, by the way. You saw me talk about, where did we go? I think I closed that window, so I'll bring it again. So here, yeah, there we go. So right click, copy video address, and I'm doing the same exact thing, by the way, here. I'll now go into detail about what this thing is. So what you're looking at is, I am also serving that video, CloudFront, looking at a bucket called screencasts that stores this video file called setupworkstation.mp4, and it has a policy to, this hash generates on the fly and exposes itself. The signature generates on the fly and exposes itself. The key pair ID is the account I use for my Amazon account that also attaches itself, and if you attach these things together, then that particular file will actually become visible to you. If you don't attach anything in line number seven all the way through 13, you will not be able to see the file, so try that without those things. Try this segment without the remainder, you will find access denied, here we go. It says file does not have any style information, some error generated, missing key, error generated saying missing key pair ID required for blah, blah, blah. So the error generated, because it is protected file, the file itself is on CloudFront in a bucket accessible, pretty much similar to what we just discussed, except you know that you can play the video when you open the website. You try to go to the video directly, it doesn't work because it is secured, secured video protected based on when you log in, then the key generates and the key will allow you to see the video, otherwise the video will not play. Same idea in the case of any service, by the way, uses the same idea. This is called, what do you call them, S3 dynamic content, I'll come up with the exact name that they use. I know the code, but I don't know what they call it. So it is basically dynamically generating and protecting a content in a store bucket and generate that signature that I show you here, policy and signature. It is dynamically generated for every user that has the appropriate authorization to watch that content and then that signature is appended and then the video shows up when you click that button. And that is conceptually similar to what Amazon will do when they are showing a user that selects a movie on a web server, I want to watch La La Land, that request gets transported through the content delivery network with appropriate tokens generated for the user for them to be able to access that S3 bucket containing La La Land for their target device. Otherwise the video will not be able to play without the appropriate authorization and your credit card on file with Netflix and a membership. So that's the end-to-end pipe of how a user will see. There is more to it, so we'll discuss more. But I would like to pause at this moment for questions if you have any and we'll discuss more in greater detail, some of the subtleties involved. No questions. Hi Yulesh, the last part of the signature and the, can you go back to the link, the break up, how you broke it in three parts, the signature. So does it change every time, even technically the same movie, does it change for every instance? Every click. Every click. Okay. Yeah, so let me, let me identify this exact name for it. So there is the documentation. So serving private content through CloudFront, that's the topic and here's how you use it. I'll paste a link for you. You should read that. The idea is to generate a token on the fly based on user's credentials and then giving them permission to access an object in the bucket and serve them through CloudFront. And that is the developer guide for that. You should read if you're interested, I'm pasting a link and you can implement this if you like. I have implemented, but I'm not fully using it yet because I'm still building things as we go along. So what you, what you see most of our videos, not this one though, but most other videos are on YouTube. Like the ones you watch, they're on YouTube and YouTube is not secured, it's public. Anybody can access it. So this video is actually secured properly. So I'm building a method to store it like this going forward. And you will see more of these videos in the sequence. If you follow, I don't think you follow this sequence at all because it's not ready yet. But the idea is to, so here is another video pops up. So I can right click on it and copy the video address and you will see that if I open it up in a browser, it will open and it did and it will play. And so I'll stop it by the way. But if you go back to this new link that I just copied, paste it here and then understand the aspects here. We have a key pair ID. We also have a policy of signature and a policy. So three things. And now the rest of it is the link. You try to access this link directly, hold on, cancel, discard, discard and paste. You try to access that link directly. It will not open, says access denied, missing key pair. But if you do the same link along with the key pair, not in this window, where was that? Here along with the policy and token and everything in place, it will open up and show you the video. And that's how, is one way to secure your video if you like. Netflix does a lot more other things other than just this simple thing. This is quite simple to implement by the way. Netflix is fairly elaborate, but concepts are similar. That you protect your private content, you don't let people pirate it and still allow genuinely paying customers to watch it. So now more, more discussions in a slightly related topic. The idea would be now imagine a situation where you have this teenager in your home. The teenager is watching it on a full TV. It's a Samsung TV. He's watching La La Land. And the movie is into, it's a two hour, two hour, eight minute movie. And imagine that it's 24 minutes, 37 seconds into the movie that you have watched up until this point. And then the teenager decides to go sit in a car with you in the back. And he has his iPhone in his hand. Now what do you think Netflix will do? And how will they implement it? That's the question. It's a question. So they're watching this movie, the teenager that you have is now in the back of your car, you drive and he's there. He wants to watch. He wants to continue to watch from this point onwards. It was playing on the TV in the home. And now he is on the iPhone. What do you think needs to happen to continue the experience? Some sort of caching. Caching what and where? Caching the state, the time to which. Yes. You want to cache this detail. Right. Right. So it should log like what the user was doing last time and they logged in. Yes. Yes. Yes. Exactly. Exactly. So the idea would be to remember what a user is doing the last time they paused. What point they pause at that they will pause at some point in time randomly. So at that time, the device that you have should be smart enough to report back to Netflix. Hey, the user seemed to have paused at this time at this time stamp. And they were watching on Samsung TV and they paused at 24 37 at that time stamp they paused. That information needs to go back to Netflix. Netflix needs to know. And so the reason one of the reasons why Netflix does not receive typically too many phone calls from all of us is because they receive continuous information about what we are doing on our devices almost instantaneously. They are watching us all day long. What are we clicking? Where we are pausing? What are we rewinding? Every single what are we browsing? Which movies are our favorite? Which we don't like? Which we like? Every single thing. In fact, even inside a movie, they will monitor what segments of the movie you watch over and over again, rewind and repeat and give that information back to the creators so that they will know that those segments are interesting. And that's the kind of extent of information by the way, this is just the user usage patterns. They will also do some device level measurements, measurements that actually will understand whether there are glitches in the video as it plays. If they notice glitches in the video as it starts to play, they will drop the frame rate on you, FPS will go down, automatically scale the video, make it poor quality, little bit and that will allow a thin pipe to carry your bad quality video that you will not even notice or perceive immediately. So you will not complain essentially. The idea is to make you not complain too much or not even have you notice errors or aberrations or glitches that may happen because bad things will happen, that's a fact of life. The idea is to be able to measure what is going on and take corrective actions appropriately. So this idea that I am discussing right now is called capability negotiation. The idea of capability negotiation is to negotiate between the company, the backend and a user and their device in their hand is to continuously negotiate the end-to-end pipe between them and you and understand what can flow through, what bandwidth they have, what latency they have and what device, what form factor, what screen size, audio-video codecs, frame per second, all those characteristics that we discussed should be continuously adjusted. Sometimes what happens, Comcast will throttle your traffic. For a simple reason, because they want more money and so they will throttle your pipe just because you are watching Netflix. They used to do that by the way, deliberately and they will shrink your pipe because they want you to watch Hulu because they acquired Hulu, right? So they want to watch Hulu, if you watch Hulu at the same location, same time, everything will be fine, just fine, why? Because they are a business, I mean they want to extort money if they can and so there are cases in which, there are the cases in which if you look at a survey of people who are least satisfied from companies or organizations or agencies. The number two agency was talking to the IRS, do you know who the number one was? Comcast, people would rather get audited by the IRS than talk to Comcast service, that was the situation and that is still the situation apparently and yes, I'm using Comcast right now but whatever. So the idea is, it is pathetic and they do that deliberately by the way, this is just throttle you to extort money not from you but from them, they extort money from Comcast and there is a case on that. And so, lots of news articles by the way, there's a court case also, Comcast wants to limit your Netflix binges. They want you to watch their cable I guess, right? You don't even have cable, right? I mean I don't have cable for so many years. So how will they do that? Because people are moving away from cable because of these providers, right? So they want to get you back probably. This is just what you see in the industry, it's happening. So what Netflix will do in that situation, they will continuously negotiate capability of your pipe all the way to you and adjust the characteristics of the feed that's going to you to the best possible extent they can. And that requires the device to report back every single, every few seconds, it will report back what's going on. How is the device playing? How is the bandwidth? How is the latency? How is the resolution? How is the smoothness of play? How is the thing latency? There is audio video lip sync. You know what AV lip sync is when the mouth moves faster or slower than the actual sound? Audio video lip sync, sometimes you see older videos going out of sync, probably you don't see that right now. I hope you don't. But you see me and you see my mouth moving in the same way that I sound. I hope that's not out of phase or out of alignment. That issue is audio video lip sync that might happen. So if that is noticed at the device level, the device needs to continuously report back those characteristics, which means there is a separate service that Netflix needs to run that doesn't fit on this picture. But it is about monitoring. Every single play of every single device that is playing a Netflix video at all the time. It is receiving constant data about what people are doing, what devices are doing and continuously collecting that information and proactively taking action to the best possible way that Netflix can continue to serve the customer. And while doing that. Is the audio and video not in the same channel? They're separate. What do you mean by channel? Because the latency issue which you're talking about where the lip sync doesn't match with the audio. I've seen with other services, but not with Netflix. I was just curious, is it because Netflix is monitoring more and managing the network better? Yes. Both. Yes. They go great extents to make sure that AV lip sync doesn't happen. And that's what any audio video service will attempt to accomplish, but not everybody succeeds. It really requires a lot of work. And I used to work in AV and Link, which is known as Skype for Business. I know this thing quite a bit, quite level of depth. And so the idea would be to think from a machine's perspective, like here, a collection of machines just looking at incoming metrics as to what the users are going through at a collective level and taking corrective actions. So when push comes to shove and Comcast really, really wants to talk to you down and your Netflix really starts to hurt, what do you think Netflix will do at that time? I mean, really thin pipe and maybe you are using an iPhone and you're driving on an area which doesn't even have AT&T signal. And maybe it is not a 4G LTE, but it is dropped down to 3G or whatever, 2G signal, whatever that thing was before. If you are in that area, 2G, what do you think Netflix will do? It drops the frame per second. It will basically tell you that your internet sucks and pause the video as opposed to showing you bad quality video, it will just tell you upfront, sorry, dude, go call AT&T. That's what it will do. So it is better to just tell people that your internet sucks than have them see bad quality video. That's what is the action that they will do programmatically and tell you that you know what, call your provider, have them or move to an area which has better signal or drive to an area which might have better signal, something like that. They will send some message, tell you that you know what, do something about it. You don't seem to have internet connection good enough to play a video. And so that is another action you can take very, very spontaneously. There are lots of other things that Netflix does in terms of monitoring. And what I would like to ask you is to come up with a list of things that you think that Netflix needs to measure. What do you think Netflix needs to measure in order for them to provide a good quality service? So this is by next time when we meet again, I would like you to give me a list of metrics that caters to making the customer happy. Stay focused on this. What metric will make customers happy because any other metric is useless. If you're not in the business of making the customer happy, don't measure it. Don't worry about it. Don't bother. Just strictly focus yourself on what makes the customer happy and tell me, what do you think Netflix should measure? What metrics should Netflix measure from a monitoring perspective, from all the devices globally, wherever they are? And come up with two or three things that you think are most relevant from customers perspective, keeping them happy. What do you want to measure to delight a customer and keep them delighted and happy? That's the whole goal, right? Keep the customers happy. So what do you want to measure? So you take it real time as opposed to over the course of our life. Type that. Type that in Slack chat. So type what you just said. So it should be a clear metric. I want to measure it in some number and succinctly write that in one sentence, one line. Just chat. What is that metric that you want to measure that will make customer happy? Or if it at least detects the customer is going to be unhappy, both scenarios are valid. So happy and unhappy, as long as they belong to the customer, what do you want to measure in concrete metrics? It should be a measurable number that you can monitor on a device, collect that data and take corrective action because all of it should be automated, by the way. So that's what we want to do is to be able to service or make customers happy, not this. And for that to happen, you need to measure these things. So what are those metrics? Is what I need you to think, not right away, but by tomorrow. By tomorrow, I mean the next time, which is, when is that 25 Saturday session, eight o'clock in the morning, 11 o'clock Eastern time. By that time, I would like you to have at least three metrics in Slack chat to tell me what metrics would you like to measure or other metrics to measure to keep customers happy or detect if customers are unhappy. And these metrics should be measurable with a number specific. I'll give you some hints. Nilesh, can we go to the competition which the modeling thing which they had of the 10% performance and look at the different variables, which they have used for modeling out there. Can you tell me again, the question? The competition which they had, the performance for 10%, to increase 10% in performance of probably people watching Netflix or whatever it is, can we look at those dependent variables what they use in the data model out there or is that- I'm not sure I understand what you're talking about, honestly. So simplify for me. The $1 million competition which they had. So the data model which finally won, can we look at those variables and pick from there or- Yes, you can. You're open to cheat. I mean, totally. My program is totally open to the internet. Do whatever you like. Come up with something. And so you can read this. Read whatever you like. I mean, I don't care. Don't have to use your mind for everything and just search. Here is the link. I'll give you a link if you like. Here's the link. There we go. So yeah, go read whatever you feel like. I don't put any restrictions whatsoever. And so some of the responses I have seen here on SlackChat are real-time or over the life of a customer, but I don't know what the metric is. So I did not understand David's comment. The next one is hours of content watched. Yes. Yes. Accumulated statistics. How much hours they have consumed. That's a good one to measure. The next one is Tyler says, use the time of length on the show they are watching and measure that against the actual time it takes for the user to watch. Yes, that's a good one. Yes, indeed. If there is any lag, yes, aggregate basis. That's a good way to catch. If there is a lag, that will tell you that the video is sluggish or typically you can detect right away. You don't have to wait for the whole movie to finish. You can detect that lag immediately in a couple of seconds. Stopped playing a movie in the middle. That's a good metric to have. You will observe that metric over time based on consumer's choice. Sometimes people browse. And so the way people differ in males versus females watching is that females tend to watch what is, sorry, females watch the TV and males typically watch what else is on TV. I don't know if you get the joke or not, but that's the behavior that people watch. Males will typically change channels to find out what else is on TV. And that's a human behavior. Apparently I have noticed that. It depends on who has the control of the remote. Correct. That's also true. So, you know, people who have control on the remote will tend to watch what else is on the TV versus yes, hiding the remote indeed. So a top playing movie in the middle is actually not necessarily a qualitative metric on part of Netflix. It may be just a bad movie, right? So they get bored. They just stop. They never come back to the same movie again. If that is the reason why they stop watching, it may be not tied to customers happiness or unhappiness with Netflix, but with the movie itself. Let's read some more. It also indicates that Netflix is putting content which is not appreciated by the group. I mean, not necessarily true because that user may not appreciate, but then there are millions of other users worldwide and the whole purpose is to provide a comprehensive choice of long tail. So you have to have good movies and bad movies as well. I mean, bad as in bad from that user's perspective may be bad, but it may be good for some other people. So you have to have a large collection. That's the whole premise of having the entirety of movie collections available. And that's what we are looking at. So let's see some more. So Tyler says then you can compare that to the residential areas and the national regions and look for outliers. Bingo. Beautiful one. Number of changes or breaks performed in some unit of time. Yes, that's also a good one. The way it translates to happiness or unhappiness will be why are you changing? I mean, is that just you're browsing around or a reason for that or is there is a qualitative problem in the way the Netflix service is rendered to your device? Is there a systemic problem in how the service is functioning or you're changing because you want to, just because you can. And that's a good thing, just because you can and you're changing and it allows you to change. That's a good thing. But if it's making it kind of relate to something what Raj said, I mean, because if I'm not happy with the content they offer me, I'm going to want to keep browsing and correct. We might be or might not be good. Right. So yes, yes. In fact, the ability for Netflix to have you binge watch original content is the new direction that Netflix has taken. House of Cards is a classic example that is available only on Netflix. They created themselves. They don't even rely on Hollywood. They do it themselves. So when you do it that way, that you're basically taking control over Hollywood, you're trying to influence Hollywood, which happens to be a supplier, by the way. So if you look at Netflix and you have Hollywood that supplies them movies, you are exercising some power on this guy, the Hollywood guys, by saying, you know what, yeah, we know you create movies, but you know what, we create top of the line House of Cards movie ourselves and just calling it a movie, it's not a movie, but it's a series. And so House of Cards, we create ourselves. We don't need you. Yeah, we want you, but we don't need you. That's the message they're trying to send by creating in-house production. The House of Cards is an example. Another metric that you will see Netflix do in competition with something like Hulu or Amazon or Apple is to increase the user stickiness. You want people to come on your platform to browse around and to keep coming and keep coming again and again every day, every week and goes on like that. That stickiness factor is what they are trying to analyze with the data that they collect from all of us. And there is a big data implications on that. We'll discuss that also a little bit next time, and I will now stop recording the recording. So I'm stopping recording now and I will read, there are more questions, more comments. So let me capture those comments also. So hold on before I stop recording, the hide the remote, don't forget the rating scale, the stars they give you at the end, the stars are given to the movie, not to Netflix. That is something that I want to understand. Make sure that you get it, is that we are talking about the service that Netflix provides, not the movie itself. The movie can be good or bad. We don't really care too much, and at least in this discussion. Of course you want to get good movies. But the idea that we can stream a movie properly with our cloud solution to a target device is the key idea that we want to measure. So also the age of audience watching, kids or adults, classification, more movies, more things to rate your users so you can suggest more movies to watch, bingo. That's a good one. And that's suggestion. So you rate movies so that you can then generate patterns of movies that you like and based on your patterns and other people's patterns, you can then associate and create similarities to create recommendations. So if there are A movies or B movies, server lag to deliver content from asked, yes, that's a good one. So what Raj says, server lag to deliver content from when you ask. I would rephrase what Raj just typed to say, time to first view. When you click play button, it should immediately show the movie. Is it possible to do that? Time to first bite, actually that's the measurement. Time to first bite. What Raj just said, when you click the play button and the time the first bite shows up on your device, then it starts to render the movie. The time it takes for the first bite to arrive on your machine is what you want to measure. Because people expect Netflix behaves as if the movie is stored right here and you just click and play. That's the expectation people have. It's very unreasonable, by the way, very unreasonable to expect, but people have expected that because of a thing which is called, let's see, I'll show you an example of this Amazon TV specification. So here is the Amazon TV specification, Amazon Fire TV. And so let's go see the Fire TV Gen 2, this one. And so let's go see how much memory they carry. I think they carry about two gigabytes of RAM right there. And what they do with that storage that they have two gigs of RAM and eight GB of storage, what they do is aggressively prefetch content. The fact that you are hovering on a particular movie, like Gleason, for example, the fact that you're hovering on it, your search results show and you click on this, for example, and I want to now watch it. I can, but the fact that I'm here, it already prefetches that movie right there on your device. That behavior translates to improving this metric, which is time to first byte. What Raj says, server lag to deliver content from asked. And then another one, playing faster than content delivery or buffering. Buffering is another metric that you want to track because if it is buffering and slowing down, there is a network problem that you want to address in some fashion. Cool. So now I'm stopping recording and I stopped.