Hello, good morning Mukesh, how are you doing? You're muted. You should unmute yourself. Hey. Good morning. Good morning. Yes. It was taking long to unmute. How are you? Yeah. Pretty good. Saturday morning. I'm just going to turn myself towards you. Just a minute. Just setting up a couple of things to get started. Okay, I think I can see you now. You should be able to see me just fine. Yes. Yes. Beautiful. So this virtual box, you had some question about, right? No, no, no, no, no question. I was just trying to ask whether there was something else like when you said, just remember then just like enough OS thing, if there was something else for Windows version, and then you can just log into some Unix on cloud. Right. It is this thing that, that thing you're looking at, this thing is available now. It is just recently came out with this product. It is still in, I think, beta. It is shipping in Windows 10, but beta, it is branded as beta. Oh, okay. Okay. It is a technology branded as beta. What they're doing is running VM inside with Ubuntu and exposing that bash shell. So here I have a little demonstration on that. It is part of Windows 10. You can get it directly. That's one way. There are a couple other ways also, like minimalistic approaches. So there is this thing called PuTTY. Oh, yeah, yeah. I use PuTTY. Yeah. So PuTTY is also possible, you know PuTTY. So that's another thing. But PuTTY is more like a, you know, a software to emulate secure logger terminals. Another one you probably know is MinGW. That could be on Windows. You can go to minimalistic install of GNU for Windows. That's another possibility. Have you seen this? Seen this thing? No, I have not seen this one. Okay. So like these, these are, you know, implementations of minimal Unix like system running on top of Windows. So this is just a couple of, I think this one is the cleanest one. The best, best, what I would suggest is this. You are using Windows and you want a minimal environment. This is a good one. Yeah, with PuTTY, the thing will be like you will directly log on to your cloud servers. Right. Right. Yep. Yep. You may not want that. You know. Yeah. Another thing that I did the other day was to be able to morning get some of the login improve. Hey guys. We were four of us. Okay. Oh, hi. So I was able to fix the login improvement that I was trying to include. I think the login experience that we had was bad, you know, really bad with this new layered approach that we had, right. If you remember, I talk about, let me just draw it. So just to be clear, I think I have succeeded in cleaning up the login experience. What does that mean? It means that before the problem was that, you know, you had to go to, let's say office and then authenticate through Google and then authenticate through cloud genius. That was the multi hop approach, which was lame. And so this technology we got like seven days ago, we just signed onto it since I was trying to say if I can use a single sign on approach. And so what I have now done in the newer scenario, which starts immediately, which is a slight deviation and it's a lot simpler is that you go to office, you sign in there and that's it. There is no other step. You just automatically are signed into cloud genius, that step I have cut out completely. You can see a demonstration right here. I'll just check. So what I'll check is, you know, here, this login page, I'm going to log myself out like this. And what I will now do is, it's still logging out. Okay. So log myself out. So I am logged out of this cloud genius thing and I need to now log in. There is a new button here now, which is basically says sign in with your cloud genius account. So you click on it and that's it. You should be in provided you are signed on to office already. That's the only thing. If you're not signed on to office, it will take you to office and sign you there and then bring you back to the cloud genius site. So demonstration for that will be simply something like, I'm going to open a new browser and in there, I will see if I can log in. And so here I have a different browser and not logged in, I think on office either. So it takes me to office and here I can just type in a name. I'm just typing a dummy name here and there we go. So he's going to log in and he just logged in and you can see the dashboard where he is not enrolled in any courses. And so that is because he's a new student. He does not have yet any enrollment. So that needs to be fixed, but he's not really a student. He's a fictitious student. So that's the demonstration I wanted to quickly give you. The login flow has improved and I think you should have less trouble, meaning it'll just a one hop to office and back at logs you in everywhere. And that should be working except for Mukesh who wants to use Gmail. That is fine. You can type in your Gmail here and password here. That will work for you as it used to in the last couple of times. You went in, you can keep using that if you like, or if you want, you can use the other method. Tell me what you want and we can enable that way or accessing through Gmail will continue to work unless you tell me otherwise. So let us talk something interesting today, which is bastion or tunnel or jump box. And let us see how we can systematically organize our own virtual private cloud step by step as we understand underlying concepts and see how we can construct out interesting scenarios for our customers. So that's our discussion for today. We'll go step by step in clarifying detail about what it is, how to make use of it. And just quickly checking if everybody's online, I think we have four of us, which means lots of people are not there yet. Which tells me it is Saturday morning. That's the hard part. I will miss today. I am visiting LA. Okay. Please record. Yes, we are recording. There are messages people are sending that are missing. We will have attendance. Not much today, I guess. But let me just quickly check a couple more messages here. I think we are good. I think we should just continue. So if people will sign on as they get up in the morning, it's understandable. They will just show up. So on the whiteboard section here again, again, from instructor, what I'll go do is in this section, I will describe these concepts, which is what we are talking about right now. So the concept that I want to now go, the last time we did with a quick recap, what were we able to do is just understand these basics of have a machine here. This is your laptop, for example. And through that laptop, you're able to connect to some machine away, some computer, you were able to connect to it using simple methods such as the secure shell protocol. And there's some computer you were able to get in Digital Ocean or Amazon, it doesn't matter which cloud you get it from. But the difference in these types of clouds versus these types of clouds is that they have another security layer, like Amazon calls it security group, which is a clear concept that you already know and are aware of, which basically means a security group basically is a collection of rules. As you remember and recall collection of rules, these rules are basically taking you from a source and allowing you access to either TCP or UDP protocol and on a specific port number. Port number this, port number that, that's how it really shows from a collection perspective. So this collection of rules, this is one rule, and you can have many such rules like that. And they will all collectively apply to a set of machines. So that's what a security group construct is. It is also similar to what your network concept in Azure and firewall concept in Google. So basically that's what it is. It is the IP tables rules, IP tables is a open source software that those, that's what essentially it is. It's a collection of rules of IP tables that apply and control security access. So security permissions, firewall, that's what it is all about. And that applies to a virtual private cloud. So VPC that we have, essentially VPC would be an island where this outer perimeter is secured. Can you see me on my face or as well as my desktop? Can you see both? Are you able to see both? Yes. No, just the desktop. I can see the face on the side. So you, you have the choice today, I'm doing slightly differently today. You have the choice to see me or not see me. There is a place to hide my face if you want or bring my face back up if you like. So I'm doing two streams, there is a desktop stream and a video stream that I'm sending and you are sending your video stream only if you are, if you decide to. And so I've given you the choice to look at me or not look at me. You can do that accordingly. So having said that, uh, this VPC idea is basically a, an island network. It is like your home or like your office where the network inside is isolated. Unless you decide that one such network is going to be public. So that's a different scenario. You may decide that. And in that case, that network itself becomes a public network. And so it will, if you put a machine on this network or another machine here on this network, in that case, these machines will get a public IP address. So that means they are routable from the internet outside. So you can just get to them. If you create a machine on the internal networks and you can have multiple such subnets, isolated subnets, uh, these machines that you will create on this subnet or that subnet, uh, they will not be routable, no route. So no nothing. You cannot go there directly. So this basically means that we have to have a structure that basically builds out a complete route. If you want to actually go from your own office, this, your own office that you have somewhere, which is probably outside the Amazon cloud, you don't live in Amazon. So you work and live elsewhere. You probably, you have your own facility. So you are outside practically in the public IP domain. And this person, which is you, wants to access your own VPC and it is not possible directly. And that's a good thing. You don't want it to be accessible because you want to go in as a, as a jump box and then go in. So that's, that's what the concept that I, I will now begin to build towards a live demonstration, which means I'm going to go to this location that is Amazon cloud. I'm going to log in and give them a token if they ask me for. And so the token is right there. And I'm going to paste the token and now I'm in. So here we are going to focus on one such region. We'll select any, any one region that we like. I'm going to stick to the Virginia region today. And here I will go to this easy to section. And in that section, I know that I have one VPC, if you remember right from the last time I can go and confirm that with you. So, you know, if you go look at the network and security sections, this is the place where you can identify, you know, your network related components. However, there is a separate tab in here, which is about networking in the bottom side on the bottom left side, which is more specifically dedicated to virtual private cloud. I think I lost it. I'll bring it back. So left bottom, that's where you will see, you should be able to see it yourself, which is this networking section. And that section has a dedicated reference to virtual private cloud in the region you have chosen. So I have chosen the region, Virginia. And in there, I'm going to look at the VPC specific things right here in Virginia. And I'm going to look at my virtual private cloud setup. So in that, I have these different blue colored items. So let's go look at what these mean. And if you go deeper in here, you'll notice that I have one VPC, one. And that VPC is given to me by default. And that is happens to be my default VPC. And as I remember, if you remember, I said, do not delete your default VPC. So I'm repeating that caution again, do not delete your default VPC. You can delete other VPCs, no problem. But this is a bug, according to me, it's a bug in Amazon's design. If you delete this, then you have to create a support case. And they'll give it back to you. But it's a hassle. That's why I'm saying do not delete the default VPC, other VPC, no problem. OK to delete, OK to delete if you like. This is a careful thing that you have to do. Don't delete this one, which is given to you already. Like this is the one that I have been given. How do you tell if it's a default or not? You just go to that VPC's name, detail, and you'll say tenancy is default. But that's the tenancy level. The default VPC yes or no question answers whether it's a default VPC or not. So that's the idea I want to point out too. Inside this VPC, let's give it a name, a proper name. So I'm going to call this name Cloud Genius VPC, just the name of the VPC I just changed. The ID number, the CIDR notation, the option set, the routing table, network access and all these things, they're just left like default, whatever was given to me to begin with. I will also go inside that VPC now and look at my subnets. And in here, I see that I have four subnets. And these are the IP addresses ranges in those subnets. You will also examine a couple of other things such as route tables. And there is only one which is given to me by default. And I'm deciding to ignore it for now. Then we have Internet Gateway. This is also I'm saying ignore this because I would basically create our own VPC from our clicks of mouse or maybe later on through API calls and automate that creation of a VPC for a given application. And we'll do those steps as we build it up. But first time, manually do the whole thing. So just to understand the things. And so these builds that you're looking at are given to us by default. And I just want you to give a quick overview of it as we begin talking about these things and eventually building our own separate VPC that we will do. There are some elastic IP addresses assigned to us. If you can, the elastic IP is like a static IP that you get. So your IP address does not change. This is the place to get it. Endpoints are defined, NAT gateways are defined, and we will create those also for ourselves. And like these are the components associated and given to you by default in your default VPC, which is this one. Now having discussed briefly about VPC, what I want to be able to do is in our dashboard for EC2 where we get access to computers or machines, I'm going to now create two machines. One that gets a public IP address and one that does not. What does that translate to is that I am basically putting machines in somewhere like this. You have multiple subnets in your VPC. So I'm going to get one box here that gets a public IP address and another box that does not. So it has a private IP address. And this box is not routable unless you are inside the VPC. In that case, you can go here, go there, but you're not there. So you're outside. You're not here. So you have to be outside somewhere here. And then you can connect to the public IP address to this machine and then hop. And that hop is what I'm going to establish. So I'll build this machine first like the last time we did, if you remember, for which I need my virtual machine. So I'll start virtual box and start that and keep it running as we go. And let me cut down a number of windows open here because unnecessarily too many windows open. Yeah, it's less now. So in here, I'm going to launch an instance. And in this instance, I'll just use an Ubuntu server. So select. And the cheapest machine is a good enough machine. So the T2 Micro is free for you. You should select that. It is eligible for free tier. So you can actually continue to use that machine for one full year if you like. And it is free under the free tier promise that you probably have signed up already for with the Amazon free tier offer. So that's the machine I'm going to select. And it is a very small machine with one CPU, 2.5 gigahertz Intel, one GB of memory, gives you elastic block storage. And that's the machine I'm selecting as our jump box. So that jump box, once I go and read the details, I will see that I am going to put that in the only VPC I have. And I will select to put this in our subnet, which is this one. So 172.31.00 subnet. That's the one I will decide to put it in. And in the public IP assignment, I will say, yes, please enable that. And so I'm going to say, please give me a public IP address. And rest of the settings, I'll just leave them alone because that's I'm not interested in right now. All I need is a machine with a public IP address that I can use like my jump box. So that's exactly what I did. I selected my VPC here and the subnet. I just selected one of them and said, give me a public IP address, please, in this section in here. Give me a public IP address. That's all I wanted. So I'll add some storage, which is given to me by default, eight gigabytes is good enough. SSD is OK. And I move on and call this machine my jump box. Just a name. I'm assigning as a key value pair, tagging the instance as jump box. And after having done that, I'll proceed forward, assigning it a security group. I have a security group already. Last time I created it one. It says, let me in via SSH, if you remember last time. And that allows me to go in via SSH on TCP protocol, port number 22 from my IP address, this location. So it lets me in through that address and I will review and launch. When I launch, I should first review the screen and then I launch it. When I launch it, it says, yeah, give me a key pair. So I already have my key there. I will just select that IDRSA and I said, click acknowledge. This is the same key pair that I have here, by the way, in the machine that is running right now. And if you remember and recall, that key pair is sitting in the dot SSH folder. And the file name happens to be this, IDRSA.pub. That file is the public key that we have given to Amazon in here. That sits right there. So that's the key I want to use and say, launch an instance, it launches the instance. That IP address of that public IP address of that instance will pop up. Commentarily, it will show up here in the public DNS section. And I'm waiting for that to happen. Now while it creates, so we have begun creating this unit. I want to now create another machine here, which does not have a public IP. That's the scenario that I'm drawing now. So while this machine boots up, I'm going to launch another machine and say that I want to use Ubuntu. And I will choose another cheapo, T2 Micro is a good enough size. So that's the one I'll choose, cheap one. And I'll go next configure the details. And in this scenario, I will like to make sure that I'm getting one instance that I am getting my network as cloud genius, which is the same VPC that I'm talking about. Now in the subnet section, I will select a different subnet, such as this one probably. And so a different subnet I'm selecting, which is not the same subnet like the last time. The last time it was this number 32 was zero. This time it's a different subnet. It's one C, the other one was one D, just the placements of these subnets. Now in here, automatic assign public IP, I will say, no, no, no, please don't. I don't want a public IP. And so it is disabled by design. So I said, no, I don't want one. And then everything else I'll just leave alone as is and proceed forward. And you remember that I'm not getting a public IP, which is exactly what I want. And then I add some storage to it, which is at the storage number one. So this, sorry, not number one, but the machine count number one. Size of the storage, 8 GB, tagging the instance as internal machine. So this will be internal machine. That's the name I want to give to that internal box that I have. And for this internal box, so this machine was called jump box. And this internal machine, I'm going to call it internal machine for simplicity, internal machine. And so that machine does not get a public IP, but a private IP is going to be available to it. That's okay. Now we have assigned a security group here. It says, let me in via SSH, that's the name of the security group for this box. I will create a new security group for this box, new security group. And that I will do next. So this security group is going to be slightly structured differently. The idea behind this security group will be something like, let's call it the internal security group. That's the name I want to choose. Now the rule set for this should be something like, from the source location that I want to come in, allow protocol, TCP protocol, and port, allow 22 port, and then get to this internal box. That's what I would like to do. However, the source machine should be that jump box, specifically that box. So we should be able to say that anybody who is a member of the let me in via SSH, if you're a member of this security group, then that's okay for you to go in into the internal machine. But if you're not a member of this security group, then you're not allowed, meaning me in this office location, if I want to go in, I should be denied access. Anybody else except unless you are the jump box, you would be denied access. So that's the rule structure. And that is what I'm going to implement in this internal machine. And I think instead of calling it internal machine, let's call it our application server. That's better name. So application server, it's a better sounding name, because we are really going to put in the internal security group, we are going to put our applications. So I'm going to call it the application security group. And this machine becomes my app server. That's a better name for it, in fact. So with that better name understood, what I will do now is this application server, I will now create a security group that I will assign it just for this internal application server. And if you look at our collection of security groups, we have only two, which is the let me in via SSH for the jump box, and a default one. But once I just generally ignore all of them most of the time, and that is not what I want to play with. So I'll not use that one, which is given to me by default, but instead, I'll create a new one. And in there, I will say, you know, this is the application security group. So that's what I want to use it as. And in there, I will allow SSH access, the TCP protocol, and the port number 22, as long as this thing is coming from the let me in via SSH. That's the source that I want to put. So the custom, I think it is too big. So this custom selection that I want to put, I want to actually put the name of the other security group that I have already chosen. How do you get that name? You want to go to the console in a different tab, and go to the easy to section, and go to the left side where you will find a list of security groups. And that's where you will get the name of the group or the group ID. That name that we have called let me in via SSH, the group ID is actually this. So that's the group ID I want to grab. And this is a little tricky, so I'm just pointing it out to you. The scenario that I am doing is I'm grabbing this name for the group ID from here. And I'm allowing in this security group, that group ID. You see what I'm doing, do you? I'm selecting this group ID here, because I want to select any machines in that security group called let me in via SSH, which by the way, contains only one machine right now, the jump box. And that ID is what I'm putting there, like right there, like this one. And by the way, I just found out a shortcut. The shortcut to finding out this complex number is as simple as selecting custom, and then type the letter S. You get a drop down of all the groups that you already have. And then you select the one you like, like this one, let me in via SSH. And so if you are a member of that group, which is the let me in via SSH, the jump box group, in that case, I want to allow you to come in through SSH. Other people, no, there is no rule for you. So you cannot, by default, everything is denied. Unless you have an explicit rule to come in, then you can come in through that location. So that's the layout of this security group. And I will review and launch, and I will launch the machine. And I will choose my existing key pair that I already have, which is the ID RSA. And so I want to quickly summarize what we just did, in especially the context of security group. The first rule we did was that we had one security group already, which called let me in via SSH. And this one applies to the jump box. And that one, this rule itself allows me to come in from my office to go into the jump box via SSH, port number 22. The second security group that we have created just now, it's called the application security group. And in that group, I'm saying if you are a member of this group, you can come in to SSH. And this is where I'm running my application server. So that's the difference in this security group versus that security group. It allows me to come in from my location to go into the jump box. And this one allows the jump box to come into the app server. That's the scenario. With that scenario understood, I'm going to launch these other instances. So I have now two machines running in here. I can launch them. I can see them. So this is one of them. And there are the other two machines. One of them is running already. And this is this IP address that you have. Public IP address is available to me. I can make use of it. If you look at the other machine, this machine is not going to get a public IP address at all. Only thing I will get is this internal private IP address. And this address is private. As you know already, 172 addresses are private. Just like 192 are private. And that machine is still pending. It is booting up and getting loaded. And the layout right now, it is running now. It is status check initializing. So right now, we have a scenario where we have two machines, like machine number one with a public IP, machine number two with a private IP. And this security group is called application security group. This security group is called let me in. And I am sitting here outside the firewall. Now I will go in like this. And I have this rule. The application security group rule says, if you are a member of this security group, you can come in. And this machine is a member of this group. So you can go like this, 22 SSH. So that's the scenario we just built. Now let us test this. We have actually created machines. And so let us test them. So we are going to grab this public IP address from this location and copy it into our buffer like this and go to the terminal here. And in the terminal, I will go to CD followed by tilde to go to my home location. And I am in home. I can check whether I'm in home or not. Yes, I am in the home location. After having said and understood that part, I am in the home location. So don't work in this folder. Work in the home location. Now having done that, I will say SSH and then give it the public, sorry, the private key that I have, which is in that location. And then I will type the address, the IP address of that machine followed, username followed by IP address. So this should let me into the jump box, boom. And I'll say yes. And I'm in, into the jump box. It will connect. It takes time. Now I'm connected to the jump box. Now you would imagine that this method that I just said, if I want to go to this machine directly, I cannot because the IP address is private IP and it is not routable from my location. However, I'm connected here and so this jumping, you would think is easy and possible. However, I told you one critical thing to remember, which is never share your private key. So you would imagine that, you know what, now that you are already in the jump box, getting to this machine application server is as easy as running the same command that you have here, you're already sitting right on that box right now. So you can just say SSH and then private key and then IP address of this box. And you should be able to go in and that is technically correct. Yes, you can. However, that's a two step process. That's a hassle. And more importantly, you are violating a principle that you gave private key to the jump box. You should not, no. You should not get private key even to your own jump box. So how do you now get in? How do you go into the, excuse me. You would generate a key from the jump box. No, no. If you generate a new key here, then how do you give it to this machine? Because yeah, it doesn't accept a key. The only way this machine can accept a key is through this interface. Look at the time of creation, you have to select, place my key there. So that new key in the jump box, bad idea. No, no, no, no. Don't do that. You're not giving private key to the jump box either. So what's the alternative? And that's what I will now show you. So you don't need to give your private key even to the jump box at all. But instead there is a specific command. That's what I will bring up now. It is a configuration in SSH that allows you to simplify. So there are multiple ways of writing, I have a very simple example already available to you. So I'm going to paste that for you. Give me a second. I'll grab it. And here is the example. And so that link, I'm going to paste it in the notebook right here. And in that notebook, you will see that this example, which is, I'm going to paste it also in Slack chat. So you have it quick and easy, the Slack chat link. So this link has some code that allows you to one command jump straight into this machine without sharing your, straight here, without sharing your public, private key at all. So you will be here and you will run this customization that I suggested on the laptop on your workstation. And you will go like this, like that, automatically one command just takes you in there internally. So that's the code, that sample that I have given for you. Let us read that and understand what it means. So here is the raw form. So in the raw form, I'm going to expand it. So here is the code, I'm going to grab it, put it in our virtual machine right there and exit out of this jump box right now, back in our cloud genius workstation. I'm going to go to the SSH folder one more time. And in there, I will like to list out these files that we have. And I need to see one more file in that location. And that is the file that I've pasted in GitHub. And I need to create that file right in this location. So I'll say atom and then dot. I'll open the atom editor in this folder, dot means this folder. So I see these three files. And I'm going to open a new file by pressing Ctrl N, new file, and saving it what I copied from GitHub, Ctrl V. And that's the code that I will modify. But I will save this Ctrl S to save. And I'm saving it in the dot SSH folder. And remember, the file name I'm choosing is config, C-O-N-F-I-G. That's the configuration. So now I have four files in that folder. Sorry, not this command. But instead, I'll say clear and say list it out. And I see that I have now created a new file called config. That's the file I want to now read. So here is the configuration. Now, in this example, let us simplify it a little further. And correct example to match what our OS choices are, Ubuntu. And the public IP address of the box that we have, which is what do we have? We have the public IP address, this one. And so I will place it right there. And then the internal IP address of the machine is not 10 dot. But let's go to the internal IP address and grab it from here. And paste. So now I have structured the command configuration correctly. Now, let us read it. What we just did is that in this particular example, we are defining a configuration for SSH proxy command. This proxy command uses SSH method through the jump box without sharing your private key. So your private key that you have here, IDRSA, it just stays there. This does not go out of the machine. It's the private key. It stays right there. The jump box connection gets established through this location. And it will forward its agent ahead as you run the proxy command. When the proxy command executes, it will attempt to connect through and through the jump box that you have connection to and attempt to go to this location, which is where you really want to go. And this is the internal app server. This IP address is the public IP address of the jump box. This IP address is the private IP address of your internal machine app server. Both of them are using your identity file, which is sitting locally. This is your local laptop workstation folder. And the same thing here. And that is the IDRSA. By the way, to select multiple lines, you can just press Control D and D as in dog to select the same thing over and over. So you can say double click this, double click that. And you can even modify multi-cursor. So this atom editor is pretty darn powerful. You can do multi-line editing in one command. You just type it once and it changes like this. And you can undo the changes. So it's fairly powerful. Now having... We'll talk about atom later. So this command that we are constructing is a configuration to SSH proxy command. Now let us see it in operation and then we'll come back and discuss this some more. So I'm going to close this file and minimize atom, go back to our terminal, come out of that folder. We don't want to be there. So CDE tilde come out, PWD to check where we are, we are in the home location. And now I want to connect to the machine internally. I have that IP address, which is this one. I'm going to copy it, which is the private IP address. If you try to do a trace route without the SSH configuration, it will not be routable because it is private IP address. However, in our configuration, this SSH configuration allows us to go directly there like this SSH. And then you can even forget typing everything, you can just type IP address and it will take you there. Let's see how it works. And you can see that I am now connected directly to the machine inside and this is the machine inside. I can confirm this is the IP address, which is the internal IP address 4262. Let us go compare that with this 4262 right there. Let's go do some more testing in terms of really understanding what just happened. So come out of it and then list out the folders that you have in the dot SSH folder. List it out. See what we have in there. And so you see that this known host file exists. I'm going to delete that file completely like this rm and then tilde dot SSH and the known host file. I want to just get rid of it. When I did that, I don't have it anymore. Like this, I deleted that file specifically to show you the steps that happen in one hop jump with one command proxy command. That's what we are doing. And I want to illustrate some of the concepts underneath as to what is going on as you jump. So that is simplification a whole lot. So I'm going to kill that known host file completely by doing this. And then I am listing it out to show that I have no such known host file. And now I will connect that SSH command one more time, which is a strange way of just connecting to the box internally without even specifying the username and the key and nothing because we have it specified already in this file. This configuration already knows a bunch of things. What does it know? Let's go read. So I'm going to adjust my screen a little bit. And in here, and this screen, editing it, modifying to suit the window right there. And so in this configuration, you will see that it already knows that this IP address is a machine defined already. That's why we are just using the IP address directly and it knows how to go there. The route to go there happens to be proxy command, which is line number nine. And in that line, as you know already, is a reference to the jump box, which is right there. And the word jump box is defined in this line, which says the IP address of the jump box itself is this IP address, which happens to be publicly routable. It is not a 172 or a 192 or a 10. It is 54. It is a public IP address. It is routable. So you can go to the jump box and use the same key that you have locally stored. This key file is locally stored in both these command portions. It is still not copying over anywhere. It is sitting right there in that folder. And that's where it belongs. It doesn't need to go out. Now having said, let us see some action and let us understand what's going on. So in the terminal side, I will expand a little bit and run this command SSH IP address. This is the internal IP address. Now I will click. And you will see that there are two sets of questions being asked. First one, are you sure what you're doing? Do you know what you're doing? Do you really want to connect to this machine? I don't know this guy because I all the deleted the known host file already. I deleted that file. So I don't know any of the machines anymore. I used to know, but now I don't. And so I have deleted that known host, which means I'm getting prompted with this question. Do you know this? Do you trust it? And I said, yes, I'm going to say yes. And so it is going to basically make an entry in the known host file as soon as I say yes. However, it is not satisfied yet because even though you have a known host file with one entry, which is this entry corresponding to the jump box, you still haven't reached the other machine, this one. And so that is the next question is asking. You said I can go here, but I don't know what is this. Do you know what you're doing? Do you understand that you want to go there? And it is going through no host IP for proxy command. I cannot even go there. It's going through a proxy command. Do you want to connect to that internal machine? And you say yes. And so I'm going to say why yes. And it will connect all the way straight to the machine inside. And it is going to create another entry in the known host file. So there are now two entries in that file, one and two. These two entries correspond to, first one is for the known, for the jump box. The second one is for the internal machine that you have here. And now we are in one shot connected to the machine inside. Let's do it one more time. SSH, IP address, enter. And we are straight going in there, inside the internal machine, one command with one hop in between through the jump box, and we are in. And we can run whatever we want there. So that's the scenario that we have. Give me a second. Any questions on this? Hi, Niles, David here. Hi. Quick question. How is the presence of that private IP being published or managed? It is managed through an inventory file that you collect and keep. This inventory of all these boxes that you have internally will become large over time. You will have a large number of machines in your internal networks. So all these are subnets, subnets internal. And you have to keep inventory of these things. So you have machines running in all these internal subnets and you have one public subnet, maybe two. And these are your jump boxes and add boxes and other things and internet facing machines that are outward facing that exist, but they're not too many of them, a couple handful, maybe three or four, depending on what your use cases. Now you have to keep track of the inventory of these IP addresses internally. And as I said, these machines will come and go, so they will come and go, meaning they'll die. They'll get created and then they'll disappear. They'll create and then disappear. These things will happen automatically, dynamically. And that's what you're looking at this screen here. These internal networks will have machines like these get created or disappear. On demand, they will shrink and grow as we go along. And so you have to have a systematic method of keeping track and updating this file. For example, there are even better methods. This is just a starting method of dealing with this problem. So the question is, if that were the case, you must have some sort of utility. You can SSH and execute through the jump box to collect the machine ID, right? So I want to understand your question very clearly. You said SSH agent on the jump box. I don't think we want to run that. Yeah, but how can I dynamically keep track of the inventory then? So as you create a machine or you destroy a machine, that's where you keep track of it through another software such as Chef or other methods. So you know, Chef is just one. In Microsoft technology, there is desired state configuration. You can use that. And so there are many such tools that are available that as you create the machine, you have to keep track of the inventory and you keep that in an inventory file to understand what machines you currently have at that moment. And then you dynamically assign this method or other methods. In fact, even simpler methods are using a VPN that you will put in the cloud that you can connect directly between your office and the cloud. And that basically gets rid of this method. You don't need this method in that case, because then you are VPN connected and we will do a VPN example also at some point, which is a little bit more involved, we'll build these concepts like bring everybody up to speed on the same level and then we'll go to the examples. So we have examples of VPN, examples of how do you manage inventory of machines because this is a dynamic inventory that machines will come and go like I described here. So they will disappear, they will come back and you have to keep track of what do you have running and how do you manage it. And you have to keep track of your security groups and understand who has access control and understanding which ports are open, keeping everything else closed by design. So that's to improve security. From a security standpoint, it is extremely important to understand what you have to begin with and then make sure that everything is exactly how you define it to be. Because that definition that you design something, right? So that's a good digression. So here, if you design some architectural layout, you need to keep track of what your design is. And also, more importantly, think about this one thing, which is that you specify something that your design should be and then you run some software to implement this design. So the software will implement it. In real practice, you will probably never, I mean, maybe you will, but I don't go to launch an instance using my mouse. I don't do that. I'm doing it right now to show you. But in real life, I don't do that. I will never ever, not ever, but never launch an instance like clicking these buttons. That is boring. I mean, it is so hard. It is so manual that you have to keep track of everything by hand. What was the IP address? Let me go look. And then, oh, that IP address, did I copy correctly? All of that problems are bad problems to have. So instead, what I do and what I recommend that you do is something like this. That you design your idea, your blueprint, whatever you call it. Basically, whatever you come up with is the thing I want in my cloud. Good. That's a design. It is also known as your specification. I will specify that's the configuration I want in my cloud. I need these subnets. I need that set of machine plays in that subnet. This routing, that security group, you lay out the whole thing in your specification. However, you're not doing it online at the same time. You're not launching these things and creating subnets. You're not doing it by hand using your mouse. No, don't do that. No, don't do that. Instead, you run some software. There are many software that are very new and very interesting. They are all open source, most of them, that will implement whatever your specification is in the cloud. You design, you come up with an idea, it will do it for you. Not only that, it will keep track of it. If you don't have to do this by hand, that's the design that you should get to. Not only this, there are further incorporations of the same idea even better in that. As you design something and you run that magical software, I call it magical, but it is not. It's a beautiful software. That software will maintain, not only create and implement, but also maintain the state, whatever state you define. You define a design state. If you look at this product, desired state configuration product from Microsoft, this one, the name itself tells you desired state configuration. Basically, you desire something here. My desire is to have this state in my cloud. This is the cloud. You desire that my cloud should be like this, should be implemented like that, should function like that. Okay, I got it. That's your desire, great. Let me implement it for you. It goes ahead and creates the cloud, creates the subnet, creates the security group, creates everything, implement it, and waits for you to see if your desire changes. If you change your desire, my state was state number one, now it is state number two. You can change the desire and if that desire changes, it will change the state again from state number one to state number two and automatically adjust to what you desire. Also, if something goes wrong in the cloud, it will proactively correct it, fix or heal. If something goes wrong, it will fix it. I mean, it will run that software itself will run to correct the design according to what you define or you desire, whatever you desire, it will correct that if something went wrong, it will detect the problem and heal it and fix it and bring the scenario in line with what you wanted to do to begin with. There are software and this is just one of them. You can use this through Chef, you can do this through Ansible, you can do this through a bunch of different and we will do exercises like this. The second segment is dedicated to this exercise. This DevOps thing that we have, the one that will start in like six or seven sessions after that one is dedicated to automation, so it is going to be doing this thing, we'll be doing a ton of work like that. There are exercises built in for you and you will see that. So I think that answers your question precisely in terms of, I hope that answers. So this mechanical, manual, boring using the mouse method is a good one for first time, maybe second time, maybe third, but that's it, not anymore, automate. Design first by doing it raw by hand and understand how much painful it is and then you automate those and you reduce the pain. So that's the design. So this proxy command, jumpbox configuration that you saw, it is in the GitHub link that I gave you and this is the example that I'm running right now. So I'm going to copy this example just for you and paste it in your Slack chat. So you have already a reference, a quick reference, something like this. So here it goes and I should actually paste it like a snippet. So create a snippet and send it. That is a better way of sending. So you have a raw snippet, you can just see and that's the snippet that you can see in your Slack chat. So if I have more than one jumpbox, how will this file handle that? So you should use one jumpbox to go through at a time. So if you have two different jumpboxes, you should decide which configuration that you want to use and which jumpbox you want to go through. At one time, you can go through one location, not two jumpboxes. So you can balance them out, but at a given time, you can enter through one door. If you are entering through one door, it is not possible for you to enter through another door at the same time. And that's the idea. So the reason why I was asking is just in case if one of the jumpboxes is down. It requires extra code. It requires code for you to detect whether the jumpbox is down or not and decide one of them as master, the other one is in a backup and if the master is down, then you change the jumpbox to the other one and you have to write code for that to modify this configuration and that will work. So this is just a simple and straightforward thing. This is just a basic thing to start. It works. You saw it working. It works in the example that it illustrates this concept that first thing, never share your private key. Second, you can automate simply by running it like this so that you don't have to worry about the jumpbox existence. In fact, all we are doing right now in this example, sorry, in this example, if you look at is just say SSH, an internal IP address, we don't even have to think whether the jumpbox exists or not and we go and it lets us inside all the way one shot without sharing our private key to anybody, not even to our own jumpbox. It just goes in into the internal network and that is just one way of dealing with it. More evolved methods involve VPN access for which we can run a Docker container in the cloud as our VPN. So we'll get to that. There are examples that we will have in our site as we go along. We'll do these examples in the bootcamp that you will see. By the way, I will open up access to the other two segments also tonight. After this session, I will open up that part for you. So you can start exploring this reading casually as we go along. I'm going to open up access to all these three for all of you. So I'll do that sometime later today. I think I was busy fixing the login issues, which was hurting me most. I think most of you did not, in fact, none of you complained actually, except Mukesh was having difficulty in logging in, but apart from that, that took me some time to build a new piece of application to include one sign-on, single sign-on login. So we have it now included in its running. It is tested and I hope it will not cause trouble. So that was this design that we did. So a quick question I have for you is, how many of you are not familiar with virtualization per se? There's this thing called virtualization and we have some videos for that here in the module number three. And I'm going to assume and correct me if I'm wrong, but I'm going to assume that you already know what virtualization is, which is basically at a high level. What is virtualization is that you are creating a smaller machine out of a big machine, practically. You're emulating the underlying hardware and you're cutting it bigger box into smaller boxes and smaller sizes and using a portion of that until you find that it is still useful, not anymore. Yeah, that's the kind of minimum size that I can get. So that is still maybe useful, maybe not. If that is not useful, I'll go to this size and call that my minimum. Maybe not, then this size is my minimum, but as long as it is usable, I will keep dividing the machine, big box, into smaller boxes and use the size of the machine that is still useful. That's what you're looking at in this example when you go in the Amazon console and launch a machine. You are looking at these sizes, T2 nano and T2 micro and T2 small, T2 medium, T2 large, M4 large, this and that, all these tons and tons of sizes that you're looking at. The question that I have essentially is for you, do you know virtualization already or do you want a quick description on that? Quick description is good. Okay, cool. Let's do that. So not spend too much time, but let's go understand a couple of things. So let's go do a look at this thing. This is a virtual machine. Here I'm going to shut it down for simplicity and understand what tools and techniques that I currently am going to use to show you the big size computer, that is this one. It has 16 gigabytes of memory that you can see here and it has a hard disk, it has some graphics components, I don't use graphics much, but then there is a display, there is storage available, it has one terabyte of storage available and things of that nature. So that's what you're looking at from a real machine perspective. Now, given that real machine, how do you virtualize it? That's the concept that I want to talk about. Essentially like I discussed in this sheet, divide that machine into smaller parts until it still remains usable. As long as it is still usable, you can keep dividing into smaller chunks. So if you look at examples of a rack mount server example, you will see that these kind of machines that people sell, like Cisco sells or Dell sells or other people sell, these machines that you're looking at, for example, let's see, Cisco rack is a good one. If you find a Cisco rack, this is an example rack. I hope it's not going to play a video for me. Let's go see. So these machines that you see, there's these guys walking and the machines are sitting on both sides. These are basically computer systems, really large computers, and they will have some specification like I showed you here. This machine is not so large, but it is reasonably big for me to run a couple of machines, maybe a ton, 10 or so machines, I can just comfortably run on top of this computer. What essentially it means is that I'm dividing that big 16 GB RAM into smaller chunks. I'm dividing it into four parts. So this each of them is four, four, four, four, and then this is two and this is one and this is 0.5 and like that. I can keep going dividing like this until I get to a point where I still find it usable. So practically, having this machine size is probably useful for about 256 K, but then below that, it is not really useful of a machine. I can have so many of machine sizes that I can generate out of my dividing my real computer. And so that's the technology that is enabling this thing is called virtualization. It emulates hardware, emulates hardware. It means that you have every single thing that you would otherwise imagine in a real computer, but not physically present. It is fake, fake as in emulated, but it works like real, but it doesn't actually exist. Let us go check the settings of this machine. You will find that in there, you have a system with a base memory of 1536. I can reduce the memory to 256 MB if I like, or go it higher, make it 8192 MB to make it really big. And that's the kind of scope that I'm living with. In fact, this green arrow in the green colored bar indicates that I can go all the way up to this point and give it about 12 gigs of RAM to the virtual machine. And I can still be okay because the underlying machine has a totality of 16 MB of RAM. It can survive with this much. If you are going beyond it, you are really stretching it. And if you go really high, I mean, it will complain. So you need to leave these resources, some of it, of the underlying operating system. This example shows Macintosh for it to run correctly. And similarly, apart from this memory resource that you are dividing, the processor also can be divided. In this machine, there are eight virtual CPUs, so we can say that I want to give all of them to the virtual machine. That's a bad thing because what will Macintosh run on? Macintosh needs some. So the machine is suggesting you can give up to four virtual CPUs to the virtual machine and leave the rest for the Macintosh to work correctly, so 50-50 divided. You can also put execution caps if you like by adjusting this pointer. And some acceleration methods, so in terms of technology use, you can see which technology is getting used. The virtual machine, virtual box, supports a bunch of different technologies for underlying virtualization. And here are some examples. I'll just go with the default for this case. However, in the context of cloud, the cloud companies will choose their own methods of virtualization. For example, this KVM is used in your OpenStack. You can see that there is another virtualization method called ZenXEN that Amazon uses. This Hyper-V virtualization method Microsoft uses. And there is another virtualization method called VMware, which is what VMware uses. And so there are more different types of virtualization methods. And we are, in this example, just using the default, which is what VirtualBox likes to use. Similarly, if you go to the display section, you can assign video memory out of your memory available to run the display if you need a display. But most cases of cloud computers don't need a display. In our example, we do because we are actually interfacing the machine directly. That's why we have given it some video RAM. You can have storage allocated. And so you can say that, you know what? My virtual disk, which is this Cloud Genius Workstation disk, is actually 127 GB of virtual size. But the real size is only 6 GB expanded. And the compacted size, if you look at, is very small. If you go look at the workstation itself, the OVA file is about 2.21 GB. So this OVA is 2.21, it's expanded to this virtual size of 6.87, sorry, this actual size of 6.87. And the virtual size is 127 GB. Let us understand what this actual and virtual and real things are. On the compressed side, when the OVA is compressed and given to you, you have this size of 2.21 GB. This is the real file that you download or synchronize through GetSync. When you expand this, import into virtual machine, you are seeing, look at, 6 point something GB of real size of the machine. However, the virtual size is defined to be 127 GB. This is not really allocated. It is actually allocated up to this limit. In an example that I would like to now show, which is going to be a different machine, I'm going to say new. And here I will say that it is going to be a new machine. It is going to run, say, Linux version Ubuntu 64, continue ahead. I can assign some memory to it. So let's say I'm going to assign some 2048 of RAM. And I will create a virtual hard disk right now. I'll say yes, please. And I'll create the virtual disk, virtual box disk image type VDI. I can decide to select other disk formats, different types of hard disk formats that I'm going to emulate. Again, there is no real hard disk, but it is just using a file and treating it as a disk. And that file is in these formats. I'm just going with the default format. VHD is the Microsoft format, VMDK is the VMware format. This is the Parallels hypervisor format, HDD, and a bunch of other technologies. So we are going to just use this one for virtual box for now and continue. And it says dynamically allocated. That means that I don't need to allocate the entire hard disk right now. So if I continue, I can here, in theory, select two terabytes. So I can just move the pointer all the way up to two terabytes. And I would like to point out one thing, which is the real computer doesn't even have two terabytes. So what is going on? Can anybody tell me? So the actual storage is only one terabyte, but this virtualization machine is letting me create a two terabyte hard disk. What is going on? It is letting you decide what is the maximum size your hard disk. You want it to expand to that level only when the needed disk reaches. When the need comes, it will expand up to that size, up to that real size. But I can technically create a two terabyte disk on top of a one terabyte real storage. And that's craziness. But that is what virtualization is. So I created a two terabyte hard disk. Are you kidding? No, I'm not. The real size is very small. This new.vdi is a file that actually is really small. So let's go see its size. We'll go and open the machine in Finder. And the new VDI file size is 9.4 megabytes, really small. But it can be expanded up to two terabytes. And if you actually go to that limit, we'll see that this machine will complain because the underlying hard disk doesn't even exist up to that size. It is actually a small hard disk, one terabyte. Having understood these aspects, let's finish through the settings section a little bit. So storage, we saw that there is this new VDI, two terabytes. You can also expand and include some sound card. You don't need a sound device in a virtual machine. I can say no. But you can play sound if you select it. Similarly, in the network section, there are these adapters available that you can enable. I enabled only one adapter. And these are not enabled. You can add more network cards if you like. And then this other question relates to how you intend to connect these adapters to the world outside. And so in a virtual machine scenario, you are looking at something like this. I'll show that to you. I'll draw it for you. So in a VM, this virtual box VM that we are looking at, there is a router inside the software. And the whole virtual machine configuration can go behind that router. What does that mean? It means something like this. That you have your office, which might have a connection from outside, from Comcast. And then you have a router. And then your router will give private IP addresses to your laptop and your desktop. And we are on this laptop right now. In that laptop, there is this virtual box software, which happens to run its own router. And then inside virtual box, you can have multiple virtual machines. And here, each of these machines can have up to four network cards. And you decide how you want to connect these network cards, whether you want to connect this card number one directly to the router, and then goes to the laptop. Then laptop in turns goes to this router, which is a real router. This is a soft router. It's a virtual router, a virtual box provides. And then through this router, you can go out to the internet. You have a method of choice of selecting how you connect this adapter, whether you want to connect to the router and then go, or you don't even want to connect anything. So you just say, no, I don't want to connect, or you can decide that I want to connect it directly to the laptop only and no connection to the world outside. That is also possible. You can also decide that, you know what, not to the laptop only, but instead, I want to connect it through and through the laptop to the router directly, this router. You can also do that. And that means that your connections will be established like this. And so all of that is possible. That's what this interface lets you do. So we have this new machine, this new one. And in the settings section, you'll see the network connectivity. There are four adapters, one, two, three, four, only one is being used right now. And now the connectivity choices are right there. Let's go see each one of them. You can have the not attached at all, meaning you can have the adapter enabled it, but not connected. So it is not connected. The VM cannot go do anything. It just runs and do nothing. It cannot go to the network at all because it is not connected, not attached. That is that example. The second example is a NAT, meaning it is behind the router, this guy. The third example is a NAT network. And here in this example, you're creating a little network of virtual machines that all these VMs that you're looking at are actually VM 1 and 2 and VM 3. They can all participate in a local network behind this software NAT. NAT is network address translation. NAT also means router for practical purposes. There are specific things that are included in the router that is not just NAT, but other things also. And we'll go into details of that later. But for our purposes in this example, we are using the NAT part of the router. That's why it is generally referred to as network address translation. And that's what you're looking at is creating a little network of this VM, this VM, and this VM while collectively participating in a NAT network. NAT network is basically a network created by this soft router inside the VBOX. And that is this example. Then we scroll down to see bridge adapter. And this bridge adapter is basically connecting the VM directly to the router via bridging it straight to the router. So this means it needs a bridge to connect to. This bridge is going through the Apple Airport connector that I have here, which is this one is running right now, which is how we are running the screen sharing. And that is how you can connect this to the airport router in the office provided you bridge it. Other choices include an internal network, meaning within the office, you can have this machine participate as a first-class citizen, this VM, and become a first-class citizen in the internal network if you like. It is again participating in the router that is in your office, the physical router. Then you have host-only adapter, which is connecting just to this Macintosh and nothing else. And it has another mode called generic driver, and you can specify details about it in the advanced section and keep going. So if you want to customize something specific, you could do that. This network capabilities, which I think are fairly elaborate, although most people don't pay attention to that area, because most dominant usage is just behind and that's the dominant usage. That's a good one to keep, unless you want to change it, which is fine. You can enable serial ports and USB ports if you like, and these functions just like real ones, except you have to use the underlying physical machine port and map it to the virtual machine. You can also do folder sharing and change the user interface if you like. That's the concept that you saw in this example where I'm using the VirtualBox virtualization software. And by the way, just like VirtualBox, there is VMware, and there is Parallels, and there is Hyper-V. And then there is this, Windows has this thing, what is it called, PC mode, something like that. XP mode. I think they used to have XP mode. I think they got rid of it probably. So you can get the Windows XP, virtual PC, that's the thing. It's an old technology. I don't think this is how they name it these days. You could run Windows XP on top of Windows 7, and that was a VM. These days, I think they're just using Hyper-V services and provide a Hyper-V interface. And that's how you can run Bash on Ubuntu on Windows. So this is a new software, still in beta, from Microsoft, available on Windows 10. You can get Ubuntu running and access to Bash from a Windows box. It is very nice. It works beautifully. You can get full-fledged Linux command line, just like this, I mean, not like this, like the cloud genius workstation, just like that, on Ubuntu, running on a Windows machine, a Windows 10 machine, in a Hypervisor. That's what you will see. It is not a full-fledged Ubuntu implementation. It is just the Bash portion of it. So you cannot do certain things that are possible otherwise. It's not a full-fledged VM, as I understand. I have not played with it too much. It is a recent beta announcement. I have installed it. It works. I have not had the chance to run through the entirety of its usage, but that's another scenario. The bottom line is, what you just saw me discuss was the concept of virtualization as it applies to desktop usage experience. The same concept is kind of sort of similar to a server virtualization scenario, where the differences include no need of a GUI, you don't need a graphical user interface, and all you really need is the way to connect in some CPU, some network, some storage. That's pretty much it. Who needs a GUI because we don't need to spend money or time or effort or energy? So cut the GUI elements from this desktop virtualization example that I just showed you and don't use VirtualBox because that technology is not useful in the cloud. Instead, cloud technologies for Hypervisor are these ones, like Xen is a good one. Xen Project, which is an open-source project, it is what Amazon uses, I understand. Under the hood, what you're looking at is this technology, EC2, is Xen virtualization, the Xen Hypervisor. Similarly, you have this Hypervisor called VMware, which is used by EMC and its sister companies. You have this technology called Hyper-V Hypervisor, which is used by Microsoft Azure and other solutions that use Windows technologies, so this is what an Hypervisor essentially does. It gives you a fake computer, and this technology of virtualization itself is not new. It is, I think, 50 years old, roughly. So when in the IBM mainframe days, this is like old time, 40, 50 years ago, this machine still supports this concept of what is known as a L-P-A-R, logical partition. You take a partition of the IBM mainframe and basically divide that big box, the IBM mainframe, into smaller pieces until they are still usable, and each one of these units is known as a logical partition, or an L-P-A-R. That's when virtualization concepts started about 50 years ago. It has taken shape of PC desktop virtualization, taken shape of server virtualization, commodity hardware has kicked in over the last several tens of years, maybe 30 years, and this mainframe is still there, it still works. Many companies use it today, however, mainstream virtualization has been enabled through these technologies that I mentioned. As a consequence, what you see is this product, taken shape of this product, called EC2, which is what you see as one of the most popular clouds out there. You will also see this example of Hyper-V, so I'm going to log in there to show you if you don't have an access. Let me just go there. I think it is called manage.windowsazure.com. Let me log in here, and log out of this guy. You will see similar things in this scenario where you will have... What I'm doing is generating tokens. I think some of you know this and do this already, but you should do that. You should absolutely do that. It's a hassle, but you should totally do that, is to enable two-factor authentication for everything that you do. Here is virtual machines technology from Microsoft, and you can pretty much do the same thing, create a virtual machine. In there, you can select to create quickly from gallery, or one of these two choices you can choose. In here, you can select to say that I want to give it a name. I will call it Cloud Genius, hope that name is available. I will select my server and the size, whatever size I like, a bunch of choices available, and pretty much these are the sizes, the breakdowns of a real big box that Microsoft provides in smallest one is this one. You can select your own password if you like. I'm going to select a password, and from the same password, and select the region I want to put it in, and then click to create a virtual machine. This password is too simple, apparently, it doesn't like it. Now I can just click to create this machine in Western US, and I have a login name and password, I can just log in. That gives me a virtual machine, and this is going to run the Microsoft Hypervisor virtualization technology. It's the same thing like we just did here in the Amazon example, EC2 server, and you have these two machines running there. You can effectively destroy them like this, and it goes away. The concept essentially is you take a bigger machine divided into smaller parts to create a virtual small portion of a machine. That is how you can dynamically do this. You can do this programmatically. That's the key part. The most important thing here regarding virtualization is you can generate these virtual machines on the fly by users clicking, deleting, launching, or to API calls. That's what makes this really possible is that you are creating machines on the fly, on demand, and self-service mode. That's what translates to the original discussion we had about the definition of cloud computing is to be able to create these VMs on the fly as we need to go. That's what you're looking at in this example or other cloud examples where people are using cloud services, they're not getting physical boxes, however, you are getting virtual machines. That's what virtualization is. That's what I wanted to make sure that I cover, and I think I did. Any questions on this idea of virtualization before we begin proceeding in the other direction or take a short break, either way, whatever you prefer. Can I take a quick coffee break? Yeah, break is good. Break is good. Right now it is 9.29. Let's resume at 9.40. That's a short break, and coffee would be a good thing. We'll continue that. We'll build our understanding in depth, in understanding infrastructure as a service, in further detail, understanding creation of a virtual private cloud, nuts and bolts by doing it manually, and later on we'll automate the whole thing at some point. Taking a break, I am disconnecting myself. I have this new method of hooking myself up. I'm going to make some noise right now. I'm sorry for that. But I need to remove this cable, which means my sound will make noise, might make some noise. Sorry about that. I will come back in just a minute. I will walk away. I'll leave the mics and everything open. The camera is open. Mic is also open. And I can hear you, and you can possibly not see me, because I'm going to actually walk away. Give me a second. I'll be back. I'm going to walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to actually walk away, and you can possibly not see me, because I'm going to share this with the copy Slack chat, and I'm going to paste this link directly to your Slack chat. This is a different link. It is not expired, it just generated, and that should let you in. You see that now? Yeah, I see it, good, let me try it, okay, yeah, I see it, good, let me try it, okay, yeah, so you should now see it synchronizing, and that should give you continued access to the video recordings that we do. Perfect. Cool, I think that resolves that, because I noticed that you were missing in the list who was downloading it, so I thought I should just contact you and tell you, so, okay, that's good. Great, thank you. Yeah, so let's go, begin, let's continue, so what we just briefly covered is this idea of virtualization, which I want to expand, because these things actually become a variety of infrastructure services, right, so the idea essentially that we discussed was around, how do you take a compute resource, a big one, and then split into smaller chunks, so you get a T2 micro, T2 small, T2 nano, small, large, medium, like that. You can divide however big or small compute horsepower that you want depending on an example, and here are some other examples, like you can get A0, A1, A2, A3, A4, any of the machine sizes that Azure provides, you can basically get D1, D2, D3, like that, so F1 and G1 and all these, so they have a meaning to it, let's understand that meaning as to what these guys mean when they want to divide compute horsepower, so this compute horsepower, how it gets divided in terms of consumption model of compute capabilities from the underlying infrastructure, you can get the appropriate size of the machine, and that's this element that we are talking about is how do you break that into virtual machines, also known as VMs, so you break that into VMs and then use and consume them and relinquish them, when you're done with it, you release them, so this idea that we are looking at, let's go look at some of the example sizes, so you saw these things in both Amazon as well as Azure, these sizes are available, but let us understand that they are a little bit different in its flavor of the layout, there are actually too many variations to the family and the type and the CPU, the memory, the storage, all of them are differently optimized, for example, let's go see T2 micro, so I'm going to search for it and look at all the instance types available from this company Amazon, and so let's go look at a quick review of what these sizes actually mean, so T2 in Amazon is general purpose usage scenarios, so they give you basic CPU, basic storage, basic RAM, memory, and that's the classification of this T2 type, prior to T2 there used to be T1 micro and T1 things, but these days it's a next generation T2, which is you get decent burst performance, for a short time you get a very good performance, over time if you keep hitting it, the performance will not be as good, and that's why if you need transient burst power, you would use this T2 model, the other examples are mainstream general purpose machines and they are M4 instances, prior to them there was M3, M2, M1, and so historically this technology is the mainstream usage of a computer, that you get some decent amount of memory, decent amount of CPU horsepower, and some connected storage, and some bandwidth. If you keep looking at this technology type, you will find that there are some other types of machines available, known as C4 machines, C4, and these are optimized for computation, so they are heavy on CPU aspect, so if you see C4 large and compare that with an M4 large, you have two CPUs and 8 GB memory in M4 large, in a C4 large you will find two CPUs and a little bit of less memory, so it is optimized around not memory, but computation, so two CPUs but less memory, four CPUs and some memory, even bigger CPUs and up to 36 CPUs, but not too much memory, so certain application types require that characteristic where a number crunching happens, you need lots of CPU, not much memory, but not too much memory, then you use this one, this C4 model, C4 compute optimized models, so you have C3 also, C3 variations for older technology, and then these machines are X1, they are memory optimized, meaning you get lots, a ton of memory, how much memory is that, 1952 GB memory, this crazy memory, but that's the memory you get on a X1 32 extra large machine, and that is optimized for high memory usage. Let's see some other examples, somewhat of a less than this monstrosity of memory is this R3, they're also memory optimized machines, you get decent amount of memory to begin with, so you're looking at a decent amount of memory on a smaller size of CPU and that's the use case. Some specific computations that are requiring GPU horsepower, graphical processing unit, the special kind of math that I mentioned the last time, these machines come with a dedicated GPU associated, they're not really having a display adapter to connect to see any display as such, but they are doing the mathematical processing needed on the graphical processing units to help you run applications that require that kind of a math processing capability. These machines, G2, two extra large, and G2, eight extra large, are specifically useful for machine learning, 3D application streaming, video encoding, other GPU compute workloads, that's what they're defining the use case scenarios for, and they have graphics processing capability, not that it has a display, but just it has the ability to process, do the underlying number of math, the technology used in GPUs is slightly different than a regular CPU, and that's what these machines come with a NVIDIA GPU included, with four GB of video memory dedicated to that. Then there are IO optimized machines, so these are I2 extra large, where the machine's hard disk performance is optimized already, so you're looking at high-performing SSDs connected, and these are good for usage on databases, so things like Mongo, Cassandra, scale-out databases, data warehousing, Hadoop clusters, where you need high-intensity, high-performing IO at a reasonable cost, you should receive, you should get an I2 machine type, where the focus is on IO. Then the next one types is data D2, dense storage, so you get a focus on storage, hard disk storage, HDD, so you get a lot of storage because it's hard disk, not SSD, and those machines, just big fat machines from a storage perspective, locally available, three hard disks, two terabytes each, like that, or 24 hard disks, two terabytes each, if you have a lot of storage that you need, locally available for your data storage, for example, your Hadoop deployments, for storing your HDFS file system, you need a lot of local storage, you get these machines types for that use case, which is a valid use case, so this is the case where a magnetic hard disk is probably useful, this is probably the only case where you really want a really gigantic hard disk collection, a pool of hard disks allocated to you, and it is possible to get that kind of a storage available locally attached, and that's the scenario where you get dense storage instances. That's this description, a quick overview, very similar to this is the Microsoft example, where you have this definition of, let me make it big. You have this A0, A1, A2, they have a definition of, I think, let's go to Microsoft's front page and you will find it there, Azure machine sizes, Azure machine sizes, we'll describe that as to what these things are. In this paper that this person has written, they're defining these types of several series, A series, D series, Ds series, F series, G series, Gs series, and the considerations are defined here. They are conceptually similar to what you just saw in the Amazon case, where the idea is to focus on compute-intensive workloads, or high-performing CPU, or more memory or more local SSD per core examples. Depending on what your performance considerations are for a given underlying application, you should then select appropriate machine type. By design, the machines types in Amazon do not directly, there's no one-on-one mapping. Machines from Amazon, machines from Azure, machines from Google will differ slightly, and sometimes substantially, and the pricing will be different, and it is just to make things different and somewhat difficult for people to directly do a comparison between competition. They do it deliberately, apparently. I don't know exactly, but I think that's what may be the reason why the machine sizing, the machine pricing is a little bit cryptic among multiple cloud providers. They don't standardize towards one common way of doing things. That might sound confusing, but at the end of the day, when you're deciding a machine type and a machine size, you need to really think about what is the underlying application? What do you want to use it for, depending on those characteristics that I mentioned the last time, which are, do you need a lot of CPU, a lot of GPU, a lot of RAM, a lot of hard disk? Maybe you need slow or fat disk. Slow and fat is the HDD, hard disk. Sometimes you need fast, and that may mean small, although that is not necessarily true, but fast and small are SSDs, typically. Then you may want to have, from a storage perspective, this is the machine that gets these local disconnected directly to the machine, like your C drive in Windows, for example. Then you can also get network capability, how fast and how wide the network is. From a wide width perspective, is it broadband? Is it really wide bandwidth available or not? That capability, whether your performance of the latency of the network is very low latency or big latency, is it fast or slow from a responsiveness perspective? Those characteristics together, you have to really define which of these characteristics you want to get in what fashion. In one such cloud, if I remember, Rackspace does a great job in doing what I'm describing right now. So I'm going to go there and show you what Rackspace visualization is for this type of a selection. So let me log in there. And we will do the same thing, which is to create, not this, no, not this, but this, this login, and I'm going to get a text message right now. So I'll go and look at my text. I will receive a text message from Rackspace to validate me. That's what I'm going to grab right now. And there is it. So it expires in 10 minutes. That's good enough. Paste it, verify. And I'm going to log into Rackspace. Now here, this is another cloud. And I am going to create a server, a cloud server to be specific. And that user interface that they have when I go and create a server is kind of nice in that it lets me choose the operating system that I want. So let's say I want Ubuntu 14.16.04. That's great. And then I want the machine type. So it is letting me choose this machine flavor kind of graphically. So look at this part of the picture here, this specific part, which is basically this diagram that shows me how much of a resource type that I want. So this blue indicator is focused more towards disk IO and not so much towards VCPU or disk. So let us adjust that a little bit. I can say that I want to get a machine that is focused more on compute. And then I can maybe increase the amount of horsepower I have. As you can see that the blue diagram here grows in its height as I increase the number of virtual CPUs that I am assigning. The blue arrow, blue segment goes up in this CPU direction if I adjust. And that UI is actually one of the good ones. So from an understanding perspective of what am I focusing on when I'm getting a machine. Similarly, if I'm adjusting my memory and I will want to get more RAM versus less RAM, I can see that my numbers, my pricing information, as well as the pictorial representation changes as I adjust the amount of memory that I want to get. I can understand the pricing as well as the relative mix of these characteristics that I want to be able to get from the underlying infrastructure. Similarly, IO. I can adjust this same way and I can see that the whole machine performance seems to be increasing because that's what they are doing to make sure that the entire machine is able to perform from an IO per second, IO performance perspective. And if you choose the standardized sizes, you can of course see these things. The sizing is available right there as you tweak. The reason I'm showing these to you is basically this diagram, which tells me how I am allocating my resources. And I tried to draw the diagram by hand, but I did not succeed too much. I'm not an artist. However, this is a good one to represent systematically as to what resources are you getting from the underlying machine that you want to get. It gives you a good perspective as well as get you the estimation of a price. How much will you pay for it when you are going to that particular machine and select to run it? And so in this example, we can choose the machine to be in Chicago or Dallas or Sydney or any of these locations in Hong Kong and Northern Virginia and things like that. And choose the appropriate machine sizing, choose everything else associated and then create it. So that's the reason I logged into Rackspace just to show you what this thing is. Now having done this, let us understand some of the other capabilities of using infrastructure as a service. So from a compute perspective, I think you get this idea as to how big or how small of a machine that you will get at what price. So what size and at what price are you going to receive that unit, that computational unit? Now sometimes or actually in many cases, you want to deal with this particular machine as you remember and recall in our example when we ran this jump box and then the internal server, internal application server, I did not even bother giving that internal machine a name and I just used its internal IP address and we just dealt with it like that without even naming it. The reason we do that is a philosophical one and it is like this. So it's another example of comparison between the classic data center where many of us have used in the past and this new model of using cloud machines. So I know some of you in the group are from India or at least have Indian origin, so pardon me for using this example that I want to give which has something to do with cows. So just excuse me for that, there's no religion involved here. So the reason I want to bring up this discussion on cows versus pets, like you have a pet or you might have some cattle in your ranch, if you are a Texas cowboy for example. So a machine in the cloud is like cattle and a machine in your data center is like a puppy. That's my analogy. And this analogy works for most except as long as you put the Hinduism aside, then it works. So just keep your religion aside for now and just understand that you are a cowboy in Texas and you have a puppy. And so if the puppy falls sick, what does it take to take care of the puppy? You have the whole family of three, go to the vet and spend time and hours and days to make sure that the puppy is back doing well. And that's the scenario of a data center machine. It is like a puppy. You mean you deal with it, you manage it, you make sure that it works correctly and it is your puppy. I mean, you take care of it. That's the idea. Puppy is like a machine, your machine that you pet in your data center, that you make sure that it runs all the time and all the good things happen to it because that's how it works. That's the reason. That's the reason you have a name associated with the puppy. You give it a name like Bingo is my dog's name. And that's great. Now just the other side, a machine in the cloud is like cattle. So imagine now you are a cowboy in Texas and you are on a horse and you're shepherding 10,000 cows, like 10,000 of them. And you want to take them to the ranch and they're just going and you're one, you're just one alone. You have a horse with you. You have a gun, of course, and a bottle of whiskey on the other hand. Now with this scenario, imagine one cow falls sick. What do you think you will do? And you are a cowboy in Texas, right? No Hinduism involved. So what do you think you will do? Tell me. Anybody? Anybody? I think terminate it on the spot, right? Exactly. You will take your gun and shoot the cow and move on with the remainder of 9,999 cows and you go to the ranch. That's what you will do as a cowboy, you know, because you don't care. Cows are disposable to them, right? And so that's exactly how cloud machines should be disposable. You don't pet them. You don't even give it a name. You just deal with them as an IP address. You stamp them, brand them, deal with them. If they fall sick, you shoot them and move on because it is so easy to create a machine is that you just click on it and replicate. And you want to shoot a cow? You shoot it. Well, you have to do an offline postmortem though. Yeah, no, don't worry. Just move on. I mean, who has time? If you are managing like 10,000 servers and one of them falls sick, you just shoot it and create a new one and move on because you're not creating these machines by hand. You're creating them through automation. So you don't even have to do postmortem because all you do is the postmortem in your specification. You specify that I would like my design to be like this. And you never actually create the machine by hand because it is done through automation. So the only postmortem you will do is to the automation itself or check whether your design is bad or not, but you will never ever go to the machine itself to see how the cow is doing because it is not worthy of your time. You have created this cow through automation and you specified this design and that is how it got created. So if something is wrong with the cow, your design is broken or your automation software is broken. One of those. But the cow is a cow. It's dead. Move on, kill it and move on and create a new one that's easier. It's more efficient. And I think it is the right approach to deal with cows because they are cows. I mean, come on, and you are a cowboy on top, you're not a Hindu Brahmin. So don't worry about that Hinduism part. But the idea is data center machines are puppies, but cloud machines are cattle. They are not puppies. So don't get associated emotionally with a cow. That's what I'm saying. That's what I'm saying. So no emotional associations and no postmortems about the cow. You do postmortem here and here. That's okay. But if the cow gets... Mukesh, in this scenario, I'm guessing that the customers may have slight outage or something. Probably not. Maybe, maybe there is a case of outage. If the cow goes down, there is outage. But here is the picture that I want to show, which is again the same picture I keep drawing all the time, which is the customers deal with this called the load balancer. And the cows are here. So if one of the cows goes sick, I mean, whatever, you just find that this cow is sick, you just kill it. And this cow is sick, you kill it. That's how it works. Customers will automatically get load balanced across this collection of servers. For the big customers, what about small? So small customers, meaning customer who? Customer is... Meaning like us, we go and create one or two small virtual machines on that cloud. And then something goes wrong in that big machine where we created our virtual machine. In those scenarios, possibly I may have some hiccup, right? Yes, you will have hiccup. If you are a small entity, small business, and then you deal with this scenario where you have not so many, but maybe only a few. So let's go kill them. So yeah, go, go, go, go, go. Let's say you have only two boxes, right? One of them behaves badly. So let's say if this box behaves bad, then you still don't have downtime because you have the other alternative, right? That still works for your customers. However, if both of them go down, then you have a problem. And yes, you will have downtime. Yes, you will. No doubt about it. There will be downtime if you have both boxes bad, your design is broken simply, as simple as that. So you should go look at your design, which is this, fix this design and recreate your boxes again and see if it works. And if it works, that's great. If not, go back to the design and make sure that it works by automating and always create these machines through automation and run them through automation. So you don't have to worry about who makes mistake because the only culprit is your design. And so if your design is bad, the implementation will be bad. And you just erase your implementation and check your design and run it again. That's the way to operate in the context of cloud. You typically do not. You can go do post-mortem, but I think it's a waste of time. It is better to just focus on your design itself and make sure that the design is accurate. In that case, the implementation will be accurate because it's by definition. So that's the model I would suggest that we stay focused on dealing with pets versus cattle. In fact, this topic, this idea of pets versus cattle is a well-known discussion in Hacker News. And you can see this multiple places. You can find it. There is this Hacker News discussion. You can read more about it. And that's the summary of what we just discussed is that cloud machines are cattle. So don't worry about them. Just move on. Focus on your design as to what you wanted to automate and fix that because that's where the bug is. If you have a bug, it is probably right here. So fix it and then create it again. You will be OK. Well, in this case, we can safely assume from a data center perspective that it's mature enough and they don't really care whether your machine died or how it died. It's just a given. Can you elaborate on what your question was? I did not quite understand. Data center mature enough as in what? So from a data center perspective, right? So say for some reason, virtual machine 1001 died. Do they see that happening and try to see what happened to possibly find a failure on their side or do they just don't care at this point? When you say data center, do you mean from the cloud side? Yeah. Yeah. You just assume that the cloud provider is doing OK most of the time. Most of the time. That's an assumption that the problem is probably with your specification. Because that is fairly thoroughly tested by Amazon and by Google and Microsoft. They're doing it. They're doing a great job. I'm not saying that they're good all the time. That's what I meant in terms of maturity. Yes. Yeah. They will be OK most of the time. Most of the time, they will actually voluntarily declare that they have a problem and I will show you how they do it. And so they have this health dashboard, the service health. Every provider of service will have some problem to deal with, like right now, we are looking at exchange online services degraded. And so they're acknowledging the fact that they have a problem to deal with. Similarly, Office 365 is degraded and this is Microsoft admitting their problem. Skype for Business, again, degraded. Every other service seems to be OK. And not just Microsoft, everybody else will go and show you that status. So AWS has a health dashboard. You can see what services are operating normally or not. And they will voluntarily disclose this detail to all of us as we are their customers. So they will tell you that services seems to be running OK in most of the places. And I think right now it seems like all the services in North America are running OK. And the time right now is current status. So you can see how it's going on in South America. Seems to be OK. And history of status. So you can see when things go bad, they maintain that history of how and when things were bad. You can see historical data about history of status by going back in time and identifying a place where things went bad. And so here is a bad scenario happened. So something bad happened right there. So EC2 in Virginia. So that's where you can tell that something bad happened and they are correcting it. They say that we found an issue. Oops. There. So click on it. It says we found an error, elevated error levels for new launches. That's a sign of a bad problem. But they are being honest and transparent in admitting that they are having a problem. See that? And that's my reading is most of the time, most of the time, these providers are being very transparent in telling you what is going on. And both companies that you just saw, Amazon as well as Microsoft, will tell you that something is wrong right now. So blame us. And yes, we'll blame them because they are telling us that they are bad at that moment. And then they took about less than an hour, more than an hour or something and then fixed it. And so that issue of creating a new machine launch in US East one region was resolved on about one and a half hours on that particular day, which happened to be August 30th. And you know this because they told you and you have historical record of this. They maintain it and this is not all nice and beautiful 100% of the time things go bad even with them. But most of the time they are okay. It is typically we make mistakes. They have let's put it that way. Companies that are running professional cloud services have a lot more capability than smaller or other companies that don't have that core competence. That's the way to put it. I'm not saying that these guys are gods. Not saying that they also make mistakes. They admit that nobody's God. But the fact that they know they've been doing it, that is their core competence to run cloud services is something to be acknowledged. That's what they do. And Boeing Company is known to be an expert in making airplanes. And that's what they do. I don't question Boeing whether the plane will fly or not. I just trust it, right? But you know what, as a fact of life, even that company has occasionally problems and you see something going wrong here and there once in a while. Hope not because we want to fly safely and all that. But that is their core competence is to be able to build planes that fly absolutely problem free all the time globally. That's a big problem to solve and they have solved it. Same thing with Microsoft and Amazon and other big companies that are doing it because that's what their core competence is. So I really don't have the guts to question, although I will, if I really find evidence, I will question big companies that, you know what, you are actually wrong. You are broken, not me, because I can prove it to them, but I need evidence. So I will open a support case and show them that my design is correct, that your implementation is bad or your infrastructure is bad or you have the problem. I will have to prove it with preponderance of evidence that I have to actually produce. And that requires due diligence on my part, which I will have to do if that's the case. So I think that answers your question. Typically, okay, most of the time. And they're being transparent. They're not cheating. No, nobody's cheating here. So is your image cloning part of that desired state management scenario? Say again, please. Image quality what? Is your image cloning? So if virtual machine 1001 goes down and you want to transfer that image to 1002, is that cloning part of the desired state config process? Okay. I think you bring up another valid point, which I think I should mention is this thing. You mentioned image quality. And you saying that, you know, can I take this image from 1001 and apply it to 1002? Yes, you can. It's very easy. You can just take a snapshot of one machine, apply it to another, totally possible. So there are tools like that built into the tool set here. Just click there and say, take a snapshot like this. And you can create a snapshot that you want in any machine. Just create a snapshot for any particular disk that you have with any machine and just keep a copy. The image gets created and you can use it. So you can do a snapshot like for this machine, you take a snapshot and then you apply that snapshot. It becomes an image at that point. Once you take a snapshot, you can use this image to start a new VM. If you like that technology exists, no problem, no problem, however, however, I recommend and this is not a don't do it, it's a recommendation. I recommend that you don't use this imaging method at all. It is totally possible instead, instead what I suggest is to never rather not never, but most of the time you don't create snapshots or images at all. Here use the images provided to you from sources such as Windows or Ubuntu, plain vanilla, just enough OS, just use that and build everything up on top according to your specification, just enough OS plus your thing and no other crap, literally the word crap. Don't get any crap because you don't know what crap this thing has and you're taking a snap and your image now has crap because you don't know where it came from, who mugged with it and who's tinkering with it. You don't know what setting might have changed. You don't want to even deal with that image because I think it is dirty, honestly. It is dirty because somebody touched it. I would rather go to the source like Microsoft for Windows, Ubuntu for Ubuntu, Red Hat for Red Hat or Suzy for Suzy or whatever that operating system that choice, go to the source, the fountain, the origin where the operating system is coming from and get just enough operating system and use that because that you know where it is coming from. These machines that you have in the cloud running, I don't know what it has been. If you have a large company and large group of people with at least more than one person touching it, ouch, you don't know what that other guy did. You have no idea and so I don't want to touch that machine at all. In fact, I would not even bother taking an image because to me that image is dirty. All I care about when I reconstruct another machine is to again look at my specification and play it on top of just enough operating system. I get the J E O S and play my story, my specification through automation and I get exactly what I want because I can go and inspect what the heck did that specification do and on the foundation I have just enough operating system, no ugliness, no ugliness. In fact, I would encourage that your people in your team don't mess around connecting to the machines directly at all. Don't connect SSH, although we know we can, but don't let the automation deal with it. Let your specification make changes to the design and play those on the machines. Don't touch them manually. The moment you touch a machine through SSH creates what I call entropy. If you read this term, let's see, let's go see entropy. It's a mechanical engineering thermodynamics term. This basically means a lack of order or lack of predictability or gradual decline into disorder. That's what entropy means. Over time, as you have this virtual machine 1001 and you have two guys, let's say A and B on your team, and this is you, three of you, and you are connecting to this machine occasionally. And you're disciplined and all that good stuff. But again, at the end of the day, you're people, right? So you might make mistakes. And as a consequence over time, this leads to entropy and disorder. You will at some point move something around to test something, play with it. Don't, don't, don't work with it. You want to play with it, create a separate VM for Mr. B and have him play here, do whatever you like. I don't care. But this is production. Don't go there. No messing, no messing around at all. You want to play, go play here. You want to play Mr. A, you play here. Your own private VM, go mess with it. Do whatever you like. Delete, destroy them. I don't care because I don't really bother. All I worry about is what is my design? So I will work on the design and this guy A, this guy B, this guy, me, I will use the design that we call share in a Git repository and play in our own little sandbox. But nobody is mucking around with the production VMs. Nobody's touching it. The only one actually touches is not a person, but it automation. It takes the design and implements it. If you have something goes wrong here, you delete it and create a new one. You want to experiment. You take a new VM, use the J E O S to instantiate this, apply your design and go to town and do whatever you like. I don't care because that's his private machine. You just do party. I don't care about anything because it is not production. I care about the production setup and this should not be touched. Not even by you or A or B who might have privilege and access to get to that VM if they want to. But no, you're not going there. Even if you are the super duper admin, you're still not going there. Only way to go is through automation. And that is the reason why I say no need to create an image because you don't need to now because you have this automation that builds the whole thing from scratch based on just enough operating system. Well that's also where you implement just enough access on your nodes. Yes. Yes. Very limited. Yes. That's why it is limited to only these guys, you and A and B, those short, small group of people who are trusted and are the super duper admins on your production infrastructure. And each one of them promises to the other people that they will not mess. That's a promise. And that's why they don't go there themselves. All they can do is through the automation and if something goes wrong, you check the automation as to what did happen and then you fix the automation, fix your design and run it again. Maybe run it here in the VM first locally and see how it works. Maybe do a staging area, testing area, figure it out. But not the production. Nobody goes there. Nobody. Not even you. There's no need to, no need to. That's where you kill terminal server and throw away the mouse. Yes. So let's keep discussing the other aspects. The other aspect that I want to discuss now is this idea, which is probably known to you, but I want to call it out very, very clearly. And that idea is a disposability of the machines. So the idea of disposability, meaning you can actually implement this scenario of having this machine. Let's see if we can put it on top. Bring to front and let's see if I can bring this to front. No. Bring to front. Yeah. Okay. Yeah. So this machine that I have, I would rather not store material information in that box. That's what I mean to say. And I'll describe this in a little bit better fashion. So here are these people that you know, these are customers, right? They come onto your site and they will want to visit your load balancer right here. This is your load balancer. And your site is a collection of a bunch of machines that all connects to a database on the backend. And this load balancer connects to these machines. Okay. To simplify, I will draw this picture a little bit differently, just to simplify. And that picture would be, ouch, I made a mistake. So I'll revert from it. And okay. So this box and this box, only two, only two, just to simplify. Now I will illustrate a point, the point being like this. I have a picture that I took. So this is me, by the way. I'm a customer, right? And here is my neighbor. So all these guys, let's get rid of them, too many of them. So here's my neighbor and his wife, and this is me. And we have, let's say Facebook is implemented here. So this is Facebook, the load balancer for Facebook. So I visit here, these guys go there, they all visit the same front end load balancer. And this is Facebook application running on these two machines. And there's a database on the back end, just for simplicity. And the reason I bring Facebook as an example is because what I want to be able to do is upload a picture of the party last night, right? So I take this picture, that I have this picture called P, this picture P, and I will upload it to the Facebook application, which in turn will associate me with this unit. And I might send my picture here. Yeah, so my picture P gets loaded up here on this box, and it's safe here. Now when this other guy comes in, my neighbor, and says, you know what, let me see the picture you took last night. And so the neighbor says, show me the picture. And the load balancer decides that, you know what, you, as a new customer, should go to that box. And so me as the existing first time visitor goes to this box. It's load balancing, right, LB. It's load balancing one here, then here, then here, then here, round robin load balancing. It's happening. So this customer number two, which is my neighbor, comes in and says, hey, where is the picture? I don't see it here. I mean, there's supposed to be a picture, it seems here, but where is it? It is not here. It is here. So how do you solve that problem? The neighbor will get surprised. You know, my picture, you said you uploaded it. I cannot find it because this guy sent the picture over to this machine and it is sitting here locally on the hard disk. This picture P. When this guy comes, he gets assigned to this machine. And where is the picture? That is a problem. You can solve this problem in multiple ways. And one of those is to synchronize the file system between these machines. As long as the underlying file system is synchronized, it is possible that this other guy will see the picture P when they log in because by that time, the file system is synchronized. Beautiful. That's one solution. But there are many other types of ways of handling the same thing is what I'm hinting at, which is even if you don't have a method of synchronizing file systems, by the way, Amazon does provide you with a method of synchronizing file systems. And that thing is known as what is it called? Elastic file system. And they're basically using another open source product under the hood, which is called Gluster, if I understand right. So the Gluster file system is a file system for your cloud. And it is available in open source storage for your cloud. And it synchronizes the file systems on machines like this, like this and this. And you have more, and it will do that just like that, open source, this one, Gluster. And in similarity to this, they have this product, very, very similar, which does the same thing. The problem with that is that you have to make sure that you're using it. If you just create a machine out there in EC2, it doesn't automatically use it. This EC2 launcher machine doesn't give you that capability of synchronization to begin with unless you ask for it. By default, it doesn't. So the other methods, which are a little bit simpler, is to not store, do not store picture here. Not store it here, but instead, store it somewhere else. Somewhere else. Now, where do you store it? And that storage is called storage as a service. So we'll make use of that, which means you have a picture available. So when a person uploads a picture, this guy, P, the picture reaches this machine. You do not store it here, but instead, store it here. This is a picture storage as a service. Storage as a service. And every cloud provider will give you this ability to store stuff as a service outside of the machines, not on the boxes, outside. And that scenario is what we are going to talk next. Do you know of these services already? Do you know what I'm talking about, storage as a service? Anybody who's used it? Google Drive. Google Drive is a good one. And other examples that are more developer-friendly and well-suited to the cloud. That's what I was asking for. So Google Drive is indeed Dropbox, Box.com, High Drive, S3. Somebody said that. Yes. That is more developer-friendly from what our usage patterns dictate. So Simple Storage Service is the service that Amazon provides known as S3. So let's go look at it. Simple Storage Service. And that is this service from Amazon. Very, very similar is a service from Microsoft called Azure Blob Storage. Very similar service, and here it is, Azure Blob Store. Similar service from Google, Google Storage. Not Google Drive, but Google Cloud Storage. Not this, not this, but this. So this is Cloud Storage, and this has a specifically different characteristic than Google Drive Storage. So it is slightly different. Let us understand that as to what it means and how it's used. So the primary usage pattern of these services that I generally collectively call them as... What did I use? The name... Yeah, I think I erased that somewhere. So the name, I prefer to use these services, I call them as Storage as a Service. You store stuff as a service, meaning they give you a storage file system, you just put in there and you forget that you have it there, and it is there, it is going to be there practically forever until you decide to delete it. And those examples like we saw as three Blob Store from Microsoft and Google Storage, Google Cloud Storage from Google. So those are three examples. They're basically the same thing conceptually, not actually, but conceptually. So let's see a quick usage. So we'll go to one of the clouds. Let's see this one, we sign into the console for simple storage. Service basically is the same console, and we land in this section here, which is Storage and Content Delivery section, this second segment, in which the very first one is S3, this one. And that's what I want to click, I will click on that and go in. It lets me into the interface for simple storage service. Now realize that there is no machine that we are getting, we are not just getting the use of a storage area. Some folder, think of it as a folder, that's the best way to think of. Think of this thing as a folder, which is not on your desktop, not on your laptop, not on your server, somewhere away, but it is there. Then you store stuff in that folder, and you extract content from that folder, you expose that content on the internet, or you decide to not expose it on the internet, or you may share it with only specific people. And so this access control methods are very well defined. What does that translate to? Is that, in fact, most of these cloud storage providers will introduce this concept of a bucket. And this bucket is where you will put object. So literally, this is a bucket, and you put the object into the bucket. That's it. That's how you use it. So you have a new picture, you put that in the bucket, that's how you store it. And when you want to see the picture, you just read the object from the bucket, and you get the picture. Now this bucket is managed completely different and away from these boxes or these instances or these droplets or these god knows whatever you have, but they are stored in a separate service, known as storage as a service. In the examples that we have, implementation examples, we have the three names that I said. Amazon has three, Blob Store from Azure, and Google Cloud Storage from Google, and many companies provide similar examples. But this has no bearing on this or this, it's independent. You get a picture from this guy, you'll get it here, you say, you know what, I am not going to store it here because these machines are cows. I may want to kill them at some point or eat them, sorry, because it's Hinduism. I'm not Hindu, but whatever, just saying sorry. But yeah, I might kill the cow and the picture goes, get lost. So I would rather not store the picture here, but instead move the picture away from here to here and put it in that bucket. And then that picture is safe because then my picture sits here. And so if this guy comes in and he gets assigned to deal with this machine, this machine knows where the picture is. It can always go to the bucket and grab the picture and show it to him and he's happy and all that good stuff happens. And that's the usage scenario. So you're basically not storing what is known as user state, a wide storing state on these boxes. Don't put the state here. Instead put it somewhere else, somewhere outside, like a database as a service, as a service. So some service provider is going to give you a place to store your database to put it there or some storage as a service, which is going to give you a bucket and you put that in the bucket. So it's the state of the user, state of the application, do not belong in the machines. Don't put it there. You will be happy because you don't have to worry about a lot of things that these service providers will deal with for you. They will handle it. So the core thing to remember is to make sure that you can somehow get this stateless if possible. That is difficult in very large scale applications like some of you deal with in Boeing, for example, very difficult. But the ultimate goal is to get to this point is to get the machines to be stateless. And then you can deal with them like they are cows, cattle. You go sick, I'll shoot you and move on because I can create another one. I don't care. My state is safe somewhere outside. Now how safe is that safe? Let's go see. So let's go see S3 durability. Let's understand what does that mean? How durable is that storage bucket? They say that they give me a bucket. What happens if the bucket gets lost? What's the durability? Let's see the claim. The claim here is 99.99% of availability. And let's say durability is even higher. So I think it's 11 nines. Let me just see. This is the 4-9 number for availability and 11-9 number for durability. Let's go check that durability. So here is the answer. Amazon's claiming that they are designed for 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11 nines of durability and 4 nines of availability. What does that mean? So it translates to this, that this bucket, as claimed by Amazon, is going to be available to me with this level of availability and guaranteed to be there even if not available right now at 11 nines, meaning it is 99.999, all those percents. That's the percent of durability. So the chance of data loss is really, really, really small, really small. That's the data loss durability aspect. Durability is the chance of losing an object, 11 nines, 11 nines. The chance of not being able to access an object, availability, is 4 nines. This difference between durability and availability should be clear to you. If not, I will clarify again. This is the chance of loss. This is the chance of access loss. It is still there, it's just that you cannot access it. It's not going to die, but just wait a few minutes, it will come back online. It's the availability part. It will not disappear, I promise. That's the availability promise. The durability promise is a very high promise. The chance of actually losing the object forever is extremely low, 11 nines. The chance of you getting disconnection or not able to access the object that still exists, by the way, it's just that you're not able to access, is also reasonably high. And that's the promise that these companies provide. Similar promises are available from other companies, Google, Microsoft, same thing. Very similar things. The idea is that this storage is a service, meaning somebody is professionally managing it for you, that you don't have to worry about the data loss or the chance of loss of availability or chance of loss of access. But all you do is you take the item that you have, the object, and you put that in a bucket. And you forget that you have to worry about maintaining and managing all that disk storage or nothing. Just don't even worry. And this bucket is, I should say the word, huge, sorry, it's not the orange colored huge, but just plain huge. It is actually virtually infinite. This is the illusion of infinity, like I mentioned the last time. It is huge. So there is no virtual limit to how many objects can you put. In fact, you can see S3 limits. Let's see, S3 size limit. And the objects can be five terabytes long, big gigantic storage, five terabytes, terabytes. Each object can be five terabytes, up to five terabytes. And the number of such objects, there is no such limit as I remember, however, this is old article, so they must have updated it. This is like this 2010 article, so really old. So five terabytes is the limit, excuse me. Let us make use of it. Let's do something with it. So I am going to grab a picture, literally, and put that in a bucket. For right now, I will begin with creating a bucket, which is very, very simple. Just click on the create button right there. This bucket called create bucket, click. That's all it takes. So I will just create a button, create a bucket. I will call it garden, for example, and select a region if I want to, or just go to the standard region, that's good enough. And then create. So I have a new bucket created, but that name is taken, so I cannot use that name. It's unique. It has to be unique globally in the Amazon property. So that name is not available. So I will add a prefix, a suffix like cloud genius, for example. That's probably going to be available because it's not very likely not taken. So I will click and create that bucket. The bucket becomes created immediately. You can see that the bucket is ready, and I can actually see the bucket right there. You also see it here. So if you click on it, currently the bucket is empty. That's expected because we just created a bucket. Creating a bucket doesn't cost you a thing. The service gives you bucket creation. It doesn't cost you. The fact that you will store an item in the bucket, that will cost you something, very tiny amount. It's extremely cheap, but it is non-zero. So now I am going to upload an object into the bucket. So this bucket exists. I am going to upload something to it. So say upload an object, and the limit is five terabyte size. I don't even have an object that big, so I cannot even upload that size and test it. And by the way, it will take forever to upload a five terabyte object, so I don't want to even try. This I'll get a fiber connection at one day. I will probably next month, I hope. But let's see. Until that time, we are living with Comcast, and Comcast cannot upload a five terabyte file successfully. So I'll just upload a picture. So I'll add some files. This is not rocket science. This is simple. I'm just getting some picture from this location and uploading it. So here is the picture I want to upload. And say start. So the picture gets uploaded. And there it is. I see the picture, but I cannot access it because I haven't given myself the permission to read it. So the picture exists. If I click on this link, I expect to see the picture. But no, my access is denied to me because I don't have the permission to access my own picture through the web browser. So access denied because you have to explicitly assign permissions even to yourself or to people on the internet for them to be able to access this item that I put in the bucket. Now, where did I put it? Let's go see it. If I go see this URL, if I go and read that URL details a little bit, you will see that the URL is very simple to understand. It is s3amazonaws.com, then the bucket name, and then the object name. Very simple. The structure of the URL is very simple. It is easy to construct. However, permissions are not in place. So I cannot access my own object, which means I need to adjust the permissions. So I will go do that here in the permission section, and I can grant everybody the permission to open it. I'll go to add permissions for everyone to be able to open and download and things of that nature. So I'm doing basically that and saving it. Now, when I do that, I immediately allow everybody on the internet to access my object, which I'm going to copy it right now and put it in the Slack chat right there, and you should see it. You should be able to open it up in the browser, any browser, even without logging in, just click and you should see the picture. The object is visible and available on the internet, and this does not require a machine for it to be rendered on the internet. You see it right there. You see the URL that I constructed like this, like this. So it has the bucket name, this location, and the object item that you put, whatever object you put. And that's how the syntax for the URL, which is given to you, by the way, they give you the URL right there as a link. Now what does this translate to? It basically takes you back to this discussion which we had, where is that, here, is that we are using this storage as a service. So it is basically some storage that we want to use, and we don't have to worry about managing maintaining the disk or managing maintaining anything except that we have to pay for it. So there is this service available that we are going to use, and we're putting the picture in there, the P that stores here, because this user uploaded, it needs to go automatically all the way to this bucket. And then when this user comes, he may get assigned to this machine, which can always go back to that machine and grab it and show it to the user. The same picture becomes available to multiple people. And at the same time, these machines remain stateless. Stateless as in, if they're not maintaining the user state of the picture, where the picture came from, this is user-generated content, user-generated content. So this UGC belongs to the user, so you want to keep that separate and not lose it. So this machine, you can delete, destroy, create again. They are identical. They are cows. And you are practically keeping your user-generated content away in a storage as a service. This is not software as a service, but storage as a service. And that is a very, very simple, trivial way of using it. So you just have that item. You can delete it as simple as selecting it, right-clicking it, and say delete. The item will get deleted, and you can do these things through API also, and the bucket exists. And if you try to click on that link again, the bucket is empty, go to the Slack link and try to open it. Everything will open because it doesn't exist anymore because I deleted it. Even though the permissions were there, the file is deleted and it's gone. Because you deleted, you killed it, it's gone, gone. Now on the topic of this concept, what I would like to now show it to you is a scenario where you can create a full-fledged static website onto these kind of storage services without having to start a machine. Have you heard of such a thing ever before where you create a website, like a static website where there is no server because you don't need one, because you are using the storage as a service, and all you are doing is just exposing the disk to the internet and creating a static website right there? Have you seen or heard of such a thing before? Any of you? Is that where an example of Amazon with the DynamoDB comes in? No. No. No. You can use DynamoDB with a static site, but right now we are talking about a static website which doesn't even have a server. All it uses, all it uses is this storage service, for example, S3. You can do the same thing in Azure and same thing in Google, but the concept remains is that you don't need a server. You don't need any other service, just storage as a service and construct a site very, very simply in the same bucket like we saw in this bucket, and I will create one for you right now. Here's how it goes. You create a page in HTML, basically, and you write, say... I'm making it big, so you can very clearly see what I'm doing, and all I'm doing is basically including a picture there, so just a very, very trivially simple HTML, which is an image tag. This is my head. This is my body. This is a line that I'm writing there, and this is an image tag, and this is the end of the body, and this is your HTML file. That's it. Simple file. Not much in there. Head is empty, body is one line, and save. I'm going to save this file as index.html in the downloads folder, so index.html, so saved it. Let's test it first. Testing will fail because I don't have the picture yet, but that's understood, so we'll just run it right now. Open the Finder, go to the downloads folder, and double-click on index.html, and you see that the image link is broken because the picture doesn't exist, so we'll upload that picture once again because we deleted it. I'm going to upload the picture one more time and also set its permissions exactly the way I wanted it to be, which is for people to be able to read, for everyone to be able to view permissions and save. We just view permissions, enabled it, click on it, the picture should show up, yes. Now let's go to the download folder, open index.html, and you see the picture. This is the simple HTML I created. What I'm going to do now is upload this HTML file that I just created by hand, go back to the downloads section, index.html, open, and upload, and it is uploading, and the upload finished. Now I'm going to adjust its permissions and add permissions for the index file for everyone to be able to open and view and save. And this, along with the concept that this bucket itself, and remember to note this carefully, what I'm going to do now is to convert that bucket into a static website here. So I'm at the bucket level now, and I'm adjusting bucket permissions to enable static website hosting with an index document named as index.html. That's it. That's all it takes. So what I did was enable static web hosting on the same bucket and gave it an index.html file and save, and I get a website. Here it is. See that? I'm going to paste it in Slack chat so you have access to it. So this is a website I just constructed out of no server, no server whatsoever. And it is possible. You saw it. What we did was we took a bucket, so this bucket, we put an item in the bucket. Item number one was a picture. Item number two was an index.html file that I wrote by hand, and I will paste that code for you also if you'd like. Let me do that. So I'll paste this code out in Slack chat so you have it there. Very, very simple code, not a big deal. Create a snippet, paste it, and create. And so the snippet is coming to you in Slack chat, this is the snippet. And what I did basically was added these two items in the bucket. And then the third thing I did was, by the way, permissions, permissions here again. And then third thing I did was change the bucket representation into a static site. And a static website always expects that you have some HTML file to show the website. So I said, you know what, use this. And that's it. I get a site like that here. You look at the URL, it is very simple. You can actually understand where it's coming from by reading it. And part by part, it is on the East1, US East1 section of Amazon. It is an S3 website. On the top level domain, amazonaws.com. And the bucket name is garden cloud genius. So it is very easy to construct sites like this. And it doesn't cost too much. In fact, my storage pricing, let's see storage pricing, S3 pricing. The pricing is probably free for you for the first year. So that's a given. But check the pricing, it is ridiculously cheap. So Virginia pricing, for the first terabytes per month, they will charge me 0.03 per gigabyte of storage. A gigabyte of storage, I'm paying 0.03. So pretty darn cheap, you know, just a picture with 100 kb. So it's not even a gigabyte. So I'll probably get a fraction of a bill or really a fraction of a fraction of a cent is the bill that I will expect to see this month. That's totally irrelevant. Now that's the pricing. Now the availability you saw already 99.99 availability, it's pretty decent, better than a machine that you can think of. The durability is rock solid, 11 nines, it's amazing. So we were able to get a get a new site like this, running without spending almost nothing, almost nothing, practically nothing. In fact, this is just a trivial example I just wrote by hand as I was talking to you. I'll give you a little bit more involved example. Here is a site. This is my site. I've been running it since 2012 or something. And it does not have a server. It is a static site hosted in a bucket. This one. It is a blog. And it functions like a blog with all the hyperlinks and everything and you can see all that. But the point that I want to go to next is this, which is you can also run a site like this. And I would encourage you to run a site like this on your personal blog by your own domain that you now have, most of you have. So create a bucket on your own name in Amazon, you will not pay a dime for the first year. And for the next year, you might pay a cent because you are not getting traffic. But this thing scales very nicely. In fact, it's a globally available service. And it is highly available as you saw four nines and highly durable, 11 nines. And you can run a decent static site with global presence for less than a cent every year given the kind of pricing that you saw. However, if Oprah invites you to her show, you might spend maybe $10, $100, depending on how many people hit your site. But if you're really popular, you might get a hundred dollar bill because people are going to hit your site too much. And that may be the case if you decide to go on Oprah, which may be a good thing depending on how you monetize that because people will want to buy from you or maybe just chat with you, talk to you. I don't know what you do. But that side, it can handle the traffic of global audience size because the services are designed to be scalable. And you don't have to worry about managing, maintaining machines. Many new sites are actually constructed like this without the need of a server. As long as they are static, it is totally possible to implement. And I've been running it for, I don't know how many years, for 2012. And my total bill is probably three cents a month. So that's the kind of cost of running this site is. And it is beautiful. Now to construct this site also is automated and we will do that exercise next. Is it today? Probably not. The next time. So the next time what we will do is construct site like this through automation and we'll put that inside. In fact, for that, there is a exercise already somewhere here. Where is that exercise? Yeah, here. Nine, number nine is the exercise. This module number nine has that exercise in which what we are doing is, let's see, why is it not adjusting properly? Module number nine, cloud-based storage services, storage services in the cloud. Yes, you will have this, the entire module nine is what we intend to do next time when we meet, which will be on 20th, right? Yeah, 20th at five o'clock PM Pacific time is when we will meet again. And that time, I want to show you this idea and you can read it as already written down. But the concept that I will illustrate is that you can use storage services without spending time or effort or money on starting machines. And you can still get to a point where you are having a full-fledged functional site, which is a static site, doesn't have a database, but it works. To the extent it can, works just fine. You can connect to a database if you like. And that is also possible these days. Therefore, what our focus will be is to understand how do we use storage services in the cloud in the context of using, just putting content, when you put static content such as HTML file like this one or some JavaScript or some CSS to make it look nicer. And we'll generate JavaScript and CSS through automation so you don't have to worry about coding that by hand. And you can modify it if you like and some pictures like this one. And so these elements collectively construct a decent looking site. It's not as beautiful as you want it to be. You can further tweak, but it's a functional site. It works and it is pretty darn high performing. In fact, it has had no downtime, zero downtime since 2012. And I'm paying how much, three cents every month, sometimes two. So literally no downtime and I'm measuring it. I'm measuring it through sites like Bingdom, which is a service to monitor websites. And this is what I use. And I have downtime multiple places, but not this site. I don't use this site much anymore, but there is one exercise that I will be doing next time, which is to run the site like this for you and you will also do it. You will get to do it and you will understand how to do it yourself. But beautiful availability and durability with absolutely no downtime as of date. Not that it will never go down, but it has not gone down for me yet. It probably may not. It's a very decently done service, Amazon storage service and Google storage service and Azure storage service. Now, one quick discussion on the Google Drive part or Windows Live Drive, what's the new name? OneDrive or those kinds of tools slightly differ in their usage. So let us go see the Drive scenario. So drive.google.com, this is my OneDrive. I'm going to upload a new object, new image. Where do you image? Okay. There is no upload image, but whatever, I'll just take that item and upload it into the OneDrive. So I'll just do that manually. I'll go to that index HTML and drop it into the My Drive. Okay. Index goes. So can I open it? Yes, I can see it. The problem that goes is that I cannot generate a clean URL, which I want to publish. If I want to decide to make this available to the world outside, I can share it with the world outside and say that, you know what, get a shareable link. So I get a link there. I copy it, and this is available to anybody, anyone with a link more, anyone with a link on the public. So I can share it publicly. And so this is anyone with a link, public link. So I can open a new browser, Firefox, and see if I can open that link available to public. And yes, I get it, but you will see the difference in that. I don't want to see this opened up in a browser for editing. I want to see it like a site, like a web page. That's the difference in between how you share and render. This is totally possible for me to download, and I can do it. But this representation is not what I expect from a developer usage. I want that developer usage should be something like you saw here. That is a classic usage of representation like a website, like you see here. And this is not what you see here. That's the usage model. This is a consumer-based storage system, Google Drive. It is also unlimited for you, as you know already. And Google Drive is not developer-friendly. It is user-friendly. And so it is useful for consumers like you and me who are individually using to store stuff on the internet or back it up on the internet if you like. However, for development usages, this model is a much better model, but much better suited for writing new applications. That's why those services, which are known as storage as a service, are different from Dropbox, Windows Live OneDrive, or Google Drive, which are more like consumer storage solutions. That's what I would call it. These are consumer storage solutions, consumer cloud storage, let's call it consumer cloud storage, whereas these services, storage as a service, is a full-fledged developer-friendly service that you saw, that you will be played with it and you will also play. By the way, you should do this on your own, on the Slack chat. Post a link to your own static site with your own image, just to test. And we'll modify it next time and actually update it with a full-fledged, properly constructed site, which we will develop through automation. So there are instructions written down for you. You will have a site just like this one, coming up on your own, for yourself, using tools that we will use in this module number, whatever that number is, nine, yeah, nine. Why is it broken? It's not showing me properly. Oh, the browser size. Okay. So I'll go to a different browser or maybe just reset this to adjust it. Yeah. So module number nine, cloud-based storage services, that's the exercise we will do next time to understand and generate code, not write, generate a good-looking, decent, not good, decent-looking site that kind of sort of looks like this for yourself, on your own domain, in your bucket, in your S3, in Amazon. That's the exercise we'll do. And you can understand the capabilities it provides and how do you make use of it in the context of using underlying infrastructure. And the key thing to remember, the whole purpose of doing this is to make your machines stateless. That's the one thing you need to remember. As long as you are able to get these machines stateless, that you are storing the state outside somewhere, that's the whole purpose of using storage services, primarily in this context. So remember that piece and let's do the exercise next time, which is Tuesday, 20th, five o'clock Pacific time in the evening, same method, same number. And any questions? Just a quick question, I don't know how simple it may be, but how does that S3 service from Amazon monitor or manages the uniqueness of the link for your storage? Yeah. The uniqueness is managed by the fact that your bucket name is globally unique. You cannot randomly create any name you like. Like you try to create the name Gagan in the bucket, try to create a name called Cloud Genius, I've already taken it. So a person by the name Gagan may have taken the name Gagan already. And so it is not available. It is globally unique right at the name of bucket level. So that's the bucket level uniqueness. Now within a bucket, if you have a file or an item or an object in the bucket with a name already, and you drop in another object with the same name, it will collide. It will say, are you sure you want to override the object? And you can say yes or no, but that name collision is avoided because you are careful in not overriding your own object. And that's how uniqueness is maintained across a global footprint available in the service. So it's pretty much based on DNS, I'd assume. Yeah, DNS and bucket name and file name. Yes. Okay. Cool. Thanks. You're welcome. Cool. Was this interesting, the storage idea, was this interesting to you or did you know of this already? I should have known of this 10 years ago. I'm about to cry. Cry? Don't cry. It's amazing. Yeah. Very interesting. Please don't cry. Wow. You're hilarious. Yeah, we do use some VM images, virtual box images in S3. When we set up Vagrants, we download the image from there. Yes, yes. Vagrant boxes. It's a good thing. So that's another valid user scenario where you have Vagrant boxes sitting in. So Vagrant boxes are basically images that you download and run it locally in your test situation for Vagrant testing. And that's a good use case to store stuff. It is commonly available within your company to your developers and so everybody can share. You can even make those images publicly available and see these Vagrant images, Vagrant boxes on this site, Vagrant boxes site. You will see that. Oh, I'm not sharing apparently. Okay. So let me share. So Vagrant boxes is this site where you can download a bunch of boxes publicly available. And it is all open source. And so that's how people are actually doing. You can see that these boxes are sitting in various locations. Some of them on Amazon Cloud, some in other places, bunch of boxes, huge collection. And you can put your own box there. So that's a use case for Vagrant people, Vagrant developers. Absolutely. Any other questions? Nope, nope. So the exercise before the next session is to have some static site created. Yeah, exactly. The exercise will be that you create a static site, post a link in Slack chat with your own sample static site that you want to have. Just a plain index HTML, I gave you a sample. Use that. Put your own image, upload it, create a static site in a bucket, share a link in Slack chat. And next time we'll do the little bit more involved exercise of generating automation code, automation of generation of HTML, CSS, JavaScript, and send that over to a bucket, make it a static site, and make it look somewhat useful than this lame thing that I just developed. And I developed this lame looking site. Where is that? Here. I just did it by hand. That's not a site site, it's just some silly stuff. So this is what we will create, a better, more usable site through automation of a tool known as, what's it called, forget the name of it, Octopress, like WordPress, like Octopress. So Octopress is a software, open source, and documentation site here, and you can go to GitHub and get to the source of it. But that's the software we intend to use. The source code is available in GitHub, right there. And it's fairly popular, like 3,000 forks, 9,000 stars, so it's a pretty decently popular application. And this is what I intend to use the next time. This is, by the way, the automation tool that generates a decent looking usable site. Not you having to code HTML, CSS, JavaScript by hand, you will write in English, and it will do the work for us, we'll see that next time. Thanks Nilesh. Cool guys. I'll see you Tuesday, five o'clock. Thank you. Bye bye. Thank you. After that, can you give me a little help? Yes. Yes, please. Yeah. So let me stop recording, and I will let you share your screen with me. So give me a second.