Hey, guys. Hey, Mukesh. How are you doing? Good, sir. How are you? Pretty good, pretty good. Good afternoon. Hello. How are you doing? Good. How are you? Pretty good. Let's begin. I'm just starting to share my screen. And what do we have here? Give me a second. Okay, I think we are ready. Ready. There we go. Just Mukesh and Ram right now. But that's cool. We'll just get started. So let me open that OneNote thing and I can maybe draw easier. So here we have that OneNote notebook opened up. And I think that we would like to continue discussion around a variety of recommended architecture diagrams and illustrations. We'll see how it applies to variety of business situations. So we'll go to the same location which is where you have been, I guess, by now, which is this thing called architecture application architecture center. That's where they have some beautiful diagrams, illustrations that they have made for us. There is quite a bit of some reference deployments as you can see. If you want to deploy a particular type of application, then you read that particular illustration. So if you want to set up link server, then you read this pretty much. That's what the idea is. It is very specific to that particular application. So you read the entire guide about that. And if you want to further actually make it standardized in terms of how you go about implementing, then this is the button you actually click that takes you straight to Amazon and puts you in a new tool that is not new. It's an existing tool. It's called CloudFormation. And that's what you will see that it actually takes us straight to that location and gives us a ready-made template, ready-made blueprint. So using this template, we will just proceed ahead and it will construct the whole thing for us to enable us to start this particular application deployment scenario as described by this document here. So this is the best practice accumulated between Microsoft and Amazon on setting together link server on the AWS Cloud. And this is a very documented practice. It has been updated as late as May 2016. So it is fairly recent. And the idea here that you're looking at essentially boils down to this concept where you have, oh, we have David just joined. Hi, David. What we are looking at, what this illustration shows us, is that as we start looking at these what I call reference architectures produced for us, for people, everybody, primarily so that we can make some scenarios, some business use case. So there is a business use case of some sort. Let's say we want to establish communication and collaboration between our team members. And there are various ways to enable that. One method that people use is to use Skype for business. Skype for business. Or rather, we'll call it Link in this example. Link 2013 is the way to set up communication and collaboration between your company, employees, and other people outside your company, things like that. So for which you will use this as the platform and for which it requires you to set up some blueprint in some cloud. And that blueprint that you have is provided to us based on best practice. Best practice accumulated by two companies. The one that create this, so Microsoft and the cloud provider here, which is Amazon Web Services. So these two guys together have created this blueprint for us. That's what you're looking at. But what does a blueprint actually take? You know, what's the concrete steps that we can see and actually understand what this is? I see some write-ups and I see some pages after pages of information. But what does it boil down to? So this is what the picture really looks like in terms of what they have as a recommendation for quick start architecture for link server. And so if you start looking at the scenario of usage, which is this business use case I'm talking about, which is about communication and collaboration between your employees and your partners, or people outside. So basically, three types of people. Employees. And then you have some business partners that you collaborate with from other companies. So these are partners. And then, you know, public outside, worldwide. So it's worldwide public outside. Whatever people are on the public PSTN network, phone network, people who do not have phones, but they are on the plain Skype network. You can all contact these people whether they are in the company, in another company, or public at large on the phone lines. You can just call them, send them messages, things of that nature. And also, those who do not have phones may have Skype. You can call Skype the regular plain Skype. This is Skype for business link. So that's the blueprint that is illustrated in this diagram. So if you start looking at this picture a little bit deeper, and so you see that there is this gateway, which is where the boundary between public outside, meaning people on the internet, companies, customers, wherever they are. And this is the internet gateway where you are entering the Amazon cloud. So here we have a typical illustration of what is described as a virtual private cloud. So if you start going deeper in that, they are illustrating two different availability zones, one and two. And inside there, you will find that these two availability zones are pretty similar in how they are laid out. Basically, they are two different locations where you have exactly similar deployment. And they fail over. So, if you now focus your attention on one of those regions, let us say the AZ number one. We are looking at a NAT gateway, which is conceptually very similar to what we did in the other exercises early on, which is where we prepared what is known as a NAT box or a jump box or a bastion box. We used a variety of different names for these three things, for a box that lets us not only enter the VPC, so this will let you enter the VPC, but also let your boxes inside the VPC go to the public network outside the internet using the NAT function, the network address translation function. So that's what we did. We actually deployed a T2 micro device that we converted into a NAT machine. In another example, if I remember right, I think we just used an AWS AMZN NAT AMI, if I remember right. And so that Amazon machine image that was already made, given to us, that we chose to build a T2 micro box that we put in place in our public subnet. And that is what we used in constructing a NAT device. We also used the same device to jump or to get inside the VPC from outside, like a bastion jump point, jump box. And what Amazon has now done is they have constructed a new service to do this thing, exactly this thing. They call it the NAT gateway. So it is a paid-for service where you don't have to hunt down a box and then convert into a NAT and all that. You don't have to worry about all this thing, because as you know, it is quite complex. If you go look here, you know how to find that machine and start looking for it. Let's say launch an instance, and you don't know which one instance you want to launch. So you come here and search for NAT, and then you find it in the 2014 version. There is a 2016 version of NAT. That sounds interesting. So we'll take that, NAT, AMI, VPC NAT, HVM 2016. So we'll select that and then we go along and then we have to enable this disable source destination check, put the public routes and private routes and all that complexity we have to take into account to convert this box into a NAT box. But what Amazon does is they give you a ready-made solution these days called NAT gateway. That is already available, by the way. Somewhere here you can actually see that NAT gateway solution is available in the VPC area. You can just click a button and it gives you. So where is that? VPC. And somewhere here there is this NAT gateway option. You can just click this button and that's it. You get a NAT gateway. That's the idea. So that's what they're talking about here. NAT gateway. That lets you, actually lets these devices, the link FE1 and DC, CGI, DNS, the active directory domain controller that these guys have at the back end. And the link server standard edition front-end servers. This box, the blue box here and the blue box there. These two boxes are needed for the user, this guy, to be able to actually use the link server services on the back side. So at the end of that, you will have this software, the second one, not the first one, the Skype for Business software can then, you can then use this software to let's see, start the Skype for Business and here I have the Skype for Business running, configured and it is separately having trouble connecting so all that. But the idea is this is the Skype for Business software which all of us can use to actually connect. So here we can do a video call with Ram for example. And so that's the idea. I can start this call provided provided Ram is online using the same software known as Skype for Business. But to enable this, the Skype for Business on the back side, you need a setup like this. That setup is provided to me through Office 365 cloud services that we are using. But if you want to set up your own, that's the layout you need. You need a server box, this blue box here and other box to enable your active directory domain controller. And then you put these things on one AZ, put a similar setup in a different AZ and then have the ability to fade over. So you basically load balance between this segment here and this segment here. So it's a very simple design. So it's actually very simple. And to be able to reproduce this entire design, the entire layout in a fashion that makes it easy for somebody to quickly replicate and set up the communication collaboration scenarios using this software by merely click of a button as long as they're ready to pay for this kind of a CPU horsepower and some memory and some sort of disk. You can accomplish that scenario very, very easily by clicking that button that I was showing you here. So you go to the PDF file and then you find that there is this template that you look. And actually this template, it will come down on my desktop. So we'll just save it here and we'll see what it does. So this template file, apparently this one is some kind of a JSON file. I'm pretty sure. So let me see what it opens up as. I'm pretty sure what I downloaded is very likely a JSON file. But let's see how we can open it. So we'll just open Atom Editor and see what that file contains by dragging it in. And so here we have a JSON file representation, which is basically the way that Amazon describes a template of infrastructure that you need for your application architecture to be deployed in the Amazon cloud. So this is basically a JSON file that you're looking at. You can download this yourself, by the way, if you don't have it. I'll just give you the link we are using. And you can examine this specific by the way, for every architectural illustration that they have here, they actually accompany the same idea along with an JSON file that you're looking at, that you can reach from this location. And you just go down to the link server, go to the PDF file, and then look for that link, the template link. That's what I have opened up inside this file here. So if you now start looking at this file, this JSON illustration, it's long. It is actually 200-300 lines long. But don't get exhausted by reading that, because what it is actually doing is illustrating how the thing that we saw, the picture, the image that we saw just a few minutes ago, here and here. This illustration that you're looking at, there we go. Next. Yeah. That illustration. So whatever it takes to build this entire scenario out is written down black and white in form of JSON description in this file right here. That's what this is the picture representation, this is the JSON representation of the same architecture. Now this may be the first time that you're looking at a JSON representation of a computer architecture, but let us try to understand what it is trying to accomplish. And for that, I will walk you through here. So before I go in this, which is actually fairly easy to understand, let's understand what these things are. So what these things basically belong is that you draw some architecture. You actually build something actually. So use your mouse, go to AWS, and then you build out something interesting that you like. And this is the real stuff. Whatever you build. So you build some machines, set some code, you configure ports, configure security groups, lay out your entire application suite in a variety of machines, multiple subnets, multiple availability zones, failover, NAT boxes, everything that you can think of. That's the real thing. Whatever you are deploying is the real thing. What you can then do is to extract this information and put that in a file. This entire design can be extracted, can be extracted and put that in a JSON file. That's what we are looking at basically here. This illustration that you saw in this picture, once you go into the Amazon console and actually create these things item by item, meaning create an EC2 machine, and then you select the instance, select Microsoft Windows Server, and select the appropriate size recommended, like M2, whatever that size is that you need and configure the details. Do the whole thing, step by step. And then repeat the same thing on the other side, on the second availability zone. And then you have a gateway in between, NAT gateways, put the routes, put security groups, create the public subnet defined here. Like you see here, public subnet right there, then this private subnet constructed, this private subnet constructed. If you do all these things by hand by going to the site, and then you want to repeat the same exercise for your friend. So you build the whole thing by hand. You know how to do it. So you build it. Then you design it, that's great. But now you want to repeat that thing. It is very hard. Repeat the same thing now this time. So say for example, I do it, and I want now Ram to do it, and now Ram does it, and then he wants Mukesh to do it. That becomes a pretty tedious job if you want to repeat the exact same design over and over and over again. And that is a problem, is what this thing tries to solve. Is that once you build something that you know that you have built correctly, that you want to take a or rather extract all the interesting useful information that is stored in the actual implementation out in form of a JSON file. And then you pass this file to your friend. And what your friend can do is take that JSON and apply to the cloud. And so when it does, then you have another real implementation. It becomes available once you apply. So these tools that we are going to talk about that are actually under the hood, that's what is used in constructing this JSON. So they have a word for it. I'm going to look for that word and actually point out the tool that is being used under the hood here. The tool is called CloudFormation. And in this CloudFormation tool, the idea is that there is a template that is also a stack. These two things, template and stack. What is a template? Template is basically this. And stack is whatever you implement. So this is a template. And this is a stack. And so you can construct a stack if you have a template. That's the easy part. Like I just give you a template, you just reproduce that entire template in form of a real implementation. That is the stack that you build up. That might look like something like this picture. That's a stack for a link. And the converse is also true. If you have a real implementation, you can go this way. So you have a stack ready. You want to give it to your friend and say, here is how I did it. Go do it yourself. You construct a template to generate JSON. And then give it to your friend. And the friend can apply. Apply to create his or her own stack. That's the concept. Now, what this cloud formation tool does is basically looks at a stack, create a JSON. It gives you a file. If you give it a file, it will give you an implementation stack. Both are possible. By the way, I must caution that this is unique to Amazon. Unique to Amazon Web Services. That's where it runs. There is no other place that this exact tool will not work anywhere else. It is specific to just Amazon. So that's one of the problems you need to know, be aware of because if you are using this tool, then you are basically married to Amazon Web Services. You may not want to get married. You may want to remain independent. It's your choice. So if you like Amazon, you want to get married, go ahead. No problem. But having understood that piece, let us see what this tool does back here. There is this idea of design a template, which basically says, I will give you a template that tells Amazon CloudFormation which resources are going to be provisioned, how to provision them, and then when you build a stack, you must submit a template. So we are going to use the design template. We click and we will provide a template. So there are lots of ways to actually illustrate these things, like in the illustration here, let's begin with something like let's say load balancers. Load balancing. You get a load balancer in place. Then you can put some machines like we need EC2. So we can get a host machine here, and an internet gateway here, and a NAT gateway with this machine, and pretty much like that, we can start designing actual implementations. And as we build these things over, you will see that once you establish your relationships and your connections, you will start to see a template getting generated right here. You see that template got created as soon as I start putting things in the picture here by using my mouse. It starts to build the template for me. If you want to actually not use the graphical illustration and you don't like the graphical approach, that is totally cool. You can actually download that diagram as an image. So it will give you an image down as a PNG file. If you want to actually open a file that you might have, so you just locally open that file that we have somewhere in our desktop. It's called the link template. So we just open it up. And here we have a link stack. So the template itself got loaded up in that browser at the bottom. The same JSON template you see here, the same one I loaded up here, and it shows me this illustration. That's what you have. So if you now expand that thing, there are two different stacks. One is an AD stack, one is a link stack. There are two stacks defined in that picture. So that picture basically is drawn based on what you are looking at in this description. This template will deploy two link servers. It also deploys the required AD architecture outlined in the whatever PDF file that it shows. That's what this, let me adjust the view a little bit. So soft run. And so here you can read the whole thing now. This template actually creates machines, Windows machines, and related resources. It will basically cost you. You will be billed for the AWS resources you use to create the stack using this template. So if you now take this design and say, you know what? I want this built. It will build for you. You can actually go and build this illustration. You can validate the template. It is validating the template. We provided a ready-made template. So the template will probably not be wrong. Now the other mode of, once we go through this template and say, you know what? Let's do it for me. It will construct an AD stack, meaning the Active Directory stack for us, and also the link stack for us. It will construct these two stacks, connect them appropriately, just like described in this illustration, and you will have a functional implementation based on the design that you have in this picture, or in this JSON file. So now let's take a quick look at what that JSON actually has inside. We'll start looking at from the top-level perspective. The top-level item is the template format version. There is a description here, some parameters. There is a key pair that we are going to use. So there is a key pair name you provide. There is an AD instance type, so Active Directory instance you can select, and for extra-large, that's the first Active Directory instance. A second AD instance is also going to be M4 extra-large. You can allow values are the smaller ones, so this is M4 large, extra-large, M4 large is okay. That's what they're saying in this example. So second AD, first AD to failover. Then there is here remote desktop gateway. So there is another device, basically, the M4 extra-large, and then we have here link front-end instance type. This is the solver that will be running M4 to extra-large, and this will create the standard front-end server on the cloud. We'll have another one, edge instance type. So here is another link edge server construct, M4 large. Then we have domain DNS, FQDN for the forest root of the domain. For example, example.com, you need to basically change these things. For example, you're setting up for your company, you need to go change this thing pretty much like I did, like this. You can then define the domain DNS for your... You need to modify that template before you actually execute the template. The moment you execute this template, it will actually create and configure your cloud services exactly like you described in the JSON file. You keep going down the chain, you will have net bios DNS, daily server DNS bios name. So item by item, it will define the strings that you want to use in configuration for that particular link server. So here is the net bios name defined, a DC1, DC2 names defined. These are specific to the application. You want to use a password for a separate administrator account. You will specify that password pattern here. Domain administration user, there is a default value for stack admin. Again, password patterns for the other. Then we have CIDR block for the public DMZ subnet located in AZ number one. Same for AZ two. CIDR block for AD server tier number one in AZ one. Same thing in AZ two. So you keep scrolling down and you will find that this actually also includes every specific detail that you would need for you to be able to set up at the top level this picture which basically helps you and your company in communication and collaboration using this software provided you want to run this thing yourself. So this is a self-hosted, you host it yourself in your cloud, in the Amazon cloud to be specific. And that's what this picture does. There is another solution by the way from Microsoft which is called Office 365 which you can just click and get. So it is a whole lot easier to actually implement the whole thing and have it ready for you as long as, you know, some people like the all done for you scenario, they'll just use this method that I'm illustrating to you the approach which is Office 365 based approach. It is already hosted for you. You don't have to worry about even this thing or this thing. It just gives you a ready-made solution out there. That's what you will see once I reach that point. Admin center and you can get to that location Skype for Business and this location gives you access to your Skype deployment for your company. And this is it. This is where you have a bunch of users configured and all of you, by the way, have an account here which you don't probably use. But you have it. So you have a Skype for Business online account and those people, all of you, actually have a user account that I don't think you are interested in using so I have no problem with that. But this is one way to implement it. Just go ask the company, can you give me a solution? They just give it to you. Other way is to take their software and implement it yourself. So this is the implementation method that you basically take this JSON, apply it to a cloud and then extract something out of once you apply, you get a ready-made deployment. Now I want to see this in action. I want to see this, what I just described to you, this link server may not be very easy, immediate for us to really understand what's going on under the hood. So what I am intending to do is to actually delete that file. But instead, I will do exactly what I described to you to begin with. Which is back here a new page. You can construct some stack in the cloud and then generate a JSON. Or you can give it a JSON, give this tool a JSON file and it will build a stack for you. You saw these tools. It will construct that stack for you based on what this JSON actually contains. That's the tool called cloud formation. Now, one caution and this part is slightly difficult to comprehend. But bear with me. Here is the idea. This concept called cloud formation, this tool itself needs to run on some computer somewhere without which it cannot produce this JSON. Can you give us a little more JSON background? Yes, please. Thank you. Sorry. JSON background. Let me identify a simple one to drop some JSON. How do I illustrate that? XML. Yeah, but you know these can be vague for people who have never used XML or JSON. It is really, really vague. People talk about JSON and people don't understand. I don't want that to happen. So let's go to this site. And here let's understand what JSON thing is. And so I think it's a good digression. So very short one though. Very short. We'll not go too much into it, but give you enough to understand what's going on. I think it is important. And so what is JSON? It is a lightweight data interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate. It is based on JavaScript. That's why it is called JavaScriptJ. Yes. JavaScript object notation. That's the name. Now what does it contain? Basically at the top level there is a definition of some object. Something. Instead of calling it object, I call it things. What is a thing? A thing is a key value pair. What is a key value pair? Let's go understand this in sheet of paper. Whole lot easier. So I am going to describe a key and a value. That's essentially what a thing is described. Now what is a key? Key is a way to find something. And what is that thing? Is the value what that thing is. So whatever that thing is can be found using this key. So this is the thing representation. Now I'll give you some examples of writing some things. The best way to write this thing first time is actually go to a JSON validator site. So we'll use this JSON link website which is where you write starting start writing JSON. So this is the validator tool. Where you write JSON. JSON begins with a curly braces. So you open and close. Then you give it some key. Right like this. Key and some value. Pretty much like this. Then you have a comma followed by some other things and some other value. Some other name Nilesh. Like that. That is a JSON representation. Now let us check whether this JSON representation is valid. And it says it is valid. That's the simplest you can think of. So if I want to actually send information to somebody saying what is the name? The answer is Nilesh. Thank you. What is my address? What is my email? I'll say my email is this. This is the simplest way to think about JSON. So I'm basically passing information. It's written like this. I think this is clear. I think this is the simplest way to describe somebody what JSON is. This is it. You're passing information through a structured method. There is a key colon value. Key colon value. It's a table basically. Basically it's a table like you see in Excel spreadsheet. Simply. Simply speaking. Very, very simply speaking. There is a comma in between and there are curly places on top and bottom. That's it. That's the JSON idea. So that's what you're looking at. It is a site dedicated to that. And it is a validator tool for that. And we just are going to use this file format. By the way, the file format is JSON. So that's what we are going to use..json is the file extension that we typically like to use. And you can use whatever file name you like. And Atom will handle it. By the way, just like Atom, there is a new Microsoft tool called Visual Studio Code. Pretty similar to Atom. Actually based on Atom's engine. The Electron engine. Atom has an engine under hood. It's called Electron Atom. This one. This is the class platform, desktop applications. This is the underlying engine that builds the Atom editor. And very, based on the same tool is we have Visual Studio Code. And this actually looks like Atom for all practical purposes. Slight deviations. It is designed by Microsoft for Mac and for other people. Like Ubuntu for example. So we have Windows, Mac, and Ubuntu. All three platforms. It is able to run class. You could use this if you are a Microsoft person. You could absolutely use this tool. Which you will find delicious. It's very colorful. Very easy to understand. And they are describing how to write TypeScript. You can use JSON and read that comfortably. So now what we intend to do today is we have some illustration that we will actually construct for real. And then create this file based on whatever we do here. That tool we will use in the context of Amazon is called Cloud Formation. And as I was saying, the catch that we need to understand. It is slightly hard to understand what I'm going to describe. And that's what I'm going to take a few minutes to explain that. Is construct a JSON out of a stack. And use a JSON to create a stack. We have this tool. We have this tool. Now what I was saying is this tool needs to run somewhere. In the cloud. So you're basically running somewhere in the cloud like another box. This red dot will run this tool. And then start looking at whatever you have in your cloud right there. Excluding the red dot. So anything but that red dot. And then use that it will survey whatever you have constructed in your cloud. And dump that information in this file. And then extinguish that red dot itself. So you have an accurate representation of what you have created in your cloud. So the S, the stack that you constructed. You want to take a information extract out of it. You use this tool called CloudFormation that itself will run here. And then survey whatever you have exactly like you have done. And dump this file. This one. And then extinguish itself. That's what this tool does. And that's what you're looking at in this picture that I'm going to open now. Somewhere here. No. Here. No. Let me open a new tab. So AWS Management Console. So you're looking at this CloudFormation tool which is a management tool which is what I'm going to now show you what it says. Here is the design template. So you have to give a template in this situation. You click this button, it generates a template. Back. Back in here we go to CloudFormation one more time. And we look at the next blue button which is create a stack. And in this situation, in this scenario, create a stack scenario, you are giving this thing a JSON. So you create a new stack. Then you have to give this guy a file. And it will construct whatever layout that you really want in form of an application stack running in the cloud. So you have to give this design template. I think they've changed a little bit here. Let me go back again to that place. So if you create a new stack, you have to provide a template. You can upload that file like here. You put that template itself into Amazon S3 and then select. Or you can choose your file from your desktop and send that file over. One of those methods. And then you go next. When you do that, you will have a complete stack constructed for you. That's what I'm describing in the second illustration. Here we are creating a stack. Here we are creating a JSON. So two methods. And now the third one, which is I need to run this guy. The tool itself needs to run somewhere. That's the third method for which I need another template that constructs this tool that in itself will be able to do this or that. Whatever you like. So this is a specific, unique tool. This is a specific, unique JSON file. Just to start this thing. This is provided to us. That's what you look at the third item at the bottom. So we go back to the front page on that transformation tool at the very bottom. Here. You have to create a template from your existing resource. To do this task, to create a JSON from your existing resource, you need to use another template. Which is kind of crazy to understand and digest. It says create a template from your existing resources. Whatever you have in the cloud. You want to create a JSON. You use this CloudFormer tool which will specify this exact JSON. We have given it for you. What is that JSON? Let's go see. Let's go open it up in the terminal. We'll download that link in the terminal. Download the terminal. We'll go to the desktop and download that template. Now we have it on the desktop right here. What that thing will do is open up in Atom. Atom and then template. This is a CloudFormer template. What this template does is actually starts that CloudFormer tool. Let's go view and then softwrap to read what it does. This is the AWS CloudFormer beta template. It's a template creation prototype application. It allows you to create a template from AWS resources in your account. It generates a template from your resource. For which you need the CF application. To start a CF application, this is the template. Back in here, what we are really looking at is we want to do this first step for which we need to have a JSON template that creates a CloudFormation tool that will then look at your stack and then create a JSON. That's what we are looking at. That's the third convoluted way of using this tool. Somewhat convoluted. You want to create a JSON, you have to provide, begin to the JSON then create a tool and use it. Then you run the tool here and then you are looking at your Cloud and then it will dump this JSON out. That's the example I would like to play right now. We'll begin exactly like step number three. To look at this example ready-made tool to run, the tool will then look at our Cloud and create a JSON. But for this, we don't have anything laid out right now. What I want to be able to do is put some things in here, actually for real. We want to put some real machines out there in the Cloud to construct some stack layout just for us to understand and then we will go back here, construct the tool and then let the tool work on our Cloud like here and then produce this JSON and we should be able to read this JSON and understand exactly what we did in this Cloud. So we will put a very simple layout architecture and then run the tool to see how the JSON looks like and then I will pass that JSON to you and you can reproduce the same exact thing on your Cloud very, very easily. The way this tool is designed is kind of somewhat difficult to really understand what's going on as to how we use it. That's the hard part. But using is not as complex. It does say it is beta. As you can see, the tool itself is beta. However, the product is not beta. If I understand it came out of beta but the application still says this thing is not beta. CloudFormation tool is not beta anymore. Otherwise, it will say beta. The template still says beta, whatever. To use this tool, we need to actually have some resources. So we'll create some. We'll go to our region, say Northern California, select our VPC that we have there already. So here we have one VPC which is an existing VPC. We can use the existing one if you like. It's easier to equip or to operate or we can create our own VPC and just build the whole thing like we saw before. And then we put some machines out there and launch an instance, lay it out in the configuration that we like. So I'm taking a very simple example. What I'm doing is first of all, going back to that VPC that we have. In Northern California we have one. In there we just examine what we have. So we have this available VPC. We have some subnets. These subnets are this ID 4581 and the two subnets, one is 1A one is 1C. And then we have some route tables. Basically this is the default one given to us. So to actually illustrate this point very clearly in terms of how do you take a snapshot of an existing cloud and make it reproducible for another team member, another customer and other partner to play. I think it makes sense to actually construct a separate VPC so we can actually create that JSON that becomes meaningful. This default created VPCs don't make much meaning. So I'm actually beginning with creating a VPC just like this. Very very simply calling it cloud genius giving it a cider block like this that you probably know this already 10.000.16 and say create. And VPC gets created inside that VPC. We'll just put two subnets. So we have one subnet here which is, sorry you don't have anything so create a subnet. We'll tag it, call it public and put this in the cloud genius VPC and the cider block is 10.000.24 and create that. Basically creating a couple of subnets and putting a box in one of the public subnets. Very very simple illustration I'm doing. So creating a public subnet. We have a gateway that we don't have any so create one call it internet gateway IGW create and then associate that gateway with your the gateway that you have detached. This needs to be attached to the VPC so the cloud genius VPC gets a gateway Now we have to create some routes so in that route we have two two routing tables that we have. So we need to filter by cloud genius VPC where we have this one default route. We'll just create a new route call it the public route and associate that with the cloud genius VPC and say create and I will then look at the public route and make sure that it is modified to go anywhere you want on the public internet by using the gateway that we have created and save and similarly the association that we have done we have to make the public subnet gets associated. You have done this thing before if you remember right and so that's what I just did I constructed just one public route a public subnet with this public route that is able to go anywhere on the internet and also a association with the public subnet that we have created here we have a gateway defined that's pretty much it. That's the extent I'm playing with the VPC here and now I will go to the EC2 section, start a machine launch an instance, select this server and select micro, select one in the cloud genius VPC in the public subnet we have and enable public IP access and then I will proceed to add some storage and tag this instance my test server and then I proceed to configure security groups I have an existing security group that I want to use so that one is good and I will say all effective is open except that this is not really open so I will modify the security group later but I'm for now saying that you know what this is the security group called default that I want to use, proceed review and launch and I will say launch it and so here I'm going to select this existing key pair I have and proceed with the machine launches that's great, machine becomes ready we have a VPC laid out in our illustration what we have in the diagramic illustration what we have done is simply created one gateway IGW and then we have just one public subnet in there we have one machine starting up right now and this machine is able to go anywhere on the internet through the gateway, that's simple very very conceptually simple illustration that you have that's what we constructed by the way we also have a get out table that says that you can go out and this anybody on this public subnet can go out and we have this machine that we are going to start it is starting right now, that's our implementation, very simple, very trivial right now, just because I want to take a JSON out of it, so I will construct this and create a JSON that's what I want to really do for which I just created this simple stack, so this is my stack it is inside a VPC by the way and so all of it is inside a separate newly constructed VPC and I have constructed a stack and I want to create a JSON now, to construct this I need to use a special template to construct this cloud formation tool which will inspect this cloud and then give me a JSON that's what I will do next, and this JSON I can then pass around to my team members the idea behind this JSON business essentially is that your team can reproduce this stack easily, that's the whole idea behind all this exercise and this reproducibility, it is you know, building this one is kind of complex, you know, you're doing anything by hand, you're experimenting, understanding and then making sure it works and once it gets working, you want to reproduce this as quickly as you can because you can basically tear down the whole thing and say, okay, do it again it doesn't take much time, once you have it in a JSON format you can just go this way and apply the JSON create a cloud, up and running in few seconds, maybe few minutes and so that's what I intend to do, I will choose this sample template by the way, my machine should be running this is running right there there we go, and so the rules security groups are kind of closed, I want to open them up so I can access the machine so I want to edit this, and I will say you know what, let me come in from my IP or just say anywhere, so anybody can access so that's the rule I changed so it's a very widely open rule right now, but just so understand that this is open, anybody can access that machine, that's how I have constructed, that's my design okay, now let's go apply a special template to construct a cloudformer tool which is the third item here at the bottom, launch the cloudformer which requires this special template that will run the cloudformer, that will inspect what do you got here and then produce this output that's what we really want, because we want to make the whole thing reproducible so let's begin cloudformer and they already give you a template so just use it, next next and so we'll call it the cloudformer, I need to use a password to protect my cloudformer so I'm just saying my password is going to be A the user name is going to be A it's a very simple A that's my login name and my VPC needs to be here you can create a new VPC or just use the default we'll just use the default VPC we have the setting will control if the cloudformer web server will launch in the default VPC or if a new VPC will be created, if you wish to launch an existing non-default VPC, so you can basically choose what you want, a new VPC or use the default one I'm saying my default VPC is unused go do whatever you want there so basically start a new machine with this login name A and password as A and that will be the cloudformer tool that will run in a separate VPC this is the default VPC by the way and this is the newly constructed VPC that I have, my cloudformer tool is running here using this template and this will then look at this cloud and produce this JSON, that's what I want so let's go, next and it says I want to specify some tag, I can say key value pair if I like some IAM role I can specify identity and access management roles I can specify those details but none of these are important for me so I'll just skip and I'll say you know what, this is going to construct my AWS cloudformer for me so I'll say I acknowledge the reason I want to acknowledge is because there is a cost to it so it is telling you that there is a cost, so here they'll estimate that the monthly bill for this cloudformer tool will be $49 that's monthly, that's understood and then we say you know what, this is the cloudformer tool that runs in my default VPC and I create so now I will have a separate machine started in a separate VPC that is going to inspect what do I have in my newly constructed VPC and generate a JSON for me and I will do that I'm waiting for that creation to complete as you can see that this creation is happening through a templated method we are using this special template to construct this tool right now this tool that runs in the default VPC for me so this tool blue one here is running actually start getting started so it is creation in progress you can see lots of events happening as the creation proceeds is that this is basically create complete it just suddenly completed that creation which means we should now go and examine the output outputs are not there yet so we should receive it still shows create in progress and I have only a part of it got completed the security group got completed this role got completed but not everything is completed still doing things that's why you don't see any output there when you see everything complete when this status gets complete and green in color which shows creation completed for the entire stack that's when you have an output here which will be a link to go for you to access your cloud former tool for which you have chosen the password to be A login name to be A so very simple just for me to remember the password nothing else creation is happening I am impatient so I need to refresh the screen and that's how just that is just me so I am being me right now nothing else so cloud former tool is going on the user started doing it and steps are in progress it takes about a minute or two so this should finish once this finishes we should have an output in this location where you will see a link to go to the newly constructed cloud former tool and so that special template that we used to construct this cloud former tool that is now running in a default VPC for me we will go and inspect the cloud genius VPC and identify whatever we have constructed in there whatever we did and dump a JSON so I am waiting for this to start then I can log in here and say you know what go look here and then produce this that's what I will be doing as soon as this thing runs so let's go see three more events to display creation in progress by the way while this does let's talk about something else very very similar in nature but without any vendor login that's what I want to talk about so while it is going on we are waiting for that cloud former tool to start this tool is basically an Amazon tool you get tied up once you are dependent on using this tool you are married and I don't really like to get married with cloud companies or rather any company so no marriage so I like to remain independent of companies that's the concept of neutrality and so I personally do not prefer using this tool what I use is terraform which is conceptually very similar except independent so when I have a terraform tool I can do this situation so you have some template you have a template you can apply the template to a cloud to build something you want we have done this by the way already we have done it at least two times in an exercise that was called dais we used a template and we applied it to construct our own platform as a service in the cloud technology segment you've done it so you basically create a template out of something either you do it by hand or use a tool so once you really understand how these templates work you can very easily apply them to any cloud the idea is any cloud so let's go see what this terraform tool does so this terraform is by this company called HashiCorp which is a group of smart people they have come up with creating this product for open source known as terraform that we have used and it is available for you to download from this github location at the bottom line of using this tool if you go look at the documentation is that it let you work with a bunch of different things you see that it has AWS it works with AWS it works with cloud stack digital ocean docker github fastly google cloud let's see azure azure legacy open stack and a bunch of others all the famous people that you can think of are available here provisioners you can use these chef tooling for provisioning your services using terraform you can put these things in combination and use them together and that's the idea that I personally prefer to use this tool as opposed to using this tool, not this this tool which is still getting created in terraform there is no such thing, don't have to wait for it to start you just run it it's a binary available here source code available here so while this runs let's recall what we did in using terraform in our exercise we did so if you go look back in time we did this so scrolling back in time we went to the platform as a service exercise at the tail end of it and here here what we are doing essentially is we are checking out some design this specific design is with the terraform command here, the terraform apply and then we give some parameters and you apply it to digital ocean so if you look at the code that gets downloaded when you run this run this git clone command basically there is a design already built in in that git repository we are just using that readymade design in this example so we go to that site and see what that design is and in fact this particular product gives you a variety of different designs in a variety of clouds, so if you go to the days and then contribution folder you will find that you can run days on any cloud like AWS Azure, bare metal I don't know what this thing is bump version, so it's not a cloud, Core OS Digital Ocean, Google Compute Engine, Linode, OpenStack and Vagrant, Vagrant is not a cloud, it runs on your laptop so you can run it on the laptop if you like and so a variety of different readymade blueprint or they don't have JSON they have a slightly different format we can just go and see the format so I just went to days, contrib and digital ocean, that's the one we used, so if you go see this file particularly, days, digital ocean here is how they describe what do we need in that particular cloud for this to construct a platform as a service implementation, so we constructed our own platform implementation based on some template design that was given to us by these people at days and we just apply that to digital ocean cloud and we obtained our platform if you try to do this on your own, that becomes difficult so why not just use a readymade template if somebody has done the work already, just apply that template and you get what they are trying to give you readymade platform as a service is what we used in that exercise and we used this tool terraform to basically get to the same point so this is what that folder itself is a way to describe how to provision a days cluster on digital ocean and this is the file that you use, these are the instructions that you run and these instructions are given here for a variety of scenarios, there is a version 1 and version 2 and all that but we just use this simple one line command, a multi-line command which is this, you go inside the folder and run this and you get the get the stack ready at the end of the exercise we again use terraform to say that you know what, please destroy the whole thing for me and it destroys the whole thing and you are done with your exercise, the reason we destroy is we don't want to waste money, it is expensive, 3 boxes it generates and if you keep them running it's going to cost you money for no reason, that's why we destroy and so this creation back in the cloud former tool has completed which is good, let's go see what that completion output says, so we will have the events showing all complete for this stack, they are all complete, at least the top level status share is complete we should be able to find the output in this segment here and so it says, here is the URL that you should visit to access your cloud former, so now what we have done use a custom specific template this is the special template use that to construct our tool this is the cloud former tool now ready, we will just go to that site to see what it is done and then you run that tool, this will look at our cloud and whatever we have constructed here it will dump a JSON, that's what we are interested in looking at and so that's what the output we want to see from this tool and so we will go visit this tool now back here, there is a link so I click, it asks me for a certificate, which I don't have honestly, because it never gave us, but we will say that you know what, I don't have a certificate so it says proceed and now it is going to ask me for a username and password so I will give it one username is A, password is A and I log in and now I am connected you may see the warning up on top, because I don't have a proper certificate that's why you are seeing that warning which is it can be avoided if you actually use a proper certificate on that machine, but I don't have it we did not give it to us this machine should have given us some direction on how do you access the certificate I did not bother so we just bypass that and use the login password we already know and that's why you see the red sign here at the very top we will talk about that specific thing which is a security thing, it is known as SSL certificates in another exercise which is a critical thing to run especially in the context of cloud communications, all communication needs to be HTTPS TLS, SSL by default, it needs to be like that, like you see in this site, like click it is completely SSL so end to end SSL, you can view the certificate and you see that the certificate is in place for this company and so that's what we will show, we will do these exercises for ourselves maybe in the next year or two so back here in our tool which is running, which is the cloud former tool we are going to say that you know what you, dear tool please look at my California region, because that's where I want you to look, because that's where I created all these things that I have here, is in California I have this special VPC that I have created and this machine is running right there and by the way this is the machine that I started and I see there is another machine what is that other thing this other thing is the cloud former itself this cloud former tool actually is running in this T2 medium machine in my cloud, except in the other VPC so not the cloud genius VPC but the other VPC so you see that here the VPC name let's go see VPC name should be VPC ID is E6FE that's the VPC ID for this particular machine that doesn't have a name but I will now label it as my cloud former tool CF tool, I'll just label it like that so I'll know what this machine is and this is the VPC ID number E6FE let's go see this test server that I have created it's in a different VPC V42 delta so bravo 42 delta that's a different VPC that's the newly constructed VPC that I have created for myself and this server belongs in that other VPC this tool is running in my default VPC which is this one so that's what this machine actually is running here in my cloud and it is saying that I am ready for you to use in this situation what I want to accomplish next is ask this tool to look at my cloud and give me a template that's what it's saying, create a template because I want you to look at this cloud and give me a template so go look at the California region please and create a template so it will now go and analyze the account it will actually look at what do I have in my Northern California region except it will exclude itself meaning the CF tool will not be counted it will count everything else, except the CF tool itself which also happens to be in the same Northern California region except in a different VPC but it is not going to count itself it will count itself out so in the cloud there is one machine here that is actually running this tool and this tool, although it is running right in the same cloud, it will not count itself, that's the point I want you to remember and having understood that piece let's see what this tool does for us, so we go back here in the tool and start the template information so we are basically building the template with the help of the tool and so we can say, introduction great, thank you, you can describe the template description here so this is my awesome cloud period, that's my description, continue and I say, you know DNS names you don't have any, so it doesn't find anything because we don't have any names specified, next are there VPCs that you want to take into account? yes, which one do you want to take into account? you want to take the one that is default VPC? this machine is in the default VPC, do you want to take into account this E6FE? I don't think so I don't want that E6FE, no not that one, so I said not this E6FE, not this one but look at that one, that's the newly constructed VPC I have, brother 42 delta so yeah, take everything from that VPC please and I say, continue and then it says you know in that VPC you seem to have some subnets so you want me to select the other two subnets? probably not you want to select the other gateway? probably not, because this gateway the 4 delta 7 belongs to the other subnet, let's go see and compare those two side by side, that will be easier, so I'm putting these things side by side, here is one side and here is the tool on the other side, so let's go see VPC and the two VPCs we have, we have E6FE which is the default VPC, I did not select that I selected B42 delta, which I did now inside that B42 delta VPC we have subnets in that subnet we have to select the one that I want which is this subnet, and that is the B, sorry 65 delta 1 already selected there on the left side here, already selected next internet gateway we have one there you don't actually need to go look, compare side by side as your constructing template, it will automatically select the right ones for you however I'm just showing these to you side by side so you can relate to what is going on you will see that this gateway got picked up, 64 alpha and it did not select the 4 delta 7 by itself, so we don't want that other one selected because it is the default one, our newly constructed gateway is this, that's what I want it's selected it also notices that you don't seem to have any customer gateways or a VPN gateway, are there any DFCP options, yes we do so there is a DFCP option set given to us by default which is selected already that one, next do you have any VPN connections no we don't, we'll have to create a VPN connection eventually and maybe another exercise we'll work with, is to the scenario essentially is that you have some cloud and it may be in a virtual private cloud with no way for you to actually go in and so there is no jump boxes no public subnet nothing, there may be situations like this where there is absolutely no way for you to get in in that case and actually in most cases, jump boxes are the easier way to go into a VPC but most large companies actually deploy VPN boxes inside their VPC and then logically place the data center, the building they may have, so this building that they have here and so people are working here wherever they are they will have a method of logically placing this entire VPC inside their building, logically and that concept is VPN, so you're basically logically there, if you are standing here physically you are logically standing here so you can just access any machine you like here, this one or that one that's the concept of virtual private networks many of you use it without thinking too much about it but the moment you are outside your company you try to go back to your corporate network, you have to VPN back in that's what we are talking about the same technology is used in the context of cloud where your entire VPC is brought inside your company logically, not physically logically, so that's what you're looking at in this example VPN connections, we don't have any but eventually we will create one not right now peering connections we don't have any so it did not find anything, we can proceed to the next step it says we have some network access control so we have them defined here automatically for us it picks up that correct one by itself we have some route tables defined that got picked up we continue forward and then we see that you have some network resources to worry about for example you have some cloud front distributions do you want to include the cloud front distributions in part of your design I don't think so these cloud front distributions are the ones that are deployed already for other projects that are happening in my account so they are not really related to what we are talking about, so we can skip all of them and that's what I intend to do is to not select any of these so basically unselect anything because these cloud fronts distribution endpoints are for different projects that are going on, so we are not including in this particular output that we want to construct this JSON output that I want to construct I'm not including that cloud front in there we proceed forward do you have any managed services, like things that Amazon will do for you is to manage an application stack for you using a tool called Beanstalk this is an auto scaling tool that they have provided to customers so you want to include that it will help you automatically scale out if you like, and that Beanstalk is here, so there is an elastic Beanstalk you can construct it grows and shrinks by itself that's why the name Beanstalk and if you have anything it just picks up here next, managed configuration so as you can see the terminology, the concept the content in that output that you will eventually see in this file is specifically tied to everything that Amazon does so here we have some elastic Beanstalk applications that we tried in the previous exercises there is nothing actually running there, so we are not going to select any of these guys, so we can maybe eventually delete them, so we can go back here and say my first Beanstalk applications, I have some saved configurations, no I have environments, no maybe I have in other regions maybe in Virginia, or maybe in Oregon somewhere you will find that there is this, so there are a bunch of things that I have tried in the past that shows up, so we'll not worry too much about your Beanstalk application versions, we proceed forward with the next step, and now it will ask you to select the compute instances that you created, so we have them at least one that we created by ourselves, and that is this EC2 machine that you know is this my test server and it is in the California region and it is running right there it is called the my test server machine that one that is number FA65 that's the same one you want to select that, yes I want to, because I want to include that in my illustration, in my design in my JSON, so I continue, I want to add some storage if I have any in this, I don't want to because I have not used any storage as such so I continue forward there are no storage configurations defined so I continue forward, no application services used, no simple queuing service, no simple name server topics, no kinesis streams, so we'll skip all of that because we're not actually using now we have a security group that is called out, so we should correctly select that security group which is already selected we do not have any RDS relational database security group, so we can skip that, similarly we go forward and identify any operational resources like scaling policies we have included, any monitoring activities that we included, any audit activities so this is scaling policies scaling schedules monitoring to monitor your cloud infrastructure, we have specific discussion on our monitoring as well at another day here we have audit trails as to who's accessing what, when and why those things we have not used in this example, but they'll get picked up in here, we continue forward and then we get a summary sheet in this summary we have basically selected one VPC one subnet one gateway, one DHCP option set, one network access control and two route tables and one instance and one security group that's it, simple, so continue forward and here is the JSON it constructed the JSON for you like that it's a long file, but not too long, it is still readable so I'm going to save that template, and first of all before I save I just copy that template out in my one in this notebook here, so I'll save, yeah, here is my template so let's go see first of all save this, so saving it and saving it as this file called untitled and that's where I should rename it to something useful .json once you name it as JSON it starts to color code appropriately atom editor will color code appropriately and will make it a little bit readable so that's the output you're looking at from this tool that dumps the JSON right in the screen for you as well as gives you a method, so you can save that template, so this will be something useful template file that will automatically save in a bucket, so this bucket is where it gets saved when you click the save button, it saves in your S3 bucket, so we'll go and look at our S3 here S3 and then look at that particular bucket which is saved in and that bucket is this one and in there you have this something useful template and that you can just download from the storage service if you like and that's how it saves and now you are ready with the template, at this moment we are practically done with this machine, we don't need it anymore we have constructed a template and saved it to S3 now what we need to do is go back to that service that started it, that service is this CloudFormation service which is this one and in here we have this particular machine running the CloudFormer tool with all these things create complete we used it we are done using it so we should be able to just go and delete the stack and that means the entire thing that got created as a consequence of this special template that constructed the CloudFormer will be destroyed this is very similar to what you saw in Terraform destroyer you delete the stack this will delete the Terraform itself we have our stack constructed in form of a JSON that looks like this so let's go read this what we did is basically this is described succinctly what we constructed in our cloud so there is a template format version and at the very bottom you will see this is my awesome cloud description and that's the description I wrote by hand that's the description for the JSON itself, if you go all the way to the top again, you will find that there is a resource defined here which is basically a virtual private cloud it specifies the cider block it says tenancy is default and enable DNS to true, enable DNS hostname to false the tag is CloudGenius, that's the value for the tag for this VPC so that's the name that you are preserving, the other things essentially I typed as I created that VPC with this cider with this name, then we keep going, you have created a subnet the subnet gets created with this cider block that I typed by hand in the browser, it is preserved information is preserved in this JSON availability zone, I selected usb1c and the name I gave to this subnet was public that is also preserved in the tag you keep going forward, I created a gateway, it is called IGW then I created it was given to me, DHCP option sets created to me, created for me with this domain name, automatically constructed, we just selected that we have some access control defined, that were also selected they were given to us we have a route table that we created in that route table we named it public we will also see further that that route table is associated with this particular VPC ID that is matching at the very top if you keep going deeper, you now see that there is an instance defined this instance is a machine that we started which is T2 micro it is using the Amazon machine image image ID, AMI which is the Ubuntu image that you know already that I used to start this particular instance, the key name I used was nilesh the tag I used for this machine was mytestserver if you keep scrolling down, it has a primary interface connected with the subnet ID and a private IP address 10.000.207 that was given to it based on the subnet definition of 10.000.24 that's what matches up with your 207 that's the private IP address we got for this network interface, for this machine that has a key called nilesh, a name called testserver and that's the machine ID or machine image ID T2 micro size if you keep scrolling down below you will find that there is a mention of a security group the security group itself was labelled the default default security group and here are the rules that we used, we said yeah, allow everything from anywhere egress, true let anybody go anywhere outside and now for the other rule which is tackle number 2 again the same, go anywhere you want allow going forward in the same sequence this association, the network access control association was given to us by default, we did not change anything these are just references given to us we also have a VPC gateway attachment this is the gateway ID number included we scroll down, we have a subnet route table association, we are basically saying that this particular subnet associates with this route table ID this subnet associates with this route table ID and here is the ID again and it says destination, you can go anywhere you want and you use the gateway called IGW the one we created use that gateway to go anywhere you like 0000 is anywhere on the internet you can go through that gateway that we constructed so this definition is the route definition we also have some DHCP options defined, which is given to us by default, we have also a security group for incoming route, so this is incoming traffic, ingress the name is SG default and the cider value, allow everything and this is again the outgoing, egress outgoing security group definition, allow everything, all protocols allow everything, so this is all protocols allow everything same thing here, so basically open security group, that's what we have constructed and this is my awesome cloud, that's the definition that we have, now we have saved it here, what I can do is go back to my S3, which is somewhere here so we'll go to that S3 bucket and you find that this particular location has that JSON file stored already, so I'll modify its properties and I'll say you know what, that file, I need to give everybody access so that other people can access this, so I'll say you know what everyone should be able to open and download and save excuse me once I save this for everybody to be able to open and download, I will then copy this link and give it to you so that once you click this link, it should open up the JSON and it is coming straight from my bucket to you once you have this bucket this JSON shared with your team members you can then go and delete the CloudFormer stack so it says go bye bye the stack deletion in progress which means that CloudFormer machine will get destroyed, meaning this machine is gone now, so you can close the window and back in our implementation what you're looking at is this ECTune section you will see that there is only one running instance because the other one is dead it is shutting down this is the CFTune going away and I have now managed to share my entire Cloud design to you in this link at the bottom so practically you can take this same link here from SlackChat and then construct exactly what I had in your account for your customer, it is basically repeatable now so you can go to CloudFormation and say that you know what, create a stack for me so create a stack I will give you a template so create a stack and use my template this one something useful that template I just gave you in Slack that you have you just specify it here and say next and give it a name and next and you choose key value pairs if you want, but go next and then estimate the cost this cost estimation will basically look at the template itself and find that it has a T2 micro and T2 micro costs me $12.45 which is understood once that is, by the way for free tier, it is first 12 months are free so you can ignore that cost practically, what I would like you to do is take that stack that I have given you and invoke it to construct exactly what I did in my account with you what we are really accomplishing with this exercise is to understand this concept and now that we have used this special template to construct the tool and the tool ran in the cloud and it studied whatever it had and then put this output that is stored in S3 and I gave you a link this link gets you the JSON that you need and that's how I am sharing this JSON to you sharing with you, now that you have this JSON and you want to repeat this entire exercise, it is fairly simple now you don't need to construct this cloud by hand because all you can do is just use this JSON and say you know what, hey cloud formation, take this JSON create a stack and that's what I will illustrate to you, in fact you should try this yourself on your side, you can just take this template that I have given to you and invoke it, now before I invoke, I want to clean up, so I will clean up first before I invoke, you should also invoke on your side and see how it works out, it should be fairly simple, just click the button to reproduce the exact cloud that I had in my illustration you will be able to reproduce the same thing no matter how complex the design layout can be, it will be able to handle that complexity because it is constructing very granular level of detail in everything that you have done for your cloud and you are basically capturing that level of detail inside in your JSON file so I just terminated my machine, I will go back in the VPC and get rid of that VPC also just to clean up the whole thing so here is my VPC that I am getting rid of so here are the two VPCs I will say my cloud genus VPC should go away, delete VPC and say yes delete, it says could not delete because some things are still in use so you have to wait for the gateway to be detached, we will go to the gateway and say you know what, you should detach from VPC and by the way, there is some volumes mapped, so yeah it needs to be, the machine needs to be disconnected, it must be detached first so say detach first please it says network map public addresses that machine is connected apparently, that is why it is not a link machine and so zero running instances both are terminated, any volumes probably not, no so we should be now able to understand why it is not letting us delete and that gateway I need to disassociate this gateway needs to detach from the VPC and it did, now I can delete that gateway, gone and now I can go back and delete the VPC also, so this VPC should go bye bye and it should now delete correctly and now I have the whole thing deleted, including my associated subnets, routes, gateways everything else that came along with it all of them, I am deleting them you should try this creation of my cloud in your account using the template, just like I have and I will do that exercise myself also, so what I will do now is take that template that I have provided to myself which is this and then use that template and go to my cloud former tool like here and say I will give you a JSON would you please create a stack for me, the complete stack that this JSON describes, and so here is the button, create a stack here is the button to provide that S3 link so this something useful template and I will proceed, give it a name and next and next and the cost I know already and next and here it goes creation in progress, so you have the whole thing constructed based on a templated method that's what we are really looking at, so it will finish in maybe a couple of minutes you will have basically reproduce the exact thing, I deleted the whole thing, you just saw me delete and recreating whatever I had before using this JSON which captures the essence of the cloud this JSON was captured through this method that I just used, and so now I have the essence captured, I can create a stack that's what I'm doing right now and this will basically reproduce the exact same cloud that I had before same thing as before and you should also see the same thing if you try on your side, you should be able to see that entire stack getting constructed, and it is happening right now for me now the while it does, we'll talk a little bit about Terraform again so this takes a little time to actually implement, so the differences between Terraform and cloud formation simply speaking cloud formation is specific to Amazon, and Terraform is general purpose the new problem the beauty of this tool is that it is able to look at your cloud and produce a JSON this guy doesn't have that capability as evolved as you would like it to have doesn't the problem with this as you know is tied to Amazon, so it works only with Amazon this difficulty that you have is to look at a cloud and construct a JSON is somewhat less than optimal than what you would like to have, but there are other methods that actually allow you to use this tool better than this tool because it lets you really define item by item what is needed in your cloud and use this in a editor in your atom editor for example in your atom editor you can specify what exactly you need and by the way there is an exercise for that and so in that exercise we will actually construct a fairly complex deployment but using Terraform tool, so that exercise is coming up at some point where you have the description of your cloud written down in some file it is not JSON it is some other format, it's the Terraform format, so it is a.tf file it's very simple to read, it's not complex it's human human readable code, like JSON was human readable as you saw, this is also readable by people, so it's very easy to read, you will have a.tf file that is what essentially captures the essence of your cloud and then you apply the good thing about this method is that it keeps track of what you are doing to the cloud and then if you reapply it will make changes, if you make changes here, make some change here, make more changes in the document in the design itself, it will apply that new without repeating the whole thing, so it is very intelligent in terms of understanding and that aspect itself to be able to incrementally develop whatever you are developing in your cloud makes it a whole lot more beneficial to use the Terraform tool, so you can begin very small very simple with a small simple.tf file and you apply and then you have that simple implementation in your cloud and this is neutral to any cloud, you can apply to any cloud and then you make it a little bit bigger, so say medium size complexity, you add more detail to your design in the .tf file, you add some level of detail, add more complexity and keep applying, your design becomes more complex and you keep increasing that small, medium, high you make it a really highly complex highly laid out configuration for your cloud and it takes shape of exactly what you would like to have in your implementation from the same method as you grow, you can keep applying as you go along and you improve the depth of information stored in your file that will basically get applied to your cloud with increasing complexity, so as you build things, I think this model is a whole lot better than this model that you're looking at right now, the JSON model, so this creation of my stack called Cloud Genius based on my template that I gave you here that looks like this in JSON is now complete what does that mean? It means that we don't have any outputs here because we did not define anything but we have a complete replica of whatever we started with right there with one running instance called my test server in using the key pair that I have in my machine image that I chose, same machine image the newly constructed VPC security group which basically allows complete access just like I had in my original implementation, complete access for outbound you will also see that it has constructed a virtual private cloud different from the default cloud, so the default VPC that we had is this one it constructed a new VPC called Cloud Genius in there it also constructed a gateway and it was attached to the Cloud Genius VPC it also has subnet called public, it also has a route table called public which has a route to anywhere on the internet using the gateway that we have created, I actually the template created this one, not me the template created the whole thing we also have subnet association just like we had for the public subnet, we have an internet gateway that got created and basically what you are looking at is a reproduction of the original implementation that I had to begin with but we constructed we basically took this route, what we did was we began with some cloud we used the CloudFormer tool we used a special template the template created a new box that starts the CloudFormer tool we run this tool, this tool looks at this cloud and produces a JSON this tool itself is a machine running here by the way, but that's ok, you can ignore that so it automatically ignores itself and then we use this JSON, so after we have that JSON ready, we delete the whole cloud and delete this one also we just keep this in S3 and share it with our team once we have that JSON ready, we can then use the tool again, and say you know what take my JSON, run it CloudFormation when you give it to CloudFormation, it produces the same cloud that you had before the same cloud here the same one exactly like you had before it reproduces based on this information captured which is the essence of this that reproduces here, that's what we essentially did in our layout now at this moment questions what questions do you have on what we just did any questions or any types so, you're welcome if you have any, we'll also take a short break is the only output you see of the CF, the JSON format or are there other formats in this tool, it is just JSON format file extension is called .template the file extension stored by default is called.template, but the file itself is JSON, like you see here something useful, I just rename it to JSON, it works that's the extension that Amazon has chosen to use a different extension, they call it.template but the file is JSON so we can rename it as JSON, it works the extension is irrelevant .extensions are all irrelevant and this is the entirety of one file that produces the whole thing and yes the format is JSON, they call it.template they call it whatever, I don't care but the format is JSON so, did anybody attempt to reproduce this stack on your side? you can do it later if you like but more importantly what I would like to do is this was a trivial exercise we would like to build a little bit more complex and a little bit more useful one that you will see when we do it, it is given somewhere in the links, you will find it in the flow in the cloud architecture segment so what I am doing right now is looking at the time we will take a short 5 minute break and we will resume at 6.55 maybe 6.56 something like that so 6.56 we resume and we take a short break until that time and I will in the meanwhile clean up this whole thing so I will gateway detached from the PC so it cannot be detached because the machine running there so easiest way to get rid of all this now that we have the whole thing is to actually go back to cloud formation and say you know what kill yourself so it says delete the stack and the whole thing goes away that's a whole lot simpler now to clean up so if you say delete the stack, it deletes the whole thing whatever it created it deletes I am walking away from the computer, I will be back in a minute I am walking away from the computer, I will be back in a minute I am walking away from the computer, I will be back in a minute I am walking away from the computer, I will be back in a minute I am walking away from the computer, I will be back in a minute Hey, guys. I'm back. Are the templates zone-specific? Yes, they are. They are. You can actually paste the error that you're running. I can understand what the error is. Very subtle issues that you occasionally run into in running those templates. And you try to reproduce that in a specific account that you will see sometimes some errors pop up. The errors should be visible to you in the Create Stack with this CloudFormation tool in that window. It will show you the errors. So if you see the errors, you should be able to paste them or just give me the results. So you can see what's going on. And I think it's a good idea to just see what you have. So I'm going to stop sharing my screen and let you share your screen, David, so that we will get a good perspective on what possible errors that you occasionally see. It is related typically to use of subnets and also the use of specific availability zones. Certain things may not be available. Certain things may not be available. That's the likely error I'm expecting you will have. So if you just click on Roll Back, it rolled back the whole thing. So if it goes an error, you click on the 84, More Events to Display. And that's where you should be able to find the errors themselves. So click on the 84 number. And it will expand. The number 84, 84, More Events to Display. So click on the More button. Yeah, that one. And you scroll down on the red items all the way to the bottom. That's where the first red item at the bottom is where the error is. All the way to the bottom, the first red color. So keep scrolling up a little bit and find the first red. That's where the error is. So here, it says Create Failed. The key pair, nilesh, does not exist. Obviously, it doesn't exist. How would you have that? That's the error. So that's one. We'll also go up a little bit and see that it failed to create the resource, because your key pair did not exist. I never gave it to you. That was the problem. So you can keep scrolling up and see what the error is. What else? That's the only error. So you can basically create a key pair by the name nilesh in your account and redo it, and it will work. The exact key pair combination that I have, you don't need to have that same key. But the name nilesh is what I used in the context of my cloud. So you want to reproduce the same name in your cloud by constructing a key in your Amazon account in the region you are working on. So you can do that very simply by letting me control your machine, and I will tell you where to do it. It should be simple. So you should go to the C2 console that you're running in. So you go to the key pair called key name. You go to the Oregon region. Is that where you're playing in Oregon? I guess so. We'll create a key pair. And this is the bad way of creating key pair, by the way. Nothing wrong with it. I just don't like it. But I'm just calling it out as the bad method of creating key pair, because you're asking somebody else to generate a private key and give it to you. This is a bad idea. It's a philosophical thing. It's not really bad, bad. It's just a comment I'm making that I don't like this method for a philosophical reason, is that somebody else generated a private key for you. This is a private key. They're giving it to you. And you can save it. And this is your private key that you saved, nilesh.m. But you know what? You did not generate this key. And that is the only reason I don't like this key pair generation method. That's the only reason I don't like it, because somebody else is generating a key pair. How do I know that they did not keep a copy? Well, I trust Amazon, so there is no reason to worry about. So it's OK. But as a general practice, I don't like this idea of people creating key pair for me. I would rather create my own private key. That's the only philosophical reason why I do not like this blue button. I would rather import a key pair. But it's OK. So having said, we should go back to here and all the way to the top and say, delete. And then Cloud Formation. And then Appliate. And then paste that link. And next. And then give it a name, David, and next. And here we go. And create. That's it. This should work. This should work this time, called David, because you have the key pair in place. That was something that I did not give you, which I should have, which is assume that the team that you're working with already knows. Actually, it should be specified. I would generally call this a usability issue in this tool, is that it should actually capture this information that the key pair name has to be Nilesh or some other name that you use. And then it will work. So I would classify this problem that you ran into as a usability problem. By the way, this user interface is completely different than what I see. What's going on? Are you seeing this interface or something else? What do I see? I don't see this at all, ever. This UI is completely different. This is what is going on in A-B testing. Maybe it is A-B testing. Maybe this is a new version of the entire console, the Amazon console that you're looking at. It's completely different than what I see. I don't know if everybody sees the same one. Only I have the different one, or I don't know what. But whatever it is, all these guys are sitting in the orange cube there on the top. And you can go to CloudFormation and play with it. But they would also roll back in progress. So something went wrong again. What went wrong? Let's go see the events. And let's scroll, scroll, scroll, all the way to the bottom and see what's going on. The issue is, did we get failed? Huh? Oh, you're in the California region. No, we created in the Oregon region. How idiotic. OK. That's the problem. So create a new stack. Stack name. So I'm going back to the Oregon. Sorry, the California region and say, you know what? You kill yourself, please. And so it says, kill myself. That's great. Back in Oregon region here, we should see that this one should actually succeed. This will succeed. We have the right region, and we have a key going there. So the key pair is, I hope, in the Oregon region, if I remember right. Is it not? Key pair. Village. Oregon. There are usability problems. Occasionally, you see them at times. But this tool, it is also died. Oh, well. Yeah. Yeah. It's human error, people error, all those errors coming to play. So this time, it's a different error. The error is probably associated with choice of subnet. It says the value 1c is availability zone is invalid. So I have seen this kind of error occasionally pop up. That's what I was talking at the very beginning. This is the error I anticipated. And it occasionally happens when you transfer the file from one account to another. It may be configured that your account is configured. Let me describe this. Your account seems to be configured in a fashion that says right here. It says in this blue segment. It says subnets can currently only be created in the following availability zones. US West 2 Charlie, 2 alpha, and 2 Bravo. And I have it in 1 Charlie. So we are using the 1 Charlie, which is the California stack in the Oregon region. That's the problem. So you should go to California. And in there, you have a key created first called NELESH. And so by the way, this is a good reminder that all of us, it is specific to a region. So you should, by the way, in Amazon, the design itself is such that they think that a region is an independent cloud. And that's what you are running into when you have a legacy-based system to work with the last seven or eight years of cloud that they have. They started off with just one region. And they thought that that region itself is every cloud. So you have this key pair has to be generated in every. I'm going to overwrite it. And so the idea is that you have to have a key pair in every region. Every region is treated as a separate cloud, which is a challenge in case of Amazon. That's the design that they have. You cannot do anything about it, which means we'll do it one more time. And this time should work. And we will then give it a name. This should work. And go next. And go next and create. So when this should work, works. In the meanwhile, we'll go to the Oregon region. By the way, there's a new Ohio region now. And in Oregon region, we want to kill the other stack that is kind of halfway incomplete. And that should die. So we are going to Oregon and kill the other. Come on. This guy needs to die. So we'll say go, bye-bye. And then go to our California. And hope this one succeeds this time, because it should succeed. If you see the errors, the best way to go is look at the event log. It begins at the bottom and goes all the way to the top. That's what you go and read like this. So you basically begin at the very bottom. This is where the user initiates it. And then you scroll all the way to the top looking for errors. There should not be any this time. And then as a consequence, you will have a machine running in your California region here. And so this region is this one that got created. This is your existing machine I don't want to mess with. I will leave that one alone. I think this was existing before. And I suspect you are doing something with it. This is the machine that just got created. It is running. It is also in your newly constructed VPC, which is going to also be there as soon as you see it. You should see that there are two VPCs. One of them is called Cloud Genius. And this is the one that you have created. This one is the old one that you had to begin with. It's the default VPC. And I'm going to just name it like that. And so this is the one that you created just now with the tool. And the tool should say that it succeeded, which means this tool should report success by now. And yes, it does, which means we should be able to connect to this machine in the California region using our key that is stored on your computer on the desktop. So this computer, you should be able to connect on that public IP address just fine using the nilesh.pem key that you have on your desktop. So this is the public IP address. That's what we reproduced. So you successfully did that. The gotchas that you saw where I'm going to take control away and show my screen now. And I am going to project my screen here, share my screen. And what the gotchas that we ran into primarily is because of a couple of things. Amazon thinks that every region is a separate cloud. A region is a separate cloud. That's the thinking that they have, which is that every region is a separate cloud. Which is probably not very user friendly, but that's what they are designed to operate like. So it's a separate cloud, which means generate a key pair in Oregon that is not available to you in California. How nice is that? And then you design something here that you try to play here. It will not work because all the references are here. So it says US, West, 1, C. And you try to use this design reference in here, it fails. That's one of the two issues that we saw. We created a key pair here, and we tried to use this design from this cloud, California cloud, to the Oregon cloud. It fails. It is expected. Because they treat like a separate cloud. In fact, the reason why I think they should rebrand it, but again, who am I to decide these things, is that they should just call their regions as separate clouds. So instead of confusing everybody the heck out of Oregon, they call it Oregon as a region. And we should call it the Oregon cloud. We just call it like that. It reduces confusion somewhat. So people will understand they're separate clouds. But then at the top level, the branding, positioning, all of that get destroyed because the cloud becomes small. And that's the problem that Amazon has. But just saying that this is what you're running into. OK. Any other questions on this cloud formation thingy before we switch back to our architecture discussions? So I guess they're not questions, not many questions. So back in here, we have this PDF that describes this illustration that you saw. And along with this PDF description, this illustration, comes this template. Where is the template? OK, here. Here is the template. This template, as you saw, comes down on the desktop. And I will open it up in my editor. So this editor terminal. And here is the JSON that basically shows you how to deploy a link server for a large company. This JSON basically is a JSON representation of the picture. Which one? This one. This picture. If you want to implement link, you will run this JSON. And it will do exactly what the picture says. So it's very easy to reproduce this. That's the concept that Amazon has tried to evolve. And it is doing a decent job in what it has done. So you have the architecture description. And in English language, they describe what is being built. And also, they give you the ready-made template that just does it for you. So you click, click, click, and it's done. It is done. It implements the whole design. You can always modify and configure, reconfigure, and re-layout certain elements. So let's, for example, in the context of this link discussion, if you have decided that you want to run link server, 2013, for your company. And so you come up with that. You land on that page. And you find that PDF file. And then through that PDF file, you look at that beautiful picture illustration. And you also find this JSON template. OK, great. I got it. So then you take that JSON. And then you apply it to your cloud. And you get a real implementation of link. Link running for you right here. OK, sweet. But then I want to modify some things. So how do I do that? There are two ways to doing it. One of them is to understand this JSON and modify it. Change, change. You want to change whatever here? You change it here. Another way to do that is to actually change here in the real cloud. Whatever you want to make changes here, just go do it. On the console, go to command line, go use Chef, go use Ansible, go use whatever you want. Make the change. The real cloud changes will actually happen. However, now you have a difference between this and this. They don't actually relate to each other because you just changed only this. You modified something here. This JSON doesn't actually have that modification. So you need to take a dump this way to produce another JSON. You call it JSON Prime. And this is the new JSON that actually includes this red color change that you made in the real cloud. And you need to track your changes so that you know what you're doing, at least in the initial design step. You need to exactly track what changes are you making in your real cloud. And you put those inside your JSON file, maybe a different version of JSON file. Make sure to track the change. Otherwise, you will have no idea of what you did. Ideally, you don't make changes here at all. I don't like this. I like to make changes here. So I can put this in a Git repository, and then just play, and then make more changes. In this JSON that I now have, JSON Prime Prime, and I click. And when I apply, I get the new design here. And then I make more changes. And then I apply. That's how I would actually operate. I don't really, really like making changes directly to the cloud, because it's a bad idea. A bad idea, because you have to remember to bring the changes back to your Git repository, and then keep track of it. The tracking happens automatically the moment you make changes in your JSON, and then keep applying as you make modifications. So your JSON iterations are basically kept tracked in a Git repository in the form of various Git commits. You make a mistake, so you say, oops. And then you go back to this one, and apply. And you're back in business. And this oops, this oops went away. You roll back, and you're back in business. That's the ability you retain if you don't make changes here. And I don't really recommend making changes to the real cloud by hand. Don't do that. It helps you in the longer run. As you keep modifications, keep modifying, keep tracking, keep iterating, keep improving, make sure that you are addressing what the customer needs, you retain the ability to go back in time when you will have this oops. By the way, this oops will happen. I can guarantee that. It is a guarantee to happen. You know this. Mistakes will happen. So the fact that you have this method of going back in time to another Git commit previously, and you can then apply that one, brings you back in business quickly. If you don't have that discipline method of tracking changes, you can never go back quickly. It will take a lot of time, a lot of effort, a lot of wasted money, time, and customers. You don't want to lose them. You want to quickly go back as quickly as you can to be back in business so the customer is happy, and that is possible only if you put this discipline in place to not mess with the real cloud. Only mess with your files. Put them in version control and go operating on that. That was this design. Let's go look at some other implementation. This was a very simple implementation of just using communication collaboration software, using link software. That's what we looked at. That was this. If you want to run Exchange Server 2013, here is a diagram. Simple, not super complex. Let's go see what the diagram looks like here. It's a two-dimensional representation of the same thing, basically conceptually very, very similar, except the only unique difference between this implementation and the previous implementation that you saw was that this is an Exchange Server as opposed to a link server. It also has the Active Directory and Active Directory, so pretty similar to what the other example was. Let's put them side by side and you will understand what I'm talking about. So here we go, click. On this side we have Exchange Server, and on this side we have Link Server. You can see it's very similar, except one is a 2D diagram and one is a 3D, but that's the only difference. Here you have Exchange Server, and here this blue box is your Link Server. Pretty much similarly out, pretty much similar. 2D versus 3D is another difference, but all of it, two EPCs, two different availability zones, and you have your Exchange Server running with an Active Directory in one zone, another Exchange Server and another Active Directory in another zone, and you have a gateway and you have users. That's the design, very simple to implement, and the key thing to remember is to capture this design in a Git repository so that you can reproduce this thing. Let's go look at some of them more, some more. So Windows PowerShell DSC. This DSC is basically like your Chef Server. This idea that you are looking at here, Windows PowerShell DSC, think of it as your Chef Server, although it is not Chef, it is DSC. So it is describing how to build a Windows-based PowerShell DSC pull or push server environment on AWS using ADM DevO Desktop Gateway. Click on the picture, you'll see the picture illustration. It is very, very similar again, Poggy, and so you will have same thing, except there is a similar illustration like you saw in the picture over here with link or exchange, very similar, except you have two extra. So you have a web here, web server here, web server here, in addition to the DSC server here. So these are, think of it as your Chef Servers, these two, also known as, in this context, known as the DSC Servers. You have a web server tier, separate additional tier other than what you had in this illustration. You have an active directory scenario, you have an active directory scenario, the same thing that you have here, and a NAT and a remote desktop gateway. So the remote desktop gateway basically lets you connect through terminal services session like MSTSE, for example. The Microsoft people will know this thing called remote. This tool, that you can use this tool to connect to any remote computer, and so you could connect to these boxes internally if you have a RDGW. If this RDGW is unique to Microsoft environments, it lets you connect through remote desktop that you probably know already. And so closing this window back to looking at some other illustrations, some other designs in the context of what Amazon gives as the ready-made templates for a variety of applications that get deployed in the example of, here is another one, SharePoint, SharePoint Server, slightly more complex as we go along. So it will increase in complexity as we look at pictures. Here is a quick start template, that's a JSON file. The template itself is here. So you click, and you download a template, open it up, and it shows up, and add a editor right here. Here is another complex, fairly long SharePoint deployment, and you want to deploy SharePoint in the cloud, you run the template, like this, click, and it goes. As simple as that. Not that I want to do it, so I'm skipping that. But if you look at the picture illustration, you will see what exactly gets deployed if you run that scenario. You will have something like, come on, yeah. So a lot more tiers, web tier, application tier, database tier, and active directory. So four items right there. And repeat the same thing in another availability zone. So you have two of the same, so it gives you redundancy, higher availability, and you load balance across the two. And your users can then use your SharePoint deployment across these two availability zones, which basically are these four tiers. So if you think about those from a tier perspective, at the front end, there is a NAT gateway, which is basically like a NAT device that lets these boxes go outside if they want to. And the remote desktop gateway, or DGW, which is specific to Microsoft. This is like your junk box. This is like your SSH junk box that we had in Linux. RDGW will help you connect like this. Like this. So basically you put in a new entry for the connection name and then put a PC name, that's the machine name, inside. And then you provide login password, and you open up a window that will let you play with that window in any of these boxes internally. So application server, front end server, primary replica, and DNS. So active directory tier there. So four tiers here for SharePoint deployment. Basically you run those SharePoint servers, and SQL server instances, and the domain controller, these guys on four different boxes in these four tiers and connect them appropriately, and that deployment happens fairly smoothly. Like just click this button, then you click that button, which is the launch button, and this will deploy a complete SharePoint for you. You can do that if you're interested. I'm not sure if you are interested in looking at this, but if it's the case, I would like to show you that. So anytime you want to do some specific experiment, please tell me. There is a lot more content than we can actually fit in the entire program, so that's why I should say that. I should say this, that you know what, if you think that we should cover something specific that is of interest to you, please bring it up, and I will actually cover it. Even if I am not plying for it, I will still cover it. Otherwise, I will actually cover core aspects that we have discussed already. I will just give you an overview of what we have to cover. I want to cover all these architectural diagrams that we have from Amazon as well as Microsoft. So they are basically good reference architecture, so it gives you a good perspective on a variety of business scenarios, how will we put that in the context of real deployment that will help you understand what architectural best practices out there are available. Then we will cover aspects of security, which will involve TLS, involve SSL certificates, involve basically HTTPS, that's the dominant, probably the only way that I recommend that going forward we should use, which has to have this element, the secure element. We'll do some exercises in this context. We also do exercises on VPN, so to establish virtual presence, which is basically logically placing, logically moving you or your machine, or your machines to another location. So logically you place yourself in another place. That's what a VPN does. We'll talk about that. We'll also discuss monitoring of your cloud, so how will we monitor performance characteristics where we are falling short of available resources. You have to throw some money, throw some resource on making sure that it actually performs the way we would like to have it perform. So the aspect of performance and monitoring, those aspects we'd like to cover. What are the best practices in the industry in terms of ability to monitor your own cloud using your own tools? So there are open source tools that help you monitor. There are some commercial tools that are also useful that you can use to monitor. So that's the monitoring aspect we would like to cover. And we will have Terraform exercise, another one. This Terraform exercise that we will do is going to be reproducing the entire exercise that we did in the very first segment, cloud technologies. Cloud technologies segment. We had a section dedicated on the virtual private cloud. The entire thing we built, quite a bit. So go back there and you will find it here. You will find it in this module that scrolls down. I think it's number 10, if I remember right. But I'm not sure. I keep changing these numbers occasionally. But yeah, this section, data center in the cloud, module number 11, where you begin doing a whole lot of complexity and understanding the exact nuts and bolts of a VPC that you create from scratch. And there is this series, that you saw, you've done it before. You did it by hand, you did it manually. You'll do it through Terraform. The same thing, Terraform. So this is module number 11. And that is in this segment. We did it by hand. You would like to do the same thing this time through Terraform. And that is in module 11. It is in module, what's the module number? The module number is, come on, eight. So it's a fairly short exercise. Fairly short, it just magically happens. And I'll walk you through this as to how you can. Basically, what you did before, except using Terraform. That's this exercise we'll have. The exercise is already there, so you can just read it if you like. And I'll walk you through, because although it might look short, there are a lot more steps to actually understand. So it's just one page, that's it. But there are things to understand here that I will walk you through as to how do you automate a cloud, a complete architecture deployment that you will construct. And then you take basically the same scenario we discussed early on, which is you put things in a Git repository, and you keep iterating on it, and you keep applying that design to your cloud, and you implement whatever you want, and you keep iterating. Except we can do this in JSON and use the CloudFormation tool, or we can use the tf files for Terraform and use the Terraform method. Any of these methods will work just fine as long as you're putting that in a Git repository, and you can apply your designs to... The benefit of using Terraform is it can be any cloud. And with the CloudFormation, it is JSON, sorry, CloudFormation, it is AWS only. That's the only challenge. So it's specific to Amazon, this is any cloud. So that's what this exercise is. We'll do that one. We'll have some specific things with Docker. And what I'm really trying to communicate to you is there is a lot more content than actually fits in the remaining five more days. And so I would like you to actually identify things that are of your interest that you want me to cover that I will. But you need to tell me, that hey, I'm more interested in this versus that. And you saw this list, I'm writing it down here. That's what I intend to cover in the upcoming days. And we have more content than it actually fits in the program because the world is huge. Not everything can be accommodated in a program like this. So there are things that you want us to cover, we will do that, even if I have not planned for it. Because I will actually find how to do it and show it to you. I have no problem doing that. If you have an idea, bring it up. Talk to me personally, one-on-one, Slack chat, group chat, whatever it was. So the only thing that I was planning on asking you is if you have planned anything other than deploying WordPress, anything a little bit more complex than... About Rails? Sure, yeah. A Docker, we'll do that. You have an exercise. In fact, I just did that for myself. So I can just repurpose it. I have constructed a new Rails application here that I can see. And so it basically, it is running Rails and it does something, right? I'll show you how it works. And can we do this using Chef, building Chef recipes? One thing that I want to practice is to build a couple more recipes that gives a cookbook, sorry. So specific on Chef, is that the idea? Or you want to use Docker? So Docker is a whole lot easier. That's why I'm suggesting you go Docker. So Docker is a whole lot easier. It is a whole lot easier, that's why I'm suggesting you go Docker route. It is a whole lot easier than using this tool. This tool is fairly complex. You know this, you've seen it. Yes. Yeah, it is fairly complex. Using the Docker way is the new way and it is a whole lot simpler. Okay. That's why I'm suggesting you go that route. Sure. You've seen me do like a multi-tier architecture in like one file. Try that, try doing that with Chef. You'll fully have hair. I don't have any hair left on my head. Same here. So yeah, I remember this. So remind me if I miss on this, I will make sure to cover the Rails aspect on Docker. Okay. Is this example application that I have in the grid, the event kit is a software that I'm running here. So that's the event kit software that I'm running in that site and it is available on GitHub and we can see how to run Rails applications on Docker, which is different from WordPress. WordPress is of course WordPress, but this is custom applications. Right. So you need to understand how to customize. WordPress is standardized, so this is custom. So this tool, I should paste a link for you, is this exact kit, which is a Rails application, which is what I will be showing you in Docker containers. The container itself is also open source if I remember, or maybe I did not open source it yet. Let me see if I have it open source or not. I do have it. I have the OS private. So yeah, there is some private information that I want to make sure that I clean it up before I give it public, but that project is private right now. I'll extract some useful information out of that and then make it public. And you should be able to understand how to run that Rails example. This is an application. It runs like this, and gives you some settings and some things like that. So it does what it does, so that's the application. It is custom written by this company, and so it basically tells me, so let me tell you what it does. This tells who is clicking on my emails. Basically, that's what it says. That's what it does. You see what I'm doing? So, who is clicking on my email? That's what I want to find out. So there are lots of people that subscribe to CloudGMS emails. Out of them, certain people actively are interested in what I'm going to say or do, and that's these people. So you can download them as a CSV, and this is live information coming from people. It's a list of things that I just downloaded, and I can see in that file who are these people that are clicking. So I can know that this guy, Chandravan Empati, or Avani Indra Verma, or this guy, Nilesh himself. These people are clicking, Shiv Ganesan, Kelly. Kelly, you know, from Microsoft. These people are actively clicking on the messages that I send. And so that's how I'm generating that insight into what users are doing. So it's a data analysis application for me. And so that is running in a Rails application in a Docker container. That's what this exercise would be. Other ideas you want to cover, please bring them up, please bring them on. And you don't have to do right away, but as and when you get time, please bring it up so that I can actually address it. Like Ram said, Rails, or custom application, not standardized applications. So WP is very, very standardized. It is, you know, just well-known. How to do it is very well-known. It's systematic, you just follow along, it happens. When you're running with random custom-created applications, you have your variables to deal with, and you have to understand how to make it happen. So that nuance is what Ram is looking to accomplish, and we will take care of that. Please bring up other ideas if you have them. But that's what you're looking at coming down the road. So top level. Reference architectures, we'll have discussions around it. Security aspects, VPN aspects, monitoring and performance aspects, and Terraform exercise in depth, which actually is a very short exercise, but it needs a lot of understanding of what's going on. We have done that exercise ourselves by hand. We will automate it, and we'll do Rails. Please bring up other ideas if you have any. Okay, let's continue. So we have a few minutes still remaining. I want to actually cover a couple of illustrations that Amazon has put together, which is on the AWS architecture. There are lots and lots of content available. And here is more. So here is, by the way, how to deploy a Chef server. Here is, by the way, how to deploy a Chef server. So this is the deployment of your own Chef server if you want. And I have our own method of deploying Chef server, which is using Chef to deploy Chef. It's very meta, but this uses a CloudFormation template. So here is the template. And you bring it down, and you can see that the template is actually fairly simple. And it has only 113 lines, because the design itself is like a simple box. So if you go look at the box itself, which is, where is the box? Yeah. And so the illustration is actually fairly simple for a Chef server, so it should not be, yeah, there it is. So let's go make it big. You just have one availability zone, and you have a Chef server right there, this one box. You have a Chef workstation, there we go. So these guys are smart. They're putting the workstation right in the cloud. And then they're managing a node in the cloud again. It's actually very, very simple. In our example that we have been using Chef with, this workstation was on your laptop. It is outside the cloud, it is with you. So it is a cloud-generated workstation. That's what we used in our case. So we used the Chef workstation that these guys, what they have done is moved the workstation itself in the cloud, which is fine. But in our case of the cloud-generated program, we want you to have the workstation with you, so you can actually do other things with it. That's why we have it outside. In our examples, our node was not limited to this availability zone. In fact, our design was that you can have this node created anywhere you like whatsoever. Like you can have it in DigitalOcean, in Google Cloud, Azure Cloud, God knows what cloud, Microsoft Cloud, Amazon Cloud. Bring your own cloud, put it near the garage. That's the node, that's this box. And that can be anywhere you want to play the cookbook. That's what it plays. The node will play that cookbook for you. The Chef workstation is actually your cloud-generated workstation that you have on your laptop and the Chef server is like a box. That's it, that's it. Just a box running Chef server. And that's this illustration. It constructs that through this diagram, which looks like this in your JSON. If you now see this, it's fairly simple to read. Not too long actually, so I'll just adjust my view, to wrap and then read. So this is a template that deploy a Chef server and you must obtain the Chef server AMI from the marketplace before you launch this step. So you must subscribe to an AMI, which is a subscription available. You have to get that subscription to Chef server through that cloud service that they might charge for it, I suspect. So let's go see if there is a marketplace. There's three marketplaces, go to EC2 and look at the marketplace. So where it is, marketplace. And here you say, Chef. And so there it is, first five nodes are free. It's a commercial service. You may or may not want this. Some companies want to pay for it and that's great. Nothing wrong with it. So for that situation, just select this, five nodes are free. However, if you are really into open source, you don't want to use this image. This is going to limit you and it costs you money. Why? If you know how to use open source, you don't go there, you go here. You go to GitHub and use this Chef server and it works. And it works. You know, here's how it works. You can go to cloud genius site, go to the DevOps section, identify the cloud DevOps portion. And there is an exercise for how to run this open source, Chef server, right? In any cloud you like. And so here it is, setting Chef server. And here is, build a Chef server for your organization, all from scratch. And this is basically, let's go read a couple of lines here. So let's see your own Chef server. And in that exercise, you probably have not done this yet, but essentially what you're doing is, using Chef solo, setting up, using Chef solo to install Chef server. So all of it open source end to end, using the cloud genius workstation, using a gem file and it just boom, boom, boom does it. Pretty much gets to you running this Chef server without having to get into commercial applications. That's this application actually is open source under the hood, you know it. And here is the source code. So use it, or otherwise just go subscribe. Both of these options are available. This design that they illustrate here is commercial. So the Chef server is actually commercial server that you have to pay for, that you understand. Similar to the Chef server, there is an Ansible server called Ansible Tower, which is another commercial product available in this illustration. And the diagram should be very simple again. It should be looking like, where is the illustration? There we have, simple. Here, there is a T2, which is the Ansible tower, which is the Ansible server on the back end. This Ansible client is basically the cloud genius workstation. That's it. This client, Ansible client, is your workstation that you have been using all along. That same one you can continue to use. All you need is a box in the cloud that operates like a tower. Like Ansible tower is the analogous thing to Chef server in Ansible. And it is a commercial application. Fair enough. So just so you understand that some of these are commercial, like here, Puppet, which is a similar tool to Chef, and DSC, for example. And so, if you keep scrolling down, there are lots of applications that are given here for you. You want to run Docker data center. And there is this layout illustration. Again, these guys make it what they have in mind. And it is based on the illustration that you see here. That's how they would like to construct two different availability zones and construct out your Docker deployments. And it really boils down to what you would like to see for your design, maybe very different from what they think should be the design. And so, it really is up to you how you want to construct a Docker-based data center. And you don't necessarily have to follow their instructions. Just, it's a good idea to read and understand what they are accomplishing. So in this example, what they are doing is basically using Elastic Load Balancer. If you look at just one of the availability zones, there are two of them, one is here, one there. So just look at the two availability zones, look at one of them. You have a bunch of instances here that will actually run Docker. And these Elastic Load Balancing, this particular activity of balancing your load will happen through the Elastic Load Balancer service that Amazon provides. So it's a paid-for service. You can construct all these things, even the load balancing itself in a Docker container and put that on an EC2 instance. In that case, you don't have to use this service. Because as you can manage your own load balancing, which you can, all inside Docker, you don't need that level of complexity unnecessarily. And so sometimes you will find that commercial entities like this one, for example, Amazon or Microsoft, they will like to push their agenda. Nothing wrong with that. Nothing absolutely wrong with pushing your agenda. It's their cloud and it's their site. And they will like to sell their services. So nothing wrong with their approach. But what I'm trying to point out is sometimes these illustrations can actually lead you to a design that suits Amazon better. That's what I'm saying. So they might have some illustrations here that may attempt to benefit Amazon in addition to benefiting you. Whereas if you think of it from a neutral objective perspective, there may be cases in which you may find that certain services that they may have included in a design does not really need to use the services that they are indicating that you should use. For example, was this PDF, Docker data center PDF, that one, that shows that you need a load balancing service? Not necessarily. If you know how to construct your own load balancing service, you don't need them. That's what I'm saying. So occasionally you will find in this illustration that they will like to push their product, which is totally understandable. It's a commercial entity at the end of the game. So just be aware. And you saw this in this Chef example. They're trying to use the official Chef server AMI that supports five nodes for free, up to 50 nodes and all that good stuff. But under the hood, they make you basically hint at buying that commercial entity for which there is an open source solution. So if you go look at this quick start template that you saw before here, they want you to buy this AMI in the marketplace. There is no need to. There's totally nothing wrong with buying it if you're interested in buying, nothing wrong. But just remember that there's an alternative solution available called open source. And parts of it, you need to understand how to make it run and read the license terms and conditions before you actually make use of, but then otherwise there is no issue. Just make sure you consult your attorney. That's what I'm basically, when it comes to open source, just read the language, read the contract that they might have. Every project, as you will probably know, has a license file. You should read that, make sure to read that. So this project, the Chef server project, for example, has a license file. And so you should read that license. And in this example, you will see that it is Apache license. It basically means you do whatever you like. Yeah, that's what it boils down to. So it is open source in the true sense. So there's no problem in using this comfortably. Just go and use Apache licenses open to everybody. Of course, consult your attorney. That's what you should really look for this license file. It tells you how to use this product. If you are able to legally comply with this license, go ahead and party. And you don't have to pay anybody if you know how to use it. Otherwise, the other option is buy the server from these guys, which is fine. Other illustrations. So if you want to run, say, Bitbucket, which is a GitHub-like software, you want to deploy Bitbucket yourself, this is how you do. Now, after this list, there are some better illustrations of not application-specific, but business-scenario-specific illustrations. That is what I intend to cover. There are actually quite a bit of them. I think we covered only one up until now out of these illustrations. We discussed only this one in depth in the context of Netflix. If you remember this picture, you probably do. And so you remember this picture of the black units here, how they operate, and how they are looking at the ingestion pipeline, upload web servers, and the white-colored units here, which are the media distribution servers, how the media actually gets distributed to the end user. We discussed this a little bit in depth. Similar to that, for a variety of different types of business use cases, there are these illustrations that I think we should cover. And I will cover them beginning from this point, web application hosting. And we go all the way up to the bottom. There are some very interesting applications that real businesses deploy in a variety of contexts. We will make sure to at least give you a perspective on how a variety of different businesses, not just technology businesses, but other types of businesses, apply these concepts to using the cloud for their business applications. This is, again, not application-specific. All these charts are specific to a business usage as opposed to specific to applications. So up on top here, all the way up above, you see that these are talking about specific applications, like how do you run CloudEra, Hadoop? How do you run MongoDB? How do you run SAP HANA? And they are specific to applications. Whereas in here, it is references for general purpose business illustrations to understand a variety of scenarios, how businesses would use without any specific reference to a given application. So that's what we will discuss next time when we meet. Any questions up until now? And if you have any ideas on what you want us to cover, please bring them up in Slack chat, one-on-one chat, phone calls, meetings, whatever you like. Does that make sense? Was it useful? Yes. OK. So we will continue meeting our schedule on the 26th. We have a session at 5 o'clock. That's when we meet next. So I will see you at that time. Thank you, guys. Thank you. See you later.