Hello, how are you, David? Hey, doing good. How's it going? Pretty good, sir. Just starting to share my screen. And you saw that, I guess, this vulnerability that was disclosed today. Yeah, that's crazy. Yeah, it is crazy. They describe how to hack it. It's exactly described right there. Pretty scary. And they then go on to say that Chrome is good and all that. This is marketing. But yeah, the problem is, yeah, they gave only seven days notice, and then they just published. That's the thing. If you look at the vulnerability itself, the actual database that is there, the CVE database, that doesn't describe what the vulnerability is. It is reserved. The description is reserved. And that is reserved by Microsoft, because Microsoft needed more time, apparently. That's the challenge. Let's see who else is here. Mark. Hey, Mark, how are you doing? Good, how are you? Yeah, we are talking about the CVE that just Google published. You know that, right? You probably know, or internally. I don't know. This just happened. You know, Google published this document. Oh, yes, I did read that. Yeah, Friday or whatever. Yeah, that was today. Today is it? Yesterday. So they disclosed it yesterday, and that is just unfortunate. That's what it was. No, that was, let's hope we get a fix out. Yep, fix is coming, but it's not there. And the information got disclosed as to how to hack. That is the bad part. That's terrible, yeah. How to hack it. Which is lame. Yeah, lame, lame, lame. OK. Yeah, that's how competition is, I guess, these days. And so they're using security vulnerabilities and all of those things to push Chrome browser on top of Windows users. That's what they basically are doing from a business perspective, is basically highlighting the fact that Windows is bad. But you know what? Everybody can be bad. It's not about one product versus the other. It's just the technology, it will break. Things will break. I mean, come on. Right, because Android doesn't have malware, right? Exactly, exactly. Everybody will have a malware. Some problems, some there, or the other, right? Yeah, everybody runs into problems. What's the big deal? Right. The big deal is to behave properly on the internet, basically, and not disclose. I mean, come on. Give a couple more days if needed. And Microsoft and Windows is large. It's just the fact that it's not just some ordinary, small, simple library on GitHub that needs seven days. It is a large product. And so we have to understand the implications of what does that mean, and all those things come into play. You can just not disclose things like this. That's what Microsoft is complaining about. But on the other hand, I have seen the same comment, like, got to love Windows 10, David says. And then there are other people on Facebook saying different things. But the bottom line is that things can go bad, and they will, unfortunately, will go bad at the most difficult time. That's the unfortunate state of affairs. Security is a critical thing. Everybody is impacted. And so we just have to prepare for it. Is prepare for Mr. Murphy. And yes, I'm just generally saying Murphy is a part of security game. But this guy strikes at odd times. Yeah. So let's see who else is there online yet. We have five people. Diego, Mark, Ram, David. Hey, guys. I'm just going there immediately. So I guess now that I remember one more thing, some of you have had difficulty in logging in or maybe trying to remember your password and other things, like what was the email address I used with Cloud Genius. So when you try to log on to this site, you will have this issue about some of you did have. Because I said that I couldn't make it because of my training. So what I was saying in that context is that if any of you need to change the email that you use, you want to change, you need to change. Hey, Diego. Diego, we can hear you. Hold on, hold on. Let me just mute a couple of people. OK. Muted. So what I was saying is that if any of you need a change in the email address used with Cloud Genius, please tell me. I can modify that. You don't have to use that cloudgenie.us email address. That is just one more email address that you, I don't want to force it on you. That's what I'm saying. You don't have to use this email address. We can change it. If you need a change, you want to have a preferred email address which is different, please tell me so. I will make that change so that you don't have to remember one more email address or one more different login. And having said this, just tell me on Slack chat and I will modify that and come back, give you a new password and give your preferred email with a new password. We'll do that. Having said, there is also one more thing that I want to point out, which is please, please remember two-factor authentication and strong passwords. And then when you use those things, strong passwords, it is important to put all these passwords, especially for your bank. I'm more worried about banking and financial institutions and brokerage houses. They don't seem to enforce this. So ask your bank to enforce strong passwords and strong security with two-factor authentication and start using them, especially with banks and primary email that you have and things like social media if you are big into that. But focus here, enable two-factor, and now that you will accumulate complex, very difficult to remember strong passwords in a large number of them. I have something like 1,700 or 1,800 of them. And so that becomes difficult to remember, and everybody will have their own unique passwords. So this is one solution that I use that actually creates. It's called One Password, by the way, this thing that I use. It creates a vault that vault is stored. It is stored locally, and it is encrypted, and it doesn't go to the cloud unless you want it to go to the cloud. In that case, it's a different story. So this is the solution that I use. There is an open source solution also available if you are into open source and you want to use an open source product. This thing is approved by Boeing in your company. It is available internally in your software depot, whatever you call it. And this key pass is a good product, open source, and also approved by Boeing for use internally. So you should use that as another key store if you want. And there is a free product available called Last Pass. For those of you who would like to have a vault on the internet, this is one solution. They keep their vault online, and it is encrypted and all that good stuff. That's where you store your passwords, so you don't have to worry about remembering them. I know there are some. You can always have a debate about where the vault gets stored. Is it OK to store it online? Things like that. You can always debate and discuss these things. But choose a solution and enable a vault. Store it locally. Store it in the cloud, your choice. But please use these things, like two factor, like I mentioned in this description. Use that specifically for your banking and financial institutions and also your primary e-mail. Let's keep talking a couple more things about security and about access controls. We keep hearing about security issues all along. So what I want to be able to do is make sure that I address a couple of aspects regarding security, specifically in the cloud. So when we're talking about cloud and the aspect of security, a couple of things that I would like to point out is the way of thinking. First principle of thinking about security is open something like a port or access or anything else that allows the traffic, allows access. Open that thing only when necessary. That's the first principle. Open this exclusively only when necessary, only for those people for whom it is necessary. So limit it. Less is more. The bottom line is less is more. That's how you should think when it comes to security. The less things are open, the less vulnerable your infrastructure is going to be. This is a fundamental concept to think about. Less is more. Another thing that I would like you to remember when it comes to cloud is each cloud provider will provide their own methods of securing things. I will talk about a couple of methods now. Briefly, here, let's go to one of the providers. You will look at this one, which is the Amazon's method. And you log in. And you will have a system in which these guys provide security to their cloud infrastructure. We'll go to another cloud, which is manage.windowsazure.com. Let's go to this one. And these guys provide a different, let me log in here also. So we'll illustrate those points to you exactly where it comes into play when it is about security. So how these companies provide you access to methods that help you in ensuring security. That's what I will be illustrating with a couple of clouds. And another one is this cloud.google.com. So each one of these guys will have their own methods of providing security controls, whereas the other guys like this company, they leave it to you. They really leave it to you to provide security controls. You've seen that. And I want to make sure that I identify specific differences and how to put things in perspective when we will be dealing with this for real. So we have four cloud examples. We'll begin with one by one, beginning with AWS, then go on to Microsoft, Google, and DigitalOcean, for example. The fundamental way that this company, Amazon, provides security essentially is this concept of security group, which is something that I think you understand. And I think it is always important to really, really get to the bottom about what exactly is this thing called security group is. And I want to make sure that that part is clear. In fact, I want you to be super comfortable in dealing with this concept of a security group. That is what Amazon calls it. In essence, they provide you with one security group by default in your VPC. So you have a VPC. They give you one in every region. They will give you one VPC already. You have it. Sometimes you will have more. Like in my example, I think I might have one VPC right now here. So all of these are the same numbers. But then there are a couple of different security groups. As you build complex infrastructure, as you build complexity in your application, in your architecture, you need to really understand security groups. Otherwise, your application deployments will not work together. Things will not work properly. And so let's understand some of these things. I think let me ask you a question. The question is, how many of you feel that you already nailed this concept of security group? You know it already that you think that this discussion that we intend to have right now, focusing on a little bit deeper discussion around comparisons of security technology across multiple cloud companies, is something that I want to discuss. If you think that you know this already, please tell me so, so that I can skip ahead. I don't want to skip this necessary element. I do want to cover it. But if you think that this is something we should not discuss, I'm OK with skipping. But I want to know that you feel really comfortable. And so I will let you just tell me if you think that I should skip security-related discussions as it pertains to security group and corresponding technologies with these other companies. So we will do that. I'm assuming that you want to discuss this. And with that assumption, I'm proceeding forward with a concept to think about these variety of methods that these different companies apply to ensure what is a building block that Amazon wants to call security group. So from a building block perspective, I don't have a name for it, but I'm just calling it a building block. It is a concept that I will describe. The concept is that an ingredient that constructs a security group essentially is a collection of rules. This building block that I will define is actually a collection of rules of traffic or access. That's what it is. It is a collection, meaning there are more than one things. And these are basically rules. That's a building block. One building block will have one collection of rules of some sorts that relate to traffic flow or access control. That's the fundamental building block behind this concept that we are talking about in security. And different people will call it differently. But we'll just begin with one example. So that building block, let's go look at exactly how that collection is constructed. And at the foundation of these collections is a rule. So basically, you will have a rule followed by another rule, followed by another rule. And collectively, this is a collection of rules. And this is what I am calling as a building block of a security group. This rule basically applies to access or traffic. How will the traffic flow? So for something to flow, there has to be two points. A goes to B. That's the destination, the direction of flow. This is the source when the traffic generates from. And this is the destination where the traffic lands on. And so this is how you should think about a rule, is that it is about access. It begins from some point and reaches another point. Whether you allow or don't allow is the rule itself. Do you want to allow this traffic from point A to point B? Yes or no? That's the real rule underneath, under the covers. Now, what is A and what is B? These are basically some machine, some IP address with some port number or some protocol defined. Some machine, some IP address, some port, some protocol is an example of A or B or maybe C. These are locations. That is where traffic will either originate from or land on. So either you start from that point or you reach the destination, source and destination. That's how you should think about traffic flowing from A to B or A to C or either way. Any way it can go, this way, that way, or some other place. It is not limiting. It is not limiting. You have to collectively construct rules around flow of traffic. That's what we are really looking at from a security standpoint. So if you decide that you have, if you want to write a rule that says allow traffic on this port from A to B, as long as the person, this A, is only in a specific IP address range. So you can specify a range of addresses. If it is within that range of addresses, only then allow this traffic to this location called B on this port number 80, which is basically the HTTP protocol. If you want to allow traffic that originates from this source and reach this destination, you want to basically call that out in terms of a rule that says so. These are the range of addresses. So I might have a range of addresses 192.168.0.0 all the way through 192.168.0.255. And I will write this in form of 192.168.0.0 slash. Do you know what number should be here in the CIDR notation? What would that be? Here, what should I write? You mean the port? At 24. No, it's not the port number, it's the CIDR notation. OK. OK, here is something that you should review. We go there. Module number eight has a little link that I'm pulling up that will give you a quick view into CIDR. So this, or module 11, this video should have it. That will give you a very clear idea of what I'm talking about. And I'm going to paste that link for you to review in Slack chat. And which is the group that we use? This one. OK, here. So read that later. That will describe what I'm talking about succinctly. But let's focus on what this means is that you can actually, let me quickly tell you. If you have an IP address range like this, you're basically ranging from 0 through 255 in that span. This represents eight bits. This represents eight. This represents eight. And this does represent eight. So in a CIDR notation, which is classless internet domain routing notation, the idea is that you want to write this entire range of addresses, which has about 256 addresses. You can write this entire range of addresses succinctly like this. And this 24 is a representation of one, two, and three. And this does not change. You're basically saying that don't change these values, but change whatever you like here. That's what the representation means. In another example, I will make something like this. And the whole range can now change from 0 here to 255 and 0 here to 255. These things will not change. If that's the case, then these eight bits will not change. These eight bits will not change. These will change. These will change. And you have a range of about 256 here and about 256 here. That basically multiplies to make it 256 times 256 addresses, which is 65535. That many 65536 addresses available that you can fit into this number changing as well as this number changing. And you can represent this in CIDR notation like this, 16. This 16 basically indicates don't change this. Log this up. Log this up. So eight and eight are logged here. That's how you should think about CIDR notations. And so in our discussion regarding defining a source, a range of addresses, the idea that you describe is basically in the range of 192.168. I'm just picking up that number, but 0.0 slash 16. I can define a range succinctly by writing it like this, CIDR block. And this is what I'm saying. And when the source is in this range of addresses, allow it to come to port number 80 onto this machine called B, which is my destination as long as the source, this A, is in the range. IP shorthand. Yes, it is shorthand. Exactly. It is exactly shorthand is how you describe complex address spaces in a succinct fashion. So that's a way to define one rule. That exactly is what is going on here in this cloud. So we can define a rule, a new one. And we'll say, call it a name. So we'll call it, let's say, whatever. And description is IDK. But I'm now adding a rule. The rule is a custom rule that says TCP protocol, port number 80 is allowed from custom location, such as 192.168.0.0 slash 16. Allow traffic source. Allow traffic as long as it is coming from here. And let it go to port number 80. That's what this rule will do for the inbound portion. That's what we are defining. So that's how you create one rule called whatever. Maybe name it better. Like allow for, let's have a short name. So short name. And then description says, allow everybody from 192, or rather, I should say, from my office to this machine, to this resource. So that's how I would define, allow everyone to access port 80 from my office to this resource. So you basically define it like this. And the assumption is your office is getting allocated addresses in this range, this 192.168. By the way, that address range happens to be a private address. So it doesn't really work in the context of the cloud. You cannot say, my office, and use this name, or use this 172 series numbers, or use the 10.series numbers. Those addresses are reserved as private IP. So you don't actually use them in the context of calling them as my office. You actually have to use public IP to denote your office. Like I mentioned here, this one is not able to really denote your office, my IP address. My IP address is this. You see, it is not 192. If you look at this router in the office, the address is 10.0.1.1. It is 10.0.1.1. That basically is a private IP address. So you're looking at this range of numbers. It is, again, private IP address range. It doesn't actually denote things on the public internet. If you look at this machine itself right now, which is here, and you look at the machine's IP address, it is a 10.0 address. This is, again, internal only. That's the problem that you cannot really refer to machines on your office or in your private area, like this one. Cannot use this specific number. Those two numbers, like 192 is reserved, 10.0 is reserved, and 172 is also reserved. That series is reserved in, I think, RFC 1918 reserves it. RFC reserves private addresses that you can read more about in here. Having said that, the idea that we are discussing applies to using that kind of number sequence when it comes to the cloud. Inside the cloud, you can have private areas, and then you can use these numbers, actually. Inside your cloud, you can actually use this kind of numbering scheme, no problem. It just will refer to whatever you have in your cloud. It will not refer to your office, because it is not routable, cannot reach there. That is something you have to understand. Now, having said this part, the idea about constructing a collection of rules, let's continue to build. So essentially, what we just said is that in my office, we have this address. What was that address? The IP address was this, the 67, 183, 67, 183, 157, and 68. So inside this building, any machine that you have will have this IP address from outside. And you want to say that my range of machine IP addresses inside in this building denoted by a series of addresses that begins with this and ends with this, meaning there is only one IP address. One address. And so there is only one address. How do you notate this series of addresses with only one entry in a CIDR notation? You say 67, 183, 157, 68, slash 32. You're basically saying all these guys are logged. All 32 of them are basically frozen. You cannot change it. Thereby, you're indicating that there is only one address. This one. And you can go and use this number. Copy this number from here, like this, and take it to your configuration and say that, you know what, this IP address followed by 32. So that IP address is the one that will be the only one that will be the source. And now your description suddenly becomes correct. I can now say that this actually represents my office. So if I set a rule like this, nobody who is not in the office can actually reach that resource that I'm defining and applying this security group to. And that's how you should think about the concept. And these guys call it security group. These guys call it something else. So let's log in here. And I'm going to assign and authorize this one more time. Where did that go? Authorization came, but it disappeared. Let me log in. Where is the thing? Hold on. Two factor hertz, but not really. It's just a little bit of inconvenience, I would say. So here, if you look at this different cloud, they also have similar concept. And if you go start a machine and create one, it will ask you for specific things, like what do you want to do and which ports need to be opened. And so by default, everything is very secure. And so if you take from the gallery and say that I want to create a Ubuntu machine in the Microsoft Cloud, this version, and say select and proceed forward, you can choose the release date that you like. Choose the machine name. You can give it a name. And then assign some memory to it, or maybe more. And then give it a username, if you like. And provide a password or choose a certificate file. Either of one of those is fine. So you can upload a certificate, if you like. And you can send your ID RSA certificates, write this guy that will let you connect. However, the concept that I'm trying to illustrate is not about starting a machine, but more about getting to a point where you can control access to this machine, for which I need to just start a machine to give you the idea about how to do it. So I'm providing it a password and making it somewhat difficult so that it actually creates a machine by the name Ubuntu and gives me access to that machine. So I will proceed. And it says, OK, use this service. And go ahead. And here is how they describe their security group concept. Azure calls it Endpoints. That's the name Azure uses it. You can see that we are basically doing the same thing like we did here, in the case of Amazon's security group concept. We're basically doing the same exact thing here in Microsoft. What we are doing is they have automatically chosen SSH port for us, which is they're assuming that we want to connect to this machine using SSH protocol, which is exactly true, by the way. That's why they chose it for us by default. However, in the same location, we can add other protocols, for example. Like, for example, here I'm saying anything that comes into port number 80 from outside, please connect it to port number 80 on the machine inside. Let's go one more rule here. And now we have a good collection of small collection of rules. Now here what we are defining is the same structure. Port number 20 to public port, port number 20 to private port. What does that mean? If you go back and look at the illustration here, what Azure is telling you is that they will create a machine for you, like here, inside the Azure Cloud. And this will get a private IP address, private IP. It is the Azure Cloud. However, they will give you a public IP. And you will be able to set a, what's it called? The name that they use is called endpoints. You will be able to set an endpoint, which basically is a collection of rules saying that if somebody comes in, wanders around, and reaches this public IP address at port number 80, please allow that traffic from this public location over to the private IP address and port number 80 inside the machine in that box that you're going to create right now. This one, we are creating a new box right now. And to associate with that, we are putting together this rule, this is basically a rule. And in that rule, what we are saying is SSH, which is the number of 22 from outside to the 22 inside. That's the kind of collection that we are putting here in form of an endpoint. That's the definition. That's the word that Azure uses, as you see in this little table for public port, private port, basically a port mapping between what you have outside to what you have inside. And you can keep adding more, or you can custom create something if you like. Or you can custom create. And that's the concept of security groups, or endpoints, or these guys have a different name for it. These are the Google guys. They have something different. But same concept called something different. It's in the network section. Where is that? Networking. And here, they call it the firewall rules. And so here, concept is very, very similar. Let me make it a little bit so you can see from a distance. Here we go. So here, you're looking at the concept called firewall rule. Basically the same idea, like you discussed in the two examples. The first name was security group. Second name was endpoint. Third name here is firewall rule. And we are basically describing, giving it a name. So giving it some name, some description. And then choose a network. You can have multiple networks, like a network is a VPC, for example. In Amazon's language, they use the name VPC. In Google's language, they use the name network. It is an island. It is a dedicated area just for you, private, only for your machines, only for this account. And whatever you do is totally, totally private. That's a network. Here, you can say source filters. And you can define IP ranges. Or you can say allow from anywhere. So it says 0000 slash 0, meaning allow from any location on the internet, which is not a good idea. Sometimes you have to do it for necessary reasons. But in case of secured access, you want to narrow it down to something like that I did this, my IP address. This address I want to take and limit access to only my machines. So I can say slash 32 here. And allow only from these IP addresses. By the way, this is only one address, but that's OK. And allow port number 22. And that's how you construct one rule that is known as some name. You can give it a name for some name. Essentially, you're associating with a network, which is like VPC, Virtual Private Cloud. In Microsoft, they call it Virtual Network. In here, they call it Network. In Amazon, they call it VPC. So different people, different names. But the same concept is what you're looking at. And you can construct this by clicking a button. It opens up that particular IP address. So it basically translates to you allowing access to anything inside that Google Cloud from your location. So you have this location that you have, where you have your machine sitting here. From this location has a specific IP address. And from any machines in that location, you're allowed to go in on port number 22. And that is a specific firewall rule that we have to construct. These guys call it firewall rule. However, so we discussed a couple of different names for the same idea. We saw this thing is called, what was the name? Security group in one cloud, endpoint in another cloud, and firewall rules in a third cloud. Now, let me caution with some other types of clouds like this guy. Do not have such a concept. They do not have such a thing. No. No such thing exists. However, that basically means if you want to use that cloud, you need to provide your own method to manage security. And you know what that method is? What would you do if you have to use this cloud? What would you put to use? What method? Anybody? What would you use to provide security because they don't give you? SSH? SSH is the method of connection. It's a protocol to connect. But what tools would you put in place to protect your infrastructure? The protection has to come from? Security group. Yes, but they don't give you. The problem is these guys don't have it. So they don't give you anything. All they give you is machines directly. They don't call it anything else. The DNS. Yeah, you could use a firewall. Yes, you have to actually use a firewall. And in Linux, the firewall comes in this library. And this library is really hard to use. So therefore, there is this thing called uncomplicated firewall, UFW, which is basically implementation of this library. And that is what you need to use on every machine. Security group. On every machine that you will instantiate in a cloud that does not give you native tools for security, such as those guys who will not give you anything security group, not give you anything about endpoints, or not give you anything called firewall rules or whatsoever. Big clouds give you that. Other clouds don't. So do it yourself. OK. And we have done an exercise like this, if you remember and recall. Let me identify that exercise quickly for you. We did something like here. You remember, I ran this binary to install UFW. And then I allowed simple rules like this. I said UFW allow 22, UFW allow 80. And by implicit meaning is that you allow from any location, any source, if you don't call out what needs to be allowed, you can actually read the UFW documentation. Or maybe these guys will give you exactly. And so here is an example of how you would allow. UFW allow from this source to any port 22. That's how you would have to write that particular range of IP addresses in a command and execute it. That's one way to handle. But again, remembering to run these commands by hand is also a chore. So you want to automate these things through IP tables cookbook, for example. So there is this IP tables cookbook available that you can use in Chef. It's in the supermarket. It will help you automate the deployment of security rules around your machines in a cloud that doesn't support it. There is no underlying easy method. You don't have these things. You're on a naked cloud. And you need to use that for some specific reason. It is OK. You don't have to depend on these things because there is this fundamental tool available in every machine. Every Linux has it. Just making sure that you use it. That is the thing to remember. And by the way, the configuration of IP tables is exactly how these things actually work. Under the hood, they are all IP tables. They all require this thing, a source. And where do you want to go to? So which port? And whether this traffic is allowed or not allowed. That's basically the collection. You can see that in this example. Allow from this IP address slash 24. So it's a range of addresses into any port. That's allowing. Another example, allow from this IP address range to any port at 73. Another example would be allow HTTP. Because you do not want to limit port number 80 to any particular range of addresses. You want to allow it to go anywhere on the internet. And that is where you need to modify and not have any reference to a range of IP addresses like this. And by that, it automatically opens up to the entire internet, if you don't specify anything. So allow HTTP, the whole internet can access. Allow 80, it's the same thing. HTTP and 80 are the same thing. Here, HTTPS and 443 are the same thing. So you basically have to manage these things by hand. If it is a large infrastructure, make sure that you're using some kind of an automation tool. And this is something that I think is a fundamental to using any cloud. You have to make sure that you're using it. And in many of our exercises, we use this cloud, which we made our life easy by going to this cloud. Because it doesn't actually make us disciplined about putting these rules in place. It lets you be dangerous, which is OK in learning, but not so much appropriate when you're actually deploying services for real, for production. And that is something that I want to now discuss back in our exercise that we had, again, a while ago. I think I pasted a link here. So this particular exercise that we did, if you remember, we did this one in module number 11 that begins with a small series of videos, very short, one minute, two minute long. But these videos walk you through how to construct a small, it doesn't have to be small. It's a virtual private cloud that you construct out of GimmisRF, out of mouse clicks. So basically in this exercise, if you remember, if you have done it, we walk you through an example in the Amazon cloud to construct a virtual private cloud basically by clicking some buttons, like go to the VPC section, create a VPC, like that. You've done it before. I'm pretty sure you have. That's what this exercise actually is. Now what I want to be able to do today is to help you understand how do you put the aspects that we just discussed about security groups, about security in general, and apply and basically play this exercise, this exercise that we are doing in module number 11, but this time doing through automation. How will we automate complexity that we have? And by the way, that exercise, if you do it the first time, it is not easy. Step by step, you will not miss any step along the way. Only then it will work. If you miss something along the way, you are screwed. You have to go back and see what went wrong and then fix it. And so to address that scenario, it is a good practice to automate and replicate your design of your infrastructure in any form that works for you, that suits your business. So we have done infrastructure, and we have constructed in the last, I think, two days before, we constructed a JSON file based on a given architecture. And then we played that JSON back against another cloud, and we were able to reproduce that architecture in another account, another cloud, another person, and somebody else. We saw that. We used this tool called, what was the name? Formation, Cloud Formation. We used this tool, and we constructed a JSON file, and we then used that file to construct a cloud back. That is the Amazon's method of doing it. There are other methods that I want to show you, which is slightly more evolved and more generic in its nature. But essentially, that's what I want to be able to show in another example, which is somewhere in here. This one, I think, this is the example, which basically deploys a multi-tier application running a variety of Docker containers, and we will use this generic tool that is agnostic to any cloud. This tool basically will let us operate on any cloud because it is not tied to one company. It is neutral. And that is the exercise I want to walk you through. This is just one page, but it is something that you have to really understand what's going on under the hood before we run it. You can blindly run it, but it doesn't help you. And so what I want to be able to do is walk you through step by step in understanding the details under the hood as to how to get this to work in a repeatable fashion where we want to construct a cloud according to our design, an infrastructure, an architectural layout according to our specification that we have codified and that we want to execute to reproduce the same exact thing in another location or another project or another customer or a partner. So basically, that's the idea. It's about repeatability of a given blueprint of an architecture. So in our example, what we want to be able to do is get our workstation going. So I have that workstation. Let me bring it up. You probably have it already. I'm just starting it up on a side window. There is this update available. I'm not getting it right now. I'm just starting the workstation, and we will use this one. So that machine, it starts. I'm putting it on the left or right half of the screen. And on the other half, I'm going to start reading a little bit, which basically says that there is this source code available. So we want to grab it and then play it in our machine out there on the right side. So that source code, if you look at, is on GitHub, which is basically this location where we are going to clone this particular repository. And what this will do, I will walk you through as we go along. So first thing first, I want to clone it. I'm going to clone that in our workstation right here and say git clone. So bring a copy. I think I did not copy properly. So git clone and then paste. So it will bring the whole folder down and is now sitting in a folder called nt-architecture on AWS using Terraform. That's the folder I want to open. So I will actually go inside that folder and then open up Atom Editor to see the code. What do we have there? That's what I want to do. So here is the thing that shows up. And I want to read what do we have. We have a folder called Terraform. And that is where a blueprint is set already. It is given to us already. Now, it is somewhat difficult to understand these things on the very first attempt. But I'm going to walk you through step by step. And what I'm going to do is to actually hide most of these elements, like one and two and three and all these guys, in a whole folder. This whole folder that I have basically is designed to hide complexity from you when you're looking at it for the first time. And that is exactly what I intend to do, is basically begin by hiding these elements. So I'm going to move this item into the whole area, one by one, like all these guys, just holding them out. And you'll see that I'm basically hiding things from you as we understand as to what to do with it. And so we have, in our example, basically in this whole folder, we have a bunch of different files that I have hidden from you. And there is some files sitting here called application that runs the application here called NGINX. And then there is this binary file, which has some steps to do in the readme that is described. It describes what to do with it. So we will deal with the binary and cloud configuration later. But first, understand that this entire folder, I have hidden many of these things that we want to actually study in a whole folder. So they are not really operative because I'm hiding them in that folder. If they are sitting in the top-level Terraform folder, they will actually be useful and actually act based on whatever is written in that code. But right now, I have just moved them away in this location. And the reason, the purpose behind doing this is to help you understand what those things are because I will bring them back out one at a time. So I'll bring maybe this item out to Terraform one at a time and then hide the rest and play with Terraform right now. And then compare with what do we see in our cloud in here in Amazon as to what this tool is doing to our cloud. That's what we really want to see. This tool will automate quite a bit, quite a bit, as you will see. And we want to see what it is doing so that we can make use of this tool ourselves. In order for this tool to start working, start getting functional, it needs a pair of access key and secret key. We need to get that. We can get that from the Amazon Cloud by going to the Identity and Access Management section. And here, we want to create a new user. We will call this user, let's say, Terraform. So in that form, we'll create a new user called Terraform and then say, OK, give me one. So here are the credentials. So we grab the credentials from here and put that in the section. We'll grab the secret access key from there and copy and put that in the secret section. So we now have the credentials in place. And that's great. However, these credentials do not have any authority. So we can close them and we can run Terraform, but it will not work, because even though the user is created, this user doesn't belong to any group. The user doesn't have teeth. It's just a dummy user. So we need to add this user to a group which has the ability to do things. Like, for example, the SuperDuper admin, this group has teeth. So we add that user to the group. So now this TF user suddenly has teeth. It can do whatever it wants. So now this thing becomes active. Having done this part, I want to make sure that we validate and understand how this is going to actually run, for which I want to make sure that I hide this item back again right there. I'm hiding it. And only file that is available is this variable references that I have in the keys. That's it. With that understood, what I want you to do is see if we can go back to the right up here. Where is that? R-A-M here, back. And so here it is described, what to do with it. So this particular exercise, we will actually build the whole exercise we did in the cloud technology segment, basically construct a virtual private. What are these things? Let me pause it. This pop-up is notifications, too many notifications in the way, so off, turn it off. So this is going to construct a VPC and a bunch of other things. So what will it do? I will walk you through as we go along. But I'm going step by step really slow. And we want to use Terraform. For which we need to have it installed, which I believe we do. So we'll open up this window and say, CD Terraform. That's where we are in that location, Terraform location. And we'll say, Terraform, do you exist? And it says, yeah, I do. So I just asked whether Terraform version exists. And it has this version of Terraform 0.7.4, which is out of date, but no big deal. We can update, but we don't have to update. It will work. It's a minor update. We have 0.7.8 versus 0.7.4, so not a big deal. It will work as is, no problem, I hope. Having said, we have Terraform installed. Now, if it is not installed, that's what you do to update it. And then we have this folder here that we want to actually make this name a little shorter. It is occupying too much territory. So I'm going to rename this folder a little bit to make it a little simpler name so that you can see better. That's what I'm going to do, is to actually modify the name called ntier to tf. So just simplifying the name tf, and then go inside into Terraform. And then we can see the listing of the folder that we have, and then open it up in Atom, and keep working with Terraform in this window. As we look at, our holding area is all hidden, as you know already. And these files, I'm hiding them, by the way. As you know already, you heard me say, I'm basically hiding it for a specific reason, to show you how things will build up. So having said, we have added our variables to the infrastructure file. As you saw, I added those two variables that are these things, these variables I have added. And then we want to create the entire stack that we have defined. But I am hiding the whole thing, so it will not do anything, because nothing is actually visible to it. So if you now run Terraform apply, it will basically do nothing, because it doesn't know what to do, because the things are hidden from it. So if I say Terraform apply, it says, configuration file not found, no configuration files found in that folder. That's what it complains to us, which is exactly expected. In the home user tf-terraform, there are no configuration files. They're all sitting in the hold folder. So I want to get this file out into the Terraform folder like that. So now we have one file. And we will attempt to run Terraform. And before we actually run the apply command, I want to see what will Terraform do for me when I apply. So before I apply, I want to see what will you do if I apply. And for that, there's a slightly different command called plan. Show me what the plan is. And so here it says, hold on, there is some error going on. So I want to see what that error is. What just happened? Hold on, let me fix it. OK, I may have to get those keys again as a threat. I hope not, but maybe I'll just reserve these keys separate. Let me just copy these guys over and in new file and keep them separate. I'm keeping them safe in another machine. I'm going to get that whole thing again down. So maybe I made a mistake somewhere. So I'm not saving anything here. Close out, close the whole thing out, do the whole thing. Clone the whole thing again, basically. That's what I intend to do. So not saving, not saving, don't save, not saving, and 3D dot dot, and then rm minus rf t. Then 3D dot dot, and then rm minus rf tf. And then clone the stack again. So here we go. We are going to clone it again like this. So it clones. Now we have it. Then we go into the Terraform folder to open and open Atom Editor in there. We should see those files again. They are all right there. Now in here, what we want to be able to do is I want to hide these guys one more time, all these guys. So app server or VPC, keypad, NAT server, outputs, private, public, security groups, and the variables. This file was needed. That was the root of the error, that I have hidden this particular file, which I shouldn't have. It was expected by this file was actually expecting that other file called variables.tf. That was the reason for the error. Now to address and prove what I just said, I will grab these things back from here, put that in that workstation, in that exact file, and overwrite those change this values with a new value here, save it, close it. Back in Terraform, we would like to see the plan. So cd, tf, and Terraform. And in there, we will say Terraform, please tell me what your plan is. And so it is telling us what our plan is going to be right now with our things hidden from you. The only file that is available to it is the variable declarations. So if you now look at this particular, let me adjust the screen a little bit again. If you now look at the output carefully, when I said Terraform plan, tell me your plan. And it says, refreshing the in-memory prior to plan. And it says, no changes needed. Your infrastructure is exactly up to date, meaning we have nothing to do and that you have nothing. So it matches. The nothing matches with nothing in the cloud. So in the file that we are exposing right now, everything else is hidden in the hold folder. This file basically declares variables. It defines what is the access key. The access key variable is defined in this block. Similarly, the secret key variable is defined in this block. In here, we are defining a region called AWS region. And the default value we want to select is US West 1. What we can basically infer from this is that anything that this particular piece of code will do is going to actually make an impact for real in this region, hold on, in this US West 1 region, Northern California. That's where it's going to impact. If you go look at the VPC right there in that Northern California region for my account, you will find that I have only one VPC. And it is this default VPC, which I don't want to bother about. I'm not even touching it. If you also see that I have some security groups, I have two of them. One of them is the classic security group, which I cannot get rid of. This is like a seven-year-old security group stuck in my account. It's a legacy thing. I cannot get rid of it. I can do nothing about it. It's just there that I cannot remove it. So just ignore that one. In legacy, ignore. This is the default security group that corresponds to the VPC that is given to me by default. And now, in our example that we are running, which is basically this example here, it is going to, when I say Terraform apply, it is going to actually apply itself to the US West One area in my account. It is going to use the VPC CIDR block with this numbering scheme, 10, 128, 00, slash 16. The slash 16 means this can change. And that is going to be the CIDR created. That's the defining of variables that I'm doing in this file called variables.tf. Here I'm saying, give me a subnet, for public subnet, and assign this CIDR block, 10, 128, 0, dot. And then you can change whatever you like for the last byte. It is slash 24. Meaning these three are frozen, the last one is open. You can change whatever you like as long as it is between 0 to 255. And that is going to construct a public subnet for me in that CIDR range. Similarly, a private subnet in that CIDR range, which is only a tiny little bit change in the third byte here. The difference between public and private is only one different byte in the CIDR range. I'm also specifying machine images that if you are going to be playing in the west one, use this machine image. If you are playing in the west two, use this machine image. East one, use this image. So I'm basically calling out these variables. And this file basically uses these variables. Where I'm saying, access key, use this image. Use this specific number, which is this one. And I say, secret key, use this specific number. And when I say region, use this. CIDR, use this. Public subnet, use this. Private subnet, use this. Like that. That's what I'm defining. But it doesn't actually involve any action to do something. And that is what we will do next. So if I say teraform plan, it does nothing. If I say teraform apply, it will say that I did everything you wanted me to do, and you wanted me to do nothing. So I did nothing. So I added nothing. I changed nothing, destroyed nothing. And I complete. That's what it just did for us. Apply complete. I did nothing. Great. Because you never told me to do anything. So now I'm going to tell this thing to do something for us. So in this particular file, we are defining variables. But now I'm going to go back to that whole folder and pull something out. What I want to now do is actually automate creation of a virtual private cloud, VPC. So here is the definition. I want to pull this file out into the teraform location and hide the whole file. Now have this file available to me. And in here, if I run teraform apply right now, it will actually construct a new VPC by the name automated. Automated. And use my access key and secret key and the region defined in the variables file and do it for me, basically. That's what we do. And so if I were to just ask without actually doing, what will you do, please? Tell me your plan. It tells you the plan. It goes to the Amazon account, finds out that you don't have a VPC called automated. This name. You don't have a VPC already by the name automated. So the plan will be green sign plus to add a new default VPC by the name, whatever you say, automated. Because that's the name here, line number 13, automated. So if I now run teraform apply, it will actually go to the Amazon cloud and create a new automated VPC for us. Let's go see first here. In the Amazon cloud, in the California region, if I refresh multiple times, I don't have a VPC by the name. What's the name? Automated. The default exists. That's OK. But there is no automated VPC here. But now, as soon as I have teraform apply, it will create one for us. So let's go do that. Let's go apply. And we'll go watch what's happening on the Amazon cloud. After this finishes, you will see that it goes to Amazon, creates a VPC, and comes back. It says, I added one. What did I add? I basically added this guy in use this side of the block you mentioned and the ACL routing table security view. Basically, whatever you said, I did that. And that is preserved in this file that got created called tf.state. You can see the state here that preserved the state of the cloud right now. How is the cloud constructed right now? It is given down in a JSON doc. You have it saved for you to refer to. However, you will see here in this window, if I refresh, you should see a new VPC showing up called automated. And here it comes, automated. It's constructed with our chosen sider block, as you can see. We chose our sider. It should be 10.128.00. That's what you got. That's all it did. Now back in here, we'll add a couple more things outside of the whole folder. We'll pull them out and put them inside our main working folder, which is this folder. That's what I want to be. I don't want these guys to be sitting in that folder. I think these are hiding here. I'm going to bring them out from here one at a time, put it in the main folder, and operate on Terraform, ask Terraform to do things for us. And so we just did one thing right now, which is to create a VPC, a Virtual Private Cloud, by using this definition, which says, call it automated and use the sider block defined in this variable, which is in the variables file here, and use the enable DNS host name true. It's a setting that you saw. You can see that setting back in here. Say enable action, enable DNS host name. It is yes. And this yes comes from the fact that you wrote yes, true, here, enable DNS host name true. And then you will have other things like access key, secret key, and region. So the region variable is US East 1. So it goes to the California cloud and does it there. Now, a couple of other things. The next thing that we need in order for us to operate on our cloud here is to actually have some essential ingredients, like you need to have subnets, route tables, all these things in place. All these things need to be filled up. And it is boring to do it by hand. That's why we have this automation. So we'll actually start building those things in form of ready-made examples that we are just going to use them in this exercise. So we are going to begin with something like, you know what? I need to create a private subnet. So I will take this private subnet file and put that upside here. I just moved it. So in our listing, you should see this private subnet file came out. It was not there before. And now I'm hiding the whole area. We have this private subnet. Let us go read it. What does it say? It says, create a private subnet by the name private. In the VPC ID that you know what I am using, the one you created, that one, yes, that one. Not the other one, but this one that you created. CIDR block should be used, the one that we told you to use. So use that one. Availability zone, I would like to get this private subnet in the US West 1A. So go to the 1A section. Create a new private subnet. And also set the attribute that don't give me a public IP, because this is a private subnet. I don't want a public IP address. It is going to be set as false. And I will also say, depends on, I'm setting a dependency. For this thing to operate and connect to the world outside, it needs a dependency. The dependency is a NAT device. It needs a router to connect to the world outside, inside the home, inside your private area, inside a VPC, inside your office. It is on a private subnet. It needs a NAT. That's why line number seven, dependency on a NAT. This means, if I execute this particular private subnet Terraform file, our output in Terraform will fail because we don't have a NAT. And that is the reason why I want to call out that there has to be a sequence in which this collection of files that I'm hiding in the whole folder, you cannot randomly bring them anything you like, but you have to have them understood in a matter in a order of dependency. So there is a NAT-related file also right there. That needs to be executed first before you execute the private subnet. It means you have to put this private subnet back in the hold first and get the NAT out like that. That has to be the sequence. So I did this specifically to illustrate this dependency concept. You know that private subnet elements in the private subnet depend on a NAT instance. So you cannot create a private subnet using this Terraform construct without having the dependencies in place. That is why I put that item back in the hold area. If you put all of them out, the dependencies are met and everything will succeed. If you just try to bring the private subnet out to Terraform and try to execute it, it will fail for dependencies unmet here. That's why I put it back. Now I will see public subnet. See if we can construct a public subnet. And I think we can because there is no dependency, except it has this thing called depends on Internet gateway. It needs a gateway for the public subnet to connect to the world outside. So we need to create an Internet gateway, which is, by the way, done in this file itself. So you can see it up on the top. There is an Internet gateway getting constructed. So resource called Internet gateway. Please create one by the name default in the VPC ID that we are talking about, the same VPC ID, this one. And create that gateway. And then use that as a dependency here. So since it is in the same file, it will work. You will also have a couple of other things here. We are running a route table called public and giving it a cider block for the route table and also associating that route table with the public subnet that we have constructed. So this subnet ID that we have constructed here, which is the AWS public subnet ID, which is this, basically this public subnet, we are associating this public subnet with a new route table that we are creating here and then associating it right here. So in this particular file, we'll do a couple of things. So before I execute this, I need to bring this particular file out from the hold area, put down the Terraform folder and minimize the hold segment. We have the public subnet file in here, which constructs the gateway. It will create a public subnet. It will create a routing table, associate that routing table with the internet gateway for the outbound traffic here, this maps to this, and associate this subnet with the routing table itself. That's what we did by hand the last time when we did this exercise. So if you go to route tables and filter by the automated VPC, you will have this default route, which is not available to use. And so it is there, given to us, but we are not using that one. Instead, we are creating a new one, which is going to be constructed through this method here, AWS route table. Create a new one called public and connect it to the gateway that you create here. So we create a gateway, create a route table, associate the gateway, create a subnet, associate with the route table, the five steps in this file. Now check the telephone. Telephone, what do you plan to do? So it will go and look at the cloud. It is studying what's in the cloud right now. It tells us that in your cloud, you don't have these green items, one, two, three. Those items, you don't have it. So I will create one for you. In that AWS internet gateway, you don't have one, so I will create one gateway for you. I will also create a route table that associates with this VPC that you have that I created early on, and also associate that route table with the public subnet with this subnet ID that you will create in this step here inside the US West 1A. So you're creating a subnet. For public subnet, you're creating a route table, associating that with the public subnet, and allowing your routing to happen through the gateway. So if you now run Terraform apply, it will do those things in the cloud. You will see that it is happening right now. You will have a new subnet associated called for the automated VPC. There is this new subnet created. If you go to look at the route tables, you will see that for the automated, there are now two subnets. One of them is this one is newly constructed. There is also a gateway created, which is this one we just created. And the route tables are already associated. You can see that the gateway allows you to go outside. You will also see that the subnet association is already associated with the public subnet, which is this subnet that just got created. And Terraform did it for us. These guys did it for us. So we have four resources added. Nothing changed, nothing destroyed. And that's what happened when we have this file inserted into the main block, into the main body of the program, when it actually can pick up. We'll do a couple more things here. Go back to the whole folder and now define one of these guys. So we can do in any order except that the NAT needs to be in place before a private subnet can be constructed because there is an explicit dependency between the private subnet and NAT right there. And so we have to construct the NAT. In order to construct a NAT, which is a machine, we need to have a key pair. This key pair is something that is needed, before which you cannot have a NAT device. So we have to construct a key pair that you have to give into the Amazon Cloud. So we'll use this file, which is ssh-idrsa.pub, which I may or may not have. So let's go check whether I have it. And apparently, I do. So I will use this public file, give it to the Amazon Cloud, and I will call it DeployerKey. The key name will be AutomatedVPC. For that AutomatedVPC, I'm using this DeployerKey. And I'm going to send this key out first. So at least I can have machines getting ready. So in here, what I want to be able to do is get the key pair out into Terraform, and hide, and come back and run Terraform plan. Show me the plan. It says there is some error. The error is some parsing error in the syntax. Let's see what the error is. Key pairs, AWS key pair Deployer. Did I make a typing mistake or what? Hold up. I think while I use the, let me go get reset. Sometimes what happens is when I'm using my Atom Editor, I accidentally make some changes. And that may be the reason for this anomaly. So what I'm going to do is basically go back to GitHub and make sure that my file is intact, this particular file I'm worried about. That's what is getting syntax error. So we'll go to GitHub and actually copy straight from there. So we have this Atom Editor open in which the editor is right here. We have this key pairs file that looks like this. I don't think it's a syntax error, but something else. I'll go find out. We'll go back to Chrome browser, go to GitHub, which is in this location. And no, not this, not this, not this, not this, this location. And we'll go to Terraform, look at the key pairs file, and make sure to get it raw. And the raw file, I will copy it over from here to our workstation. And this should be saved. No changes. Interesting. Let's apply. It says, error loading, error reading, configuration, key pair, deployer. Let me check that. So maybe I'm missing something else here. What am I missing? Apps server, NAT outputs, private IP, security groups. I'm going to try to work around and bring all these files back because I think I must be missing something in my explanation. So what I want to be able to do is put these files back in the main folder and run them all together. So bringing them back, all of them back in the main folder, like that, and see if the error persists. So I have basically put all these files back in our Terraform folder. And I'll say Terraform, see if you still have the error, and apparently there is. Something has changed. I don't know what that changed, but apparently this exercise will not run without it, which means I have to break it. And I will do this exercise later because there is no point debugging right now. I will do this exercise at another time or some other day, but not today. It doesn't make sense to do this, but the concept that I want to conclude with for right now is that you are basically defining a file for Terraform to operate against, and you're defining this.tf file, which basically looks pretty much like what you saw here. This file defines what is needed for a server to be constructed. This file will define what is needed for a VPC to be constructed, a NAT server to be constructed, things like that. It's basically defining that. Hold on. I think this deployer may be a false error. I get thrown out when there are errors. I just get distracted. So I'm trying to collect my thoughts and wrap this discussion for now. Change to another topic. I will come back to this discussion on Terraform later when we will run this exercise in a completely automated fashion with just one command, which is Terraform apply. It does the whole thing for us just like you would expect, except not right now. It's not happening right now for that, for some reason that I'm unable to catch quickly. And I'm running into this error, which I will find out what may be prompting or causing that error. Maybe it is Terraform updates. Aha. Maybe Terraform has changed. So I'll try that quick one. So I'll go to Terraform, get latest binaries, Terraform, and download the latest binary. Maybe that must be the fix. So here is the download coming. So the download is 078. So we are on Macintosh 64-bit. So Linux 64-bit is downloading right now. It did. And I will now execute the steps needed to make it work. So the steps are, where is the update steps? Here. So go to Downloads, Unzip, Remove, and this. So I'm going to install this particular binary I just downloaded into my machine. So I'll go to Downloads folder. And then what do we have? This Terraform binary. So I will go to my user local bin. And then we should have new binary for Terraform right now. And yes, we do. We'll go to the Terraform folder and Terraform apply. Still syntax error. OK, I'm giving up right now. So closing it out. We will revisit this another time. I will redo this exercise to root cause what's going on. This is not a new program. It's not a new exercise. So it should not cause trouble. But apparently, it is doing it. So let's switch. Switch switching topics right now. If you have any related questions before we switch topic, please ask away. So we'll take a short break right now before we change topics onto another idea, which is about monitoring and being able to load balance and scale. So we'll use the concept of discussion regarding monitoring cloud infrastructure. After a five-minute break, we'll take a short one right now. And right now, it is 6.30. So we'll resume at 6.35. Thank you, guys. I'll be back in just a minute. I'm walking away after I check one thing. Hey, guys. I'm assuming you're back just checking. Yep. Yeah, you are. Thank you. So yeah, it throws me off. And when things do not work, I just get thrown off. So yeah, it did not work. Sorry, we'll fix it. OK, let's talk something different. So in just a couple sentences, I'm going to recap what we just discussed in the form exercise. The idea is that you have a complex layout. You can actually reconstruct that same exact layout like you had if you are able to capture what did you do to your cloud in succinct files, like you can put them in JSON files or.tf files and apply tools like CloudFormation or tools like Terraform. This applies only to Amazon. This applies to any cloud. And that's the benefit that you can reproduce the same exact architectural footprint that you might have for your application, for your services. Make it reproducible. That's the benefit, reproducible. And thereby, you stand the benefit of dismantling and reconstructing in the event of any eventuality that you might run into. Things will happen. You want to go away from here, go there, you can. And you want to reproduce something for a test, you can very quickly reproduce the entire infrastructure that you have for a test or for a staging area. And when you have everything working correctly, you can just call your staging, rename it as your production, and off you go. You just basically have a ready-made infrastructure with your services running. Just rename it or tag it differently and have it going up and running very, very quickly. So what we were trying in this example, which we have some kind of error going that you saw, is basically the same idea where you define a reasonably complex layout. In our layout, we will have a private subnet, a public subnet, a load balancer, a NAT device here. Machines number one, machine number two here running Docker containers. We have one machine here running a VPN, which is Virtual Private Network. And from our office location, we should be able to go into the VPN and then basically virtually place ourselves, logically place our office inside the cloud. Basically, logically move yourself. You are now in the cloud, logically. That is possible through this device, which we will show you once it works. And this load balancer will actually balance the load between this machine and this machine, like this. And you can increase the number of machines you might have or decrease them if you don't want them anymore. And that scalability happens. That's the ultimate layout that that particular Terraform exercise does. In just one command, it operates and instantiates everything needed to make it happen like that. It did not work right now. However, some of the things that we want to talk about, which is the aspect of being able to scale, we depend on the ability for us to measure. Measure what? The ability for us to measure performance. That is the foundation or the underlying concept behind scaling. If you want to scale something out, like you have M1 and M2 right now, you want to scale them out. You can do that based on some logic, some very simple principle, some concept. What is that concept? What should be that concept? Let me ask you, forget complex logic. Just simply tell me, when should something be scaled out? When should something be scaled back in? When should we do this thing, back scaling out, scaling in? When should that happen? Forget computer programming, forget any kind of code. Just tell me concepts. When do you think it should scale, out or in? 85% utilization. Yeah, absolutely. Beautiful, beautiful idea. Utilization is one way to look at it. When you have your utilization increasing and it is reaching some threshold based on that parameter, you should be able to say that, you know what? My boxes are getting hammered. I want to get more. And you know what? These people, they are going on vacation. It's party time for them, so they're not hitting my machines. So I don't need to have these many computers running. I can just get rid of some of them. But the concept is what we are really talking about in here. Not in this. But in the concept going is that you have to have some kind of a logic that you put in place for which you decide at what upper threshold and what lower threshold out of a scale of, say, 0 to 100. At what threshold you will decide. And these are just arbitrary numbers I'm putting in here. They're totally arbitrary. Just putting in there for discussion. If it is 0% utilization versus if it is 100% utilization, both of them are bad. This is bad. This is also bad. This is bad because I can do nothing else. My CPU is pegged. I cannot do anything else. That's bad. And this is also bad because I'm losing money. If I'm losing money, there is nobody using it. That is also bad. So somewhere in the middle is these arbitrary numbers I put are the OK areas. Ideally, I would like to fill up my machine and leave some reserve. So this is my reserve. And the rest is my mainstream usage. And this is too low. Low, too low usage. I don't want that either. So if that is happening, if this is triggering, this threshold, let's call it threshold, the upper threshold and the threshold for lower threshold. So TLTU, two thresholds I have. If my metric that I'm measuring, whatever metric is, if it is going below 85, I'm fine. I'm OK. If it is crossing this number and, say, reaching a certain higher number above 85, above my upper threshold, for a sustained period of time, let's say five minutes, and it is staying like that for more than five minutes, maybe it is time for me to do something proactively, like scale out is how I would think, is how I would like to scale. Similarly, if I'm seeing that I scale out, I scale out, that my utilization numbers will actually change at that time. Because now, out of this scale out that I just did, my number of machines I have grow. As a consequence, my utilization numbers will fall. So it will fall down below to somewhere about maybe 60, 65, some number, something like that. That will stay there. Up and down goes on. But at some point, it might go back up again. And in that case, if it is crossing this 85 threshold one more time and stays like that above 85 for more than, say, five minutes, then I need to scale out one more time. OK, give me more. I need more. Because our people, our customers, these guys, it is like Halloween time or day after Thanksgiving. They're just hitting our servers. They want to shop. And so I want to basically do this thing. And I can do that systematically if I'm able to measure the metric that I'm interested in. The metric that I'm really, really interested in is actually not utilization. I'm really interested in measuring satisfaction of these guys. Are they satisfied? Are they happy? That is what I really want to measure. But it is very hard to measure on an automated method. You have to go and ask them, hey, are you happy? Only then they can tell whether the customers are happy or not. But that is the real metric. It is not possible in the context of cloud computing to measure customer satisfaction in an automated fashion on a five-minute metric level. It's humanly impossible to go ask everybody, hey, how are we doing? How are the machines working? It's idiotic. So don't think like that. But at the bottom, that's exactly what you want, is you want people to be satisfied. Otherwise, what's the point of all this? All of this is useless. You should not do this at all, get rid of all these things if the users are not satisfied. This is there for a purpose. The purpose is to keep them happy. But that's not a metric that we can put in place. So we use a proxy, a proxy metric. And one proxy metric is utilization, like we are discussing right now. It's a proxy metric, how good our machines are used or how little our machines are used. So like we scale out, discuss, and so our utilization grows and it sustains high for a period of time. Then we make a decision, OK, now is the time to grow. Now is the time to scale out. And so we scale. And then as a consequence, overall utilization drops. And then we watch, wait and watch, measure and watch, measure and watch, see what's going on. And then we take action. At some point, it is Christmas time and people are hitting hard. And then after that, Christmas is over. And then people go and return stuff. And so it is January time frame. And after that, returns period is over. In February time frame, we have some people doing their own stuff. They're not shopping too much. And at that time, your utilization numbers might actually trigger down, like low. And maybe you notice that the utilization of the machines overall is actually sustained below 5% to 15% for a period of more than five minutes. So you start counting, and it went below 15. And you count. It stayed below 15 for five minutes or more. Then at this point, you decide to do something. That thing is called scale in. Basically, you're going to shoot a machine. You're going to kill one or maybe more. You decide what you want to kill. And so you can kill a certain number of machines. And what will that do is your utilization will improve. And I should not draw it here. I should draw it from here. It will improve because you just killed a bunch of machines. So your utilization overall will grow over time. And that will happen something like here, 15, 85, and 100. So your utilization numbers are low. And it stayed low for five minutes. Then you brought in some kind of a weapon. And you decided to shoot your cows. And then this metric will start going up again. You shot your cows. And so it stayed below the threshold for five minutes. And so you shot some cows. And then, yeah, it came back up. That's a way of thinking about scale out, scale in. However, there are certain scenarios where these concepts do not apply. And I will give you an example now. An example is like this. You have this thing called the Redmond Fire Department. In every city, you have it. Some fire department, some fire service. And there are fire trucks basically lined up, four or five of them, sitting with the driver basically doing nothing but keeping the trucks ready and idle. And all they do is, and it's a good thing. I'm not saying it's a bad thing. I'm saying that these trucks are idling all day long, 24 hours. And ideally, I would like these trucks to be idling 365, 366 days a year, every year, perpetuity. That's what I would like to have. And I will gladly pay taxes for that. Because I don't want to utilize these trucks. And I am willing to pay money to keep it like this. I want them idle. And I want these people just busy shining their trucks, making it look good and all that. So make it shine, make it polish, all the wheels and everything. It looks fancy. Kids are going and visiting the fire truck and all that good stuff. So 0% is my goal, not 15. So it's a different game that certain times, certain cases, especially when you need something known as a very quick response time. When you need immediate response from some kind of emergency service, it doesn't have to be fire department. But I'm just saying, generally speaking, if it is an emergency-related service that needs emergency related service, that needs a very low response time, you actually want to have a decently low utilization. Keep your utilization low so that your machines can actually respond quickly. That's a way of thinking I want to remember. Because this may be missed. People tend to go and just boost their utilization every single time because that's the right thing to do. But let me caution you that it is not always the right thing to do. There are certain cases when you actually want utilization to be as low as 0. And I don't want anybody calling 911. I don't want anybody calling the fire truck at all. And therefore, I will invest my tax dollars to actually have these multiple trucks sitting and idling, doing nothing but practice. That's it. That's all I want from my fire department. And I'm willing to spend my tax dollars because it's an emergency service. And this utilization is low. And it's a good thing. On the other hand, this response time is the victim when you reach a very high utilization, say, 85%, 90%. If you use those utilization numbers, response times goes really long. And that is something that you want to be aware of, that your responsiveness of the machines that you will deploy will actually be going in the wrong direction if you are utilizing your machines too much. This concept comes from a theory called queuing theory. And you should read about this simple idea, which is it's actually a full-fledged mathematical science called queuing theory. That's where these concepts of responsiveness and utilization come from. And so as it applies to this discussion about scaling out and scaling in and bringing that up for our purposes, when it basically boils down to this, that you have to have some metric. That metric is basically a proxy to the customer. What does the customer want? Do they want immediate response time? Do they want low-cost solutions? Do they want high throughput or some kind of a very high utilization service? Whatever exactly what the customer wants is what should translate to a relevant metric that we need to measure. And then measure and keep track of. And based on this metric or metrics, I should say, that we collect, that we take action. The action may involve scaling out or scaling back in, one of those actions, primarily speaking. And that's what you will do when you have certain metrics that you need to accumulate and collect. And this doesn't happen magically. We have to actually do it. So that brings up to this topic that we want to talk about, which is called monitoring. Every cloud service today, all the big names, except not the small ones, don't do it. But all the big names give you some method of monitoring, some monitoring tool already. They give you for free. Sometimes they charge money for some premium services that you want to measure. And they will give you these tools already. Some other clouds do not, like, for example, digital ocean does not. They don't give you anything at all, except some rudimentary pictures that you can see here. You may have seen them, but maybe not. So it's very rudimentary. What does that mean? What does it boil down to? Is that we have to monitor ourselves. Now, to monitor, there are a bunch of different tools that you can use to monitor yourself. Now, to monitor, there are a bunch of different monitoring solutions out there, including commercial and open source. Here is one example of an open source monitoring solution that is in this exercise. It's called Nagios. And I would like you to run this on your own. This thing is the industry's standard open source product, which is available in GitHub. So you can go to this location and grab that source code somewhere here. You will find it, NagiosCore. So here is the product, NagiosCore. That is what is used in the industry quite a bit to monitor machines out there in the cloud. Like this, there are many open source solutions also available, like Ganglia, for example, is one. And here is the site for that, another open source product. Another one that you may have heard of or not, but here is one called Sensu. The name of this product is Sensu, and the subtitle is monitoring that doesn't suck. But that's this product. It's also open source, by the way. They have a commercial offering available, but I know it is open source. The code is open source, so you can use that one. This code product is open source. You can just download from here or go to GitHub and get it from there. So the source codes are available. Now, the commercial offerings, if you look at these cloud companies, for example, they will give you a method to systematically monitor their cloud. They have different names for it, like in this case of Amazon, they call it CloudWatch. And it is basically collecting metrics for us. So we can, in practice, browse for metrics that they will measure for us already. And so you can identify a bunch of different metrics that you may be interested in and use them. Based on that, you can start collecting. And this tool set, this service, will automatically pick up and monitor your cloud machines for you, automatically monitor your storage consumption, bandwidth consumption, CPU consumption, any process, specific details you want. I don't think this particular solution does. But then there are other solutions out there that can actually go detailed into a specific level of which particular process in your application may be the culprit for something like, here is an example. I'm going to just bring up one. I'm just going to a real live machine right now and looking at some quick performance numbers. Here we have. So we are looking at edge top, which is a measurement method to measure performance. It's like perfmon in Windows. This is a tool built into Linux that lets you monitor performance at a process level. So you're looking at right now, we have 1.2 gigs of memory used. The CPU utilization is very low. So 1.4, 1.0, suddenly it increased. It went back down to 2.6, 3.0, 1.5. So CPU performance, we are not really worried about too much on this machine. As you can see, the CPU consumption is really low. There are, by the way, two CPUs and both of them are low consumption. However, if you look at the memory consumption, it is quite high. And on top, there is swap consumption also. And so there are 888 megabytes of swap are used. Swap space is used on this same box that you're looking at. But this is a live production machine. And so the reason I brought it up is to show you this concept of edge top, which is what this is. Or you can also, in Windows, you can see perfmon or just see a variety of characters to see the performance. In Macintosh, you're probably familiar with this thing called activity monitor, which monitors activity of your processes running in the machine that you might have. So here is a bunch of things running. You can see what the CPU consumption is, the memory consumption is, the energy consumption is, and like that. So what does that translate to in the context of cloud? It's basically this particular service called CloudWatch helps you collect and act on metrics that they create for you. And so it is limited by what they have. If you need something very specific, you need to go out to some other location. Either you run your own Nagios that you can do, like this exercise shows. And so you can go and run this on your own. By the way, I recommend doing that if you're interested in getting how to monitor using pure open source solutions. This is the best way to do it. And you have it right there. You can basically apply this software. It is somewhat involved. But there is a simpler solution these days to use Docker ready-made container for Nagios. If you don't want to go through this step-by-step sequence, which is kind of long, and if you want to cut that down short, you can run Nagios in Docker and just use that for monitoring that should be available by now. I think Docker and Nagios should be there. So Docker, Nagios, which one is the best one? I think the first one should be a good one. So let's go see how many pulls it has, or rather search it in Docker, Nagios, and identify the number of pulls. I think the first one is good. So we have about 33 stars, as opposed to three stars. So take that one. So yeah, use that one, JSON reverse. Here's Nagios container was updated two days ago. That should do. So just run this one line. It should get your Nagios container up and running, as opposed to following these steps to actually build it yourself. But that's the open source implementation of Nagios. And that is exercise you may want to do if you are interested in how do you monitor a large-scale cloud infrastructure. The difference between this tool, like you're looking at a stop here, is that this monitors only that one box, whereas Nagios can monitor all your machines in the cloud, like all of them. That's the benefit, which basically means that if you have some infrastructure or some cloud, and you have a bunch of machines spread out wherever, you can put one Nagios box, and this box will basically collect metrics from every other box that will report metrics. So each of these boxes that you have in your cloud will basically be reporting to this central location. And what you can do is, as a human being, go and read. And what your control logic can do is read the reports from here and take action, whether to shoot a cow or to add more cows. You can do that through automation. But that's the overall view of how would you monitor a cloud infrastructure using complex software such as Nagios that you will run inside a virtual private network, virtual private cloud, or whatever you can. Different clouds call it different things. But it's an isolated area where you run your own monitoring infrastructure and measure it. Similarly, there are other commercial solutions like you're seeing here. And then there are more. And my favorite one actually is this one. It is called, oh, forget its name, Datadog. No. Let me just search for it, Datadog. Which one was that that I really liked the last time? How do I forget that name? New Relic. OK, I got it. The name is New Relic. This is a very good and free solution. It is available in this site. And it's actually really simple to use. And let us actually run it very, very quickly. So you will get a real life perspective. So we'll go to that site. And what this does is basically have you install a reporting engine on one of your boxes. Each one of your boxes need to run some software right here that will collect the statistics and send it over. That's the idea. So in the case of Nagios, you have a cloud. You have these boxes. And you have the Nagios server that is probably sitting right inside your cloud. And so you have these little agents that collect metrics on this box and report. And collect metrics on this box and report. And like that, report. In the context of this service that I'm talking about, it is a beautiful service that works very nicely. The name is New Relic. And it is free to use. The idea essentially is that you will have the same setup except not this. And these guys will report to New Relic. And you will visit the New Relic service for understanding what's going on in that site. You can just visit that site and find out how is the performance of these boxes. And it will aggregate the statistics for you and present it in beautiful looking graphs and all that. It does that already. So in order for us to play with that very quickly, we need some machines that run the agent. And then this agent will then interact with your service, like the New Relic service, for example. And connect like that. And then you can go and see the results, the reports, that this guy will collect for you based on whatever these machines are sending in terms of performance metrics. That's the idea. So they will collect metrics and send it over to this New Relic box. And we can get reports very easily right there. To enable this in a quick exercise, the thing that is needed is that you need to have a machine that runs an agent called the New Relic agent. And then this agent will collect performance metrics, send it over to the service, the New Relic service. And that's it. That's all it takes. So as the simplest example will be, in your cloud one box, run the agent, and that's it. You're done. By the way, this doesn't have to be a cloud. It can be any location where you can run the agent. So you have to install the agent on some box somewhere, and it will report its characteristics, its performance metrics over to the central service. It just needs to run that agent. That's the idea. So in our example, what I want to do is the next few minutes is to go to a cloud. And in that cloud, we will install this agent. So from this site, New Relic site, we'll log in and say, OK, let me in. And so I'm in. And here, I want to actually start collecting metrics for a server. So right now, I don't have any servers. So I need to create a server for which I will go to a cloud and log in and create a server in there, like this. And so what I want to do is basically create a quick server, write that, this one in San Francisco, use my key, and say, yeah, this machine, this one. Please monitor me. That's the name. Please monitor me. Create. So it's going to be created in just a few seconds. We have an IP address. We'll connect to it. And then we'll install the New Relic agent on that machine. So that's what we intend to do. So what I want to do is grab this agent from the New Relic site and load it up on that digital ocean box that I just started. No, it doesn't want to. So this email address, this IP address is available to me. So I'm going to minimize this tab and open up my terminal that I'm going to connect to that new box here called Please Monitor Me. So here is my terminal. I'm going to connect to that box, SSH root at IP address. And here I'm connected. And in that box, what I need to do is basically in this Please Monitor Me box, I need to install that New Relic agent, which is available in here. So it's an Ubuntu box, which I need to do with this. So I will follow these steps. So step number one, and then step number two, and then app get update, and then app get install. That's it. So the next one is app get install. This agent, that's what I want to be able to do. So I'm installing that agent right on that machine. It is basically installing that agent right there. And then at this moment, it is telling us that you need to provide a license key. That key can be added with this command. But we need to have the license key in here in the command. So we just go and ask them, hey, what is our license key? And they just give it to you right there. So we grab that line with the license key included, copy it over, and run the license. Then we start the monitoring daemon itself. So this daemon, when we start, it's starting the New Relic system monitor daemon. Now we can check it out, see if it is running. And you can see that it is actually running here. The New Relic system monitoring daemon is actually running right now. What does that mean? We go back to New Relic, and we'll say that it is telling us that it is searching for server data. Now on that box, as soon as the box will report back, you will have a server showing up right here. It's called Please Monitor Me. And it is actually monitoring your CPU, and load average, and physical memory, and IO utilization, network IO, this and that, everything. It hasn't done any reporting as such, but you saw that it is connected. So what I want to do is go back to that machine and install something over there so that it actually does put some load over there. That's what I want to do. So I will install some applications over there by running this Docker on that machine. And so what I'm doing basically is installing Docker on that box. And so that is what is going to happen right now. And once I have Docker, I can just quickly run some load on this machine by putting some containers. And then we can see that this report actually populates with actual meaningful information about CPU consumption, about average load, about physical memory, about how many cores, what operating systems you have, what applications and what processes are running. You can see that thing is actually getting to update. And you will see that populate just in a few minutes. Data will start to come in here. You can actually see some of that as soon as in the next few minutes, as soon as this thing finishes. We should be able to run some Docker containers right there and hit it hard as an example. And so you will see, as a result, the monitoring aspects of this New Relic system monitoring beam will start reporting your machine characteristics over to the service over there. So now we have something up and running right now. I'll just say Docker pull nginx. And then I'll run nginx as a container on that machine here. So this will basically bring down a ready-made nginx container, and I will invoke it. And then I will hit it. This will run some load, basically. So very primitive load right now. So Docker run nginx. This should start a Docker container running nginx application. So it is running right now. We can say curl localhost. So see, the connection is refused because we did not expose ports. So I'm going to run Docker command again and export minus p at core 80. And then this should actually run. Did I type a mistake? No. Oh, I did not type a D. Docker run minus D. So now we have Docker ps-a. And it is running something. So we should be able to say, curl localhost. And we should see something there. Great. So now we have an IP address that we can hit from the outside. So this IP address, if all of us hit at the same time, we should be able to see some activity reported back in Neuralic very quickly. So hit that site. And you see this page coming up. You can just hit it hard a couple of times. You can see activity showing up right in this location in just a few minutes. You can see processes reported and activity, top memory consumers, top CPU consumers, which processes are doing what. You can see that by memory, by CPU. You can see the overview of overall what's going on. You can see some network IO happened. Some disk utilization happened. Some average load that you can see. And these aggregate statistics are being accumulated because of the simple idea that we just did, which is to run this agent on that machine, which is basically reporting stats. And we are looking at those stats here in that service. That's the way, the quick and easy way of monitoring using Neuralic. And that is what you're looking at, which is very simple. Very simple to implement. And if you go and increase number of machines onto this particular Neuralic service, then they will start charging you. So this is not really a open source product. I just want to make sure that you understand that this is not open source. It may be free, but it is not open source. Similarly, in this example that you're looking at, here CloudWatch does very, very similar things for your machine that you will have in the Amazon Cloud. In fact, if you go and launch an instance in the Amazon Cloud and say, give me a machine, and say, give me a T2 micro, give me some configuration, and you say that I want to run it in my REST1C, and give me a public IP address. And then here, you can enable monitoring. You can say that I want to enable CloudWatch monitoring. And when you do that, additional charges will apply. And this is for detailed monitoring. The idea is detailed. If you want just basic monitoring, that is already available to you. So for additional charges that apply, there are certain pricing structure that you have from here that you will have to pay if you decide to use that extended detailed monitoring as a choice that you can select this. And then this will give you much more detailed information than is otherwise available for free. That's the ability to monitor your infrastructure directly within the Cloud Pro error solution, which is, by the way, a very good one if you want to launch a bunch of machines in an auto-scaling group. And that is something that I want to talk about next. What I want to be able to talk about is to illustrate to you this concept of an auto-scaling group as to how it helps us scale from a usage perspective. But before we switch into that topic, I want to ask you if you have any general questions regarding the need to monitor, the ability to monitor the solutions out there available in terms of what different types of methods that you can apply to measure performance, characteristics, usage characteristics, consumption characteristics of your resources, like machine CPU consumption, all these metrics that you're looking at right now, for example, these ones. For this particular machine, you have all these characteristics that are available. This is one way to get it. This is just one way to actually understand CPU consumption, network consumption, disk IO consumption, network IO, which process is the main consumer of memory, which process is the main consumer of CPU. You can actually tell by right here. Who is the CPU hog? It's Docker daemon. It's consuming 3.4% of CPU. And then over time, these numbers might change. This Docker daemon ran first time, and it consumed a lot because we were setting it up. But over time, it will actually shrink. It will not be as much, unless you put heavy burden, because everything then runs in that Docker containerized environment. The memory killers can be anything that you actually consume. So for example, in here, you have this NGINX container running. So there is a NGINX process getting run inside, so these instances of NGINX application are running in the container. We can see them, both of them, by going to the machine directly and saying, you know what? Show me what NGINX do you have running. So PS-EF 5.0 NGINX. And you can see those processes there. There is a master process. There is a worker process. There is another master process and a worker process. That's what you're looking at, these two master processes showing up here. And those are the two instances of NGINX container running. So you can basically, at a large scale, now imagine if you actually have 10,000 machines and you want to collect aggregate statistics across your cloud, doing this one by one is impossible. And that's where these kind of services come into play to help you understand how is your infrastructure doing, what are the hard spots, and where your CPU is getting hit hard, where your network is heavy. And in fact, you can tell by just this picture that at this time, at 7.15 or so, we were hitting the network quite heavy. What were we doing? We were installing Docker, installing NGINX container. So it is going to download a bunch of things. And the speed of download at the peak was 11 Mbps. That you can tell by the bump here, the blue bump. It was receiving data, receiving data from Docker Hub. That is what Docker getting installed. Docker image is getting loaded. You can see that. The transmission is very low because not many people are hitting your machines from outside. You can also see the disk IO utilization as opposed to disk capacity utilization. This is the IO part. So how much disk read and write are happening at the same time like in this time frame, which is matching this time frame conceptually, actually practically. So the same window of time when the network is active, that the disk is also active for a blue color. Blue color means it is going to this virtual disk adapter, VDA number one. That disk is getting hit with disk IO. That's what you're looking at. If you keep further looking into how much memory consumption, how much swap consumption is happening, you can tell that in this box, there is no swap. The pink color is missing. If you keep looking, the CPU usage, it shows you load average here, which also matches the CPU consumption also shot up as we started to install Docker and then it went down. The overall CPU consumption, actually, right now is fairly small. You can see that install, and H-top will tell us how much consumption is going on on that box. So we'll just see H-top. Show me what you got. And you see that it tells you that there is one CPU with 0.4% consumption, 0% consumption, very low. The memory consumption is 85 MB out of available 489 MB. And that's what you're looking at from a CPU percent consumption perspective back in here. That's what the picture shows you, very low CPU consumption, because nothing is happening. At a process level, it gives you details like that. And similarly, you can extend this particular service to go granular and actually help you understand application performance monitoring. So if you have some application that you write using one of these languages, Ruby, PHP, Java,.NET, Python, or whatever, then in that case, the idea is that you will then add on some extra monitoring capability within your application. So whatever language you write, let's say you write.NET. So it will tell you that if you are running in Windows Azure and you are using Azure websites as opposed to Azure VM, you can choose whether it is an Azure-related ready-made website or an Azure virtual machine that you want to run. And in there, you then run a Windows machine or a Linux machine. So you can choose whatever you want. So if you decide that in your Azure.NET environment, if you want to run Azure VM with a IIS application, here are the instructions. You basically go in, download the.NET agent for 64-bit systems, and then you get the license key, which is here. Then you restart your web server and send traffic. So wait for data to arrive. And then you can see how your application is performing. At that level, you are going to be able to receive very granular information about your specific application that you write in that particular language or that framework. And that way, you can profile your applications as to how your application is behaving in the context of your usage in that particular cloud. In the quick example that I was able to show you, which is the server-related metric, are important but very rudimentary in terms of top-level CPU, memory, disk IOs, disk space, network IOs, network capacity, process-level details. That's pretty much it is limited to that. If you are writing custom applications or if you are using ready-made applications, you need to also monitor how your applications themselves are performing, which requires custom work. That custom work basically boils down to, like the instructions show you, if I'm running a Java application, you review the license key. You get the download to Java agent. And it is basically a zip file. You are to send it over along with your application that you write in Java. And then you select your environment, which is either Mac or Linux or Windows. And then you install that JAR file. This New Relic JAR file is what they gave you when you downloaded the agent. This agent will help you and your application monitor application-specific performance metrics for your Java application if you go that route. And very, very similar to this concept is Nagios, by the way. Very, very similar. Though the challenging part with Nagios is that it is hard to deploy. Once it is there, it works beautifully. Many companies actually use this. If you want to spend money, you go this direction. Or there are other alternatives to this New Relic thing, which is called Datadog. So here is Datadog, which is helping you understand real-time performance for your applications. And it is granular, meaning it is at the level of applications, as well as your entire stack that may be deployed in the context of a large cloud. And yes, it is a paid service, so there's a pricing table for that. But at least these services help you in getting to the point where you can easily get characteristics like this that this picture attempts to show at an aggregate level. How are your machines doing? How are your applications doing? How is the customer responsiveness of your applications? Are the customers actually getting to connect to your services? Are they able to consume those services or not? How are they performing for the customer? That metric you can get from an aggregate perspective in charts like this, which are nice-looking. These are custom-made by commercial companies, so they make efforts to make it happen looking good. However, things like these, like Ganglia or Nagios, they may not be as good-looking, but they are actually more granular in getting the data that you want. It may not create a fancy-looking report for you, but it has all the data that you need and probably more. So it really is up to us to choose what solution we want to apply in our context, in our application, for example, in our business that we may be working with. If your organization supports open source, use these things. It is not that hard, and I have an exercise that helps you in making that happen. You can actually do this implementation yourself by following along this particular write-up that you're looking at in your Slack chat. It will help you understand and also implement. However, some companies don't want to use this because it is not easy to maintain, easy to manage, and they tend to outsource. And in that case, it is totally fine to use commercial services like Neuralic or Datadog. They are very good ones these days, very well-known in the industry for monitoring your cloud infrastructure. There are, by the way, many more companies like that. In fact, you can just go and search, cloud monitoring. And then you will get a lot of list, a long list of here's the manage engine. There is dataloop.io, Datadog, and 50 tools. There we have a long list, a bunch of tools come along. And AppDynamics, in fact, a person from a previous group like this just got a new job with AppDynamics. So that's what I know. I know this company from that person. He just landed a gig there, AppDynamics. CloudWatch, as you know already, AppNeta, AppTerminity, I don't know, Bitnami, BMC, CarAdvisor, Cisco Cloud, and multiple pages. So every company that you know of that is trying to play will have a solution for monitoring. And Boeing uses HP monitoring tools. I think HP, what's the name, OpalMu or something? That's what I think Boeing uses, which is systems management, OpalMu, and a bunch of other things that you already have a contract with HP for infrastructure monitoring. That's what you use internally, which is fine, fine to use, no problem there. Basically, you outsource the work to HP, HP does it for you, which is nothing wrong with their approach. But I want to point out that there are other solutions available that might make sense. Like this one, for example. You have the capability and you can potentially use open source instead, instead of using commercial services. That's a choice at the end. By the way, one of your Boeing HPC group actually uses these tools. I know the guy who is in your high performance computing group at Boeing that does modeling for your airplanes told me, and he actually had shown me your internal deployment, how you do high performance computing using Linux in a large scale cluster you have internally. And Linux clusters that are dedicated to HPC are using this software that you may or may not have heard. It is called Beowulf. And this is how a HPC cluster is traditionally built. Beowulf cluster, this cluster is the classic use in scientific computing, computational science. Basically, high performance computing uses this and you will have Nagios and Ganglia built into Beowulf. I think Beowulf, the open, the source code is available. And that's what you use, by the way, in your company in Boeing right now. So, which is not for your traditional mainstream. Just for HPC, you use open source, but not for your mainstream corporate data center. I think you use operations manager and the open view solution from HP. But you have the choice these days. Even before today, the choices were there. Nagios is not new, it's an ancient product. So is Ganglia. They are probably more than 15 years old, both of them. So, let's finish that monetary discussion. The key things to remember is that these kind of services will help you application performance level as well as machine level. They also extend these ideas into other concepts such as monitoring your mobile applications. For example, you might have iOS app and Android app and Titanium and Unity and this TV apps and all that. You can also monitor how is the performance of applications that you might have in the App Store, like the Apple App Store, for example. And so you can get metrics for users that are using your apps on your hand and devices through the same method. You can have browser-based applications that you may want. You can also collect. So, here's an old alert, apparently. I have an old alert sitting. It is telling me for this IP address in May 19, 2015, in the morning at 9.14 a.m., there was this trigger that happened, which said memory more than 70% for at least 10 minutes, and that was about 612 days ago, which basically, that machine is gone, by the way. But apparently, that message, which is an alert of violation, memory exceeding 70%, as you can see in this screen right now, was an old browser activity report that I must have received for a machine that I may be running at that time that I still see it there, which is kind of funny. But we'll ignore that. So, bottom line, you understand that these tools are there. You should use them or use open source, like this. It's a very easy one to actually run. Just one line, you have it running, like this, for example. You could potentially take this one line, like this, and run it right here, and see what happens. As simple as that. It is going to basically build up a Nagio solution for us. Ready to use, and we're going to read how to use that in here and how to access it on that port number 8080. That's where it will pop up. So, we should be able to go to that IP address and go to port number 8080 and wait for it to finish. So, this thing has to finish, and then we should be able to see Nagios right in that window in a browser, in port number 8080. As soon as that thing finishes, which it's still doing. But you get the point. So, it is easy to run. That's the bottom line I want to communicate that. I think I did that. Having said, from a monitoring perspective, the monitoring metric, the statistic, the aggregate data that you collect is actually applied in an auto-scaling group. So, what exactly is an auto-scaling group is what we want to talk about. Next, that's what we want to do is actually this discussion dependent on the previous Terraform exercise, which I could have shown you a real live example, but somehow we ran into that error, which is unfortunate, but we'll fix it. But until that time, I want to briefly discuss with you about the concept of auto-scaling for which, as it describes here, to create an auto-scaling group. To create an auto-scaling group, you will first need to define a template that your auto-scaling group will use when it launches instances for you. And that thing is called a launch configuration. So, at this moment, we have to create a launch configuration, which will basically define how big the box is going to be, for example, here. It will define, you want a small box, sorry, you want a small box or a big box. And also, you want this box to run 1-2 or CentOS or your own machine image. What do you want to run? It's really a choice that you have. What image you want to use, that AMI has to be specified. And then you also have to specify whether... I think I made a wrong button. So, it is going to give me this wheel that I don't want to receive. But, yeah, I think I saved it. So, it also decides whether you want a small machine or a big machine. That choice has to be made by you. And these metrics, sorry, this information about which machine image I will use and what size of box that I will use to scale, that piece of information is actually put to use in your launch configuration. So, you will have a launch configuration that will look something like this. Let's go create one. So, here this auto-scaling group can be constructed provided you have a launch configuration. In our example, in my example, I don't have one right now. So, I'll create one. And in this configuration, you have to specify what's the name. So, you can call it whatever you like. And you want CloudWatch monitoring or not. And I'll say yes. And then in the advanced details, I don't need to modify, so I'll leave it alone. I'll go forward to add some storage. Each of these machines that I would like to have will need to have 100 GB of storage. And general purpose SSD is good. And then I will configure a security group. I will select an existing security group I already have. So, I'll choose these guys. And then I'll review. In here, it is saying that, you know what, your improved security. Basically, it's open. So, we'll worry about security later. But right now, it is going to select this image. So, Amazon Ubuntu HDM SSD Zaniel 16.04 image. That AMI ID is what it has chosen because that's what I began clicking with. If I don't want that AMI ID, I can delete it and get some other marketplace AMI ID if I like. If I don't want to use anything in the marketplace, I want to use my own, I can go to my AMIs and use my AMI. They will list up here. You can create your AMI by first creating a machine and taking a picture, an image. That becomes your AMI. So, you can then use your AMI to start a 64-bit machine. And then you begin. You can go that way. Alternatively, you can say that, you know what, I want to use a just Ubuntu machine. So, select that one and then create that launch configuration like we are discussing right now. So, back in the launch configuration itself, let's go create that launch configuration again. So, back in here and in the launch configuration. What just happened? Okay, visit the homepage, sign in to console, EC2, launch configuration. And here I'm going to create an auto-scaling group which requires a launch configuration. So, where is my launch configuration though? I want to create one. And so, it says create a launch configuration. So, great, do that. And I will use my Ubuntu machine, which is this one as the basic image. And I will configure it to have a name. So, I'll call it cloud genius. And I will enable monitoring, add some storage, 100 GB for every machine that I will have in my scale-out plan. So, in this plan, I'm basically defining the size and the type of the machine and which operating system I have. Specifically, what I'm doing is I'm saying that the configuration that I will work with will have a machine that will have, say, 100 GB storage. It is going to be a T2 micro. And in that, I will run Ubuntu. That's generally a launch configuration that I'm specifying. This is a template that will be used to scale out or scale back in. When it knows to grow, it doesn't know what size of machine you want. It doesn't know how much hard disk you want. It doesn't know how much or which operating system you want. It needs to be told that information. And that is what we are doing, essentially, in the launch configuration here. We are saying that, you know what, use the name Cloud Genius and use Linux Ubuntu and get 100 GB disk and like that. And use this security group that I would like to have. So, that is this one. That's what I'm basically calling out. One thing that I skipped, so I want to go back there and call out. Then we actually request spot instances here. The spot instances are not available. It says not available. The reason they are not available is because our size chosen is the T2 micro. And so, we have to cancel it out and say that, you know what, create a launch configuration that uses Ubuntu server, but the size is not T2 micro, but maybe a medium. So, N3 medium is a good size, but you can see this one, N3 medium. And you can now configure this one, create a launch configuration by the name Cloud Genius, and now you can request spot instances and enable monitoring. And by doing this flag, you can actually save money. And you can say that I want to machine with a price of 0.01. And you will get it. As long as the price is, you know, these prices are below your threshold, the maximum price. If you want to have high availability, you want to maybe put a price of 1011. It will be higher than both 1A and 1C, not 1B. They have a ridiculously high price for a reason that you know already, which is they want to clean up. They want to clean up Chico machines from that particular region, 1B, so they bump the price, basically cleaning up, making room. But if we have a batch process that we want to scale, we can just select a price that is higher than 1A and 1C and put it here, 0.011. Now we are higher. We can say enable cloud watch monitoring and then add some storage. And, you know, maybe 36 GB is good. And then if you have security groups, I'll say, you know what, these are good ones for me. And so hold on. Yeah, this security group is a good one. So that's what I want to choose. And I review. And it says not eligible for free tier, so I understand it. You can improve the security. It's open to the world, so I get it. But now I'm defining a configuration, a template that I will reproduce over and over and over again as I want to scale. In that template, it is Ubuntu entry medium. And the disk size chosen is 36 GB. That's the disk I want. And this is a set image that I have set for my usage. And that image is what will be used. So it is 36 GB. And Ubuntu and M3 medium. And the price I have set is 0.011. That's the max price I'm willing to spend for one box for one hour. That's my launch configuration. And I just create one. It requires a key pair, and I don't have any. So I did not find any because I deleted them. So I'll create a new key pair and call it test, or rather auto launch config, or auto scaling, maybe. Auto scaling key pair, I just download it, and I save it in my folder there. And then I create a launch configuration. So it knows what to do when you say scale out. It will know what to do. So exactly what to do is to get an Ubuntu machine at this price or low, 36 GB memory, 36 GB hard disk, M3 medium size. And using a key pair that I just put in, it's called auto scaling. That's the name I chose. So using this key pair, it will inject into the box and get ready to scale out. And then I say, OK, scale out. That's the idea. Now, you can further define this auto scaling group now that we have a launch configuration ready. We can now start to build a group of machines that will operate together. And we'll need to have it a group name. So we can call it our group, cggroup. And we can start with so many instances to begin with. Like in an example, what I would like to do is, let's say, if I erase all this. And I will quickly construct an illustration to help you understand this concept of auto scaling by basically taking this picture. I'm taking some room here, so I'm going to kill these guys. And so what I'm going to do now is a little bit adjust this here, make it a little smaller like that, and then copy. So here are two different availability zones like this machine here and another one here. And these people are not there yet. There are just a few customers right now. And so what I'm going to do is essentially use a service called the load balancer, elastic load balancer service, and begin with just two boxes. One is in availability zone US West 1A and US West 1C. Those are the ones that are really affordable for me because I want to use part pricing. And I see that West 1B, the pricing is very expensive. So I'm going to be using machines that are like this maximum, 0.011 is my willingness to pay. And so I want to use this much amount of money, not here. So I'm not going to 1B. I'm sticking with 1A and 1C. And there I will have to have two distinctly different subnets. These are private subnets and I will have to have these machines going. In addition to those private subnets, I will also have a public subnet somewhere, like here, for example, and another one here, for example. And on these public subnets, I will have some boxes that might behave with a different purpose in life, such as a NAT box and another NAT box here. So this is a NAT machine. Let's call it NAT C. And this is your NAT A. So those two NAT boxes are basically allowing outbound traffic to the internet through a gateway that you will have in your cloud. So this is a gateway available to you. Any cloud will give you a gateway to go out. It's like the Comcast router in your home. It lets you go out. And so this NAT A and NAT B will allow that traffic for these internal machines to be able to connect to the world outside. And so this machine, if it wants to go out, it will go like this through the NAT and go. And that's like a download action, for example, or install Docker action, for example, or some kind of a Docker pull action, or install some set of something. So those things, you will need to run on those boxes. And that action involves accessing the public internet, for which they need to go out to the public internet in a fashion which is secure. And so even though they may be on a private area, like, for example, these are private subnets, but this is a public subnet. This is also a public subnet. It has this NAT device, and it also has NAT device. So the idea that you will see now is that we have constructed launch configurations for these units. And we want to begin with number two, one here and one there. That's what this illustration is asking. Start with two machines. And by the way, put that in my default VPC. And use the 1A subnet. And use the 1C subnet. So I have 1A and 1C that I want to use. So two instances. Put them in the default VPC. Call the group called C group, CG group. The name of launch configuration is Cloud Genius, which basically means that I will have two machines, 1N1A, 1N1C, by the price that I have put in place of this price. So very cheap machines. Two of them, one here and one here. That's what I'm creating right now in an autoscaling configuration. And so it says in here that each instance in this autoscaling group will be assigned a public IP address. Now, this is bad. We don't want this. So I think we have to have new subnets, because they're assigning a public IP directly. We don't want that. So we have to basically construct VPC subnets separately for us that do not automatically assign. So we have to create a public subnet and a private subnet for ourselves that do not automatically assign. So there is this 1A and 1C that we have. The routing table is through a gateway. I think we have to construct our own subnets. That will confuse you less. So I can basically do that. Let's see how much time we have. So only eight minutes. Not possible. So not possible to construct this in time for us to finish. So let me at least conceptually describe this idea to you. The idea essentially is that once you begin with two launch configurations, once you have these two machines going, if the pricing that you have set here is higher than what the market is asking for, you will get these machines allocated and they will start. Now you look at your performance characteristics at that time because you're looking at your CloudWatch metrics that you receive. And then you can set a policy for automatic scaling. Scaling does not mean grow or just grow. It also means shrink. So you have to set two distinct portions of your policy. One that defines this sheet. One that defines how to grow. One that defines how to shrink. And the policy that you have to write is basically what we just discussed at the beginning of this session, is to look at the overall metric. If the metric is crossing a threshold, there's some threshold here. If you cross the threshold and you stay above the threshold for a certain period of time, elapsed time, elapsed time, some amount of time, say five minutes, 10 minutes, whatever you decide, and after that elapsed time is over and you still are above the threshold, you're still above the threshold, then you trigger. And you trigger, in this case, you will trigger a scale out. Because your threshold is crossed, your metric is going above the threshold, you need more resource to scale out. So basically you increase the resources available to you. You add more boxes at that time. That is what you will do. You will take this box here and you will say, okay, one more. And you will say here, okay, one more. Like that. And then you will have one, two, three, and four things to balance your load against. These people who are hitting your load balancer will get distributed across four machines like that. That load balancing will happen through this load balancer. But the policy that we will use to extend and increase the number of resources that you have will basically be driven by the grow policy and the shrink policy. The shrink policy is very, very similar to this grow policy, except it is opposite. In that, you have a threshold that you see that it is actually below the threshold for a certain elapsed time of, say, five minutes, 10 minutes, you decide, and it still remains below the threshold. At that time, you will trigger. And this time, the trigger will be in, which is basically shrink. You cut, you shoot your cows, and that's how you decide to shrink because you have a threshold that you decide. This is basically driven by a metric that you know it is constantly getting that metric out for you, and you observe based on your grow and shrink policy whether the elapsed time has passed, and still the threshold is crossed, and your metric value is actually above the threshold. And once you trigger something, you want to give more elapsed time after next occurrence of the grow policy. What does that mean? So you have a grow policy, for example, here, and a shrink policy. And so you decide that, you know what? Threshold is here. I am above the threshold for a period of some time. I'm still above the threshold. I need to trigger. And I trigger scale out. So I increase my machines, which means I have to now spend some time, elapse. Elapse more time before I can take the next grow action, before even I think about the next grow action. I need to let some time pass. This time has to pass, otherwise you will have a race condition. You will start a machine, shut it down. Based on dynamic behavior, which is not really optimal. So you have to spend some time and really get to a point where you see that the threshold actually changes. You have to give this threshold to actually trigger down below and come down to a new steady state, which will happen just because you trigger and you increase the number of machines that you have. So it will trigger a new steady state. It's a transient state. So this time pass is basically the transient state. You have to go over the transient stage to a stage where you think will probably be a steady state or you hope is probably a steady state. And then you trigger your grow policy and start studying again. How is the threshold? Is it above? How is the metric? Is it above the threshold? Or is it below the threshold? And in that case, if it remains within the boundaries, you do nothing. You don't scale. Don't scale out. Don't scale in. Do nothing. Just watch, monitor, and see where the performance is going. And very, very similar to this is the other policy, which is the shrink policy that you already understand. You can see these things in action in a variety of clouds. Let me now get that other exercise working, which did not work today. I will get that to work. And then we will play this scale-out business. We'll also play a couple of other scenarios, which are very interesting and very important, like I said in this illustration. And I had a view. Zoom out. OK. Let me draw this one more thing. The scenarios that I want to be able to make sure that you get to see and make sure that you get to use, those scenarios that I'm describing are somewhere here. So I'll grab my pen. And the scenario is that in your office, you will have some office somewhere. And in that office, which is not, by the way, in this cloud. So the cloud is not where your office is. Your office is separate. You need to physically live here in your office, but logically place yourself in the cloud, logically. You can do that. And also the vice versa, which is if you have a cloud and you have a big gigantic campus in your company, which is a large company that you have, and you want to bring this cloud in your company, you can do that. It is called virtual private cloud, by the way. That's what exactly it means. You bring the whole cloud and make it logically become available in your building, logically. And that's the classic sense of the word virtual private cloud. It is private just to you. You bring the whole cloud logically inside your company and on a subnet in your building. So you have a building here. In that building, you have a network. And in that network, you might have a subnet area called out. You bring the whole cloud and dump it right here, logically. Logically speaking, not physically, but logically you can operate like this. This technology, I think most of you already use it today. Do you know what technology I'm talking about? The technology that you use today is that you connect to your corporate network through a thing. The thing is sometimes called a VPN connection or something like that. You've done it. Most Boeing and Microsoft employees I know do this. Say if you, this person, are in your home and now you want to connect to the Boeing company, what would you do? You take your laptop and you do a VPN connection back to Boeing company network. Say this is Boeing. And so you do this VPN connection that you establish and you're basically logically placing yourself and your laptop inside the company. That's what you use. What do you call that thing? You call it VPN or something else? In Boeing, we call it VPN. Okay, so then that is it. So that's what we will use. And so that infrastructure that you need to have here somewhere is a VPN server that allows these remote employees to go into the building logically. So you logically place yourself here. Logically, you are here. Physically, you are here. And so that's a remote employee trying to come into the company. That's a VPN solution. It's very easy to set up one in the cloud, basically another Docker container. In the other concept, what you could do is the reverse, which is not the employee going to another location logically, but the other scenario, which is you bring the whole cloud as if it is your private property in your company. So you have this building, which is the large company that you might have, and you have a gigantic data center. But that data center is not enough for your needs. So you need more resources until you bring this whole cloud out to your company logically. That is also possible through networking technology. That is what I think most of your companies already use. And we will see how this scenario is also possible, which is also VPN technology. You can see some of that references here in the network section where direct connection is available. This is a very expensive way to do it, but a very high performing way to directly connect to AWS, for example. So you get started with direct connection. You give it a name if you are in San Jose or Los Angeles or wherever you are, the location nearby. And this is, by the way, the Northern California region. If you switch over to Oregon, it will give you Seattle elements. So you can get a direct connection to this Equinix Seattle or Hillsboro, Oregon, or Tier Point Seattle. That's how you can get a 10-Gbps line directly connected to the Amazon cloud. And so this is basically a fat pipe from your building out to the Amazon VPC. You can construct this, it costs money quite a bit, but then that's a solution. I cannot demonstrate this to you because I don't have the physical capability of connecting with a 10-gig network in this location. But that's the idea to establish one such a method that many companies that have that need will actually use this method to make a direct connection. So you don't have to worry about this VPN technology. Basically, you just have a fat pipe going straight to your private cloud. So that's what we did discuss today. I want to now thank you and pause the recording and work on fixing the issue that I ran into, which I will do on my own and I will report back here in Slack. And you should be able to work that exercise now that we did. So with that, thank you very much. Let me stop the recording right now. Have a good night. Thank you, sir. Good night. Bye-bye.