Hey guys, we are recording this session so you have a copy at the end of the session and so we begin. What I'm talking about is this software called WordPress. It is available on WordPress.org website. The same software is installed on this website as a service, software as a service. We are not talking about this, we are talking about this software and we want to deploy this software and it requires certain dependencies. Those dependencies are Linux, Apache, PHP, and MySQL and so we will take these dependencies and actually restructure them in slightly different fashion. Something like two different parts, Linux, Apache, PHP, Linux, Apache, MySQL and that is how I should write, I should correct that, PHP and MySQL and this is PHP and Apache. That's how we will create two different roles. This is going to be one role, this is another role and we have a base role that will include some basic stuff, the base role will include things like Git, super user privileges, sys admin functions and a bunch of other things you saw probably when we created a base role function. In this role, we will call it the database role and this role we will call it the web server role. So those three roles defined in place. We also, last time I think we did this session halfway through, we created a chef file like this one and this chef file helped us gather, is that the right chef file? Yes, it helped us gather these cookbooks on MySQL, Apache, and PHP cookbooks and we have placed these cookbooks in this location. These are community-supported, community-provided cookbooks so we are just using them. We will create our own cookbooks and they will go in this location, the site cookbooks location. We don't have anything yet there. We have a bunch of roles defined here, the base role contains these things. The DB master role contains base role and a recipe reference that doesn't exist yet and the web server cookbook, sorry, the web server role is pretty trivial. The base role, add PHP to it, add Apache to it and you're done. That's this definition of role that we have three of them. We have sent these roles over to the chef server. We can check that by running something like bundle exec knife role list to see what roles we got there. By the way, while doing that, we also established that we have connected properly to the chef server. It comes back with a result that shows three roles that we have uploaded in the last time. Very similarly, we can test what cookbooks we have there. In the cookbook list, we can get all the books that we have sent. You will notice this particular list does not contain the cookbook we want to create, which is the ursql cookbook. That book itself doesn't exist yet. We want to create that book and then when that creates, it will get created in this location and when it is ready for us, we will send that book over and add to this list of cookbooks at that time. We also have done these three environments that we created, the dev environment, the production environment, the staging environment. These are just Ruby files indicating our intention to classify our environments according to what we desire. We also have created data bags, two of them, one containing the user called user with these privileges and our public key reference here, and also a secret data bag containing the password 123 in development mode, development environment, and the password 123 in production environment that has been sent over. We can check in our Chef server, bundle exec, knife, data, bag, list, should show us a list of all the bag that we have. We have two bags. One bag contains our users, which contains one user called user, and the other bag is called secrets. It contains one secret called MySQL, and that secret actually looks like this JSON file. It has two parts to it, actually three parts. The ID is the name of the bag itself. There is a dev part for the dev environment and a prod part for the production environment. Inside that, we have three passwords, one for root, one for replication, one for Debian. We mostly be using this specific thing, 123, the production password for root. That's what we will actually be using, but we have all of them. We have encrypted this data bag and put that in our Chef server. You can see that by saying bundle exec, knife, data, bag, show secrets, MySQL. You will see your secrets exposed to you. It is decrypted locally. This decryption happens because the key that we have created here, it's sitting right there and is referenced in the knife rb, the very last line. There is a reference to that. That's how it knows what to do with the encrypted bag. It received the encrypted bag, it detected that, and it decrypted that with the locally provided secret. We now have the access of the actual bag content. The bag itself shows up in the Chef server as encrypted. You can see that it is encrypted there. People in the Chef company, even if they get access to your Chef server, no big deal because the bag is encrypted. They cannot figure out, they don't have the secret, only we have it. You can see that the bag shows you the bag itself, there is an item, the item is encrypted. You saw that the last time. Then I think we ran out of time, so we switched and basically went away. At that time, we were, I think, running at the point where we were ready to create a cookbook on our own. We had defined roles as admin user, encrypting your secrets, make your, and this one. This one was the thing that we did last time, so I'm going to check that and make sure to open it up to confirm, and then we proceed to the next steps. Our goal will be to primarily make the database use passwords that we have defined, which is basically here, this password. I would like that to be using this root password. Why is it not responding? This root password, Debian, not Debian, root password for production environment. That's what I would like MySQL deployment to use. It needs to pick up that detail and install the MySQL server on a box designated as our database server with that detail, this specific detail. I would like that to pick up, for which we need to create a custom cookbook. That's what we'll do next. That is where we stopped the last time. To create a custom cookbook, what we need to do is as simple as this one line. It may sound daunting to think about creating a cookbook, but let me tell you, there are tools available that will help us construct a cookbook with just one line. The whole book gets created, which contains, by the way, nothing. As you know, when you use a template to create something, it creates you a skeleton with no meat on it. We are going to grab that as is the command and keep this editor on the side so it becomes available for us and we will inspect as it creates a new cookbook. You will see that the book gets created in the site cookbooks folder right here. That's where you will see it coming up as we execute this command, which is, I'm going to run on the right side here. The command is simple. It says, knife cookbook create. Knife cookbook create will create a book by the name you choose. I choose the name YourSQL and it creates. You can see that the book got created right there, the YourSQL book, and with all these things already in place. It basically contains nothing. If you go inside, it contains absolutely nothing. Just a structure with a default recipe that is actually empty. It's all of these things are hashtags or comments. They don't make sense. I mean, in English language, they are for humans to read, not for machines to read, so they're commented out. You will also see that this name showed up magically and a license that I chose showed up here, which is Apache license. This showed up because I chose that detail in the KnifeRB asset that you know what cookbook copyright is my name, email, my name, license choice, Apache tool. These things show up automatically when you create something. It is attributed to the creator, which is the person that writes that command. Since I wrote that command, this recipe was populated by my name and the metadata also includes references to me. You know that's how a book is written to begin with, which contains actually nothing. It's practically blank. You can see that the default recipe is empty. Our idea would be to actually modify the default recipe. We want to change this. When I change this, what I want to be able to do is make this actually execute this cookbook here, MySQL. That's what I really want to run, but before I run that MySQL cookbook, I want to make this my cookbook as opposed to the community provided cookbook. Since I want to make it your SQL as opposed to MySQL, I would like to change the cookbook a little bit and then change this a bit and then call the original MySQL book. That's the plan. We would like to change this a little bit and what do we want to change? We want to use override to use our passwords, our chosen password from the data bag. That's what we want to do is override first, use my password and then do what you do, whatever you are supposed to do. Go ahead, take control. We are basically handing off control to this cookbook at that time after we override. That's our plan. Step number one would be to override. Step number two, yeah, go do your thing. Basically run the regular plain old cookbook. After I have overridden the exact thing that I want to do, you do your thing. So it will do its thing except it will use my password. That's our plan. That's what we will do these two steps. We do these two steps in a specific way. First one is to actually override. So this segment will help us in overriding. Let's go read that portion a little bit. Then the second portion is this segment, which will basically invoke the regular cookbook for MySQL server. So I'm grabbing these things as is, copying them over and bringing to my atom editor right here and putting these two steps by, first of all, removing this because that's written in English. I would write in Ruby and so in Ruby, what we are doing is basically get the passwords from the encrypted data bag that we have stored in the Chef server. That's what we're really trying to accomplish. And then call the regular MySQL server recipe in the MySQL server cookbook, which is a community provided book in the cookbooks folder. I would like to call this, where is that MySQL, MySQL cookbook server recipe. This one is what I want to call later after I'm done. So not this actually, this one. The MySQL server recipe is what I want to call after I'm done overriding and using my passwords that I choose for my encrypted data bag. Now, let us see how the overrides actually work. It's actually intuitive if you carefully read this. So I would like to read that and I will do that with you to help you understand. This is Ruby language, by the way. So it's not that difficult to interpret. It is almost English like, except you have to read some assignments. So here is an assignment statement, secrets equals, basically I'm reading my secret. My secret is the data bag item. The data bag is secrets. The data bag item is MySQL. And I'm reading that encrypted data bag item, basically loading it using Chef. So Chef encrypted data bag item load, get the secrets data bag, get the item inside and assign the value to this variable. Now that I have the variable in my hand, I will extract its root password like so and assign it to the MySQL server root password attribute. This attribute assignment, you can see in the original official community provided cookbook is an attribute available here in which they use the same attributes, but they have chosen. I like random passwords as the password instead of your chosen password, which is not I like random passwords, but one, two, three. That is your chosen password. So it is different from what the community has chosen. So we don't want to use that password. Instead, we want to use our password. So we are basically overriding these values, one, two, and three. These three values we are overriding using first a reference to the encrypted data bag item called secrets and the item called MySQL assigning those values from root and Debian and replication to these three attributes. This is how exactly you override. This is the extent of customization we just did in our new creation called your SQL cookbook. This is the default recipe in the your SQL cookbook, which happens to be sitting in your server area called site cookbooks. In that site cookbooks folder, you have the your SQL cookbook in which we have a recipe called default and that is this file. Default recipe in the your SQL cookbook. Default recipe. That's what we will invoke. That is what the changes we need to make to create this your SQL cookbook. We save it. It's a very simple cookbook, by the way. Close it. The book is ready. We want to send it to the Chef server. So we can basically say bundle exec and send my cookbook to the Chef server. So there is a cheat sheet for that always. You can always refer to the cheat sheet. And I remember this is bundle exec knife, cookbook, upload. And I think it dash O and then site cookbooks. Yeah, that should do it. So no, it did not. So let's go read. You don't have to remember anything because the cheat sheet always says and describes you how to upload something. So here is the command to upload that. And I think I missed one of the statements. So I will clear it up, paste it. Command actually is knife cookbook upload followed by the name you want to send followed by dash O and the path to it, whatever that location is, which happens to be site cookbooks. That's where we are storing our own creations are sitting in this folder. So we want to send that reference here and give it a name. I want to send this book up. So please send. So it sends. The book will go up. It will show up here in the chef server in the cookbook folder. And you can see that your SQL book is now available in this listing somewhere. You will find as I refresh this browser, so refresh and read your SQL cookbook. This one will show up in our listing and it is right there. You will see that the content of this and the attributes, specifically the recipe that we created, the default recipe, you can see that it is the one we just edited is showing up in the chef server, which is how you basically provide your book to the chef server. Now we'll also go examine our roles, especially the DB master role, which was complaining the last time that your SQL cookbook doesn't exist. It was complaining. If you remember last time it said, you know, where is this book? I don't see it. But now that complaint has gone away. The complaint went away because we actually uploaded our cookbook called your SQL. Next, what we want to do is, and I'm skipping the Git steps as you recall, just to expedite. You don't need to, but you can. It doesn't hurt, but it's a bad practice to not do Git. But I know it's a long exercise. I'm just saying that we are skipping this Git commands just so you know. Skipping ahead, we created a book and now we want to use that book practically. So what we want to be able to do is get ready to connect to some cloud providers. For example, this example will ask us to make sure that our public key is given to the Digital Ocean Cloud, which we have given already, by the way. And I remember that I have done it, but for the sake of completeness, I will do it again. So here, this is my public key. I'll grab my public key like this and copy it over, go to Digital Ocean and give it to them. So Digital Ocean, take my public key. Here it is. So they'll take it, we'll say security settings and then security and say add my public key and say here is my key, please take it. I did not copy properly apparently. And so copy and paste. And copy paste doesn't work for some reason. No, it doesn't work. Okay. So that means I have to log in here. If copy paste doesn't work, please log in on to the services directly in this browser, Digital Ocean. That's a workaround. You will always, there is a way to fix copy paste issues. Copy paste may stop working if something updates, you have to update the virtual machine tool kit. There is a tool kit available for virtual machine. It's called the guest edition CD. That is what you need to run to update on the virtual machine so that copy paste continues to work. And I think this is something that is not up to date. That's why it may not be working. Having said, let me log in. Log in and I will provide my token and everything and then go inside and make sure that the public key is in place. On that step, it may not work for me. The solution will not complete without this critical step. So I'm doing that right now. And grabbing the token for Digital Ocean. First grabbing the password, copy and provide it here and then the token. Token copied, bring it here and verify. Now that verification is completed, I will like to copy paste my public key again and make sure that Digital Ocean has a copy of my public key and that is a critical step. Do not miss on that. If you miss, it will not work for you. And that is why I'm taking these steps to make sure double check that you have your public key there in the right location. And so here is my key getting added and paste and I'm going to give it a name and then add. It might already exist and if so, it will say you're already key in use. So that's goodness. That's a way of verification. Back here, what I want to do is the next critical step, which is you want to get this identifier by running this specific command, you have to get a list of identifiers for your keys. So that's the command and let's run it and then I'll explain what it does. As to why it is relevant. So here we get a command and what we are doing is running that command and this is the only way I know of getting this identifier. That's what we need, this ID number. That is number is needed in a next step that I want to be able to execute and this identifier is a reference that I want to provide to the key that I have uploaded here, this public key that I have given, specifically this public key that I've pasted in that window. I want to get this identifier for the key. This SSH key list command does that. I think I jumped a little bit. Let me get to the screen again. This is the only command that I know that generates this identifier that we need in a next step. There is no other way I know of in Digital Ocean Cloud that actually generates this identifier. It is unique to you. It is unique to a key that you have. Like in our example, in my example, I have these three keys, this number, this number and these three identifier keys that I have and I can see the details about it. I can edit the key and I can see the fingerprint, but I cannot see this number 6314029, which this method is the only method I know that gets you that number, which is a necessary ingredient to the next step. That is the identifier for the key. This is unique to Digital Ocean and that's a problem that you will miss if you do not save this data, this number. Number in my example is 6314029, so I need to remember that number and use it in a subsequent step. That's the whole purpose of getting this SSH key ID and this is the only method I know that actually gets you the number you need. In the website, the number happens to be this number 803301. In my current example, the number happens to be this number and I will remember that number by copying it over, by copying it like that and keeping it in my buffer. Next step, I would like to clean up my known host file. This is just for those of you who are repeating the exercise over and over again, you want to delete your known host file so that it speeds up your process of, especially in Digital Ocean, it will assign the same IP address to you again and again. If you do the same exercise repeatedly and so if you use the previously used known host with a new machine with the same IP address, it will fail. You don't want to fail so you want to remove your known host files pretty much like this and that's what we'll do is remove any stale entries from previously used hosts. I had nothing but I still deleted that. Having done that part, we'll move to the next step and remember that identifier that we need, something like this number that I have already and you can generate that by this command and that is a necessary ingredient, we need that number. Remember that number and proceed next. What we want to do next now is to construct one command that will basically build the entire stack for us. What does the stack contain in our example? In this particular step, we'll begin with just creating machine. We'll take step-by-step first in this example, we will just create a machine using Chef and then in the subsequent steps, we'll add items to the run list and then we'll have our lamp stack ready and then we'll add more items to it to construct the full stack and then in another step, we'll create a method in which in one command, we'll be able to do the whole thing from scratch, from nothing, build the whole stack and deploy our software on top and divide them into two different machines. One runs the web server part, another runs the database server parts, connect them together. All these scenarios, we'll be able to see in subsequent steps coming up. That's what we are going towards. The notepad that I want to open right now is where I will like to save that key identifier that I saved here, this number that I, oh, the copy paste doesn't work, so go back and stay in this editor instead and make some room and why is it so slow? Next paste that thing that I have in my buffer, which is the number, I will use that number and construct this command to actually create a new digital ocean droplet using our Chef method of construction of creation and that's this method. I would like to now expand it a little bit and describe to you what is going on here. What we are suggesting in this command is first of all, this item that we have copied over the SSH identifier, I would like to grab that and put that in the right spot in line number 12. This is the reference that I need in constructing a machine. I just made this change primarily like this to make a reference to the key that digital ocean knows that I have, which is this key, the public copy of that I have given to digital ocean and line number 11 is a reference to the private key, line 12 is a reference to the identifier used by the key which is necessary in this particular cloud. Now we are basically saying here bundle exec, knife digital ocean, create a droplet for me, call it Abracadabra, use this version of Ubuntu, do that in New York City, choose the $5 droplet, 512 MB droplet that you probably know already is a smallest size you can get, the cheapest size you can get is this and bootstrap the machine. Bootstrap command if you remember from the last time in solo exercises was the idea of running, installing Chef and then running Chef, that's the idea behind bootstrap. That is line number 12 and then 13 and 14 refer to the public private key that you want to use. So we will now construct this in form of a command. The only thing that I really need to change here is this item which I did already and now I will execute this command. What this will do is go to the digital ocean cloud in New York, create a machine with Ubuntu, call Chef, run Chef with empty run list. You will see that happening as soon as I run the command in this window. So back in our terminal, execute and let it go. You will see that a machine gets created in the digital ocean cloud like that. It has been assigned an IP address. This you can compare and match here, there is a new droplet ready now, it is getting started and it will start in 55 seconds or less according to what the promise that digital ocean has. It has been constructed in New York City and the NYC reference, you can see it. You can see that it is in 512 MB size. The name has been assigned Abracadabra, which is exactly what we chose. The IP address is now available and the droplet is now constructed, which means it is started and pretty soon you will see that SSH connection establishes here and that SSH established. It created a new client entry in the Chef server, a new node entry in the Chef server. You can see these entries showing up in the Chef server also. Back here, node entry called Abracadabra should pop up like that and this is where a reference to node files or in solo examples, you saw we had a folder called node folders which contains all the nodes. In Chef server scenario, we use the Chef server to store details about all the nodes we have and remember that this Chef server is free only for up to five nodes. If you cross that number beyond, go beyond that number, don't worry, they're not going to charge you because you never gave them your credit card, but it might stop working. Just caution, it is a free service that limited to five nodes, we just use one node. By the way, you will see that it installed Chef for us and then it ran Chef with an empty run list. Where is that empty run list? Right here. Run list was empty, so this empty signal asks Chef to do nothing. You will see that if you refresh Chef server again, all the attributes and details about that particular node are now available and visible to us, we can inspect without even going to the machine. What is that machine, when did it check up the last time and the details about what has happened on the machine ever since it came online. It checked into the Chef server last one minute ago. It has been up and running for a few seconds. You can see that the run list is empty and everything else basically is empty right now. The machine attributes are in long form expanded in a JSON file, you can see what shows up here. There are lots of details about the machine itself are available and you can see that detail on the nodes tab for every node that you have under management in your Chef infrastructure. Having said that, let us proceed to the next step. We basically ran Chef, installed Chef and the run list was empty, which is not exactly what we want to accomplish, but we want to do something meaningful, something specific, so we'll add some items in the run list. By the way, if you're doing the same exercise in the other cloud, say for example, Amazon, in that case, the command to write the same thing is slightly different and it is given here in this example. You can try any cloud you like. This is just an Amazon example. This is a DigitalOcean example, your choice. Right now, my choice is DigitalOcean, so I'm taking that choice and proceeding forward. You can take any route you like, any cloud you like basically, and including OpenStack cloud. Yes, I should say that. You can include OpenStack. What you need to be able to do is Knife DigitalOcean, you need to say Knife OpenStack. Before you get to that point, you need to also make sure to use OpenStack just like this, you need to make sure that your gem file contains a plugin for OpenStack. I have the Azure plugin, DigitalOcean plugin, EC2 plugin, Windows plugin. You need to provide a Knife DigitalOcean like that. What's the cloud you use? OpenStack, you need to provide a plugin just like that, Knife OpenStack. When you do that line number 86, you need to update your bundle, bundle update. Then you can go back here and say bundle exec, Knife OpenStack, server create, and follow along with whatever is required for that OpenStack cloud, which will vary a little bit depending on your deployment. We can see some examples. Here is some example of Knife OpenStack. I know you have OpenStack and I don't have it here handy with me, so I cannot demonstrate this, but here is how you do it. If I have access to OpenStack cloud, I will create a pair of credentials in the OpenStack cloud, which is where you receive it from, let's say cloud.Boeing.com, 5,000 version two tokens. That's the API URL endpoint for your OpenStack cloud that you know what that endpoint would be because that's where your OpenStack infrastructure is going to be running. You will have a username and a password, a tenant, and a region that you need to define in your OpenStack. This region is basically artificial. You can have building number one and building number two if you like. Basically, you get to choose. You have to assign a tenant name, username, password, region. That's what the administrator who sets up OpenStack will receive from the infrastructure. You will use that and then put these entries back in your KnifeRB right there, pretty much like this. Oh, no. The copy paste doesn't work. You need to copy these things from the configuration available in Knife OpenStack plugin and include that just like we have the AWS and DigitalOcean and Azure. You will add one more, OpenStack, these four lines, these lines, four or five lines. That's how you configure your OpenStack Knife connection if you like, Knife OpenStack. Then you will construct a command pretty similar to this or similar to what we have created here in our example, which looks like this, basically the same thing, except you will replace DigitalOcean with OpenStack and Droplet with server and your location with your company location and other details will vary depending on what GitHub says on the OpenStack Knife plugin. You have to read that page and then understand flavor, image, key ID. This ID number is unique to your public key provided to OpenStack Cloud. There is an example where template files may be used to read that page completely and then you have these options available, Knife OpenStack server delete, server list, flavor list, volume list, image list, group list, network list, and end up creating OpenStack server create to create your own OpenStack-based chef-driven solution set, pretty similar to what we're doing right now. That is how you will use OpenStack. In our example, I have modified my gem file and I have included this OpenStack gem, but I don't really intend to because I'm not ready with OpenStack machines yet, so I'm going to not save the changes and also on KnifeRB, not save any changes and proceed next. In our machine, we have set up, created a new machine, a new droplet in some cloud like this example and then we have chosen to keep the run list empty and in the next step, we'll expand that run list and in fact, for people who have access to OpenStack, I suggest that you don't try with DigitalOcean, do not try with EC2, but instead, try with OpenStack because that's what you have in your company. Use that and connect with the OpenStack and if you run into issues, chat with me. I'll fix it. We'll figure it out. We will have a method in which you should be able to use Chef Server with OpenStack in your company and you should succeed and I will help you in making that happen. Before I recommend, don't use DigitalOcean, don't use EC2 servers, just use OpenStack which you have in your company and run this exercise in your own stack. That is recommended and I am here to help. I'll proceed next. What I want to do is add some run list items to this particular machine that I have here running. This machine in DigitalOcean Cloud, I have it running here. I want to give it some work to do. Right now, it is doing nothing. We will do that. We will go here and say, do some stuff. What do we want it to do? We want to add it some roles. What we really want to do is, let me get the white sheet of paper. Where is my white sheet of paper? Here. What I want to do is, on that machine that I just constructed, it has an empty run list. It is an Ubuntu machine. It is running sitting in New York City, number two. It is 512 MB RAM. What I want to be able to accomplish is assign some purpose in life. The purpose in life for this machine, I would like to assign that by saying that, you know what, you new machine. I would like you to grow up and become some things in life. I want you to become three things. First of all, get some base role for you. I would like you to become a database server. By the way, on the same machine, I would also like you to become a web server. Take on all these roles. Become the base, become the DB server, and also take on the web server role. All of these three things together on one box. That's what I would like you to do. This is your purpose in life, and I'll assign that purpose in life by saying that this is your run list. Run list currently is empty. I would like to give these three roles, one, number two, and number three. Remember, sequence is important. Base is the first step. Second, DB role has to be assigned first. Before you run the web server role, do you know why this is important? Why is the sequence important? Why should we run the DB role before the web server? That we have proper orchestration setup out there? Yes. If you run some application here, and it expects the web server to be available, it will be able to connect successfully and proceed forward. If you run this thing first, and it tries to connect to database, and the database doesn't exist quite yet, it will fail. That's the reason why sequence is important. What I want to do is assign that purpose in life, and one, two, and three. That's the sequence I will take. In our example, I have written that in that exact sequence, is to basically add run list items, number one, number two, and number three. These three run list items I want to add to my run list, to this node. I have it here, and I can edit the run list, and look at the base role, and add it visually if I like. Like that. I can do it visually if I like, but let me tell you, I don't. I don't like to do it visually, because I cannot do this on a thousand machines. One machine I can understand. I can do it using a mouse. If I want to do this on 10,000 boxes, doing this by mouse will be pathetic, boring, and I'll run away. I would not do it. That's why we do it like this. We copy, and go back to the terminal, clean up, paste, add the base role. It adds a base role like this to the run list for Abracadabra, the machine that we have here. Now, if you refresh and look at the Abracadabra machine, you will see that the expanded run list shows you the base role. By the way, I did not add it visually, because I don't want to add it visually. I want to add it like a command, so I can run it 100 times, 1,000 times, any number of times I feel like, and depending on what I'm really trying to accomplish, what I just did was to add a base role to the run list, to this machine called Abracadabra. Next, the other two are very similar, so I'll just do both of them at the same time. Copying both at the one shot, copy, and paste. DB Master, I added first, and then the web server role I added. Both of them are added. Now, I should be able to see what roles do we have. I'll expand my browser, refresh my browser, and see that the run list has expanded to three things, base, DB Master, and web server. You can see that the DB Master has a reference to your SQL. This is exactly what we wanted. Next, what I want to also do is look at this machine that I have, the Abracadabra machine, and look at its environment. The environment that I currently have this machine under is by default, the default environment, but that is not what I want to put. I want to put this machine in production, so I would like to select that environment and say, you know what, production, and thereby putting this machine in production mode. I can do this visually, but as usual, I don't do stuff visually. I cancel it out. I don't want to do it visually. I want to do it through a command, and that is this command, which is to put that node in production environment. It's as simple as this one line, which will do the same thing that you want it to do, which is basically put that box, Abracadabra box, in the production environment. The idea is to run bundle exec, knife, node, environment, set, Abracadabra, production. It's a long one, but it's intuitive. Knife, node, environment, set, Abracadabra, in production. It did that. Abracadabra, chef, environment, now is production. If you miss this step, exercise will not work because it has been designed to set in production. If you don't do that, it will not happen, the exercise will not succeed. Now, let us check the status of the node. The checking status is simple. Just show me the node, please, and so it shows you what you see here visually is exactly what you will see here, the name, Abracadabra, environment, production, fully qualified domain name, Abracadabra, IP address, the run list contains the role base, the role DB master, and the role web server, three things, one after the other in that sequence. It has the Ubuntu platform and no tags and no roles and recipes yet, but the run list has been called out. With this in hand, what we can do is on that box that we have in the cloud, so this is the cloud, we have that box there called Abracadabra. As you remember, we discussed how most companies run Chef. We would like to let Chef run by itself on its own at certain periodicity, let's say 30 minutes. Every 30 minutes or so, this is just how one company can choose to run it every 30 minutes. You have a choice to decide how frequently you want to run Chef server. If you run Chef every so often, it will run Chef every so often, and if there is something needed to be done, it will do it. If you don't run Chef, nothing will happen, nothing will change. The model of usage that I am suggesting that we will have is Chef comes and runs itself every so often. When it runs, it goes back to the Chef server, looks up, what is my purpose in life, please tell me. Currently, the purpose in life was the last time we run, the purpose was empty run list, so it just did that nothing thing, basically did nothing. Now we have assigned it a purpose. The purpose has been visible to us in form of this run list. That is the purpose in life. Now next time when Chef runs on its own on this box, when it does after so many elapsed time, whatever elapsed time you say, except 30 minutes, it will run the basically look up the Chef server, identify what is its new purpose in life, and implement that purpose. We need to simulate that exact idea of having Chef run by itself. We haven't done it, we haven't set it to run by itself yet, but we need to go and simulate that. What I intend to do is just connect to the machine directly and manually run here. Manually run Chef just to simulate this automatic run. That's what I intend to do. I'll manually SSH and manually invoke sudo chef client on that box right here. This is just to simulate that Chef is running by itself on its own. You can do that very simply using crontab, but I'm just simulating it like this. What does that mean? It means that I need to get that IP address, which I have here, and connect to it. So SSH, like that, we'll connect, we get connected. This is the Abracadabra machine. What we'll do here is run Chef client. By the way, this is supposed to be happening on its own every so often. I just happen to be simulating it, so I'm running it manually just to prove a point, but it will run by itself, and when it does, see what happens. What do you think will happen? I just ran, by the way. It is going to connect to the Chef server, identify what is the purpose in life, grab the books, grab the run list, and run the run list, and that's what you see happening. You see it is happening right now. You're looking at the machine, it says, you know what, I need to understand my purpose in life. So my run list is, I thought it was base role, and DB master, and web server, whatever happened to that. So it got expanded into the ingredients like this. The ingredients are, as you can see, individual items inside those elements. So we have base called apt, and git, and build essential, and sudo, and user sysadmin, and wim, and then DB master has your SQL, and base, and then web server has these guys. So all these things got expanded in form of this run list. And then it needs to run those roles, and run those purpose in life. So synchronizing the cookbooks, and the cookbooks are delivered to the machine, the Abracadabra machine from the Chef server. So the Chef server here, wherever you have, it is somewhere in Seattle. And that is where it will deliver the books direct to this machine. As Chef runs itself, the books will be pulled down. That books will basically be including all the things that you need to accomplish your purpose in life. So the books come down, and the book list, as you can see, includes your SQL. And when these books and these things run, you will see that it will do exactly what you would expect, is to first of all build the machine to take on the base role, and then install MySQL server. But before it installs the MySQL server, it will assign a password called 123. And it is still doing that, by the way. And it is then finished doing its activity, and it reports back that we are finished. And so if you now scroll back up a little bit, you will see other activity like installing Apache server or installing MySQL server using our password called 123 that you extract from a data bag. All of these things you will see happen as you scroll through the logs. And we would like to validate that it actually worked correctly, just like we expect it to. So that's the validation next step. I'm just reviewing my logs here to make sure that it is actually doing exactly what we wanted it to do. And I think it did. I think it did, which means it is time for us to verify. So we will do exactly that. OK, come on. Stop scrolling. Oh, there we go. So I am on the abracadabra box, and I'm expecting this box to have the complete LAMP stack by now. We need to have Git installed. Is Git installed? Let's go check. Now, it would not respond to me for some reason. So I'm going to quit the connection, break it, and connect again. And now I will check if I have Git installed. I think I do. Yes, it does. It does have Git installed. I can verify that there is Git version installed. Let's see if we have Apache installed. So it's called Apache 2, by the way. So Apache 2-B should show me Apache is installed. Is Apache running? Let us see. Apache, show me the processes and search for Apache there. And yes, it is running. What about MySQL server? Is it installed? And apparently it is. Is it running? And yes, it is running. So now, can we connect to it? MySQL server, please allow me to connect to you using the root password called, that I will provide in the password I will type. It happens to be 123, right? 123. I just type that, and I enter, and I connect to the database. So I succeeded. I will exit and quit. The machine is now configured as a LAM stack. What we need to do next is the thing we will do after the break. I'll let you digest this. We've done a bunch of things, LAMP, all of that in one step. We will be able to add by adding three roles, which is the base role, the DB role, the web role. We added these three roles to the box, which we began creating with an empty run list to begin with. And we use the Knife Cloud Machine Create command. So I'm just generalizing the command here, Knife Cloud Machine Create. This cloud can be Digital Ocean Cloud, EC2 Cloud. Machine can be Droplet, Server, whatever the cloud likes. And for people who have access to OpenStack, I would like to suggest this method. Use the OpenStack Cloud here, not Digital Ocean, not EC2. No, use this. You have it. You should use it. OpenStack calls the machines as servers. So Knife OpenStack Server Create, that's how you should run. And then you create an empty server with a null list with nothing in it. Then you add these items in the run list. And then you run it again, run the Chef server again. It runs by itself, by the way. So all you need to really do is just push your role definitions in your run list. And Chef will run itself, and it will automatically construct a LAMP stack whenever it runs. So as a consequence, now we have a LAMP stack. We see that Apache is running. We see that MySQL server is running. And we did not check PHP. So let's check PHP function quickly. Back to the machine. In the Abracadabra machine, I will check my PHP function. I think it should be there. And I have it there. So PHP is installed as well. So I check PHP. I check that MySQL is running. And I am able to log in using my custom password that I chose for myself. That also worked. We see that Apache is also running. And Linux, of course, is the box under the hood. So that is the box that you got from the Digital Ocean Cloud, which is the cloud company that you chose. And that's the thing that we accomplished. And this is just to build the foundation. This is the platform that we built. And now on top, we will throw the software. The software you want to throw here is WordPress. And we will do that after a break. So right now it is Safari browser, timer, 10 minutes. And off we go. So timer starts. Let's take a short break and we'll resume with adding new cookbooks that will enable us to construct a full functional stack with our software and our platform elements that we already saw. And so there is a chat request here from Eric. Let's see what Eric says. So responding to Eric's chat message, the way you do that is easiest way is to go to the Chef server and delete that entry that you have mistakenly added. So what does that mean? This is the response to Eric, the node that you have. You will go to the run list here, edit the run list. And then you have the extra entry here. You just yank the entry out and put that here. That's a quick way to remove the entry. You have constructed an entry called webmaster. You want to remove that entry and keep the correct entry in the right sequence in here in the current run list. And then you save. That's how you resolve it. Apparently, Eric has chosen to use a slightly different name by mistake. And so his expanded run list includes extraneous items he doesn't want. So you can just clean it up like that, remove from here, cancel. I hope Eric is listening or he's not there. Oh, he's not here apparently. I'll respond to him on the chat also. Hey, David, I know you're still sounding funny apparently. So you don't have to speak today, I guess. You have to probably reboot your machine in order to get rid of that Mickey Mouse sound that you're creating. Thank you for a good laughter, but you can avoid speaking today. That'll be... Yeah, I don't know what's happening with David's machine, but his machine is making Mickey Mouse sounds as you can hear partly. So let's continue and let's understand what the next logical step in our discussion is. So our next idea would be now that we have a LAMP stack ready with a machine that we started, we had that machine running Linux already. Then we decided to put PHP, Apache, and then install MySQL Server to basically construct this as our LAMP stack. And while installing MySQL Server, we made sure that it uses our password so that we know that it's available from this Chef server that we have here. It has a data bag entry. The data bag that we have is called secrets. Inside that secret bag, we have an item called MySQL, which contains a password for the root password for production environment. And that password is what we want to be able to use as our password in the MySQL deployment that we now have. And we validated these things. All of these things seem to work. That means our LAMP stack is ready. So we should get to some instant gratification now that we have done some work. I would like to see something coming out of this stack. So I will go to my machine again here and read the next step, which is basically leading us to some instant gratification, which is, okay, show me what I have now and show me something useful, something very productive. So we will do exactly that, is to just throw our software application on top of the stack that we have built and see it live, see it running. We have already completed those testing and validation for success. I think you saw me do that before the break. So we'll skip this step. I mean, I've already done it, so I'll move on. And the next step, if we would like to add an application and see it live, the simplest way to add an application is to just do it manually. Basically download this application from here, from the WordPress.org website, click the download button and download it pretty much like this, this one. It downloads the application, but we don't want to receive it here on our desktop. We would like to have this installed on that machine, this droplet that we have there called Abracadabra, we would like that machine here to get it installed. Don't want to have it here on our local machine, we want the download happen. But on that box for which there is this two-step process that I have written down to just get us to quickly see what we got. In the first step that I'm highlighting in orange here, this step, what it does is sets up the database in a specific fashion so that this application can use the database correctly. And in the next step, which is this highlighted entry, it will actually download and set up the database. Now, let us go see what these two little scripts actually accomplish. I will download them on the remote computer we have, which is that droplet Abracadabra machine here. On that box, I will download those scripts first. So I'm getting that script. The script is now available. It is called the db setup script. That script comes down from that command I just pasted. So you will now see that command and the db setup command does a bunch of things. Basically the operative parts are to create a database in the MySQL server, database server by the name WordPress. And then also the operative parts are create a user called username on the local box where MySQL server runs. And the password for the username user is going to be password. And then grant all the privileges to that user, all the privileges to that user on local host identified by the password, password. And then grant all privileges on the WordPress database to the username user on the local host on that box remotely, and flash privileges and quit. That's the scenario that we are going to run through when you run this command that we have downloaded in form of a script. And that is this step. It has been constructed. I wrote it basically to enable this exercise. And that script is now available on that remote box we have. You can see that the script is available on the command right there. When I want to run it, what this will do is create a database, create a user, create a password with access to the database. The database name will be WordPress. The username would be username. The password will be password. And there is a specific reason why I chose these specific things to simplify another next step, which you will see. Therefore right now, I am running sudo shdbsetup.sh. And what this will do very quickly is to create a database, create a username and a password with permissions to access the database. And here we go. It finished. Very, very quickly it does. And now we are done with that part. The database has been set up correctly. Next is to actually download the application called WordPress, for which there is a script that I have written already for you. So we'll copy that, bring the script down locally. And the script comes down. It looks like this. That script, if you examine and you go open it up, you will see that it is actually very simple. It says, you know, operative lines are download WordPress, expand it, remove the zip ball, the tarball, and then move that WordPress to the correct location where Apache expects to see it and change permissions so that Apache can manage that software. And that's it. At that time, when we invoke this particular script, you will see that you have a functional WordPress running up and running already. So I want to run this piece right now. I'll run it pretty much similar to the database setup, except that I want to run it for WordPress setup. So I will say WordPress setup and run it. So it will go and download WordPress. And we are now ready. We are ready to run and see our WordPress deployment just like that. So I would like to grab the IP address of this box, which is here. So I'll copy that IP address and go to any computer, for example, this one. And actually, copy paste doesn't work. So I will go to manually typing it. And so you can see that I can get the IP address of that droplet from this location. So I can copy it and open it up in a window, in a new browser window where I see the application live and running. You can see it right now that WordPress is running. And it says, choose your language. So I'm going to choose English language and proceed forward. And here it says, welcome to WordPress. Before getting started, we need some information. Like for example, what is going to be the database name? What is going to be the name of the user that is going to access the database and has privilege to modify the database? What is the password for the database user? This is different from the root password that you had. The root user had a password called 123. This user is different from the root user on the database. The database name can be whatever you like. User can be whatever you like. Password can be whatever you choose. And the host is the machine which runs the database server, that host. In our example, we are using the same box as our database server, as our Apache server, as our Linux box. So all of these things are basically local host, the same box. Now having understood that part, let's go. And what it says, it says the file phpconfig already exists. If you need to reset any configuration, please delete. You may try installing now. I think I must have skipped something or something like that. So this file should not exist. It should get created. Maybe I double-click twice or something. But let's install. It should finish install, continue. And here I would like to now type the name of the site. So I'll say Cloud Genius, for example. I'll select a username. I'll select a password, which is going to be a very strong, complex, difficult-to-remember password. And then I will confirm the use of a very strong password. And I'll provide my email address. And then I discourage search engines like Google to indexing my site because it's a fairly new site. I don't have any content in there. So I don't want Google to see. I'm not ready to show it to them. So I'm hiding myself from the Google search bots and install WordPress. And off we go. And now I will be able to log into that website with my name and my complex password that I chose. And off we go. So we now have our site visible, active, and running with the name loaded up. This post that I have, I can maybe modify if I like. And here we go. Instead of Hello World, I'll say Hello Cloud Genius and update the post. Go back to the front end. And you see that the updates are taking effect. You can even go and go to the dashboard and update the WordPress database as well as update the application called WordPress, all of them. And it is going to update the application itself because of a new update that has been pushed out. So I just updated my WordPress application to the latest. I will also update this plugin that I have. So update now. And so now my site is completely up to date, except there are some more updates available in the theme section. So I'll select all of them and update them as well. And now I have all the themes also up to date. And now my site is fully up to date as we expect. And now you see the content. And that's what we just did. That was a quick and dirty, non-chef method of adding an application on top of our stack. We use some scripts like SH, DB setup, SH, WordPress setup. This is not the chef way of doing things, but it was a quick and dirty way of getting something to run, getting something to become visible like this, for example. And you see it here. You can also see that you can change your themes and maybe select something like a theme that you may want, maybe, where is the other theme, like customize, maybe see another activate the older theme and visit the site with a different theme. So you can basically mix and match and modify whatever you feel like in your site as you go along and the site is ready and up and running for you on the IP address you chose. You can assign it a CNAME or a proper A record and choose your domain name to go along with right here and that location will then reflect. You can customize those things again somewhere in the settings area and modify your URL if you like to. So it's very easy to accomplish those things. But the idea is to make sure that you can do the same exercise on a bunch of machines on a large scale and have them collaborate, operate together like a service. And that needs to be done without human intervention in a completely automated method. Specifically in our example, we don't want to use these kind of non-chef methods but instead have a cookbook deliver our application. So if you want this entire setup to happen using chef method of delivery, you would like to write those things in form of cookbooks. And so we will do that. But first, I want to clean up whatever I did and basically destroy whatever I have constructed. So this droplet that I have, it has references to the, in the chef server, you will find references to the droplet. So here is the Abracadabra reference, I would like to remove that reference. I would like to also terminate this droplet completely and basically clean up so I can move to the next step. So here, what I want to be able to do is clean up the box that we have, clean up a reference to the client called Abracadabra, clean up the reference to the node called Abracadabra, clean up your known host entries in your.ssh file, check knife digital ocean droplet list to see this droplet. It is not dead yet. So you want to kill that droplet also. So we'll do those things one after the other. You can actually do most of these things, copy and paste in one line and go back to my full screen mode and back in our web browser where I was copying from. And so I have these things copied over, I'm going to copy them, take it to the terminal. First step I want to do is paste them here, is to delete the client called Abracadabra, the node called Abracadabra, and then remove the known host entries, which I did. Now I want to see what droplets do I have currently running. And you will see that DigitalOcean will tell you that you have one droplet running and that entry will pop up as soon as this command runs, goes connects to the DigitalOcean cloud and comes by the list of droplets that you have. And here it is, this ID number of the droplet Abracadabra running in New York with this IP address, and it is active right now. So you want to kill that droplet. You want to say BundleExec DigitalOceanDroplet destroy as opposed to list or create. This time you want to destroy the droplet. And that's what we will do is this command BundleExecNive DigitalOceanDroplet destroy followed by dash s. And the dash s option basically expects you to provide this identifier so it knows which machine to kill. So you can provide that machine address, the machine ID number like that. This ID number you can obtain from here. And this command will go to DigitalOcean cloud and destroy this droplet with that ID number. So you can go and destroy it, which is how it did. So now if you run and check what droplets do I have right now, it will say you have nothing and that is expected. So we cleaned up our cloud. We cleaned up our Chef server. We removed our references to any nodes we have. So this Avra Kadavra node should be gone by now when I refresh. And that is exactly what I expect. It's not found because it's gone. It's totally expected. But if I go to say another location, you will not find any nodes because we don't have any. So I cleaned up that node reference by deleting the droplet, deleting the node, deleting the client in the Chef server, and deleting the droplet from the DigitalOcean cloud. So we basically cleaned up. Back in here, what we want to do is proceed to the next step of making sure that these steps that we just did, adding the application step, how do you do that programmatically in a fashion, which is the Chef way of doing things or the automated way of doing things? That's what I would like to enable. And that's what I would like to read next as to what that next action is. And we want to automate end-to-end. We want to also do these things in a fashion which will basically build the entire stack for us in the form of using the Chef way of operating. So we create a droplet, we'll install Chef, run Chef, it will look at the run list. It will not only build the LAMP stack, but also put the application on top, configure it the right way, and we'll have it up and running exactly what we expect to see. And so we want to be able to do that automation end-to-end, and that is what we will do next. To do that automation, we need to create two custom cookbooks. We have to create a custom cookbook to deploy WordPress software and the application itself. And we also need to create another custom cookbook to create a database and a user that this application called WordPress will use. So those two cookbooks we need to create. Instead of cookbooks, you have done it before already once, we'll do some more. And it's trivially simple, as you will see. I'm going to put these two windows side by side again. And here is my terminal window, which is not responding for some reason. Now it did. And it disappeared. So I'll open it up again, open my terminal, please. There we go. And now we want to create a custom cookbook for the WordPress application itself. And what I want to be able to do is go to the correct folder. And in that folder, I want to create a new cookbook. You will see that I currently have our cookbook folder contains the your SQL cookbook. I want to have a cookbook here called WordPress. And what that cookbook will do is install WordPress onto our machine that gets created when we do. And so that involves creation of a cookbook. So this is this idea here, bundle exec knife cookbook create WP. It will create a blank empty skeleton cookbook with nothing in it. We'll use that command and look at the atom editor. You will see that it does not have a WP cookbook quite yet. But as soon as you run this command, a new cookbook pops up in the site cookbook folder right here. As soon as I run the command, you will see it coming up. And it came. There's a book called WP. Now available. I want to modify that and make it our cookbook exactly how we want it to be. And how you want it to be is provided to you in the example here. What we really want to do is to use this the script that I have created already to begin with in a previous example I showed you, I want to use that script but include that in the form of a recipe in the cookbook called WP. So we have a cookbook ready here, which is empty. It has a recipe default, which is also empty, but I want to make it my own. Make it something that will do exactly what I would like to have, which is to download WordPress and put that in the right location. I can accomplish that by doing these steps. And let us read that before we copy it over and place it in the right location in the recipes folder in the default recipe. That's where it belongs, by the way. So we'll place it here and start reading to understand and save it also. And here you will see what I have done essentially is constructed a code block in Ruby like this that ends in line number 40, begins in line number 19, ends in 40. And in between that is the segment which basically downloads WordPress and places in the right location. So if you look at this segment, I'm also specifying the user should be root user, the change the working directory to a temporary location and begin executing this code that is placed in between the EOH handles. So we have two EOH handles, EOH and EOH here. These two EOH handles in between whatever we have is basically going to be a bash script. That is what we are specifying in the code block here. And what we are practically doing is injecting this bash script inside the Ruby cookbook. And so line number 19 through 40 is Ruby. Line number 23 through 38 is a bash script that we ran before. And you will notice something different and something what I would like to call idempotent in the way this has been written. If you remember the word idempotent, you will notice something very different about this particular way of writing a cookbook is anytime you do something with Chef or Ansible or Pocket or any of these tools, you need to do it in an idempotent fashion. That is a critical thing to accomplish. If it is not idempotent, it is not the Chef way of doing things. It is not the Ansible way of doing things. It is not the DevOps way of doing things. The reason we need to do it this way is that we know that Chef will run on its own every so often, for example, and you will keep running this every so often and you want to keep repeating those runs and you do not want to destroy the end result. The end result of a repeat run should be the same as your design or your desire. Whatever you initially desire, it will build your desire, it will build your design, it will produce an end result. If you run Chef again, it will not destroy the result but maintain the result. That's the goal we have in mind. We want to keep that end goal in mind. Whatever we write needs to be written in a fashion which is idempotent. I think this is one of the key learnings that you have in this concept is whatever code you write needs to be in a fashion so that it doesn't destroy the result, maintains it, creates it, and then maintains it after repeat runs. That is what is different about this style of writing. I'll show you that style succinctly. It is specifically in line number 24 through 27, 26, 24, 25, 26. Those segments make this particular thing idempotent and I'll tell you how and why. If you know the answer, please speak up. If you understand this thing, that is beautiful. If you do not, I will explain. Now read line number 26, it says something, it says, I will do nothing because you already have WordPress. That is exactly the behavior you want is if you have WordPress, this thing should do nothing. If you do not have WordPress, then I'm going to install it for you. That is what you want. If you don't have WordPress to begin with, this thing should install WordPress for you. If you have WordPress already, then it should do nothing and not destroy your setup. Line number 24 is where we check if you have WordPress. Such conditional statements are one of the ways that you can construct idempotent code and yes, this is something that you actually have to write to make sure that your ideas are implemented in a fashion that you desire. This is a way of writing something that you basically desire. It is a way of expressing your desire in form of a recipe that you create and put that in the recipe default folder in the default rb file. This basically shows your desire. Your desire is to do nothing if there is already WordPress installed. If you do not have it installed, then you need to install it. That is how you desire. You want it to be idempotent, and so you saw that already. That is what this line checks for, 24. If you have WordPress, then I will do nothing, otherwise I will do whatever you want. So it will install WordPress in that case. Now you will notice that here, this WordPress version, 453, and these are older versions of WordPress. So what I want to be able to do is update that to refer to the latest version of WordPress, and I just changed that. You saw that I did a multi-line edit, a multiple lines edited together is possible in certain editors, like this Atom Editor supports multiple line editing at the same time, for example, here, here, and like that. So I'm going to save this change I made and make sure that you understand that this idea of idempotency is clearly understood, that is a necessary condition. Otherwise, if you have Chef running over and over and over again, it will destroy your setup. You don't want that to happen. And therefore, this has to be written in a specific fashion that is actually idempotent. And I hope you understand this. If you do not understand this aspect, please stop me and make sure that you get this concept clear, and then we proceed forward. But this is the core idea behind setting up, making sure that you don't shoot yourself in the foot when Chef runs again. If you have something already installed, don't destroy it. If you don't have it, proceed. And when you proceed, you will download the thing, expand it, change its permissions, move it to the right location and done. And I close this file because I am done writing that particular cookbook. Next up, I want to write the other cookbook to deal with creation of a database user and a database. So that is a separate cookbook. And it is written slightly differently to illustrate two different ways of writing something that you would like to have. So it's the same type of idea implemented in a slightly different fashion. And that's what you will notice right now. When I see this line, I will copy it over, go to the atom editor, wait for the site cookbooks to update. When I run the command to create a new cookbook, I will do that in this window. And you will see that I will create one more cookbook like that. And off we go. We have a new cookbook called DB user. In that cookbook here, we have to do the similar kind of thing, run a script, put that in a recipe default file right here. But I will like to make a reference to another file in the files default folder. That's what I would like to do, write this thing slightly differently. So what I am intending to do is to create a file in the files default folder by the name install DB user. That's the file name I would like to have. And this name that I will use for the file that contains this script that I chose before when I ran it manually and we just got some instant gratification, wanted to see how quickly can we get to an application functioning without using Chef. And so that's what we did the other time just a few minutes ago before the, or maybe after the around the break. And so what I'm doing now is grabbing that same script that we have here. But instead, put that in a file in the files default folder. I'll show you where it goes. I'm right now grabbing the content. So copying it over from here. All the orange items I'm copying over going here, creating a new file, making it go and save in the site cookbooks, DB user files, default folder, site cookbooks, DB user files default folder. In that file, I would like to create a new file which contains my script and the file should be named install underscore DB user.sh. And in that name, I will save that empty file right now and then paste the content I have and save. Now you will also see the same thing here we discussed idempotency again, one more time, the same concept one more time, this time applied to the end result. And the end result right now is to accomplish that we have a database called WordPress, a user called username, and a password called password. You want to accomplish that as our end result as a consequence of running this particular cookbook. And I would like to accomplish that outcome and not destroy the outcome if it is already there. If a user called username already exists, leave it alone. Don't mess with it. If a database called WordPress exists already, don't mess with it. If it doesn't exist, create it. So it is written pretty much like that I just said. And that is what you can see in line number six. Create the database called WordPress if it doesn't exist. So it is the idempotent way of creating this database that we would like to have. If it doesn't exist, you create it. If it exists, it will do nothing and proceed. So this condition of if not exists, made sure that this database doesn't get created again. If it does get created again, it will fail on error and your Chef deployment will fail. You don't want that to happen. And by the way, this same idea applied to username and passwords is this piece of code, the remainder of code, which actually is written down in this fashion, conceptually is done the same thing, which is to make sure that your end result remains intact, that your username and passwords do not get disturbed if they're already set. And that's how this code block has been written. This code block actually uses the MySQL syntax of writing MySQL code, which is embedded inside a shell script. And this shell script itself is saved in a separate file that is saved in this location called default under the files folder in the DB user cookbook. Now just by placing a file in the files default folder doesn't actually accomplish anything unless you go in the recipes folder default and actually use it. So we have that files default file called install DB user. And we want to make use of that script in this location. And that is the next step. So in the next step, what we are looking at is to actually create and populate that cookbook, which looks up that file that we have created, the cookbook file inside your files default folder, and then grab this content, paste it here and understand what it does. What you will see here is this module called cookbook file, which looks at the source file called install DB user.sh. And as you know from previous examples, it is expecting that particular file here. This file is expected to be available in the files default folder. And that is where we have placed it to begin with. So it knows where to grab it from. And that file will be placed in this location in the tmp folder. And with that name inside the target machine that you have in your mind or in your Chef concept that you want to deploy. So it knows where to go place that file, it will grab it from here, this file, and place it in the right location in the tmp folder. It will change its permission to user and the group to be user and mode to be 0700. So user can actually act on that file, which is going to be placed in that location tmp with that name. So that cookbook file concept, this particular syntax, when it gets invoked, will actually place that file in the location you expect it to be. And next is this bash script, this Ruby module called bash, which will allow the user called user to change directory to the tmp folder where this file sits and then invoke it. That's what we're really trying to accomplish. This cookbook is written in a slightly different fashion than the previous cookbook in which we had the book itself like the other example that you know here, WP cookbook, the recipe default actually included the script right in the body of the cookbook itself, whereas the DB user cookbook does it slightly differently and provides you a script placed in a separate file in the file's default folder and then invoke that like this. Place it, invoke it. Two steps, a slightly different way of operating the same idea. Now that we have these two new cookbooks, we are ready to send these books over to the chef server. Maybe I should ask one question. Yes. If we go back on that particular script, what does mode 700 mean? In which file you're referring to? Where was that? Yeah, I think it's that file. Where exactly you want? Line number? Hold on, I'm looking at which file is this. Recipes default. Line number? Line number 21. Mode. Mode means CHMOD, permissions for file access. You know CHMOD, right? Do you? Yeah. It's the same thing. CHMOD 700. It allows the user to read, write and execute and nobody else to do nothing. Yeah, understood. Thanks. Next. Next, we have to send these books over to the chef server, which we will do, but merely by sending the book over to the chef server, it will do nothing. It will actually do nothing. Why will it not do anything? Because you are not using those books in a role. We have to expand the role a little bit. The role expansion needs to actually invoke these new cookbooks. I want to see my roles that I have here and I have these roles here. I would like to select one of these roles, for example, this one. In that role, we do these things that we already have. What I want to be able to do is add a couple of items here. I would like to say, please run the recipe called WordPress default and also run the default recipe in the DB user cookbook. I expanded my role definition to include references to the two new cookbooks that I have created. Those are added in line number 11 and 12 in the role definition for web server. This is how the role will expand and then you will see that the machine that you will next create will not only set up the LAM stack, but also have WordPress, WordPress user, username password, and the actual WordPress software all loaded up and ready to go when you want to make use of. That expansion of a role definition we just did here, which basically invokes the WordPress cookbook default recipe and the DB user cookbook default recipe, which are available by the way here. DB user cookbook and the WP cookbook, those two are invoked in this role. We want to update this role, which I did. I want to save it and also make sure to send this role over to the Chef server. That is expected. I need to go take my terminal and send the two new cookbooks and also update the role that we have. I will do exactly that now. First of all, what I will do is just send all the books that I have. You probably have already all the cookbooks. If you run this command, except the two cookbooks we don't have on the Chef server, we do not have the WP cookbook and we do not have the DB user cookbook quite yet. What I'm going to do is run this command, which actually uploads all the cookbooks. All means absolutely all of them, except if you already have the cookbook there, it is not going to be uploaded again. I'll say upload all of this and it should actually upload the two new cookbooks that you will see going up right now, WP as well as DB user. Those two cookbooks will be uploaded and you will see that this upload will finish with a reflection showing here in the policy cookbook section. You should see in this list, now you should see a book called WP and another book called DB user, both of them show up, which shows you that the upload of cookbooks, the two new books are there, you have taken care of. You want to also send your role, the web server role that you have modified a little bit. You want to send that role over to the Chef server. You will do that command basically to send roles that we have. You can send all the roles in one shot. You can just say, all these commands copy and update the roles that you have. You may have modified something else. You can just copy this and send all the roles one at a time. If nothing changed, it's very quick. If something changed, it updates, it updated the three roles for you. Now it is time for us to construct another computer, another cloud instance that actually contains all the things that we would expect. We will like to do that exercise again, this time in digital ocean. I will go take that command, copy it, bring it here, and remember this SSH key ID number is important. You want to keep track of that number, specifically this number. It belongs to me in my example, that's the number I want to use, so I'll take that number copy from here, put that in the right location here in that spot, paste. That is taken care of. Now, I will look at my command that I have, carefully read it. I see that I have knife digital ocean droplet create, and for those of you who have access to OpenStack, please use OpenStack, and so convert this to OpenStack like that. When you have your digital ocean cloud created droplet, it will be named not Abracadabra, but this time it will be named CloudGenius. It will be using Ubuntu 14.04 in the New York data center number two, will use a size of 512 MB, it will bootstrap, meaning install chef, and then run chef. Use my identity key file that I have the private key here, and a key reference in form of an SSH ID that digital ocean calls it this number. I want to use the environment in production mode, and then assign my run list to be the base role, the DB master role, and the web role. All these three roles, one after the other, I would like that to be assigned to the box that gets created and installed with chef, and then invoke chef, and you go and look up the chef server, grab all the cookbooks, build the whole stack for me, and also install a DB user, install a database, a password for the DB user, and then go download WordPress application from the WordPress ORG website, and do all these things, by the way, for me, and come back and report success. That's what I would like to do. This is the one command that does all the things you expect. Here we go, paste it, and invoke it, and watch it do exactly what you would expect. Got a cloud machine, and let's see, so we'll just watch it. It's getting an IP address, waiting for SSHD right now, so we got the IP address there. Now we will see what Digital Ocean Cloud is doing, and you will find that our Digital Ocean Cloud has created a machine. I need to log in again, apparently, which I'll skip and go here instead. In the other browser, I have Digital Ocean Droplet called Abracadabra is dead, by the way. I'm refreshing, so I should see a Cloud Genius Droplet here. That's what I'm expecting to see when I refresh a new droplet called Cloud Genius in New York City with this IP address available. That is what's going on here in our workstation, that the machine droplet got created. It has a node assigned called Cloud Genius. This name got assigned. This client and node got created in the Chef server. It connected to that box. It is installing Chef right now, and then it is starting to run Chef right now. It has expanded the cookbook, sorry, the role run list expanded form looks like this. You will see that it has the WP colon colon default, the DB user colon colon default. Those two entries are added to the run list already. Now it knows to get the other cookbooks like, for example, the DB user cookbook and the WP cookbook also. It downloads those cookbooks, installs them, compiles the books, and starts to run the whole stack. It begins with all the base role, first of all. Get git installed, get other things installed, get vi, sudo, all other things that you need on the base machine. Then it jumps over to the other roles, which are going to install your SQL. What is your SQL? It is basically MySQL, but with your password, so it does that. Your root password, so it does that part. Installs php, php module for MySQL, apache2, apache module for mod php5, apache for mod rewrite, and then goes on to installing the WordPress software itself from WordPress website. Creates a user on the database that you have constructed here and assigns it a password called password. That is what this software expects. The software can then connect to this database created here inside this database server. That's what you're looking at right now at the bottom. All these things are happening as we speak. It should take about a minute and finish. When it finishes, you can just visit the IP address and you will see a fully functional WordPress all ready to go. That is what I expect, and that is what I'm waiting for. You see that right now our MySQL server has been set up correctly. It has been assigned the root password of 123. You can see right there, this 123 came down from the Chef server data bag. Here we have apache2 getting installed right now, php just finished, apache mod php5 getting installed, the module for apache to operate using php. Now the recipe for WordPress getting installed and it seems to have finished it. That means that we should be able to examine our services and go inspect that IP address to see what we see there and we should see our result. That is what I am looking forward to visit that IP address. That IP address is right here, so I'm going to copy that, open it in a new tab and I expect to see WordPress running and there we go. We have it and so continue forward. It is asking for a database name, username, password and host and we say, okay, I got it. If you remember, we chose the name WordPress for the database name for a specific reason, the name username for the user for a specific reason, the name password for the password for that specific reason and same with localhost and these things I chose that in our cookbook precisely to make it easy on me to show you the demonstration without having to worry about typing things. But in real life, you will never call your database as WordPress. It makes it easy to hack. You will never use the name username and absolutely never use the name password for your password unless you are a political figure. In that case, you might and you get hacked and database host is this host. You can choose a different database server if you like. In that case, you have to change this to another IP address, so another IP in your private subnet and that's how you will connect to that but right now, we just have one box so we are just using a localhost. You may want to change your table prefix if you like and then when you're ready with it, you submit. When you submit, it says, all right, Sparky, you made it through. Now run. So it runs and now it will expect you to populate the data on the database in your site. So we'll just give it a name, give it a username, give it a very complex password and confirm the use of that complex password and put our email address there and discourage search engines from searching our site because we are not ready to expose it, so install and then we are ready with our site which, by the way, is already up to date with 472 in place. It is the latest and there are nothing else needed. You can now visit the site and it looks beautiful with the latest WordPress updates just like that and you can edit and modify and instead of hello world, you can say hello cloud genius and update and visit the site again and you can see the post right there. So that is how you do it in end-to-end deployment using one command that looks like this and this is possible only if you have complete set of kitchen available in your Chef server. A kitchen basically means these files in that systematic order with a.chef, collection of cookbooks, collection of data bags, with a secret and a user data bag, environments, we have dev, broad, stage, we have roles defined here, we have site cookbooks, these are our creations, we created them, three cookbooks, oh, I messed up, hold on. What did I do? I moved my tmp inside your SQL for some reason, it should not be here, it should be outside. That's the problem you deal with when you're using a mouse to operate on a atom editor, it just moves files around very easily. So you don't want the tmp folder actually, so I can get rid of it practically speaking. So I'll just delete that whole folder, which is basically a temporary location for Chef library and to grab the cookbooks and place that in this location. And so we don't need those tmp location at all anymore, so that it can be cleaned up. We have these three creations that your cookbooks that you have, dbuser cookbook, WordPress cookbook and your SQL cookbook, these three books in the site cookbooks folder plus community provided books in this folder, all of them together constitute your collection of cookbooks, you have these three roles here and then you have a chef file to grab the community cookbooks and place them in this location. You have your gem file to place your appropriate Ruby gems in order. And for those of you who are using or going to use OpenStack, you need to modify this gem file and include a new item like this, gem, knife, OpenStack. Please try that and please tell me somebody in your team is actually going to try this because I really want you to succeed with this OpenStack thing. So please tell me that you will do it and ask me for help if you get stuck. I'm not saving it right now, but yes, you need to save and actually a bundle update and then modify this instruction to say OpenStack and a little bit modifications rest of the way, all along the way, you will need to do that. But that one command does it all. While this command is ready, I would like to now repeat the same exact exercise in a different cloud very, very easily. All we need to do is just go here and understand how do you run the entire exercise in just one command on another cloud? You take this example, copy it over, bring it here, and this is going to be a different cloud. In our example, it will be the EC2 cloud from Amazon. So we are going to use Knife EC2 plugin to operate on the other cloud apart from Digital Ocean. The example will go and operate on the EC2 cloud, create a server, we'll use our private key here. Use the username Ubuntu as opposed to root. In Digital Ocean, they use the name root. In Amazon, they use Ubuntu as the name. This machine image is probably okay from the last run of it, so I'm not changing it. But I think I will have to validate whether this is the right machine image ID number. Then T2 Micro, a production environment, and then T2 Micro is the size of the machine. The run list examples are these three items that I have chosen. Very similar to line number 14 is line number 25. It's actually identical. And 24 is similar to 13. Line number 23 is very similar to line number 9. 23 is similar to line number 9. And your machine image reference 22 is similar to line number 7. This is the reference to the operating system. So that is something I need to validate. Apart from that, we are good to run this command in the Amazon cloud. It can go and do the same thing. I want to validate that Amazon ID number, so I will go and go to the Amazon cloud right now, right here in the Amazon cloud. Where is Amazon cloud? So, going to the Amazon cloud in the My Account section in the EC2 section and identifying the machine image. Launch an instance containing Ubuntu trustee, 14.04, that's the AMI ID number. It was 14.04, if I remember right, 14.04, that's the AMI we had to find. We can do that multiple ways. The easiest way to do accomplish that actually is the Ubuntu AMI Finder. So just go there, select your Amazon region at the very bottom. So select my example uses US East 1. And the distribution I want to use is trustee, which is mapping 14.04. And in there, I would like to choose 64-bit, so I select that. Now I get these choices. In these choices, I would like to actually select a hardware virtual machine type as opposed to EBS type. So there are two different types, a para virtualization type as well as HVM, which is hardware virtual machine type. So I want to limit to those HVM types. You will see that there are four entries, one, two, three, and four entries of HVM type. I would like to choose one of them. The instance store is not what I would like. I would like to actually get an SSD store. So that's the machine ID that I would like to prefer. And that is the machine image I will use. So I'll take that AMI ID number from here, copy it over like that, copy. And this is going to get me SSD backend, hardware virtual machine, Ubuntu 14.04 trustee in the US East 1 region. So going back here and placing this number, replacing that with this new number that I would like to actually use, not that one. And that should do exactly what I expect. So I will now invoke this command also in the terminal and you will have identical setup, but in a different cloud. So I'm going to copy that command, run it here and let it go. And let's see what happens in the Amazon cloud. You will see the similar thing, very, very similar. It goes to the Amazon cloud, creates a T2 micro machine, waits for its IP address, uses the private key IDRSA in the East 1 region. And the security groups are default. I need to go check my security groups if they are open or not, which I will do right now. In the Amazon cloud, I am going to the Amazon cloud here, going to the EC2 segment in the Northern Virginia location, look at my security groups and look at my default VPC security group and make sure that the inbound routes are open for SSH access. And I can see that they are not. So I need to modify that and allow traffic from anywhere to come in and save. And that is a necessary step that you need to do in order for your Chef connection to succeed. Otherwise, this poor guy will keep waiting for SSHD to become available. It will never actually open up. What I will notice, what you will notice is that this machine is actually getting constructed in that cloud with that private IP address and the public IP address. And SSHD became open because our security group got opened up. So now that security group is open, SSHD connections opened up and it connected to this location in the Amazon cloud and it will now install Chef. It is doing that right now, installing Chef and then unpacking it and then starting this client run. The first client run is happening. It looks up the cookbook. Okay. It ran too quickly for me, but you will see that I will scroll back up and hold on. You will see that it is jumping around. Hold on. It is basically doing the run list expansion and grabbing the cookbooks and you will see that once it settles that it is doing exactly what I told you that it would do is look at the run list. It expanded the run list and then got the cookbooks that you have, all the cookbooks that you created as well as the one that you provided from the community. All of them are downloaded to that box and then cookbooks are getting invoked in the order you have prescribed and it will become exactly what you expect it to become pretty much in line with what you thought the other Digital Ocean cloud will do. You have this one running instance in your Virginia section. It is running. It has this public IP address which you can grab from here and open it up when it becomes ready. For right now, this is the Digital Ocean droplet and this other droplet is the other machine is in Amazon Cloud is getting constructed and getting loaded with all the things you expect it to run with like PHP, Apache, LAMPstack, MySQL, your password, your chosen root password. Then it runs your database user creation, username creation, password creation, installs WordPress software. All those things you will see happening and it finished. When it finished happening, it will report back success and it tells you that Flavor is T2 microed in the US East one zone and the device type is block storage that we chose already. The public IP address is this public IP address so you should be able to grab that public IP. Go to here in another window, open it up and what do you see? WordPress and you will continue and here say database name, username, password, host. We already know these things so let's go. We have filled it already, pre-filled already, we just submit. It says, all right, Sparky, run the install. It runs the install. This is Cloud Genius on EC2 and the username is and the password is and I confirm use of a very strong password and type my email address and proceed. Here I have WordPress up and running up to date and I can visit the site and this is a machine running WordPress on EC2 Cloud as opposed to this one is a machine running on DigitalOcean. Actually what you just saw as an example is what I call a vendor neutral approach to running open source software. Actually it is not limited to open source. You can run custom software also or I should say vendor neutral approach to running your apps on your cloud. It is not tied to any vendor. You can run it on DigitalOcean, OpenStack, EC2, Azure Cloud, Oracle Cloud, Cisco Cloud, bring your own cloud. I don't care. You bring your machine in your garage and run it on the garage. That also works. So you ran one instance on the EC2 Cloud. What happens if I run five of them? You do it. No problem. They'll all run the exact same setup. So just run the same command twice? Repeat the command. It will create one more. Just repeat that one command. This one. The blue one. It will create one more box, which you may or may not want, honestly. You may not want to have the same application running again on another box. So why do you want that that way? We'll talk more about these things as to the more refined use cases for a given scenario. We'll talk more about that specifically, but I'll let you digest this and I'll let you ask questions for a few minutes before we proceed further. There are more interesting things we can do with what we just did. And I'll now talk about those things a little bit, but I'm pausing here for a few questions right now. Can we do a five minute break before we proceed? Yes. Yes. Yes. Let's do that. In the meanwhile, if somebody has a question or something, please feel free to ask if you have anything. What I would say is this exercise gives you gratification, is what I have seen. People enjoy doing this exercise because at the end you actually see something useful, something meaningful come out of it. And you can modify this to run anything else that you want to consider a cover. And so you can modify your cloud, use your OpenStack cloud, use some other cloud you like or change the application, not WordPress, do something else, change the stack you want to use, not LAMP, but something else. All of these things are possible. This is a generalized approach to taking an app and its dependencies and put that in form of a collection that you want to run and then go execute. It makes it happen. You just saw it happening in these examples. We ran it in two different clouds, this cloud, DigitalOcean and EC2 cloud. And that single one command builds up the entire stack ready for use. And if you want to have a reference copy of my folder that I have here, this, this folder, this entire folder here, you can get that, it's already downloadable in that location. You can just open that link right there, it'll take you to GitHub and here you can download a ready-made Chef report with the completed exercise so that is available there for you. So thank you. You You Hi Nilesh have a question, yeah, so We did the whole nine yards, but in reality Yeah, good people go actually to the supermarket and see if there's already a lambstack Recipe ready out there and then customize it and Yeah Okay, that is also possible. You can go that way the benefit of doing this piece meal step by step is to To help you understand how to build it Understand understand once you can put comfortable with the concepts and yes, then that's the reason why I take steps in, you know item by item build the whole thing as opposed to Okay, you want to get lambstack go get it here click one And so you just go to get that lambstack you run that cookbook and it will do the whole thing and you will see magic happened But that magic is not useful Magic is useful only when you can create your own magic understand and And that is what I need the magic. Yeah, you need to know how to know how to do magic. Otherwise magic is pointless Honestly, you know, even the real magic magic is not magic if you cannot do it You can be a magician if you know how to do it and that's why that's my goal behind this elaborate exercise is to go by piece meal step by step Okay The reason I asked is probably you know, if they are, you know, I think if you're on an agile sprint run for two weeks over a week I mean and if you three we need to start up start building the environment fast Then can you how long do you take and you know, how do you? You don't approve you don't take this approach. No No, in fact in you and I don't use this approach This is an academic exercise to help comprehend chef Okay, and and more importantly comprehend the general principles behind building something step by step It is not how you and I do it You know, you know how I do it and I'll show you by the way in a subject. I how I do it I run one command and boom this whole thing. That's how it goes and There are better ways to accomplishing the same thing these days and that thing is called wordpress library in Docker So there is a ready-made library image just use the image. That's it. I think that's the best way today Correct. You can choose to run this wordpress and you can tweak it by the way There is a docker file for that. We'll talk more about docker when we come to docker And so there is this docker file you can use Here what it exactly does what we just did by the way You can see that it installs wordpress from that location And then expand the tarball removes the tarball Changes the permission and such an entry point and off it goes from the apache So that's what this docker image already does and it's a ready-made image available. This is the source code for that You can go to docker library And go to the docker hub basically on the docker hub Docker and then in docker hub you search for WordPress And you get wordpress And there it is. So you can just docker pull wordpress and run it online That's how I do it today. I mean there's slight modification to this not exactly this but I have my own uh image I've built slightly differently. I'll login show it to you Okay what I do is go to my Repository And in there, there is a repository called cloud gbs wordpress. So this is my wordpress This is how I do it And I build it like this Which the docker file here and it builds itself on top of the wordpress container And then I add my own changes to it very little changes a couple of them are to enable cryptography library SQL client graphless client mcrypt extension to enable credit card processing And an icon an htaccess file To security permissions like you you got blocked out today, right? Correct. So this this is the file that is the The reason behind blocking you The htaccess is basically preventing bad bots from coming in so that I copy that And then I build an image and I push it and that image is available for my use in that location And I just run that one line. It brings that wordpress Off we go Then you have to worry about adding your data and adding images and writing write-up and all that But that's what I have done. So then I back up and restore and bring the whole site back It's very easy to accomplish And that's how you you do in real life but The exercise is more of learning exercise. It's not how you actually do it in in practice but This is just one way of learning something step by step Okay, okay now back here What I want to do is introduce this new idea the new notion that I would like to describe as something different than what we did before And that notion is on a piece of paper coming up What we did was in our just example that we ran is is something that we I would call All in one Approach This approach is a bad approach Practically because I said of one thing long time ago. I think I you may have remembered that I said unix philosophy Philosophy and that unix philosophy has this this one tenet the core tenet is Is Do one thing do it right right do one thing do it well, that's one of the core concepts and so there is a Dot div this acronym do one thing And do it well dot div this this is an acronym. You can find its references in a bunch of literature, but the the key idea is This is bad in any unit You want to do one thing? And for any other unit you want to that thing in its own separate unit And you do that one thing really really well Not mix and match like this It is bad idea It violates the unix philosophy And with that in mind what I am going to propose to you is that we should actually offload These two functions we have the web server function And the database function into two different boxes Box one Box two two different boxes And by doing that we are able to make sure that we are basically following another principle Which is known as separation of concerns What we are really trying to accomplish Is the concern regarding managing and maintaining data? Is separated from the concern of managing and running a web server So this is the web server concern which I separate from the database concern And I would like to accomplish two machines That does the same thing together And by doing that what we are able to accomplish as a consequence is this idea Well, I'll open up a new piece of software right now And you've seen it before but i'm going to illustrate that visually And that will look something like uh, where is that show shelf show organizer Yeah, now you will see it So imagine the scenario but I what i'm really trying to accomplish is Let's move some of these things on the side and Even these guys If i'm able to Separate The web server Separate The web server The web server role for example here is the web server And here is the database server If I can I can accomplish this scenario then what I get is the ability to scale Independently across different concerns So this is my concern boundary I'm separating my data concerns with the web serving concerns and here I have one box here is another box And they're doing two different things By separating it like this I get the ability To what I call scale on on on demand scale on demand And what that translates to is something like this If I want to scale my database I can scale my database I can scale it pretty much like this Depending on how much I need how big I want how fast I want I can make my database grow and shrink according to my desire And similarly the web server can grow and shrink according to what I want That's the scaling in Sorry the scaling up and down part The other benefit I get is I can also scale My web servers horizontally like this And still have these guys interact with the same back end database server And while creating web servers one web server number two and web server number three What I would like to do is to actually have these guys function identically they will all be Acting similarly very very similarly except That they will be all responding to one common traffic cop And they will spread the load among themselves Spreading of load Or you can call it love Whatever you call and these customers will then visit the load balancer the load balancer will spread the love Among these web machines Among these web machines and will be able to handle a larger number of customers depending on what your desire or design may be And so that is what is a necessary step for us to separate These concerns and not put all of them on one box because it allows us to then Scale it appropriately for a larger audience So what does that translate to? At least at a beginning step it translates to something like this At least at least this like, you know, you have one customer the customer visits the web server Directly, this is the web server and the web server in turn connects to the database server And you get the customer The result and the customer becomes happy That's the scenario we want to enable by separating these concerns on a web function and a database function Now you want to if you want to enable that kind of a scenario using the tooling that we just developed How would you change stuff? How would you change stuff? What would you keep to change in your implementation of the kitchen? to make This thing happen like this design Which involves separation of concerns and that sets the foundation for us to be able to scale it according to our desire If you don't separate it becomes hard to scale And you unnecessarily waste money and you know, if you want to scale it up horizontally, you cannot scale databases horizontally very easily And so that has to be scaled vertically Like this I mean like like this and so unless you do a master master master slave replication that replication becomes a different exercise for right now Database has a challenge that you cannot scale it like web servers horizontally. You cannot easily scale it like that You cannot easily scale it like that. So that challenge separation is what we are dealing with and that is why This model of implementation is what I would like you to accomplish And so if that's the desire, how would you change? That is a question for you Oh, I'm so sorry david. I'm sorry. I cannot hear and understand you. I only all I hear Is mickey mouse sounds And I think I find it amusing but I told you already so please type Is that true only me hearing mickey mouse sounds or is anybody else also hearing the same thing? Okay Yeah, I hear the same. Oh, no Yeah, same same mickey mouse. I know yeah, it's mickey mouse sounds. I'm so sorry david But something in your machine that is changing the audio characteristic of your microphone Probably your software that you just added from logitech remove that software Or something like that. So ask your question in type chat, please Oh, there is a question there. Hold on. Let's see. What is the question? Modify the run list. Yes, absolutely. How do another question? How do we pi is put into the database web server example? I will answer that question also. So Let's begin with the the modifier run list question or the comment the By the way, the solution to this Specific thing of separation of database and web server is already provided It is written down somewhere here on the website You shouldn't be able to locate it Fairly easily. Where is that thing though? Here on the website. I think that's the next step probably so the next step is that exact thing that I just described which is separation of Database and web app servers Database and web app servers in two different boxes So we are modifying roles here. What we are doing is modifying the database role in a specific fashion actually, we are changing the db role to include php in there And We are so it describes what we add we added php php and php module mysql And the db user default In the in the data dv. Sorry, let me read it again. We add php. Let me make it big What we're doing is adding the php role This this php element here the recipe for php the php module mysql Mysql right here and the db user creation process These things need to happen on the database machine So we add that to the database master role and then when we end up creating a machine instance in the cloud When we want to create a database server a separate machine dedicated to the database server We will invoke the base role And the db master role and not the web server role. We would like that web server role to be applied to a different box Not the database box And that is the separation i'm trying to accomplish You need php and php module mysql these two elements inside the database server Because without that it will have no way of communicating php needs to communicate to the database database And that is why you need this item and the mysql module for php so that it will know how to interact with the database those two things are necessary for the Box to stand on its own and respond to php queries responding to the database in that box The db user default will also create the user name and password and a database entry in the database server itself That is why we add these things to that role Called db master We also change the role In the web server and we actually delete stuff What do we delete? Let's see We remove the db user default From web server role because that has been taken care of by the other role So that entry used to be here We remove that And then since we modify the roles We need to send new roles to the chef server And we will do that step, but before that I would like to answer this question about api's So the question about api let me read that question The question is how do api's fit into the database server web server example? so as you know Api is something that you programmatically call A resource of some type some resource you will have And you can call this through command line like, you know You are running the knife command to operate on some resource. For example, a resource can be chef server Or a resource can be the ec2 cloud or a digital ocean cloud or open stack cloud or any of these resources You're operating some commands in the command line and you say knife node list When you invoke that It will do exactly what you expect it to do, which is say for example knife Sorry bundles exact my knife ec2 server list show me a list of all the servers I have running in my amazon cloud And here it is. It came down with the idea It is interacting with a server with a resource of some sorts And the response back you get is the public ip private ip of the t2 micro machine running with your private key in That instance id in your cloud that you have chosen in your knife rb, which happens to be the east coast cloud the knife rb as you can see here is Uh, that knife rb mentioned that i'm using the east 1 cloud so that us east 1 is the virginia cloud that I have having said the api examples you are invoking this command through a command line interface If you want to write a program to accomplish the same idea like for example knife ec2 ec2 server list was a command line call You can implement an api call very very similarly and write some application to execute the same concept And programmatically receive without having to mechanically type the command which you did very very similar Will be a method to call an api API It will do exactly the same thing ec2 server list but through api call And that is where api's will be applied in the context of using a cloud for example here Now if you want to take that discussion further and understand how does this api how do api's fit into the database web server example? In that context an api call would be something like the wordpress application application That you are running in your web server Will actually make calls from the web server box To the database box Which happens to be running your mysql application which is a database server in that server there is a database called wordpress So this is this server this is the database That database has a user credentials. It has a user name password for that database The server has a root username and a root password like in our example one two three And so this credential is only for that database base Database base not the server. This is for the server And so this application will also use api calls To this mysql application that runs here and it opens up on port number 3306 and that is how it is able to request for data and get a response back from data So any api call is typically in form of a request in a response Yes, you ask for something and you get a response back from it Conceptually, you will see pretty similar to like This I'm running an api call on the back end that actually goes to the amazon cloud looks up the server list Amazon server list So it makes a call using amazon's response api's for the ec2 cloud and gets the response back and it presents that response back in this little command So you're getting a response back from the amazon cloud in this example So you're calling the amazon api's you're saying give me the ec2 server list And so this api call involves two things a request that you make and a response that you receive and the response will be Whatever the service sends back Every cloud company provides api's That people can use People in the open source have collectively constructed this gem Called the fog gem to operate on any cloud you like This gem is available on github in the source code. You can see github For and it will pop up. So here we go. This is the standard ruby library that services Almost any cloud you can think of including open stack And so let's go see The website for that with documentation and you will see here It is it provides aws cloud Provides aws cloud you'll scroll down provides digital ocean cloud scroll down. It provides open stack cloud also right there Here and there is documentation and here is an example for that. So let's go see some examples Of how fog open stack will work And so if you have open stack compute Installed and set up in your company you click this and read some examples of how do you get access to? And so here is a ruby piece of code that will let you connect Using the fog library the open stack fog library for open stack connect to the Target where you are running your open stack infrastructure For example, this can be something like open stack dot boeing dot com And you get the auth url from your open stack installation You will have a username and a password that you will know and a tenant id. You will also know because it is your cloud So that you can find out from your deployment. The deployment will tell you these answers the username password and tenant Once you provide that detail You can then make an api call That is what this is going to do is make an api call to do something So when before you define an api call you have to connect And create a new instance of the provider called open stack with the password username authentication url and tenant id that you provide here In these in these four lines, you provide that detail and then you use them here to make a connection And now you operate on the connection you say compute client servers create It will create a server called lucky with image reference to this image and a function called With the image reference to this image and a flavor size of number four in your open stack deployment and If it fails, it will return an error message Designated by the letter e here and put a json saying bad request And if not, it succeeds and if it succeeds it will give you The the machine ready and up and running now next one you want to compute client dot servers dot get machine identifier And then you want to basically at the end destroy the machine you can do that using this piece of code In this piece of code you create a machine In the next piece of code you can list out all the images available on your open stack deployment And in this segment, you can find out what public ips or floating ips you might have in your open stack You can also open up console if you like on the machine that you start So here are these are examples to how do you interact using api calls with your open stack infrastructure that you may have in your organization? What this basically does is allows you to use the fog api call that is written for you by the community for every cloud that you may imagine including You know the oracle cloud and the open stack cloud and you know v sphere the vmware cloud and more backspace cloud Azure cloud Bunch of other clouds there's a whole list so bottom line Everything that you interface with any other service or a resource Is happening through an api call Whether it is a reference to a cloud whether you request for some resource and you get a response back Or whether it is your own server at any point in the cloud Whether it is a response back or whether it is your own server And asking for something from another server of yours. It is also an api call That you request for a data from the database called wordpress and you return a response back from there And it happens every time and it is what the applications do So this application will actually make a request every time it needs to read the database Then respond with the data and the application will then use that data You can see it in action right here and right now as we go to that site. Where is that site? Here we have this ip address that runs that site so you can go click And you can see that this site is running I can go and now click a button to edit something So here's the edit button as I edit It is going to make a request to the database server from an api call run by wordpress application So click And it reads from the database. This is the response you get back from the database I'm going to modify this And I just did So I modified this now I will update when I update it will make another api call back to the database and save the data The data api call will be to take my new content. I modified this Take this and put back in the database and tell me that you did and so when I update it does exactly that And it comes back to the response saying yes, I did so now you can go and see and you can see that Modify this I just did it changes come back in a third api call that reads what do you got in the database? You got this Fine edit again another api call and now put some junk here And then save so the junk gets saved and you can see the junk back up right there That's the idea behind interaction between a database and your application that you run In here, for example, this is your application that interfaces with your database server and they are talking among themselves Api calls all the time And that's just in the context of this example that's how you know, most almost every any service whether it's cloud or not will run through interfaces that you define And so it is basically a well-defined idea that you know, I will ask you this And you will respond back with that and those things are typically documented in the api documentation It's basically a collection of what I call request and response Request and response I request you something you will return me something and Along with that request you have to provide certain other things that are not so apparent for example here This is my request and I just placed but I did not type my credentials My token my security key it is all implied It is implied because it is there already in this file It is provided right there It picks up from here the access key id the secret key The region and it runs that query for me when I execute this command My credentials are passed along with the api calls So it knows that I have this machine running and I have the authority to operate on that machine So now I can actually go and run that command one more time And say that I want to destroy the machine. So here we go. I'm going to say, you know, you Please kill my machine out there. So I'll say I will say bundle exec knife ec2 server delete And I want to delete the machine out there this one So I will copy that identifier And say paste it here and say okay. Bye. Bye I want you to kill that machine So it says are you sure you want to do that? Are you really want to destroy the server? The machine that I have is this id this instance name t2 micro east one And the key pair and the ip address and all that is the is this what you really want to delete the server? I'll say yes So when I say yes, the machine dead machine is totally dead And now you can see here back to our amazon cloud amazon cloud And In the account management console in the virginia region ec2 section That machine is dead And it is by the way shutting down and getting killed so you can see that right there the public ip has disappeared And we know the public ip was This so we can attempt to open but it will not open because it is dead So i'm copying it trying to open And off we go and Did I open it right? The copy face did not work. I'm so sorry Our copy face did not work. So i'm going to copy this manually again here type it in this window And paste it here to see if it opens and nothing will open because I know the machine is dead because I killed it So it is it is showing you the process of dying right now And it is short terminated so it is dead That's how you kill a machine through api calls pretty much like what we did that is uh, I think Uh stratifies the concept of api as as it applies to Just about anything like you know, whether it is a cloud service your own machines Anything to anything communication Anything communication, uh, which is basically includes a request for something and a response coming back from them That's the transaction that you do in an api call request and response And it can apply to anything including your database in depth server example. I hope that answers cara So, uh, let's see what else we have to accomplish so the database server web server separation That's what we were trying to accomplish So i'll finish that part now I'll go to my vm And here's the vm. I have that uh Digital ocean droplet also running that you know already So let's see what else we have to accomplish So let's see what else we have to accomplish Digital ocean droplet also running that you know already so we'll leave that running but I want to do is go back in here in our Website come on respond No, it doesn't want to respond The machine is frozen For some reason so I will shut down the machine and start it again Oh, it did it did finally respond. Okay, it is sluggish So I'll close this window if it does respond to me come on close No It doesn't want to so I'll shut it down and restart the vm Nothing is operating Finally it did Finally it did finally it did Okay So I will change these roles as you see here and Modify These items add them over to my database use db user role the db master role I should say and then Remove the entry of db user default from the web server role And that's the change I would like to do in my my role definitions here so in my roles, I have the the db master role in which Come on show me No one is not showing me i'm going to shut down the machine this this vm And start it again And I know uh, we are crossing the time boundary It's seven o'clock already, but I'll finish this part I think it should be a short one should not take too long I think a couple of you had to go so you left off. That's fine. I'll keep recording So you can always see the remainder and uh, what I will do now is start that machine again And finish off this separation exercise Which we are doing right now the separation of database and web server And so the machine starts up again and I will Modify the two roles that we have like we discussed and then Create two two boxes one for database one for web server and have them connect to each other That's what we should be able to do in the next 10 minutes or so And off we go doing that Okay, so let's go ahead and get started So let's go ahead and get started That's what we should be able to do in the next 10 minutes or so and off we go doing that piece Starting the command line opening the chef rep of order opening atom editor in there opening the web browser And go into the cloudiness site And In there what I would like to do is make a modification To the the roles we have here which are these roles the db master come on open yeah And the changes I want to make are right here I need to include these lines at the bottom Be below the your sequel cookbook reference. I need to add more I just did that and save and close and now I would like to do is the second reference which is Reference which is this web server role. I have I need to delete one line at the bottom Which is this db user default line I have in the web server section That line needs to go So I delete that And then I save Close I have modified two roles right now, which means I need to send these roles over to the chef server Which I can do pretty much like this So sending these two roles up to the chef server one Send the web server updated role and the second one sent the db master role Having done that We would like to modify our db user cookbook also a little bit To include some permissions Otherwise mysql server by default Does not allow Does not allow anybody to access the box From a different box. That's a behavior by default in mysql. I would like to clarify that What happens is by default if you have a mysql server database server The only people who are allowed to access that box are need to be sitting right on that box By default if you're sitting on another box and you try to access the database server, it is refused access You want to open up these permissions for somebody else on another machine attempting to access your database server That needs to be enabled If you enable that it will then allow you to access allow the web application to access the database server from a different box This is going to be box number one. This is going to be box number two And that permission has to be enabled You can enable that permission structure by making a change in your cookbook and that we need to make a change in our Cookbook right here The changes are described In the db user cookbook the default script that we have needs to be expanded to allow permissions to let the user name called user Sitting on any location not just the local host location, but this asterisk that you see here the person sign basically indicates permissions to come and connect from any server you feel like as opposed to limited to The Excuse me, this is mouse is behaving badly Okay db user files default And that file we need to expand the permissions right now Our permissions are limited to the user name on local host I want to change that specifically make it go so that it can connect to connect from any location this asterisk Is a necessary ingredient this this person sign is important to change We want to make a change here instead of local host make it percent another one We want to make a change in this segment So basically I want to copy this portion right here segment copy it over and Just above the quit stop. So i'm copying it from here making it go replace these lines with The modified lines and these modifications basically replace local host with the percent signs here and here When you do that, you're basically allowing yourself access from a remote machine Now that we have changed our database scripts, which basically effectively changes our cookbook We need to update and send our new cookbook up to the chef server So that's the command for that. You can grab that line copy and paste and send our book up The book goes up And now we are ready to create two distinctly different two distinctly different boxes One for the web server and one for the database server That's what we will do two different commands two different boxes We'll construct them both one at a time And we are practically done with the exercise So that's what I will do now. You will remember that this ssh key id number is required So we need to remember that from the last time I have it saved already. I will take that command Go back to my location where I have it saved And it is this location. I will like to make use of that Place it in the right location here And then grab this identifier ssh key identifier and place it in place of this segment So now that I have my Construction of a web server The command to construct a web server ready This will create a web server in using Ubuntu in san francisco 512 droplet with my private key With this key identifier in the production mode uses the base role and the web server role you notice that there is no database reference here You know, this is going to be our Web server the next machine I want to create Is going to be our database box. So that is how you will basically modify the command what you will do is Make a change take that number from here the ssh key id number you place it in this location like that save And then invoke these two commands one will create a web server one will create a database server Let's begin. So i'll take this command let it run And i'll describe what it does next I'll let it run And it is going to invoke A new machine Which will become our web server while this finishes Let us understand what is going on on the chef server side on the chef server, you will remember that we have a bunch of A lot of things that we have done already. I want to make sure that We are not crossing the boundary of number of nodes that they have stipulated for us For example, they have given us five nodes to operate with for free So if you're running this exercise multiple times you are very likely to accumulate more nodes than five Which means You need to go and clean up So I am not i'm not in that position right now I am probably okay But you will see that I have two nodes here already one from the amazon time and one from the digital ocean occurrence I'll delete both these references right now Here we go. So deleting that And also deleting the node for amazon It deletes both the nodes which cleans up our chef server and that makes room for Your new node that is getting constructed this new node will basically Did I accidentally shoot myself in the foot probably not Probably not because I deleted a machine called cloud genius node not the web server node So there should be a new node popping up The new node will be called web server that is expected. So it's good. I did not create a problem for myself So the web server is getting constructed And it will become a web server because it will install php And apache and nothing about the database at all nothing When this finishes we'll then invoke the other command Which is going to create a database box for us just like this In here, you will notice that the base role is included But the web server role is replaced with a database master role and That will create a database box for us And then all we need to do is just connect the two That I will show you how and then we are off and running So back in our digital ocean cloud We have This one machine running called cloud genius, which should be killed by now because it's of no use right now So i'm killing that cloud genius machine from a previous run. It is all in one box I confirmed deletion and I got rid of it. I have this web server constructed three minutes ago It is building up right now as a full web server for us When it is done doing its thing, which is pretty soon We will be able to run And it just finished so the web server is now ready We now able to run our database invocation. So this will invoke a new machine Constructed a database server using that and here it goes. So we'll have one more That will become our Database server Its ip address will be this ip address will be the database server ip address You need to make note of that ip address because that is how We will be able to connect the web server to the database server So we need to know the ip address of the database server So this is the ip address We'll be able to tell wordpress is where your database is and you will see that happening as soon as the ssh Access becomes available this box will evolve to become a database server with your password and everything else That we discussed in the previous integrations Except there is no web server component in this box because those two are separated like this The database server is getting constructed The web server is ready And you can actually open this link right now and Proceed forward You will say that I want to run WordPress and you say yeah, let's go except you want to change this entry This entry needs to point to the database host Which you currently don't have So you wait for the digital ocean to finish deployment of the database boxes And then you can actually use this Only box is not boxes. Just one box this one And it is going to finish that deployment pretty soon But that's what we need to grab as the ip address that we need to provide to wordpress Which is this ip address we need to provide this one And we'll give it to wordpress in in place of local host. We need to say you know what use this And we can say submit only The database server is ready It is not ready yet. So we'll wait and go back and look what's going on It is doing its thing. So let it do it once it reports success We should just be able to go to the wordpress box and say, you know what my database server is ready Go and do your thing and it will go and do its thing That's that pretty much it. You will see a full flesh two box wordpress separation of concerns application on one box database on the other box running As soon as this thing finishes And it will take a half a minute more and we should be able to go run our own application right there Let it finish And we should be able to see the result So Yeah, waiting is something I cannot just do and so Sometimes I have to wait. I guess everybody has to wait at some point I Relate question uh when we create web server, we can also add the step of uh opening up the Uh the the inbound traffic right which we have which we usually do manually we can also add that Yes, yes. Yes. Yes, you can in fact, I am not even discussing anything related to security right now right now Okay, yeah, we are so bad in in security We have like all the ports open and the passwords are lame and everything else, right? So that's a problem So but we'll get to that step by step Yes, you're right You can programmatically open specific ports or you know configure it the way you want using amazon's api for that or use Net filter ip table rules for securing your digital ocean droplets or anything of that nature totally do that. Yes, you should good point So this this deployment finished which means our database box is ready Which means we come here and say you know what my database host is ready now go submit This is all right sparky proceed and you have it running up and running all people Here we have the cloud genius Web db separation And the username and the password and the email address And confirm the use of a very strong password and all that good stuff Proceed and not now and here is our new wordpress, which is running DB web web database separation as simple as that so As I said the last time and if you're doing one exercise in this devops segment This is the one that you should do And this concludes that exercise. I think If I didn't if I remember, right But if there are some more steps, let me just quickly check. I don't think there are any further steps in this exercise, but Good to check So start wordpress option. There is more There's always more steps. So these are optional steps. For example, try the same thing with azure cloud. Yeah, so Being neutral Here is how you do it in azure cloud Being neutral here. Here's how you do it in azure cloud. So there's written down. I'm not doing it right now But you can the point is it is agnostic to cloud You can run it on azure cloud open stack cloud bring your own cloud doesn't matter As long as the cloud provider provides a nice plugin it'll work It'll work so Next unit I think should be a different segment altogether not this Yeah, it is different so we finished that exercise, you know, if you want but with People in boeing I would like you to try this exercise not on any public cloud But in your internal private cloud open stack And please do that And please tell me if you if you get stuck anywhere, I'm here to help anytime But I really want you to run it on your own cloud internally not on these public clouds They don't make sense to me for you It doesn't make sense. So with that I will stop recording