Hello, guys, this session is being recorded so we have a file ready at the end of the session, your audio, video, everything will be recorded and might become public. Okay, so what we were doing the last time is we were following along this kind of sort of the sequence of beginning from here of the concepts, principles, and we jumped to an exercise in the middle of the game somewhere along the way and we straight jumped to this module number five, Run Chef in solo mode. The purpose of that was to give you a quick illustration, what I actually use the word instant gratification about what this thing is. What is it that we are dealing with? That was the idea that I wanted to actually give you and give you some quick hands-on experience about what the heck this thing is. I mean, yes, it sounds interesting about cooking and chef and cookbooks and recipes, but what are we really trying to accomplish? So this is what we're really trying to accomplish. Here, this is you, and this is your workstation. It can be a laptop, desktop, it can be whatever you have at your hand right now. That is your workstation, like this happens to be my workstation, this one. That workstation that we have been using is actually this one, this Cloud Genius workstation that you know. That is what we have been using, right? But workstation doesn't have to be limited to that machine. It can be anything that you carry at the moment. Typically, it is the laptop that you have or the desktop in your office. That is what is going to be the workstation. There is another chat message from Tyler. Let's see what Tyler says. Tyler, he's chatting with me, but he's not on the call. What's up with him? I'll tell him to please join. Okay, I think he'll join. What I was talking about is here, the workstation, or maybe he is already joined, 1, 2, 3, 4, 5, 6. No, he's not joined yet. This workstation is somewhere where we will sit and we will play the role of an orchestrator. Orchestrator. This guy is the one that is going to do those things in hand and orchestrate the magic. The magic is written down in the form of what is known as a kitchen. What is a kitchen? It's a folder, a folder containing some very specific things. That's what a kitchen is. In that kitchen, we will have cookbooks. Cookbooks will have recipes and a bunch of other things in there. That is what will be a kitchen. We basically define our desire in this kitchen. We express our desire and the desire is basically the outcome we want to accomplish. Outcome. What is the result? We want to obtain and we want to obtain that result for this person who is our customer. That person needs to receive that outcome, the result. They will be able to receive that outcome based on our desire that we define here in the kitchen. Kitchen is a folder. It contains cookbooks and a bunch of other things. We'll talk about those things. We have played with at least one exercise to get us started, but this result cannot magically happen in thin air. It needs to have machines to do the work. These machines that we will have in the cloud will actually be the one that will implement that desire and produce the outcome. We have machines like this. They will grow, they will shrink, and they will implement whatever we desire. That is the core idea that we are trying to basically build out here is have a systematic method to not only build the outcome we want, but also maintain the outcome we would like to have. The ultimate goal of all this is to allow you to go on vacation to Hawaii specifically. I'm just joking Hawaii, but the idea is to let you free yourself so that this infrastructure, all of this, will build and maintain itself, allowing you to free yourself for other things in life. For example, bigger picture thinking. What do the customers really want? What can we actually build? How we go about building it? All those things that are not really automatic. They need people to actually think, and that is where I think we should focus most of our energies and time and effort and everything on is to things that machines cannot do. But then everything else that machine can do should be given out to a machine for them to do, and they should build, they should maintain whatever our desire is so that it frees up. And there is a saying in Indian language which goes like saying that empty mind is devil's workshop. And I think it's an English saying also. And I think that the word devil may be bad, but I think the devil is good here. I want you to empty your mind so that your mind is basically a fertile ground of new ideas so that you can think more about new creations for your customers. And that is where I think we want to empty our mind because we want to not use our brain as a hard disk. We want to outsource that disk storage to a machine. And so store it there. We don't have to memorize anything. Outsource it and free up the brain power so we can think bigger, think better. And that is the ultimate goal of all this, is to let you go on vacation, think free, think about the customer and accomplish something meaningful. That is what these tools basically do at the end of the day, is to free you up so that machines take over and act on behalf of you. They build, they maintain. What do they maintain? The desire, the outcome you want to accomplish. And you can apply that desire on a collection of machines somewhere in the cloud and that they will actually produce the result for you. That's the ultimate desire that we have. And that's what we have implemented in our example. Somewhere in this run chef solo exercise that we did, what we were doing is, let's now recap of this first chef solo run. So what did we do? I think most of you have actually attempted this on your own and ran this exercise on your own, but let us now flip it a little bit differently and think about what happened when we ran that. So like I described here, this workstation that you have and that you are here and that you're playing on this machine and then I'm supposed to have a bunch of machines in the cloud where I will apply this design, the desire, desire, and then I will produce the outcome for the customer. That's what I just discussed. In our example that we ran, we did not have the cloud at all. What did we do? And what was the thing that was different? It was kind of a non-standard way of using Chef. The very first example was very non-standard. The standard way, as you know, is you have a workstation and you get Chef to run in the cloud, then it runs. That's the standard way of operating. However, in our example, what we were doing is we have that workstation which is this workstation that you know already. We use this workstation and we ran Chef directly on here to modify this machine and get the result directly from this machine and we are sitting right here and we are also the customer who is looking at the result. So all of these things together on the same physical box. The machine is right here, the target is also here, the result is also obtained here and we are also here and the customer is also here. That's a non-standard but perfectly legal and supported method of using Chef. As opposed to the classic way of using is that you are away from the cloud. Here is your workstation and then you have a bunch of things that you want to apply the design, the desire, and you apply the desire to a cloud or maybe more than one cloud and then collectively these clouds will produce the outcome, the result for your customer and the customer will then be happy. That's the standard way of operating but we started and actually used this non-standard way. Nothing wrong about using it this way. It is a perfectly legal way of using it but that is not how most people use it. We went this route to just give us a quick feel about what this Chef thing is doing. What we were doing is using the same machine as the workstation, same machine as our target node. Simply speaking, I'm now attempting to define some terminology that we will use. We have this defined terminology called workstation, the machine that we use. We'll also have the target where the cloud is typically the cloud machines. That's the machines or the instances or whatever you call them, droplets. If Chef calls this thing node, that's the name that you use. A node is something where Chef runs, meaning on all these, let's go here, let me erase some of it. On these boxes that you see in the cloud, each one of these, let me see. Each one of these will run Chef, that's how it will build and maintain itself. The Chef execution will happen right on these boxes. Chef likes to use the word nodes for these things. Some people call it machines, some people call it instances, droplets, God knows whatever you call, they're basically computers away from you in the cloud. They call it nodes, and this is your workstation. That's the terminology I'm trying to begin building. These nodes can be one or more. They can be any number as you like. That's another thing to remember. Then let's go look at the actual repository that we used. We have this thing called using Chef Solo. To recap, many of you have remembered that this workstation has all the things necessary to run Chef, but it doesn't actually have the Chef binary. If you're doing this exercise and you have missed something and it's not working, the resolution is to actually install Chef, like gem install Chef. Please do that. I have done it already. If you do it again, it doesn't hurt, and it installed. It was already installed, so it did not do anything much. It was already installed, fine. If you repeat that step, it doesn't hurt. It just says you have it installed. If you don't have it, it will install it for you. Also, bring down a bunch of other gems that are needed to make Chef run, like this, lots of gems. We have about 100 or so gems that we need to have Chef run, and so they all get installed and all that good stuff. That was a big, big digression about getting the Chef binary. That binary comes in two varieties, two varieties of binary. These two varieties are both are client binaries, but one of them is called solo client, and the other one is called the client. To be specific, I should say Chef-solo and Chef-client. That's the two distinctly different binary executable versions of the client application for the Chef software. We ran this piece, by the way, in the example. The commercial company, Chef, will never talk about this because they don't want to sell this. What they actually like to do is they want to sell the Chef server. The idea behind using the Chef client binary is it actually uses Chef server. Something like you have a Chef client that runs on those nodes that you have. These nodes, you will have multiple of them. This Chef client will run on these nodes, but then this client binary does not run by itself. It needs to interact with a central location known as the Chef server. That needs to exist in the organization, in the group, in the team, in the department that you are running this exercise with. Then this client will run on these nodes that you will have in the cloud. Basically, when these runs happen here, these nodes will in turn connect to the Chef server, bring down the kitchen, and implement, build, and maintain the desire that you have. You will define your kitchen in the Chef server. That's the usage scenario in classic cases. We'll get to that pretty soon. We'll get to that point. In certain organizations, when you don't want to invest money in here, this may be a commercial product that company sells. By the way, it is also open source, so we will use the open source implementation of this product in a subsequent exercise. However, that company actually sells this product as a service. That's how most people use Chef, by the way. Most companies use Chef. You can use it without Chef server. If you want to do that, you have to provide a way to send your kitchen yourself. When the kitchen goes, then you can then execute Chef solo binary. You can see that these two binaries are already installed here on the machine. You can say, Chef client, show me the version. It will give me the version number. Same with Chef solo. Show me the version number. It is the exact same version number you get. These are the two binaries already loaded, both part of Chef as a software. The solo thing runs by itself. The client binary expects that you have a Chef server in your company, in your organization, in your team group, things like that. Those two implementations are very similar binaries, operate pretty much the same way except this binary when you execute expects that you have a Chef server to provide it the kitchen. Whereas this binary expects that you give the kitchen yourself in a folder locally to the machine. We have that kitchen right here. This is the kitchen we used for our solo execution. This kitchen, let's go examine what does it contain. We'll open Atom Editor that will show us what it has. Here is the content of that kitchen. We see that the green items here, green colored items like this and these are modified items that we have included after the fact, after the git clone, after we cloned that original example. This index file that we added on, this folder we created, this folder we created, new entries that I have added later on. This local mode cache has been created as a cache copy of whatever we ran. We can ignore these for now. I can actually delete that one and move. It is not a copy of, I think it is. There is a new change that I just posted about git virtual file system. That is a news item I posted on your Slack chat. That's one of the reasons why it is actually preventing from trashing. It will not let me trash it, but we'll talk more about why I did not trash. For right now, let's stay focused on components. We have a folder called cookbooks. This is where you describe the how to do certain things. You can either describe it yourself like I have done here in the main cookbook. I define my default recipe and I say install, start, enable, and then add this file. Four things that I describe, how to, or rather declarative what. I declare what do I want. The how to part actually is provided to be ready made in this cookbook called the Apache cookbook. We will discuss briefly as to how do you get these cookbooks, how do you write them, and all that. We'll soon get to that. Here is a folder called nodes. This folder contains references to the target machine where you want to run this thing against. For example, in this picture, these target nodes, each one of them needs to have a line item entry in the nodes folder. You have 1,000 nodes, you will have 1,000 entries in the node folder. Otherwise, you cannot know what is going to run where. You define for every node, there has to be an item in this folder called nodes. That will keep a track of inventory under management, inventory under control of this infrastructure. That's how it knows where to go to what. You will define what is known as a run list, things to do. It's like a to-do list. You go do this for me. That is defined in your execution. For example, this DNAJSON defines the run list. What do you want me to do? Please define. If that run list is empty, if I delete this thing, the blue item if I delete, it will do nothing. It's an empty run list. For example, this. If I say, then it is an empty run list. It will absolutely do nothing literally and then come out of it, come out of execution if I run it like that. If the run list is empty, that's the concept of run list. The other concept we discussed was this idea about nodes and the idea about cookbooks. Those are the examples that we ran. Now, let us quickly not save this and not save this and go back to our write up and make sure that I am actually not skipping anything that is critical for comprehension. I'll go back here and just quickly browse through. I think I mentioned this idempotence sequence matters. I think this is important for you to remember. It is very intuitive is that Chef will execute items in the order you define. That's intuitive, but I just want to call out that sequence matters. Let's see what else I should cover. We discussed this Chef solo mode. I briefly mentioned about the usage of Chef in solo mode versus the client server mode. That I think is clear, but we'll become more clear with an example. Back in our notepad here, the way of using these two products will be Chef solo runs in isolation. It can run on your target nodes you have, bunch of nodes, any number of them. It will run individually on each of these things, but it does not expect anything else. It is self-contained, runs by itself, solo. Whereas, you will have all the same nodes again like that. These nodes can run Chef client binary. Each one of them will execute that client binary, but when it runs, it expects that you have a Chef server in your organization so that the client binary can then connect with Chef server and bring down whatever you want and connect to the Chef server and bring whatever you need and proceed to do the action, which is to implement your desire and to build it and to maintain your desire on these bunch of nodes. This mode of operation, client versus server, is slightly different than this mode of operation, solo operation, doesn't require a server. Those two methods are available, both of them in Chef. Now, let's see what else is, we did this part, I think so, yes, we did. Then we proceed to the next exercise, which is again focused on Chef solo before we jump onto Chef server and client exercises. This exercise actually walks us through some of the concepts that can be boring to read. I stay focused, describe with examples, you can go read the documentation. There'll be a ton to read, but then I don't want you to get bored. I want instead you to pick up those key ideas quickly, as quickly as you can. That's the idea of learning Chef by examples. We'll follow along example by example, and here we begin. Instead of doing this, I think I would like you to watch as to what I'm doing with these because I'll actually illustrate, demonstrate the ideas that I have here, like kitchen, for example, what is that kitchen? We'll build those things right now, and I'll put this on a side-by-side window so you can watch and understand better. Here we go, side-by-side, come on. I think this, no, no, no, not good. Let's go change one more time. This one, full screen, and then go side-by-side, like here is good now. No, oh, this is not full screen yet. Okay, full screen, and then go side-by-side, like that. Now it's better. We have this old atom editor. I'm closing it out. I'm making this a little smaller, and then coming out of that, using Chef solo exercise and cleaning my screen. Then I will prepare a place for us to call kitchen because we are chefs, and we'll be using cookbooks and knife and recipes and all that. We want to prepare our kitchen. Kitchen is the place where we put our desire, like what do we want our outcome to be like, behave like, maintain like, things like that, and that is what we'll put that in a folder. Basically, it's a folder like that. So here we go, folder created. We go inside that folder, and then we list. We have nothing there, right? We open atom editor and keep it ready for us to edit. Inside that editor, we have right now nothing. It's just a folder called My Cloud. In there, what I want to be able to do is create a gem file. As you may remember from other examples, a gem file is basically a listing of all the gems I need. You may remember there is a place called Ruby Gems. This site called Ruby Gems is the place where the community at large will put all their Ruby applications ready for you to use. That is where you can grab these gems and make use of them. You can refer to these gems in a gem file and install them very quickly, easily by calling it out by name. For example, here you want a Rails gem. You will say, give me the Rails gem. Here is the Rails gem. You can see that you can install it in a gem file pretty much like this. Say gem Rails and give you the version number five like that. In our example, we'll do something very similar and actually install a gem file that looks like this. Here is our gem file. By the way, this gem file also comes with a concept which is important for us to remember and understand. I will describe that concept. The idea is, first of all, let me create that file. This file, gem file, save it, call it gem file and save. We now have a file called gem file. The file has a specific meaning to it. First line says, get my gems from this location. Some companies like Boeing, for example, doesn't want people to go out, doesn't want employees to go out to random websites and download software. They don't allow that. It has to be approved. Even if it is open source, every software that people in the company use has to be approved by your company. Getting software from RubyGems is probably not approved. I'm assuming it's not. It's a public site. I can post something there and you can download. That's not ideal, not by policy, so you want to have control. You can define your internal location in the company, something like RubyGems.Boeing.com and then force everybody in the company to use the gems that you have approved. That way you can maintain your own RubyGems location internal to the company and have a sense of discipline about which gems are supported, which gems are not in the company. You can regulate that like this. Then going below lines number two, three, four, you will see that it's a listing. This version, that gem, this version, that gem, this version. I want these exact things. The challenge you will have, for example, let's go see something here on RubyGems website. Let's go search the gem called Chef. The Chef is a gem. This is the gem I'm talking about. You look at it, show all the versions available. There are 324 versions of Chef available. Do you want to run which one again? I don't know. Are you running this one or that one? How do you know? In a team, you need to know what other people are using. Otherwise, everybody gets confused and something's working and something doesn't work. It works on my machine and it doesn't work on your machine. That's a nightmare. We want to regulate that by putting version numbers, locking them down in a gem file just like this. Right now, you will notice that on the left screen, I have versions 12, 18, 31 available. In line number four, on the right side, I have chosen to not use that version. Am I being insane? Probably not. Why am I using that version 12, 10, 24? It's because that's what I want my team to use. I don't want them to be jumping to 12, 18, 31, which was released on January 11th. Not that 12, 18, 31 is bad, but because I am not ready for my team to jump. If I want to jump, I will jump when all of us are ready, when the customer is happy. You want to test it before we all collectively jump. In a development environment, you can change, but in the production environment, you want everybody to stick to the same game so that everybody knows what are we talking about. Therefore, this is the method that I recommend using to help us stick to this version 12, 10, 24 from May 2016. That's just an example right now. We are going to use this as an example right now. That's the reason why I have chosen 12, 10, 24 as the version. All of us will use this exercise with these versions of GEMS. However, you will notice that the GEM version installed in our computer, sorry, the Chef version, Chef client version installed in our computer is slightly newer. It is 12, 18, 31. It is actually the latest. Why is it the latest? Because we just said GEM install Chef and that obviously gets you the latest, but we may not want latest. We may want something different. How do you get that other version that you want and not some random version like 12, 18, 1 or maybe older? There are 324 of them, by the way. We want to stick to what we want and we want to force that discipline and that is where GEM file comes into play. Here you can elongate this list of any number of GEMS you like in your team in a project, lock it down, and then you load. How do you load? You say bundle. When you bundle, all these specific GEMS will get installed. For example, if I now go back here and say bundle install, it will look at that GEM file and install those exact versions of GEMS. It did. It's doing, actually. Not did, but doing. All the green items are new items that I don't have. I now have all those GEMS installed, including the version 12, 10, 24, which is what I wanted. Now, let's clear the screen and run chef client and check the version. What do you think I'll get? 12, 10, 24 or 12, 18, 31? Let's go find out. It's going to be the latest version, by the way, even though you have a GEM file. Here we go. We see 12, 18, 31. But whatever happened to that GEM file? You're talking about all these things and what just happened? I thought you're going to get 12, 10, 24. Why did we not? That's a valid question. The answer to that question is that we are not running in the bounds of GEM file right now. We have to put ourselves within the boundary provided by the GEM file, and that boundary can be enforced by prefixing something. This command, this one, has to be prefixed in the context of the bundle we just installed. So by typing bundle in bundle exec, you have to execute this command in the context of the GEM file, in the context of the bundle you installed. And when you say bundle exec, whatever, it will follow the GEM file discipline. If you don't use bundle exec, it will not. It will go to the latest. So here we go, bundle exec, and you see that. 12, 10, 24. Make it big if you cannot see it. Anytime if the font is too small, please tell me. I think it should be barely readable. So here we go, slightly making it bigger. 12, 10, 24 with bundle exec, and without bundle exec, latest. With bundle exec, the one you want. So remember this bundle exec idea. It is enforcing discipline. You can prefix this. Actually, you should prefix this in every single command. So if you run some Ruby-related command, you want to run in the context of a bundle in the form of a GEM file, you want to lock it down. That lock file has been created, by the way. And this lock file shows like this. It is a cross-dependency map of all the GEMs needed to make your dream come true. The dream is defined in this simple example, and the lock is defined in this file. It tells you that this depends on these things, and this depends on these things, like that. It's a full map, a map as in making sure all the dependencies are called out, the language, the bundling platform, all of that is basically called out item by item in a map, and you should lock it. And that's what this gemfile.lock does, locks it. And so when you execute bundle exec through something, like we did, it will follow your discipline and get you the one you want, as opposed to getting the one you don't want. So, for example, here we have GEM list, and find out all the versions of GEM that we have for Chef. And so we have Chef version 12.18.31, as well as 12.10.24. We can basically say, I don't want certain GEMs, GEM uninstall, Chef. And I can say, I want to get rid of this version. So I can say two, and so it removed that. And now I can just run Chef client, and you will automatically run the right version, because that's the only version we have. You see that? Only version. 12.10.24. Earlier it was these two versions. So we eradicated that version, we removed that completely, uninstalled that specific version, number two here. That one is gone, so we just have only one. And that's what is running right now. If you now say GEM install Chef, it will automatically bring back the one that I killed. And now you have those two GEMs again. But you can bundle exec and stay focused. You got the concept, right? Bundle execution through a disciplined approach called GEM file. And you basically list out the GEMs you need, the version numbers you need, and be happy about it. That's how you reduce entropy. This word, I think you should remember, is a thermodynamics word, and it basically, think of this word entropy as something related to disorder. You want to reduce disorder. Where do you want to reduce disorder? You want to reduce that in your production environment, like here, where the customers are going to visit. You want to reduce entropy as much as you can in this example. In fact, in an ideal situation, you never, ever, ever SSH into those machines. Never, ever muck around with them directly. Don't touch them at all. Stay away from them. Let automation manage it. Don't go in the production area at all, ever. That's the ideal state. And if you're doing that, doing that in a systematic method where your automation is managing your production, you're much better off because you're reducing disorder or entropy. I would like you to read about what entropy is, if that word is new to you, entropy. You can read that on your own later on what this means and how it applies to basically it's reducing variability, disorder, doing things in an orderly fashion without involving humans. Because when humans come, variability comes. I will do this way. You will do that way. And we don't want that variability without proper approval. We want to change. We want to retain the ability to change, but we don't want things to change by itself. We want to have a disciplined approach towards making a change for the customer, not randomly change something. That's where these tools come into play is to help you. And by the way, in every language, you will have such a method. Like in this Ruby language, we are using a gem file method. Every single language will have such a systematic method of regulating your output. So you know exactly what your customers are getting at any given time, any given point in time, and not randomize. So any team member on your team should not be able to just modify, magically change something and push it out to the customer without proper vetting, proper approval. The word these days is called extreme vetting. Okay. Extreme vetting. So let's continue. So the idea is I created a gem file and I bundled it, and I installed all the gems I need. And the gems are basically packages of Ruby language. So prepared packages are available here. Any gem you want is available here. The rack gem right there. Rack. You notice that I'm using slightly older versions of gems. So 201 is the latest. I'm using this 164 or 164. This one. I'm using this one. You can go on looking for that, like Berkshelf, like that. You can basically find out that the examples that I have are chosen carefully because I know they work with each other among themselves. It's a good collection of gems that I know work. You try changing one gem version number, it might throw off others out of balance and things may not work out. So this comes through practice and experience and testing and verification as to what works well with other things. And this is one package that I know works. That's why we are using this gem file. They come from this place. Yes, called Ruby Gems, that website. This is the place where people collectively community store their output, their gems. Gem is basically a Ruby program. Hi, Nilesh. I have a question. Yes. So when we basically deploy our applications on a fast infrastructure, then the provider, they do updates. How do we lock down that out there? So let me paraphrase your question. I heard you said you can correct me if I'm wrong. You're asking about providers. A provider can be somebody like, give me an example. For example, if I'm deploying some applications on the Oracle pass, then they say that they can take care of the updates and all the stuff. So in that case, how do we deal with this kind of situation? We don't want to upgrade without an approval. How do we be cognizant about the fact that we don't want to? Good question. I think I understood. The question, let me paraphrase what you said. What you're telling me is that you want to use this thing called a platform as a service. Whereas I am currently, by the way, talking about infrastructure as a service and everything on top. So I'm talking about slightly different things. Just remember that. What you're also saying is that this platform as a service company, Oracle, that you want to use in your organization, that you want to put your applications on top of this platform as a service, and you want to regulate here. I think you should focus on regulating here, not what they do. To you, it doesn't matter what they have under the hood. As long as your application works, that's what you focus on. You don't need to. And I would recommend against worrying about what they are doing under the hood. It's none of your business, honestly. What you should be worried about is I have this application and I'm throwing this on top of your platform. Make sure my app works. I don't care about what you have under the hood, honestly. I don't give a damn. As long as my app, if it doesn't work, I will blame you. Make sure that that thing works. Under the hood, you can put Windows or Linux. I don't care, honestly. Because all I care about is my app. And that is how you should think. Not what they are doing under the hood. Let them do what they want. I don't care. No, you should. You should absolutely not care about that, honestly. There will be people in your company questioning as to why platform services, why are you changing things under the hood. Let them do it. None of your business. Sorry for being a little bit harsh on that. But that's how it should be. Stay focused on your game. Not on other people's games. Your game is to run the app. Make sure that they deliver on the promise. My app should run. No matter what. That's the demand you should ask for. Not what they are doing under the hood. Right? How do we actually hold them accountable? Can we actually figure out before any damage is done? You ask them for an SLA. And that SLA you associate with downtime. And ask them to pay penalty if you can. You are a big organization. You are a Fortune 100 company. You can totally dictate these terms with them. Make sure that you have application one, two, any number of those that you want to deploy on top of Oracle platform as a service. That is your provider. So you being a big company can actually dictate terms on this platform. They say, Oracle, here is the method that I will use to deploy my application on your platform. My method will be a git push Oracle master. Or some other method like that. That's what I will do to push my application number one. I have hundreds of them. This is just one. And I would like to make sure that before you make any changes here, that you test this thing that works. Make tests that my app works on your changes. And it is your responsibility, by the way, that you make sure that my application will continue to work on your changed and updated platform. And if it doesn't work, you will not make those changes, obviously. Otherwise, there is a penalty for that. So remember, if my app goes down, you will pay me money. So dictate those terms if you can. And you can actually do these things. Many companies do that. Like I know the federal government has a contract with Amazon. They have, by the way, a complete cloud dedicated to them called GovCloud. You may have seen that. There is a GovCloud on Amazon. Their own federal employees can create cloud instances. They have dictated, basically negotiated good terms and conditions for their agreements on SLA. That is what you really focus on, is to make sure that your applications work. If they want to change something under the hood, let them test your apps. If you want to change, here is how you test my app. Make sure that you are testing my app on your changed platform so you don't bother me. And of course, I will keep changing my app and I'll give you how to test. And you test them on your own, on the side, not on production. Test it on the side in a staging area, development area. Make sure it works. And then roll into production. That's how you should discuss these things. But leave it to them as to how they manage their business. You should focus on your application. Make sure it works. Make sure it works correctly for your customer. That's where you should be worried about. Right? Okay. So back here. Gemfile, you got that idea. Now, let's understand this idea of Bundler. Bundler is basically an installer tool. It installs things. It goes to RubyGems, this location, and installs it. You may want to change it for your company if you like. Don't have to, but you can. That was Gemfile and the Gemfile lock that you saw. Having said, let's move to the next segment. The next segment is that we want to actually create a bunch of things that we will collectively call a kitchen. And to use that, we will have to use a tool that will basically expedite creation of files in our kitchen. So right now, we have these two files. I want to create a bunch of files quickly, and so I will use a tool. The tool I want to use is called Knife. It's just a kitchen word. Knife is basically a tool that you run somewhere on your workstation. You run that Knife tool here. And what this Knife tool will do is actually remotely execute Chef here on the target nodes, any number of them. And you execute Knife command on your workstation, and it basically remotely executes Chef Client or Chef Solo. If it is Knife Solo, it will run Chef Solo. If it is just Knife, it will run Chef Client. That's the relationship. So you run Knife on your workstation. It runs Chef on the remote computer. You will see it happening as soon as we run it. This Knife command is also a gem, by the way. It comes along with your Chef installation. There should be a Knife command available already, Knife-V. And right there, we have it. You will notice that the bundle execution part was missing. So the version number was wrong, but now it is correct. So remember the bundle exec, and the Knife command is built into Chef. You have it already. It will help you run Chef on the target, on the node. And that is what we intend to run now. So we will need to use this Knife tool to actually run on those target machines that we will have in mind. Another idea that I would like to describe to you is this idea of a place like RubyGems, like you saw here, a place where you can get gems is this location called RubyGems. Very similar to this is a place for us. So for example, you want to get a gem, you go to the site called RubyGems. And you will basically put a gem file and then you list out, I want this, I want this, I want this. And so that's a listing in the gem file. And then you will install with the bundle command. And when you run bundle, you can look at the gem file and then go to RubyGems website and download and install everything that you specify and make it ready. That's the idea behind using gems. Similar idea, very, very similar idea applied to cookbooks. There is, by the way, a beautiful site in the community prepared for us, ready to use contributions by the open source community. It's called the supermarket. And in that supermarket, there is a huge collection of cookbooks available. And we need to basically create a file like that. We'll call it Burks file, just a name. In that Burks file, we'll specify, I want this, I want this version, that thing, that version, that thing, that version. And that's a file. And you will use the command called Burks. And this Burks command will go to the supermarket, bring down all these cookbooks you need and make it available. So very similar, very similar idea. Gems, go to RubyGems, get the gems loaded up. Cookbooks, go to supermarket, bring them down. This is how the supermarket looks like. Let's go to the supermarket. And so here is supermarket. And you can search for the Apache 2 cookbook. And here it is, the one I obtained for you already. This is the Apache 2 cookbook. And you can see that in a previous example, I had this cookbook folder, which contained the Apache cookbook. This cookbook, I obtained from this location, the download button. I click that button and put it right there. That's where I got it from. And I got it manually, by the way. So there is a systematic method to obtain these cookbooks. And that method is the Burks method. And just like in open source, you have 100 different ways of doing the same thing, I'll describe the other method, very similar to this Burks method called the Librarian method. And the corresponding file is called Chef file. You have the book name here and the version number here and the book name and version number written down. You ask Librarian, hey Librarian, go to the supermarket and get me the books. It will do that pretty much the same way. So Burk shelf, Librarian, these two are executables that will run to go to the supermarket, get you the books you need. And the supermarket itself is a place where people contribute books ready for use with Chef. This is just one example of a cookbook, Apache. You will have other cookbook examples like NGINX. And every big known good open source project will have a cookbook that is already available in the supermarket. And you should be able to just use it to begin with. At some point, you will learn enough to build your own cookbooks. And at that time, I recommend putting your cookbooks on the supermarket and give it to other people for them to use. And you should do that at some point. To begin with, take. Later on, you give. So here is the NGINX cookbook. And we'll use this also. By the way, this NGINX cookbook has a method to write that in the Librarian Burk shelf method or the method of writing. Because this is how you write in the Librarian or Burk shelf file, Chef file or Burk file. And you or alternatively, you can just click the download button either way. So the idea is you write either this way or that way. Two different files with similar content, very, very similar content. You can put, specify the cookbooks you need and you will bring them down into your kitchen. That's the idea. The second idea, which is about getting the books. How do you get them? You use these tools to bring it down from the supermarket. Supermarket.chef.io. That's the name of the site. It is maintained by the Chef company and it is open source. You can go to that location and find out. Back to our example folder here. We currently have those two files, which is what we want to build on. And so what I want to be able to build next is to prepare our kitchen. I want to construct a specific set of folders. That's what a kitchen essentially is. It's just folders with files. And so I want to stock my kitchen. And so I will use the command called this specific command I want to run. And I'll describe what that does as to what it's going to accomplish for us. Basically, it will create empty files and folders in our atom editor. I will just type it here. This command bundle execute, you already know this part, bundle exec. The tool that we are really running is the knife tool and asking it to initialize something. It is very similar to git init dot. Initialize git repository in this location. Git init dot as in here. Very similar to that is knife solo init dot, meaning initialize your kitchen right here. So let's do that and let's see what it actually does when I run that command. You will see that the kitchen will get populated. So here we go. Bundle exec, which is a prefix as required, knife solo init dot. Don't forget the dot at the end. When you say dot, you basically indicate, please create your kitchen in this location, which is this folder called my cloud in this example. So when I do that, you will see in the atom editor a bunch of things automatically get created. And what you will notice is that all of them are actually empty files, empty files and empty folders. So let's go run it. Here we go. And you see that bunch of things got created and a bunch of warning messages came along. These warning messages are actually not errors. They're a warning message to somebody telling them something and that somebody is not us. So don't worry about the warnings. That's what I'm saying. These warnings do not apply to you or me. They apply to the person who created those gems. Ruby wants them to change and do something differently. So Ruby has updated. Ruby wants these gem creators to modify their gems. They will eventually do. And by the way, you will notice I'm using slightly older versions of gems. So that's the reason of the warning. You can ignore the warning. You will actually see that it has created a kitchen, created a file where we store our credentials, the knife rb file. It has creating cupboards. This is a joke. This is a joke. There is no such thing covered. And it has set up Berkshelf. And so let's now see what files it has created. It has created a folder called cookbooks, which has nothing in there except this empty file. It has created a folder called data underscore bags, which is nothing in it but an empty file. It has created documents and empty file, nodes empty, roles empty, site cookbooks empty. What these things do is basically allow us a place to store relevant things in the right places. This is the initialization of a kitchen, basically empty. You can also see that there is an ignore file, which basically says to ignore anything in the cookbook folder. When you git commit, you don't want to include this folder because you don't want to give this folder to your team members. You don't want to share your cookbook folder with your team members. Can you tell me why? Why would you not want to share your cookbook folder with your team members? Why do you want to ignore or rather git ignore them? This ignore will actually hide this folder completely from git repository commits and sharing. You want to suppress that sharing. You do not want to give the cookbooks to your team members because they can obtain themselves from here. So why do you want to pass it around? This is redundant work. You don't want to give them. They can always get it from this location, the supermarket. So you don't want to give it to them. It reduces unnecessary things to flow in a git repository. So you want to cut that down. However, you'll notice there's one more folder called site cookbooks, which is not ignored. It is not in the git ignore list. And you want to give this site cookbooks to your team members because this site cookbooks location is designed to keep the books that you create that are not available here in the supermarket. These are your creations. You will create, by the way, a bunch of cookbooks in the sequence as we go along. So you want to keep your changes in this site cookbooks location and not in this location. That cookbooks folder should contain whatever you obtain from the supermarket as is, no changes. You want to make a change. You do that in here, not in the cookbooks folder. That way you have a disciplined approach about what did you change versus what the community is created for you. And so you don't mess with anything here. You just use that one, but instead you want to change something. You override it here. Override, not modify. Don't modify this. Don't modify the cookbooks folder. You want to override those cookbooks in here. And we'll do those examples subsequently. And you also want to give priority to the site cookbooks folder above the other cookbooks that other people have created because you want to actually see your changes implemented. You can give that priority. You have to specify that priority order. Please give priority to me over anybody else so that I change something. I want to see the result. Otherwise, I will not see the result. And you can flip that priority by going to a very critical file. It is kept hidden in a folder called dot chef all the way on the top. There is a file there called knife RV, which is where you store your credentials, and it is supposed to be one of those files that you don't share with people outside your team. It is okay to share within your team. And so in your team, you will store your credentials in this location. And here is where you specify the sequence. Right now, the sequence says cookbooks first and then site cookbooks in the path. I want to modify that path a little bit. I want to give priority to my changes over and above anything else. So I would like to actually make this word appear here and basically flip them around. So site cookbooks first and then cookbooks. So I'll take this, cut it, and move it here like that. Save. That's a critical change you want to make. When you do that, you're giving priority to site cookbooks, this folder content, as opposed to these cookbooks. Just a little change. Having done that piece, I would like you to understand that you're defining here my cookbooks are stored in two locations. Site cookbooks and then cookbooks. My nodes are stored in this location called nodes, that location. My roles are stored in, and I'll describe what a role is, but my roles are stored in the roles folder. My environments are stored in the environments folder. My data bag path is the data bags folder. And then there's this comment, hashtag, encrypted data bag secret. And I'll talk more about that with an example when it comes to that point. And I will also say that my Berkshelf path is the cookbooks folder. That's where you should bring and put my cookbooks when I execute some Knife command. That's the idea. So it will know where to get the cookbooks from. That's a quick short summary of the Knife configuration file. You will add your digital ocean token, add your Amazon AWS keys here, like that. You will add those things, you will add other cloud credentials in that file. In the same file, you will see one after the other, you will add it at some point. But right now, do it like that with no credentials. So saving that file. The only change we are making is the site cookbooks first before the other folder. And with that, saving. So save, close. Now, I have a bunch of empty files and folders practically right now. That is my kitchen to begin with. Having seen that kitchen, what I want to be able to do next is to actually run Berkshelf and ask it to get me a cookbook. Because I want to run a, sorry, let me define my desire. My desire is that I will have a machine in the cloud. Let's use the digital ocean cloud as an example. Just get one machine and convert that into an NGINX web server. That is my desire. For which I need the NGINX cookbook. So I can get my NGINX cookbook from the supermarket by clicking the download cookbook button. Or alternatively, I can create a Berks file where I can mention, get me the NGINX cookbook, this version number, please. And ask this tool to go to the supermarket and get the book down for me. That's the plan I have. That's my idea is to be able to specify NGINX cookbook in the file listing here. Ask Berkshelf, hey, go get the book for me. That's the idea that I want to execute now. So it will basically go to the supermarket, grab this NGINX cookbook for me. And it will do that provided you tell Berks file that. So here is the file called Berks file. It is currently almost empty here. Almost empty. In fact, if it is not completely empty, I would like to point out that this goes to a different source location. Whereas I would like to go to the supermarket location. So I will erase that completely. I'll say, okay, go bye-bye. Make it actually empty and paste my content. I want to go to the supermarket instead of that location that we were just specifying before. And in there, get me the NGINX cookbook that I need. My NGINX cookbook reference I want to get is 276. That's the latest. So I want to grab that and mention it right here in that location. So my source location should be the supermarket. Go to the supermarket and please get me the NGINX cookbook version 276. Thank you. Close. Now when I run Berks install, it should go to the supermarket and grab the NGINX cookbook for me. That's what I'm expecting to happen. And so I will execute the Berks install command to grab that book, bring it down to my machine like this here. You will see that in my computer. I will execute bundle exec berks install. And bundle exec is a prefix. This is a prefix that basically limits us to stick to the boundaries specified by the gem file, as you know. And Berks install is the command to install that cookbook that we want to grab from the supermarket, which is this location specifies which book I want from, where do I want it. Close. Run. When I run it, it will go to the location, supermarket, bring the book down. It's bringing it. And when you do these exercises, there are some other additional steps specified that are not required. So you don't have to do it, but you can. Like, for example, here is an illustration of how do you use that Berks vendor command. It's a different way of running the same thing. As you will notice, it is generating a lot of warnings. But again, these are warnings, not actually errors. The warnings do not apply to you. They apply to the gem creators, and they have fixed it, by the way. So you don't have to really worry about these warnings. You will notice that it has installed a bunch of cookbooks, but you only wanted one. You wanted the NGINX cookbook, and it has got all these books down for no reason. Why? Let's go see. Why did you bring it down? It brought that down because the dependency map says that, that this cookbook depends on the other cookbooks. So it brings down the dependent cookbooks also. It is smart enough to know that I need to get these guys yum and raw unit and ohai and build essential and blue pill and apt. It brings down all of them. That's what it just did. All these books are now available in your computer because they are dependent cookbooks. The thing that you really wanted was this one, but it knows from the location description, the dependency map tells it to get the other things as well. So you now have them in your machine available. You can see that they don't actually become visible to you. You can see that this cookbook folder is still empty. Why is that so? It is hiding from you in a central location. It keeps those books separate centralized in one computer in a central location until you use it. This is to reduce the number of downloads you may have on different projects. If you have like 10 projects on your laptop, you don't want to necessarily clone all these books from different locations. It will keep only one copy in a central location. I'll tell you where that location is. By the way, that location happens to be. .burgshelf. This is where it stores the books. This.burgshelf is in your home location. So tilde slash.burgshelf is a hidden folder and the cookbooks are actually stored right there. You can see them in a tree that these books are stored like that. And the one that you really wanted is this. Everything else is a dependency. They have stored safe in the tilde slash.burgshelf centrally. So you may have 15 different projects you're working on and you have a bunch of different books file. You'll run books, install this books, install that. And you don't necessarily want to have like so many different copies of the same set of files in multiple folders basically spread out on your machine. You want to centralize them in one location. That is this location. And you can clean them up if you like centrally very quickly. That's how.burgshelf manages its cookbooks for you. Stores it in a central location, this one. But you will be playing in your example folder, which is there. And in that folder, you don't have anything, any cookbooks yet. And you don't have any books in that folder quite yet because it's empty right now. It will receive those books once you run it. Next, this step is not required. You can do it on your own, but I'm skipping it. Next one is the idea that we have to define a target, define a node, define a machine where you want the cookbook to actually run. So you are basically identifying something like this. A target you want to run Chef against, which translates to us getting a node in some cloud. So any node, any cloud, this machine. You want to get that machine created and then have this Chef kitchen that we have. You want to run our kitchen. And we want to run this kitchen on that target, the node. You want to specify this node in our kitchen. You do that by creating a file entry in your nodes folder. Like we have this node folder here, right? Where is that? This folder, nodes, is currently empty. You want to create a new file in there. The file has to be in a specific fashion and it should contain a run list, which is right now looking like this example. So it's an empty example of an empty run list. So what I'm going to do is create a new file and save it in the nodes folder like that. In the My Cloud Nodes folder, I would like to save this file with an empty run list right now to begin with. And I want to associate this node with a node actually that I would like to have in some cloud. So here is Digital Ocean Cloud. And I will go there, create a node quickly. And so here we go, log in and then give it the token. Copy, token, paste, verify. And then we have a machine there or not yet. So create one. And then here we go. We create one, Ubuntu machine, $5 machine, San Francisco. And then I want to select my key pair. My key pair is, don't have it here yet. So I would like to provide that key pair. I will need to provide the key pair. Remember, new SSH key and give the public key here so that it knows how to connect. So don't forget that step. But the idea that I'm really trying to accomplish is I want to know this node by name, not just by IP address. So right now I'll get an IP address as soon as I create one. I want to know it by name. And then I want to create a node reference in my nodes folder here and give it some file name, proper name,.json. What is a proper name? That's what I will describe next. So you have a machine here. IP address 102.31.2.1, some random IP address. I get this. So you can have your nodes folder contain a file. So this nodes folder you have, it can contain a file that says 102.31.2.1.json. That has to be the name that will associate itself with this box. That's how you can name a node like this by the IP address. Alternatively, you can create a DNS A record and call it web1. You choose your name here, by the way,.your domain, cloudgenie.us. And that's the name I chose, web1.cloudgenie.us. And that's the name I chose. Then in that case, my node has to be named something like this, nodes slash web1.cloudgenie.us.json. Now, what the heck is this JSON thing? If you do not know, let me describe. Atom. Not responding. Atom. There we go. So Atom opens up and I'm going to describe this idea of what is called a JSON file. So watch this carefully. JSON is nothing but a text file written in between braces like this. These are the curlies that I'm looking at. So look at this carefully. And in here, I will describe a way of writing things very succinctly. By the way, this is the most efficient way of writing something that is not only human readable, but also machine readable. Very, very simple and elegant way of writing something. For example, here I go, describe a name. And the name happens to be Eric. And his last name happens to be like that. So very simple way of describing first name and last name in a record structure, that human readable as well as machine readable. I should classify this as first to make it even more meaningful. So that's a simple JSON file. That is what I'm really building. And the example that you have here is also a JSON file. Let me save this, by the way. Give it some name. So let's say web1.cloudgenie.us.json, enter. That's the name I chose and that's the file name I have, which is web1.cloudgenie.us.json is the file name. It contains this JSON. Let us see what it contains. Since it's JSON, I'm copying it over and then bringing it back to this location. And the idea here is that I have a thing called, let me put that one on the side here. This is better. Smaller, smaller, smaller, a little bit smaller. Yeah, now you'll see that I have two distinct JSON elements. One is lines number 9, 10, 11, and one is in line number 1, 2, 3, 4. Two JSONs. First one on the top describes two elements, first and last. The second JSON I have described only one element called run underscore list. And the value of that run underscore list is empty. It doesn't contain anything. The value of last in the upper JSON is this name. The value of first is this name. And the value of run list is empty. That's how you basically write JSON. You can put JSON inside JSON pretty much like this. And any levels of depth, you can go like that. That's what JSON does. And so it might look cryptic, but if you see it very carefully, it is nothing but simple structure of describing something complex or some structured method of describing any kind of data structure. So first name, last name, contact databases, any kind of data structures you can put together is how this JSON thing is. By the way, JSON stands for JavaScript Object Notation. JSON, that's the idea. Now here, our JSON file, which is this file, is not rocket science. It's simply saying run list is empty, as simple as that. And we need to have a machine that this thing needs to point to, so we need to create a DNS record so that this run list, empty run list, it will go and do nothing on that machine. It has to know where to go and do that nothing thing. So it will need to know where to go for which it needs the DNS record. That's one way to operate. The other way to operate is to just go by IP address. You can do that way also. So you can have a run list like this, which is empty, for example, to begin with, create a new file, save this empty run list in an IP address. So you go here, say, nodes folder, and in that nodes folder, I get the IP address 102.31.2.1.json. And now I have this other file which contains my run list, which is also empty. A different way of writing the same thing, by IP address and by DNS name. This is how Chef will know where to go and do nothing or do something when you modify the add something here. It will go do that something there. You need to know what that something is. You need to define that something. That something needs to be defined in form of a recipe in some cookbook, but that something is what will run on that target node here. And you need to basically have that node now. So we'll create one. But before we create, we need to have the public key provided to DigitalOcean, which I don't have. I'll call it public key and give it 2DO, like that, and bring it here, paste it, and then call it ID underscore RSA and add the key. So now key is added here, and I can now call this web1.cloudgenie.us. That's my machine name. And it is a Ubuntu machine, $5 droplet in San Francisco. Number two uses my private public key. And the name I chose is web1. By the way, here I can choose whatever I like. It's not relevant. What is relevant is the DNS name. This name is irrelevant. This is the host name, not DNS name. It's really easy for us to remember. We can use the name this, web1, just for us to remember. But this is not what counts. What counts is this, the Cloudflare DNS name, or any DNS provider that you may have. That DNS name actually is what's going to count. And not. And so you will see that here I will create a DNS entry for web1. And that entry currently points to this machine from my last exercise. I'll delete that, keep it empty, go back here, and actually create this node. By the way, as I repeat, this host name is irrelevant. This is not the DNS name. Create. So it is creating a machine for me. I'll get an IP address right there. And I'll use that IP address to provide it to DO, to get a DNS record created for web1. That is what I will wait for. It will get me an IP address as soon as it's ready. And it is finishing up. So we will have an IP address at our disposal pretty soon, in a couple more seconds. We'll copy that like this, go to DigitalOcean, paste it here, and click. So we now have a DNS record created for our droplet right here. And so we can do it two ways. Two ways are we can either rename this one and say, OK, use my IP address, or use the DNS name. Both are valid. Both approaches are valid. Now, having described this idea of what is a node, that's what we just did. Defined, and we are going to work with a node. And to begin with, our run list is empty. But we don't want to really leave our run list empty. We want to actually do something useful, something meaningful, and thereby actually accomplish our desire. So I will expand my run list idea and say, do something useful. For example, run the NGINX recipe, or rather, run the default recipe in NGINX cookbook. So that is how I will modify my run list. I'll say, you know, that run list, grab it, put that in place like that. And save. I can also do the same thing here in place of this something. I will say, you know, do something useful and save. So two ways of accomplishing the same idea. Used by IP address. IP address needs to actually exist like this. It needs to be real. If the IP address is broken, your actions will not apply. It doesn't know where to go and target the Chef run. So this will change. This might change, by the way. So I don't recommend using IP addresses like this at all, because IP addresses will change. You will destroy the machines automatically, create them automatically. You will get a new IP address every single time. Very likely. So therefore, don't use the IP address method. You can, by the way. So I'm deleting that file completely, because I don't want you to actually use that method. It is okay to use, though. And having said, having described this, I'm going by name, which means I need to have a DNS entry pretty much like this, which means I need to have a check on the DNS method. So I would like to run command, dig, and figure out if that is actually going to the right location. So dig, web1. Let's see where it goes and hits. Is it hitting the right place? Yes, it is. Good. Good to know. So it is hitting the right place. My DNS is okay. I can close this window now and go back here. Now I have my run list defined. Why is this thing in the way? Hold on. So that run list I have defined here, which basically maps to this example, as you notice. This is the item I just added to my run list. So my run list is not empty anymore right now. So having done this part, what I would like to now go describe this idea is that we need to have that cookbook in the right spot, which we have done through Berkshelf, through this Berks file, as you already remember. Next step, what we want to be able to do is to understand that next idea. I think this step you already have done, so I'm going to skip this. This is basically creating a DigitalOcean account which you have. There's a coupon there. You can use that. And SSH keys you already understand, so I'm skipping. Okay, come on. Next. And so now what I want to be able to do is to actually describe to you the concept of KnifeCommand. And that's what I will be doing. So I'll go to a notepad here and describe the idea of what this KnifeCommand actually does. So when we say KnifeSolo, and then there is an action followed by that. So the action is what we will describe next. So we can do that action description by running KnifeSolo like this and find out what are the available subcommands for it. So when say KnifeSolo Enter, it tells us that there are these subcommands available for KnifeSolo. And the ideas that we have here are Bootstrap, Clean, Cook, Init, and Prepare. In a previous step, you will notice that we have done this step already. KnifeSoloInit followed by dot, meaning this directory. KnifeSolo initialized my kitchen in this folder. I did that already. The other four are what I would like to describe. Bootstrap, Clean, Cook, and Prepare. What do they mean? Let's go here and describe. So the idea about, let me put it mathematically, so it's very quickly able to comprehend the idea of Bootstrap. The action of Bootstrap is equal to the action of Prepare followed by action of Cook. And Clean is basically clean. It removes everything. It removes the kitchen from the target. So then you say Bootstrap, a target node. So you have this node that we have defined. Node is our node is web1.cloudgenie.us. That's our node. And we want to have KnifeCommand use the ChefSoloGem to Bootstrap. This node. And what this Bootstrap does is actually in two parts, it will prepare the thing and then cook it. The preparation part, basically in simple English language, means install Chef here. And Cook in simple language means run Chef here. That's what this thing means. So Cook means running Chef. Prepare means install the Chef executable on the target node. And Clean means remove everything. And Bootstrap means this followed by this. That's what these four commands actually do. Recapping, you have these four commands. KnifeSoloPrepare will install Chef. KnifeSoloCook will run Chef. KnifeSoloBootstrap is redundant. It basically means prepare followed by Cook. And then you have Clean, which you don't really clean. What you do typically is you just destroy the droplet, destroy the machine, destroy the instance. Just get rid of the box. That's the easiest way to clean. Just kill the cow. Be a cowboy and move on. Instead of cleaning. So cleaning is boring. I would just shoot the cow and move on. And so this we rarely use. We will use Bootstrap, but before that, I would actually go step by step. We use prepare followed by Cook to understand. That is the idea here. That's the prepare command. This prepare here, and I will go half, half, 50, 50 here. So the idea here is this command. Let's go study that in a little bit detail. So here, paste. The idea here is that we are running this command in two different ways, by the way. I'm going to type the other way of running. Sorry. I'm going to type the other way of running the same thing. It goes like this. Let me write that a little bit differently. And it will be written something like this. Yeah. So there are two ways of running the same command. The upper method, the number one through four method, is useful when you don't want to use DNS because we are feeling lazy, and you don't want to create a DNS record. And so you want to still use the IP address way of creations, and you want to use the JSON file corresponding to the Digital Ocean Droplet IP address. That's the method you will use. I don't recommend that method. It is totally usable. You can actually use it just like that. What we are doing here is you are preparing that particular IP address, but you're saying that, you know what, use this JSON file by that name. And you're saying, use my private key stored here and my private key stored here. Whereas in the line number eight, you are saying, you know what, I have my DNS in order. You can just go to that location and cook itself and prepare and cook itself. And that's what you will notice is bundle execute, knife solo, prepare that machine. Bundle execute, knife solo, prepare that IP address. Use this name. That's the only difference. And you will find that both of these commands will work just fine, except I would like you to use this method. It's more appropriate. It has using DNS and all that good stuff. So having said that, there is one more shortcut that you can put to use in your life to cut down on the amount of typing you have to do or basically expedite is to skip this portion, number three and number nine. You can skip those portions comfortably by providing a method of having your SSH key evaluated as an SSH agent. So I'll describe that by doing it. Here, what I want to do is run this specific one-time operation, SSH agent, eval, and then backtick SSH agent, backtick. And I'll type this in Slack chat also. So run it. Evaluate that SSH agent. And now it has a method of remembering your private key. You can do that by saying SSH add and then give a path to the private key and say, please remember my key. That's the key I want to use in any subsequent steps. So it remembers that. You don't have to then pass this dash i option again and again anymore. I will now paste this in Slack chat. And so that specific command is kind of weird way of writing with backticks and it's not super friendly. So I'm typing it here. It is written like this. Aha. OK. Slack will not let me type backticks because Slack has a specific meaning to backticks. So backtick is basically the one below tilde. On the keyboard, there is a tick. Below the tilde, below this button, the thing is called tick. That tick, the other way. Not the forward tick, but the backtick. That is what you have to use in eval SSH agent. It is written something like this. And you have to write these backticks in two places, these yellow items. That's how you need to write. And when you evaluate that expression, it will then subsequently allow you to remember your private key by going something like this. That's what you need to run to avoid typing the dash I and then the path to private key over and over and over again. So this is the shortcut. I will use that shortcut, by the way. And therefore, I will get rid of this line as well as this line at the bottom. So this goes away. And this also goes away. And you will notice that now my command is actually pretty darn simple. Relatively speaking, it's all relative. So the command I now have is actually intuitive. It says bundle exec, we know what that means. Knife solo, prepare my box. Hey, knife, please go prepare my box. And by that, I mean install chef. And then subsequently, I'll say knife solo, cook my box, please. It will go and cook it. When you cook, it will run chef. That's what I want to actually run. So I will run this line followed by this line in which I will say, cook. That's what I really intend to run. You can run it the other ways also. But again, I don't recommend that. The reason I do not recommend that method is that it depends on the IP address changing, which is not a good thing. But you can totally, totally actually implement that and make use of it. The differences will be lines number. This is the prepare command. This is the cook command. The prepare and cook. Do the same thing on that target IP address, which is this one, which we happen to have. By now, we have it here. The IP address we have is this IP address. I'll copy it here, bring it to this location and that location, and paste. So I now have given the IP address and the DNS name. You need to choose either the upper approach or the lower approach. One of those two, don't do both. I recommend the lines number 10 and 11 because the proper way of doing things. What I want to do is prepare the machine and then cook it. By prepare, I mean install chef. By cook, I mean run chef. That's what KnifeSolo does. So here we go. I'll now begin. Look at our description. It describes BundleExecKnifeSolo, prepare, root at, and it goes by the IP address method. But I'm suggesting that we use the domain name method and then prepare the node. When we prepare the node, what is going to happen is our kitchen that we have here, this kitchen, will be pushed to the target node and will actually get ready to be run in the next command which is the cook command. Next step. For right now, just repair the box. By doing that, I will notice that this BundleExecKnifeSolo prepare root at the domain name will go and install chef on the target machine. Here we go. I started it. You will notice that it will connect to the remote machine and then install chef and then tell us that it did and then exit, come back to our local machine. That's what you're noticing right now. It's happening as we speak. It's bootstrapping chef, download chef from this location, downloading, checking the checksum. You will notice that the version matches with our desire and then install chef and says, we're installing chef, it's back to our local workstation. We did not do an SSH connection. We actually ran through KnifeExecution. Knife is this remote execution method. It executes on the target. What happened on the target? It ran chef. All of these things that you're looking at are actually happening on the remote DigitalOcean machine, this white area. All of these things are local. The entirety of this operation actually happened remotely, but we saw it happening right in our window without doing an SSH connection explicitly. We now can go to the box out there, SSH, root at web1. and see whether it has chef there. We can see that it has chef. How do you see? It's a chef client. Do you have it? Yes, I do. We have that box now loaded with chef. We were able to do that on this DigitalOcean droplet because we prepared our box through knife tooling. We said, our box, I want you to prepare, and now it prepared. Basically, it installed chef. Now I want to cook the box, so I'll cook it. Basically, I'll go and run the next command here. I'll copy, bring it here, and clear the screen and execute. When I execute this command, I will see that this machine connects to the remote computer. It also pushes my entire kitchen. That kitchen upload is happening. My kitchen is going to DigitalOcean right now. When the kitchen is there, it will remotely run that kitchen in that location. What do you think my kitchen is asking us to do? Install NGINX. That's what's happening right now. It installed NGINX. It reloaded the NGINX service. What does that translate to? My target box, this machine, is actually running NGINX web server. How do we see that? We see a 404 page, an error page, which shows that it is being run by NGINX. If you go to their website, web1.cloudgene.us, boom. We see NGINX 404 not found, run by NGINX. This is the machine that installed NGINX application, NGINX web server, through a chef run on the target. That's what we just saw. I think we should take a short break. What I would like to ask you is a favor. A favor I want to ask you is, if you have a camera, please open, because I want to take a picture. I want to take a screenshot of all of us. We can do that. All of us, we can open a camera either right now or later. I just want to take a screenshot of all of you so I can use it in some future opportunity. If you have a camera, please open it up and let me know. Just smile and, oh, thank you. I should also comb my hair, right? I guess some of you do not have a camera. I guess I'll have to request you guys, if you haven't heard me. If you can, please open your cameras. I can take one picture. And then we have to set a time for us to meet. And we take a short break. And we have to meet. And those on the East Coast, I'm so sorry, guys. But we were thinking of meeting for, I know, I know. We're thinking of meeting, getting together for dinner. And I'm so sorry for those of you, specifically Shiva and Josh and Charles. I will find a way to get to you guys some other way. But we'll discuss that. If you have cameras open, please open it up. And I'm going to take a screenshot. And I'll give you a copy of what I have taken. So here we go. Smile, getting ready. Three, two, one. OK, we took a couple of pictures. The two ran just came on. So Sami just came along. Sami is opening up his camera. Thank you, Sami. I noticed that he just opened his camera. He's going to show up. Yeah, I lost my connection. I had to reboot. OK. Yeah, no worries. We'll do it after the break one more time. And so I want to start a break right now at this moment. I'll summarize and recap and then our discussion further. The dash N, when you give the IP address, what is, why is that? Sorry? The dash N, in the first method, when you give the IP address of Digital Ocean. Yes. The upper method. Yes. This method of describing is to be able to use this JSON file reference. So the JSON is implied. OK. This JSON, I want to use that JSON file. So get my run list from this file. That's implied. So basically, it uses that JSON reference. OK. Thank you. Sweet. So we'll start a quick break and then resume. So timer goes on. And here we go. There is a shortcut now. These days, timer 10 minutes. That's a quicker shortcut, apparently. And so here we go. Start. And we started. And we'll resume in just a few. Let's see how the pictures came out. The pictures will be somewhere here on the machine here. OK. OK. OK. OK. OK. OK. OK. OK. OK. OK. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. Thank you. So what I'm getting is when I start my terminal, it says there's a software update available. And when I try and apply that, it says not enough disk space. Okay, I think we should be able to fix that later. Let's focus on the session. We're going to start that right now. And after the session, I'll look at what's going on and resolve it. Okay. Sweet. So we are about to begin after the break. And let's recap what we did. So the recap will be useful. What we did was a bunch of small little things. We understood the idea of what I would call a kitchen. A kitchen is a place for us to store information about our cookbooks. So we can have a cookbook on Apache or other community provided cookbooks. They will all belong in this folder. We have another folder called nodes that we used. And in that node folder, we had a file, a very specific syntax, the JSON syntax. And that file can have either a DNS name or an IP address dot JSON. So both methods are valid. I recommend this approach because it's a nice way of interaction. The command becomes very intuitive as you saw. Knife, solo, prepare, root at, domain name. Knife, solo, cook, root at, domain name. Very intuitive if you go like that. On the side, we also did this eval business SSH agent. And that helped us in cutting down the number of things we have to remember to type. And this basically remembers the SSH key that we want to have it remember. SSH key add. We have to add the private key there. That remembers the private key. It can remember multiple private keys. Let me add the microphone in the right place. So it remembers multiple private keys. If you have multiple keys, you can actually put them in a keyring and it will remember. And that's what this thing does. So having discussed this, we also did a couple of things. For example, we used this command called knife, solo, to initialize in it our kitchen in the current folder, which is what created this entire kitchen to begin with. Then we used knife, solo, prepare, knife, solo, cook. And those two are the install action and the execute action. So install action and execute action for Chef. Install Chef, execute Chef. That's what happens on the target DNS name or the IP address. Whatever node you specify, it goes and attacks, basically specifies and runs whatever you describe in the run list. If you scroll back to your notepad, you will have that run list was specified in a file called.json. That file that is also defined in this location. You can see it here. This web1cloudgenie.us.json, that file is a JSON file. It has this run list, which includes a reference to the nginx default recipe. That is the reason why it installed nginx right on this droplet. If that nginx recipe was not there, if it was an empty run list, it will do absolutely nothing and still run. Right now you can see it can run and do nothing. If you run it like this, it will run Chef and gladly come back and tell you, yeah, I did nothing and I came back. And that's what is likely to happen if you have an empty run list. So let's fill this item with the run list. That's the concept we discussed up until now. And then we would like to introduce one more concept now. And that concept is just like cookbooks, like nodes, is this new idea that I'm going to talk about next is called roles. And that is defined in the roles folder here. And roles can also be described in a variety of ways. So before we discuss roles, I want to actually make sure that I stick to this write up, otherwise I can go random directions, but it can confuse people. So I'll try to stick to the write up as much as I can. And here we are running the cook command to cook the machine and then we proceed forward to see a 404.not found page run by nginx. And then next we are discussing, I think about idempotence. So we can, you know, this concept already from a previous run. So before roles discussion, apparently there is idempotence discussion. So we have, we are going to have that you can realize this concept very simply by SSH connecting to that remote machine. And there you can create chaos, create entropy, deliberately create destruction. For example, kill the nginx web server. How do you kill it? You first of all enumerate its process ID and then kill it. Enumerate the process ID. The process ID is 37853880 and 3958. You kill all these processes and they actually know the 3958 is not the relevant process. Only two. This is the master process. This is the worker process. So you have only two numbers, 37853880. These two. So you kill both of them, you have nginx dead. Now you can go back here and say that, you know, it will not show you a 404 page, but instead say enable to connect. And cannot be reached. Site cannot be reached because nginx server died. What you will also see when you come out of this machine, web1, you come out, exit. And then here you run cook one more time. How do you run cook? You say knife solo and the same cook command. Knife solo cook. And up we go. So knife solo cook. Bundle exec knife solo, not prepare, but cook. So I want to say cook. Bundle exec knife solo cook that box one more time. So this is the concept of idempotence. You're running the same function over and over and over again. And it is producing the same exact outcome you would expect to see. That's what you look at. It will go and start that nginx server on the target box defined by this name, which happens to be in this JSON file. This is the node reference. That's what you will see happening. As I run it, it will go connect and see that nginx is dead and it will start that nginx server. Pretty much similar to the first exercise we did. We manually killed Apache server and then we ran Chef again. It brought back Apache life back to life. And you're seeing that it is uploading the kitchen right now. In the first example we did, we did not have to upload the kitchen because the kitchen was right there. It was the local machine we were running Chef on. In this case, we are running Chef remotely on the droplet. And so it is going to see that nginx is died. So it basically starts nginx services. And then you see automatically the page pulled up with a 404 on the left side. It automatically says 404. It got refreshed and you see that nginx is active now. It connected and found the 404, which is expected. So the idempotence concept applies on remote machines. Basically, the idea here is that you have the Chef infrastructure maintain state for you. Maintain your desire for you. Not only just create that or build the desire, but also maintain it. That's this idea of idempotence. You can go further destructive and uninstall and do all sorts of things. And when you run Chef again or when Chef runs by itself again, one of those two, you will see that it corrects itself. Next idea, I think, is about roles. So the role idea is basically a collection. Role is a collection of things. So the thing you want to remember here is that you put something in the run list here. Run underscore list. In there, you had put some item here. It was the recipe for nginx. And so that was the run list. You can specify one thing in a run list or you can specify more things together collectively and call it a role. An example of a role would be something like, I want these 300 machines that I have in that particular cloud, I want all of them to become my web servers. I want them to take on this role in life called the web server role. All of them should become web servers for me. What does becoming web server mean? In my example, I would like that to be defined in form of a role that I want to give to these 300 units, 300 nodes. And I want all of them to become a web server. It's a web farm I'm building. So I would like them to become web servers, which means I could define what that web server means. In this example, I can make it really simple and say a web server basically is a machine that runs nginx. So it's an indirection of saying the same thing that I did before, but instead of saying a run list, please run the recipe for nginx. Instead of that, I can say, you know, you should take on the role called web as opposed to running an item called the nginx default recipe. So that is what I'm going to describe is construction of a role file. That role file becomes another JSON file, by the way. And that will go and sit in this folder. In that folder, we will define a new role file that will look something like this. It might look complex, but it is not. I'll go simplify it for you. And let's go read that first and create a new file and paste that content, save it in the roles folder like here. In the My Cloud Roles folder, I would like to save this file and give it a name. The name has to be associated tightly with the actual name I will use for that role. So I would like to name it in a specific fashion. Since I want to use the name web as the role name, I will say web.json. And then save it. So I'm creating a role basically describes the function that I would like that particular collection of machines to behave like. So grow up to become a web server. In that, you do right now only one thing. The one run list is defined here instead. And this item is the run list definition. In addition to this, I'm also including certain other things in the same JSON file. And those things are, I would like NGINX to listen on these ports, not just on the port 80, but also on port 443. So I'm calling out these things in form of attributes that I would like to override the cookbook here. Now, by the way, you will notice all these cookbooks are available locally because we ran it once. And so these cookbooks became available locally on our machine. And inside the NGINX cookbook, there are a bunch of attributes. And one of them is the NGINX listen port. And I want to call out that I don't want to use your defaults. There are some listen ports defined here somewhere, but I want to use the one I want. So I will specify them by name. I want to use this port and that port. So I'm providing a way to override what this cookbook does for me in their attributes. I'm providing my default attributes should be that NGINX should listen on these ports. I'm defining that in my role definition. And everything else is basically a description of the fact that this is a role. So the name is a specific name. This web name has to match the file name web.json here. This web.json has to match the name here. The description can be anything you like. So basically you can write whatever you like. The chef type is a role. It indicates that this is a JSON file that contains a role description. By the way, these things you can write in JSON like we are doing or in Ruby like we will do in a subsequent example. You can do it multiple ways. This is the JSON way of doing things. In a subsequent exercise, you will see the Ruby way of doing the same thing. So bottom line, this is a file. It's a JSON file. It describes a collection of things you can do. For example, here I want to not only add the NGINX recipe, but I will also include git in that box. I will add on that item in there just like that so that all my web servers will not only run NGINX, but also have git installed. I can add more things here. If I want to use some other things, I can just list them out in the run list and all of these collectively become the web role. And once I have that defined, what I need to be able to do is if I am adding something here, I need to include that in the burks file, which it doesn't. So as a consequence, my cookbooks do not have git in the listing because my burks file is missing. I need to run burks install to get the web JSON, which requires git cookbook, which I don't have. So I need to fix those things if I want to include this item. I don't intend to, so I'll remove that right now, but you can. The idea behind this web role definition is to include a bunch of things in a common collection of things that you want your nodes to become when they grow up. When you begin, you're basically beginning with just enough operating system that you receive from the cloud provider. And then you add things. In this example, we're just adding one thing, but calling it a role. It's a way of describing a collection of functions. So here in this example, what we are basically doing is sticking that content right there, saving it in form of web JSON, and closing that file. Now that we have a role there defined, inside there, we will like to make a change in our node description. In our node, we are not using the role yet. We are directly invoking the nginx default recipe. Instead, I would like to invoke our new role that we have, which is the web role. So what I would like to do is modify our reference to the nginx default recipe. Instead of using the recipe nginx, I would like to say, please use this. Instead of this, use this. I'm basically swapping it out and telling him, use the role, please. I have a role definition for you, so use that. And then I close. The description of the web1cloudgini.us has changed by referencing the role, which is this role, which in turn says some default attributes, and then calls out the default recipe for nginx cookbook. Slight modification, this allows us to get one more level of abstraction, so we can put collectively a bunch of things together in form of one role. And we can have multiple roles, like a web server role, a database server role, a database role, that role, proxy role, some other role, MongoDB role. You come up with your roles and you put them in a role.json file, and that's what you assign to a node. For example, here, we are assigning the web role to that node, as opposed to directly calling the recipe called nginx, the recipe called the default recipe inside nginx cookbook. That's what we did the first time we ran. Now, we'll run it through a role reference. And now, we'll execute. What do we need to do to execute? Same thing, bundle exec, knife solo, cook. Because preparation is already happened, it's a one-time thing. This is going to install chef. This is where you run chef. So you run this command over and over again to get your changes applied. We made a change in the role definition. We created the role reference in the node reference. And now, execute. You will see that this time, it is again going to push the entire chicken. Not chicken, kitchen. See, I'm getting cook. So the kitchen is getting uploaded. It will also send the newly created role definition, the web role. And you will see this time, it is going to expand the web role, as you will see. And I will go scroll up. It did that quickly, fairly, so let me scroll up a little bit and point out that it doesn't show me in the log as to how it is expanding a role into a specific run list item. It just straight away shows me that it is installing and enabling NGINX and all that good stuff and started the service. And so at the end of the run, maybe I was jumping ahead. I cannot locate it quickly in the log, but what I do notice is that it has successfully started NGINX at the end, which means our web server should continue to function. And I refresh and it shows me NGINX showing us a 404 page. Totally expected. So that's the idea of a role where you basically give one more level of indirection to defining something. Now, having done that part, the next step that we have here is to go next unit, and you can assign multiple roles. By the way, this step is not a part of the exercise. So I'm calling it out clearly here. This is not a part of the exercise. But instead, it's more of an illustration. You can club, mix and match multiple roles in the run list like here. Like the example I have provided here, I'll elaborate on that, is to go something like in the run list, I would like to have two things, the web role as well as the database role. You can put two roles on the same box if you like. Similarly, you can mix and match between roles and recipes, references. For example, here. Here I'm taking a web role and also running a specific recipe on the Postgres database. You can mix and match whatever you feel appropriate for that particular target node. And this is how you can make use of a role that includes a role. So this is actually a role definition that includes another reference to another role. You can put roles inside roles if you like and also recipes if you like. You can mix and match basically. That's what I'm suggesting. Let's go back to our next example, which is another illustration only. And that idea is to be able to override attributes like we saw in one example before. In the other example, here is another cookbook for the Apache server. And you will see here that the cookbook, we are straight going to GitHub where the source code is for the Apache cookbook. And you will find here that the file that we are looking at is the Apache cookbook and the attributes location and inside there is default attributes. And in there, you will find certain predefined defaults. If you decide to change something, for example, you can override them very specifically in any of these segments that you like. And you can override attributes that are chosen for you by default in the community cookbooks just like this. For example, here is, let me see a good one. So for example, how many workers do you like to have for your Apache server? The default number is four. If you have a larger deployment, you may want to have more worker processes, but the cookbook does not allow you to select or rather gives you a predefined number four for your worker processes for Apache in the cookbook by default. If you want to change that, you can include an override attribute by saying, you know what, instead of four, I would like to have eight or rather eight. And so it will let you do that. That is how you can override attributes inside your references to roles and nodes. And you can basically call out just like this, add a line like this that will change the number of worker processes that are specified by default is four, but you want eight, you can get eight, no problem. You don't have to go modify the original cookbook in the cookbooks folder. You can just override values as you go along. That was described here in this example that I was playing with back in this location. The GitHub links seem to have just subtly changed. So I need to go and dig it to identify the topic I'm describing here. This is again not a part of the exercise, more of an illustration of how you can attribute override attributes if you like. Here is one more example where you want to say, keep a live timeout. You want to have a timeout value of 30, a longer timeout in certain scenarios versus a very short timeout for production deployments. You can adjust those things according to how you want to deploy this tooling to enable a variety of scenarios. I think I'll elaborate a little bit more on that. So the examples are here. One more example I think is about environments, which is the next one here. The idea here is I think it needs a better illustration on a white piece of paper. So the idea would be, just like we did here in our example, we were sitting here in our workstation. We have our kitchen there. What I would like to do is I have this cloud. Here is a customer or a bunch of customers that use that cloud. And this is our production deployment. And so when I run my knife command, it will make sure that the production is maintained and built and maintained correctly, just like my desire is written down here. And that is goodness. But I want to actually create some mess, make some changes. And I don't want to do that here. I want to make it a separate replica of the same thing in a staging area. And I want to create mess here. I can do that. All I need to do is specify that this is a different location where I want to stage my mess and test it to see if it actually is a good mess or a bad mess. I can then decide whether my creation that I changed something, is that good or not. I can test and validate in a staging area or testing area or some other area. Just call it whatever you like, not production. And that can be defined in the environment folder back in here. You'll see that there is a folder called environments, which is currently empty. But here you can create a development environment like this and look like this. And here you can specify in my development environment when I'm making a mess, I would like to have the Nginx timeout value to be 120 seconds to basically help me diagnose if there is a problem. Usually timeout values are kept small in production. But if I want to test something, as I develop something, I want to know my code is actually working or not, I may be willing to wait and have it fail after 120 seconds of timeout. So I can tweak those values, specifying Nginx timeout value to be longer. And this attribute value will be useful only when my deployment mode in the development environment. If my environment is production, then the default value of timeout will apply, not 120. If I'm using the development environment, I would like that number to be 120. I can call these things out depending on what my end output is going to be, whether it is staging, testing, development, or something else, not production. For production, you are sticking to the production environment. And that is what you will not mess with. Everything else you can mess with, modify, change, play with, see how it works, see how it breaks, all of that is okay. The scenario essentially is this folder that environment folder that I described allows you to use Chef to create replicas of your production in alternate, basically for experimentation, for building, for developing, for testing, staging, inspection, verification, backup, restore, whatever you feel like doing, you can do all of those things by creating separate environments that will create replicas of your cloud that look exactly like the production, but with subtle changes you want to make, very, very specific measured changes. I want to make this one change and see how it actually, whether it is a good change or not, whether the test passes or not, whether the customer becomes happy or not. I want to validate that, but I don't want to mess the production area. That is where this environment folder comes into play. You can make it shape like dev.json, which will hold a different value, 120 for example, and that was this illustration. We will go to specific examples at some time in the future. In this specific exercise, let's go talk about this one last thing that I did not yet talk about is the data bags idea. We talked about environments briefly. We used nodes, we used roles. We have not yet created any cookbook on our own, but we will do that. We like to talk about this thing, data bags. What is that thing? That's what I will describe next. It's a nice little exercise, so we'll do that in the terminal. Clear. Here, let me describe what data bag actually do. What do they do? In the folder called data underscore bags, notice that the word is plural. That means we will have bags and bags and more bags, lots of bags inside the folder called bags. This is plural, this is not. In that folder, we will have something like item and maybe another item. These are items that you put in a bag that you put in the bags folder. That's the structure of the data bags folder. What is an item and why it is relevant? That's the next question. Bag is a folder, data bags is a folder, item is a file. Item is a file containing an item, literally. It's a JSON object, basically. That's where you will put something like a password you want to keep that you want to use with your database. Something you want to keep safe and secure. You want to structurally keep it in a bag, an item in a bag, and a bag in the bags folder like that. That's the structure of the layout. Now let's apply it, applying it, an example. In order to play with data bags, you need to have this additional gem. That gem is currently not in our list. This gem file does not include that gem called knife solo data bag. The reason we have to have used this gem is because knife solo on its own does not have the concept of data bags to begin with. Chef was not designed to handle chef solo with data bags. But since it's open source, other people have contributed this new gem called knife solo data bag gem. To be specific, it is knife dash solo underscore data underscore bag gem with this version number that I want to use. That enables data bag capabilities for use with solo. By initial design, it was not there. Somebody outside created it. We are going to use it. Save, close. Now that we have added our new gem listing to the gem file, we need to update our bundle. We'll update bundle, update. When we do that, it will go to that RubyGems location and grab this new gem called knife solo data bag. Now we have new capabilities with our knife solo gem. That new capability will allow it to operate on data bags. You can see those new capabilities in this listing. You probably saw this command before. We had only cook, init, prepare, bootstrap, and clean. Now we have new things, specifically here, here, and more. These items correspond to the data bag function. Let's see what these things are. There are four commands for data bags. Create, edit, list, and show. They'll do exactly what you think they will do, is to create a data bag, list it, edit it, and show you the data bag. It might sound ridiculously simple the first time we'll do it, but bear with me. The very next example will be somewhat complex to understand. The first example is simple. I want to create a bag and put an item in there. The command is simple. Let's go read it. BundleExec, totally expected, KnifeSolo, data bag, create. Create what? Create a bag called what? Called pass with an item called MySQL. I want to put my MySQL password in that item in the bag called pass. This is the bag, this is the item. I'll create that command, copy that, and run it. You will notice that it pops up in editor. Let's see, run it. It is going to pop up a new editor in the computer here. Pop the editor. The editor opens up. You see it already has something there. It's the item description. It is also a JSON file. It says ID equals MySQL. That's the identifier for the JSON item for the item called MySQL in the bag called pass. I want to expand it and include some password, something like this. I'll say, you know what, in your bag, please put my password. And remember, I also pasted a comma. This comma is necessary. Don't forget that. If you forget the comma, you will have problems. Save. Save and quit. The editor quits, and it says create a data bag item called MySQL. Now, you will see in our folder, in the data bags folder, plural bags, plural, there is a bag item. Sorry, there's a bag called pass. In that pass bag is an item called MySQL JSON. And it contains the password, super secret pass. That's the layout. Like you saw me describe this idea here. There is a bag. This was called pass. And there is an item called MySQL JSON. It is very simple. This JSON was simple. It has an ID and a password. That's it. Nothing much. That was this bag. And it was plain text, open readable. You can see it, humanly read it, and all that good stuff. And that's close. That was first example. And now let us exercise the other idea, which is these things. Like, for example, we created. This was a create command. Create a bag name pass with an item name MySQL. So we create that. We can edit it the same way. Pretty much the same way. What we can do is instead of create, edit. And then we run edit. Instead of create here, we are saying edit. And it opens up the same editor. And this time it has that super secret pass. We'll say not so secret and save and close. So now our secret item contains the not so secret password. In here, we are able to now list our bags and show the contents of the bag, if you like. It's the same way of operations. Instead of edit, I will say show. And it should show me the contents of my bag, which is this item. And it's human readable. You can also export this in JSON format by typing FJ. Edit. OK. J is lowercase. So it will give me JSON format by typing F capital and J as give me the output in JSON format. So you can get that output in JSON as opposed to line item. It's a way of interacting. Now, this was just one example. Let's go slightly complex example. I don't want this password to be human readable. I want to hide it. I want to encrypt it. So we'll go next step and understand a quick method of encryption. This encryption method will involve use of a long prime number, very long prime number, which we need to generate. I'll generate a very long prime number just like that. I'll say give me a prime number. Here we go. We got that. We'll say, OK, give me again. It gives me another number. So it randomly generated a long prime number. You will notice that there are end of lines here. End of line, end of line. And I don't want that end of line. I want to have it one long line, not those so many lines. I want to have it on one line. So what I will do now is trim those. Trim the new line and the carriage return, which you can see here written down for you. I'm trimming them. And that will put all of these end of line characters and put them all in one long line just like that. You can see this one long line actually is really a one long line. You can repeat it over and over and you will generate new prime numbers. And that is going to be the encryption key, by the way. So this, it may look daunting, but don't let that scare you. This is what we are really doing is generating a prime number and then cutting down the new line carriage returns at the end of the line characters that you saw. Here you see end of line characters and here you don't. And I want to keep this prime number saved in a file. So I will redirect the output of this to a file. The redirection happens like this here. And I want this to go in a file called encrypted data bag secret. That file should belong in the.chef hidden folder in this location. That's where it needs to go. So I will basically run this whole command in one shot, which in turn will generate a prime number and redirect the output to that folder.chef in the encrypted data bag folder in that file called encrypted data bag secret in this file where the output will go. And so here it goes. And you can see that a new file got created right there. It had that long prime number. This is the key that I will use to encrypt. And only I know it, my team knows it. This is our encryption key. It is uniquely generated just like this command. And now that I have that encryption key, I can see it. Here it is. You can also see the editor if you like. But the point is not to look at it, but instead use it. So just like we did before, I want to now create a different idea. And this time we had a bag called pass. And now we will create another bag, we will create something different, not pass, but make it an encrypted data bag. And in that encrypted bag, I want to keep an item which will keep my password encrypted hidden from plain view. The plain sight, you cannot see it. And you will understand when I run it. So the idea would be that I want to create the same way like before, except I want to this time, as I create or edit or modify or do anything with it, I want to encrypt the transaction that is happening. So let's go see it. First of all, here. Like you saw the previous command, this time it is slightly different. The command is to create a different bag called passwords containing an item called MySQL. However, there's more. I'm using a secret file. And that secret file is referencing that long prime number that I have saved in that location. In that location happens to be.chef slash encrypted data bag secret. So I want the command here, knife solo data bag create, to actually create a bag called passwords as opposed to pass. And this time creating a different bag called passwords and encrypting it using the encryption secret. And this is the create command. That is what I will execute. And you will see the result and you will understand once you see it. It's intuitive, but you need to run it to understand. So here we go, run. It opens up editor. The editor is open now. It is that window here hiding. I'll bring it up front. And in there, there is this ID, MySQL. It is humanly readable for me because I am the creator I need to know. And in there, I will actually provide my password. And since I am the creator, I need to know the password. So it is human readable right now. And I will save it. And I close the window. And I exit the editor. And now I have another bag. It's called passwords. It has an item called MySQL JSON, pretty much similar to this, which is readable. And this, which is not. It is encrypted. So you cannot really read, or even if you can read, you cannot make sense. It doesn't make sense what it is. You cannot read in human unless you have the encryption key with you. You cannot really understand what this thing is. You cannot tell what the password is even by look at it. Even if you look at the whole thing, even if you copy and paste, you still cannot get the password. Even if you have a data hack, you still cannot guess the password. You let this publicly available. You still cannot figure out what the password was or is. That's a security method. This is possible because we are using the encrypted data bag secret in a specific fashion like I described. What I did was bundle exit, knife solo, data bag create. And I think this way is better. So data bag create a bag called password with the item called MySQL and use the secret file to basically operate on it. So when I do that, I got that new bag created. Now I want to edit that bag. I can go the same way and instead of create, I will say edit. And you will see that when I edit and provide the encryption key, which I have, by the way, it will let me change. So I'm passing the secret file, it will let me read in English and modify this is secret. Save, close, close. So I was able to modify and edit my secret. I can go back to my original file here and try to read. It is still not readable. This file is still not human readable. So back here, I want to now expose my content of the bag. So I will say, let me edit that. So I'll say instead of edit, I want to see, show me please, show me my password's bag. So here we go. We see it, but we really cannot understand it. And now I want to execute the other bag, which is called pass. I can see that one. It was not encrypted, but this one is encrypted. So now I want to decipher this and the decipher can happen only if I pass a secret file. And then it deciphers because I passed a secret. You probably did not see the screen, did not show you, but I will expand it to see or maybe make it smaller. You saw that I am passing a secret file as a parameter, and then it expands the secret in English. The benefit of doing this is to be able to store your credentials in a safe and secure manner without the risk of getting exposed as long as you are able to keep your encryption key secret, truly, truly secret. Don't lose that one. If you lose that one, you're screwed. But otherwise you're protected through a cipher. And that cipher is this algorithm, which takes into effect. And you see the encrypted data here. I cannot make sense out of it. It is not really readable by me. So that's the idea about encrypted data bag. And let's go back here in this. Can you show that unencrypted password? Unencrypted password. Here we go. So you want to see unencrypted password, show password unencrypted here. So the password shows that this is secret with a space in that. Was it the original configuration? I thought it didn't have spaces, or it was called super duper secret. What again? In this one, the password shows as this space is space secret. I changed it. In a previous attempt, I have modified it here. I can do it again for you. Hold on. Let me get clarity. So here I'm going to edit. And here I'm editing the secret. And instead of this is secret, I will make it Raj. Save. Close. Quit. And now I will show without the key. And now I will show with the key. Got it? Yeah. Got it. Thanks. Cool. Yeah. So I had edited that thing before. So now it is Raj. And now it was whatever it was. So the password now is Raj. And this is a systematic way of storing database passwords. Any kind of secret information you want to keep for your infrastructure, you will keep them in a data bag just like that and encrypted so that other people cannot access it. And remember, this item here should be protected. If you lose that, you're screwed. How do you protect that item? In git ignore, like that. Something like this. You protect it so that git repository will not receive it because you are excluding it, ignoring it. Another thing you want to do is go to your KnifeRB configuration. And in there, this line number 6, you uncomment that. And you provide a reference to your... I think it should be in the right up here. Let me see if I can identify. Yeah, it's right there. So you basically provide a reference to line number 6, whatever it was to begin with. Like it was something like this that had the data bag key. But you want to remove that line, remove the comment hashtag, and then paste this. And so now your Knife configuration knows that you have an encryption key that you want to use. And this way, it knows so that you don't have to type the key over and over again, which means you don't have to type this segment at all over and over again. And you can just get the key, boom. It is implied. The implication comes from KnifeRB line number 6. It tells you or tells Chef, hey, I have a key I want you to use, and it's right there in that location. I kept the key for you. Use it. That's how you encrypt data bags. So now we have discussed all these things. Cookbooks, data bag. We created two of them. One is a simple bag, encrypted bag, environments. We did not actually run examples quite yet. We'll get to that. Node, which is a reference to an actual machine in the cloud with a specific run list, a purpose in life. This is a purpose in life. We have roles defined. The role is to run nginx recipe. And we have not yet created a site cookbook. But we'll get there in a subsequent example. So this is what a kitchen looks like. This is how you implement your desire. You write that down in a set of files and let Chef handle it. That's the idea. Nilesh, one question. Should we exclude the data bags password also, or is it all by default excluded? In Git, you don't want to exclude. You want to keep this. Give it to people. Yeah, you want to give this to people. They cannot figure out. The password. Yeah, this is a trivial example. I will delete that. This is not what you do. You move to trash. You use this way. And you give it to people. OK. The simple example I just deleted. It's not how you do it. So killed it. Next one. Now we have to do an exercise. And the exercise, I will see what time it is right now. It's almost 7. OK. So the idea would be to attempt. Let me define this exercise first. And then I'll get a solution available, by the way. So what I want to do in this exercise is something like this. In our previous example that I demonstrated to you, we took a machine in the cloud, some cloud, some machine, got an IP address, put a DNS entry, give it a name, and a record points to this IP address. Shiv is asking, should we change permission of that folder, that particular file encrypted? This file should we change the permission to avoid deletion? I think it is a good practice. So what Shiv is suggesting is that you do this. This is what Shiv suggested. So yes, you should do that, which is what I did, which is to make sure that that particular file is chmod 400. And if you do not understand the chmod business, you should read chmod. Oh, no, not this, chmod. Read that page, and that will tell you what chmod is, if you do not understand. If you still do not understand, ping me. So summarizing, in our exercise we did, we got a machine, got the IP address, and then assigned a DNS name to it. Then we go ahead and built a kitchen. In that kitchen, we had cookbooks, we had node definition, we had roles, we had some data back that we did not use, and other things, like gem file, box file, all those. That's our kitchen. And we said, knife, prepare the node. So it prepared. When it prepares, it installs chef here. And then we say, knife, cook the node. And when you say cook, it runs this thing here. And by the way, before it runs, it sends the whole kitchen. And the kitchen goes and sits here, and then chef runs on the target machine as a result. You have your desire implemented. The desire is called out in the node definition. The node.json has to match this DNS name or, alternatively, the other method of using IP address. And that has to match. And then it knows what to do with the node. So it will then do the thing that you want us to do. And then it runs chef on the target machine. And as a result, you get an NGINX web server. That was the outcome. And we destroyed the NGINX machine, NGINX process, and ran chef again, and it brought it to life. Beautiful. Next, what I would like you to do is this next step, which is attempted. I have written it down somewhere here. And the idea is to set up a node that has MySQL ready, server ready, with root password enabled. What does that mean? It means I would like you to use chef to install, get a node in some cloud, install MySQL server, and a caution. Caution is to use this particular version of Ubuntu. That's going to simplify your life. Ubuntu. That's the version I would like you to use, not 16.04. Remember, not this. Remember, not 16. That's the default. That's the default, not 16. So you see 16 is not the one you want to use. You want to use this 14.04. Remember that piece, please. 14.04 for this exercise. And then node is ready, is accessible by SSH. And I'm basically describing the result. The result is I would like to have that machine become my database server, and I need to know the root password because I will assign one. I would like Raj to be your root password, or something that I choose to be the root password for that database server. And then you run chef. I mean, through knife command. And when you do that, you should have that machine evolve into a database server that contains root login with your password that you choose in a data bank, or you choose in some other way. That is what I would like the outcome to be. And this is my desire. I have defined that in form of an exercise. Attempt to do it on your own, but it's OK to cheat. The cheat sheet is the right text. And this is the answer, the solution. This is how you do it. Please do it. I think doing is more important in this exercise than attempting to strike your head against a hard object the first time you're doing it. So it's OK to cheat. The idea is to focus on Ubuntu 14.04 to simplify. You will also notice that this exercise uses a little bit older gems to simplify life for you as you build your understanding of this tool. I have attempted to use those gems that are slightly older and simpler. The newer gems are a little bit more complex, a little bit more involved. And this is one of the first few exercises that you will like to do on your own. And therefore, you will notice that this cookbook uses a slightly older version of MySQL cookbook. If you go look at the MySQL cookbook, you will find that the latest versions are actually fairly advanced. And here we go. The versions available are 8.2.0. And I'm using something like 5.6.3, I think. Yeah, 5.6.3. I'm using 5.6.3, and the versions available are 8.2.0. And that makes this exercise a little easy. And that is why it requires an older operating system. Because this 5.6.3 did not have, sorry, 16.04 did not exist when this version of cookbook was developed. So it doesn't work with 16.04. It does not. It fails. So please use this version. As we begin to learn the complexity, we have to go step by step. There will be a lot more complex things to do. Bear with this one. It is slightly older. But it gets you the clarity of understanding how do you build something. So this is one page that does it. Not too complex, not too long. But get the point across, meaning you have to have a method in which you will define your passwords in a specific JSON file. Instead of going through the data bag route, I have simplified it. And you just put that in your web2.cloudgini.us.json. And the web2 is a reference to another node, like web1, except web2. And that web2 becomes your database server. And that is what this command will do. So you will read through this, do it, and you will find that in the next step, I think, I describe how to test it, verify that it worked. And the verification steps are defined as how do you make sure that it works. You may see that the password I have chosen is 123. So it has to use that password for the database server. And that's what you will see, that it actually succeeds. And that will give you a good, solid idea about how do you use Chef in real life to construct something like a database server that uses your chosen password. That's this exercise. So you do that, and we will meet again. We plan to meet again at 9th of February at 4 o'clock. And I would suggest that you please also select a date and time you want us to get together. And let's do that. So if you have an idea, please propose. No idea. Hey, Nilesh? Yes. Also, I think you said you can also use gem files instead of JSON files. Do we have that on the modules? I said Ruby, not gem. Ruby. So in subsequent exercises, you will see those examples also. So how do you use Ruby as opposed to JSON? So you will have one of those future ones somewhere here in the chain. OK. And so I think somewhere here, yeah, module 8 will have it, I think. Yeah, module 8 has it. Yep. Thank you. So don't jump into that direction right away. And so the suggestion is to Haiku Sushi Seafood Buffet in Redmond. That's the suggestion location that's coming up. And so pick a date. And we are all set. So suggest a date also in Slack chat. And we should get together at that time. And in the meanwhile, remember, we are meeting here on live conference on February 9 at 4 o'clock Pacific time, 7 o'clock Eastern time. That is a Thursday, which is two days from now. So I look forward to seeing you guys at that time. And I will remember to stop recording.