I'm going to project my desktop and we proceed with what we were discussing the last time going to take it forward. The last time we were focusing on DevOps and we began discussing the idea of what DevOps is and how we will use a variety of tools to enable the scenarios that is what we discussed. What I have done is written down a brief summary of what we discussed right there and the video recording is of course available in the location inside DevOps and then you go to the recorded videos in spring 2018. That's where you will find it. I think you know the location already but I'm just pointing it out to you right there. This section will have a recording. As you know the last time we covered bits and pieces of what DevOps is, where it came from and then we tried to do a decent but simple exercise a really simple small exercise. The exercise we did was, let me just get to that exercise itself. That exercise was about getting started basically first run, this first run exercise and it's easy to run really really easy. What we did was in the virtual machine, just remember that you're running it inside the VM not outside and if you run it outside it might cause trouble to your computer because it will set up Apache server on a machine that you may not want. That's not what you want necessarily on a regular use machine. You don't want to make it a server. So this exercise actually sets up a server. I just want you to be aware of that and with this understanding what we're really trying to accomplish is to use this particular tool in the example that we are discussing is the Chef tool and we are deciding to ask this Chef tooling to enable a scenario. The scenario that we had discussed if you recall the other day was the idea of implementing a desire like we have a desired state configuration that we have in mind and we would like to define it. We define that desire in form of something that I think I referred to this thing called a DNA injection. This is a biologically incorrect analogy that I used but I define my desire. What do I desire? I desire that my virtual machine that I have becomes a web server. So currently the machine is not a web server. So when I run the exercise it should transition the state of this machine from here to here and so we can run that exercise very, very simply again going back to the same VM and running or first of all checking whether the machine is already a web server which is very likely the case because I ran this the last time. So if you check the processes you will find that the machine is actually running a web server. So I want to remove that web server components before I successfully demonstrate that this state of the machine changes from what it was to a state where it is a web server. Currently if you look at the localhost location in a browser you will find it shows you running some service running a localhost port number 80 will show you I think it just shows you a default page or maybe it shows a custom page I created that should show up right in that location. So the localhost colon port number 80 and try to visit that I'm expecting to see a custom page I created that's what you saw. However the scenario that we had in mind is here is that this chef tooling that we are using this chef tool this tooling will accomplish the idea that we have is an initial state converts into a desired state and then upon subsequent runs subsequent runs of chef one run again run again like that it will maintain the desired state meaning in a subsequent state if you have a desired state already accomplished and you now run that one more time it will do nothing and you run it again it will do nothing it will just maintain the desired state that's the idea that's the idea of idempotence. What we would like to essentially accomplish is to enable a setup which maintains the desire creates the desired state and maintains it that's what we would like to accomplish imagine if you have some large-scale cloud infrastructure here and we have a desire to accomplish something let's define that xyz whatever that is just define it and then write that down in form of a a git repository that we will tell our tool a tool you take this desire and implement it in this cloud and so then this gets implemented subsequently this tool will run again and again and again and it will maintain the desire of having this xyz implemented in this cloud context so the first time is creation the second time is maintenance it will maintain just like that subsequent runs over and over you run it it should maintain the state of exactly what you desire that's the goal and so with that goal understood we ran this exercise for using chef solo in the last time and we what we did was i think sh run chef solo was the command that i ran and i provided my password and then it runs now this is the second time i'm running it the first time i ran it accomplished apache installation it accomplished that apache is enabled and started and running and right now i ran it again i ran it again manually and you saw that it did nothing zero out of three did in three seconds it did successfully did nothing in three seconds that is exactly what i expect is you know in the second second run i wanted to maintain the desired state which is i would like to have this machine which is the cloud genius workstation it became a server with apache cloud genius workstation became a server with apache and that is currently rendering on port number 80 and i run chef again with my same desire and then it did nothing no changes nothing changed which is meaning to say that the desire was successfully maintained i still have my apache server running and my cloud genius workstation is still intact that's the core idea now if something happens to the apache web server something goes wrong and this doesn't function like it is supposed to be functioning the way we use these tools like like the chef tooling or ansible tooling or any other type of tooling we will use them in a fashion that they will come and take corrective corrective action on their own so typically we'll run some kind of a crown job and run these tools on a periodic basis every 30 minutes it depends on your company's policy you know whatever your project policy is you decide how frequently you should run it and you will see that these tools will look at your entire cloud infrastructure and periodically maintain the state the state that you would like to have that will be maintained every so often it will run and across the entire cloud and maintain the state of your cloud according to what you desire and it will also at the very first time create the state for you that's the goal now having understood these parts you go back to here in the workstation and you see that currently my browser is currently showing you that it is functional by refresh my site it shows you that it is showing you this page that I'm expecting to see and I refresh it and I still see it however what I now intend to do is to basically go destructive deliberately cause trouble I am going to destroy a couple of things like right here I see that this Apache process is running so I find it and I'm going to kill it like that what's the number two five three nine and the numbers are three five five four three five five five and now Apache server is dead and we can check it oops you can test for it and it's not there anymore which means that our machine is bad it's not going to give you any page unable to connect which is I did deliberately I killed my Apache servers and you saw that my site is not functional anymore and so we can have on a periodic basis the tool comes in on itself on a cron job or some other invoked method in which it will come and run the same tooling again pretty much like this it runs again and when it does what do you think will happen it will correct the state it will detect what is missing and fix that exact component whichever is missing you saw that it found that the service was stopped so it started that particular service it tells you exactly what it did and so one out of three resources updated in three seconds as a consequence of the deliberate action I did before which is I killed my Apache server and I did nothing as a consequence my page is not loading but chef came along and fix the situation for me so it looks at what was different from the desired state and now as a consequence of that chef run we now have Apache server up and running again which means we should go back here into this site and we should test we see that chef is working again I mean the site is working again the site came back to the state that we desire that's how you should you know that's how you will see that these examples operate practically the same way the the core idea essentially is you have an initial state which is just enough operating system and you implement your desire and the desire is desire is implemented in the first time and in subsequent runs it is maintained that's the goal of any kind of tooling in what we would discuss and what we will use is you know no matter which tool you will have in practice the core idea is that you define your desire and then these tools will do it for you maintain it for you the ultimate goal of this is that so you can go on vacation that the tool will take care of the stuff for you and you can just do whatever you want like for example you want to go think like the thing that people cannot the machines cannot do right you can go and think on creative thinking maybe you can go to Hawaii and think or go to Sedona and think or enjoy life while the machines take care of your stuff that's the kind of concepts that you can put to use in practice in your business in your company and that is the idea that we are now building to discuss and we will see some live examples more and more complexity increasing complexity of such examples coming up so next one what I want to do is I I know I killed Apache server in a previous attempt what if I just eradicate the existence of Apache from this machine altogether like for example I will pseudo apt remove Apache to and let it go and not only that I will also completely remove any dependencies that were installed as a consequence of this Apache installation which chef did I did not install so I'm saying pseudo apt auto remove and completely get rid of any auto remove so I'm completely getting rid of all the dependencies that I that might exist for Apache as a consequence I have Apache completely missing and not running and you can say that apt list by grep Apache and that is also missing and so you can see that no no this is this is not the list this is a list of available packages the list of installed packages I think is different not that list so apt list is going to give you a list of entire list of possible Apache packages but you can see that Apache is not installed so with that of another destructive change that I just made what I want to now notice is this chef server I mean chef tooling coming in and running and correcting the state for me while I do nothing so right now what I'm doing is currently manually simulating the cron jobs by running this step mechanically by hand but in practice this command will run by itself you know in a cron job so you know at a clock time whenever it happens it will come and run this and when it runs you will see that it is going to fix the problem that I created deliberately I uninstalled Apache removed all the dependencies and you know it basically got rid of everything as a consequence this thing is not working right now you see that this thing oh it by the way but before I before I tried to reload the thing fixed itself so it installed Apache and corrected all that and as a consequence you have a PS dash EF grep Apache is running again so my demo of this failure did not work because this thing came along and installed it before I could even hit refresh so that's the concept that we just saw in live action with destructive behavior on my part deliberately causing trouble and chef coming in and fixing it for us that's the scenario now let's extend the scenario further what I want to do now is to look at another case is now I want to change my desire that you know this desire is too simple for me I would like to make something interesting something different and so what I want to do now is to define a different idea for example in here in this page that you see it you see this page which I want you to change it I want you to be able to change it through automation by chef tooling should go and change this page it should say you know your name or something like that something different so for that to happen what do you think will need to go in our tooling to make this change happen as you know in our example in our repository if you open that repository in a atom editor you will find that this contains a cookbook the cookbook is called the main cookbook the main cookbook contains one recipe the recipe is the default recipe and that default recipe invokes apache action install and apache action start and enable that's simple that's it it doesn't do anything else so now if you want to actually place a file containing your custom HTML content like right in place of this file which by the way this is a file somewhere located in in in the computer I think by default it should be sitting in it should be sitting in let's go there VAR www that's where the file should be sitting this file cat index HTML is where the actual file is sitting right there you see chef really works this is a file I created manually I placed it there but I want you to change it I want you to use the chef tooling to change it in a dynamic fashion whenever chef comes in it should just put your content in place of this content so for that to happen we need to modify something so let's understand what do we need to modify our desire our desire is expressed in in terms of this folder this this particular using chef solo folder currently expresses the desire that we would like to have the DNA actually this file very succinctly describes the desire it says run the default recipe in the main cookbook as the run list when chef runs that's what this reads this is where you express your desire and we have it done we have expressed it and what I want to do now is to extend the desire expand it from what is currently is which is simply what it currently does is you know install apache enable start and enable apache enable and start three actions those are the three actions very clearly written down in our desire in that particular folder these three actions one after the other they just happen if any of these things are missing it will take corrective action and restore back to normal that's the implementation of our three step desire written down in that folder this folder exactly in this recipe install action start action enable action those three are called out so now if I ask you to come back here and extend the desire to place a new file called index html let's say you want to place a file here in in this location in that file in that folder you want to place a file right in that location and you would like to say welcome Sridhar if that's the thing that you want to see in this file you want to place this file in that folder but not manually you want to use the chef tool to make this happen what do you think needs to be done to our folder here specifically this default recipe in the main cookbook what needs to happen something like I'll write some pseudocode here right some pseudocode will be you know create a file welcoming Sridhar and then save it in folder that's what I would like to see in this particular recipe expanded in its scope and that's what I want to happen make it happen and I would like to then save this file and run this particular cookbook or rather this particular chef installation or rather invocation of this run chef solo command when I run it it should then you know look at this extra item and do what it says but this is not how you write so how would you find what should I write here like what would you go and read to come up with the idea what needs to go written in this segment what would that be where would you go search so let me tell you searching where should you search you should search for chef documentation that's the best place to find and it's right there so you go there and find out how do you deal with creating a file right so that's the core idea we just search for so we'll just take the keyword file and put it here and say file and it should come up with a file resource right there and we look at the file resource and it will tell us that a file resource is a block that manages a file that exists on node so there it is so I want to place a file so essentially I want to read this segment focused on this segment so for example to write to the home page of an Apache website do this so I'll grab it take it right here and paste it and see what happens so that's what I want to write that's what I want to write in my in my recipe and so let's see what I how would I customize this to match exactly what I desire so the file resource should not have the customers in public HTML but instead it should be just HTML in our example it should be index.html file and then the content will be something like HTML and we should just welcome Sridhar like that and that's the we get a file and save it in that location and let's go see what happens now so if I save it there's still a problem the problem is that this owner and admin owner group that is not the correct owner admin for our machine so we need to make it the correct owner www data I think is the name for Ubuntu you can see this detail you can find out that detail from finding out who is the owner of Apache process in Ubuntu so we should be able to do that in a couple of different ways one of them is to go here and find out the Apache process is run by this user and that's the name we can just grab from here and put it right there owner and group this is one way to find quickly the other way to find is to look at this file location that might exist in your computer and find out who are the owners of that particular file and you see that currently the file is owned by root root so the point is root is okay to use you can use the word root here like that in place of www data you can say root that is also okay but having a non-existent name like we have from the documentation is not going to give you a successful result like this web admin needs to change to something that is an existing user in your computer in your system and this is a quick example of a syntax to place a file in a node in a machine in the example that we're looking at so this is how you would go about in real life just read the documentation search the exact key phrase come up with the syntax that is available to you and then edit it appropriately to get the desire implemented now with this change in place let's go read it a little bit further one more time before we run it so this this file block that I just added that has a content it has a mode 755 the owner is group owner and owner is root and group is also root I would rather you know my preference is to use www data again these are cosmetic issues for right now they're not critical currently but you know how to modify them you can actually you know just skip them that's also okay just delete them but let's read what is the right thing to do so we'll go read further the full syntax all the properties available now here you will see that you can put other aspects like atomic updates backup checksum content force unlinked group inherits lots and lots of capabilities of the file resource in a block like this that you can put in a recipe inside a cookbook like this so I my my think is as you as you expand your understanding of how do you use these two links to express your desire to make a goal happen is that you begin small and play with it and see how it works for you see how you would then go back to the documentation and expand it the best source to find these tools actually is right in the documentation like right here so for every tool there is a documentation page you can find these things on github the best place to find and here it is the chef's web documentation is on open source and you can of course read the file resource from here that is also available that is the best place to find documentation is right on github that is also the authoritative way to find the correct answer every other answer is a derivative answer this is the right answer because the developers are themselves putting the answers right in the documentation it is a part of their git repositories you are free to and actually said I suggested you go to the straight to the source and read it directly from the github repository for that for the product and now we have modified this so we'll save it and we'll close it and we'll understand what we did what we did was we extended our desire like this this is the new segment that we just expanded you know this this portion is new which was the initial was all these three things now we have these are new items so create a file with some content place it in that location and boom what do you see let's see what we see so I'm going to run it right here in this location I will go to my using chef solo folder it's important that I run in that folder and then I will run it just like that and see what happens I'm expecting to see welcome Sridhar show up when I visit this page and refresh it currently it shows me this and now if I refresh I see welcome Sridhar that's expected but let's see why it worked that's what I want to understand why did it work what did what did we just do so let's go see that so we'll go and read the log the log will tell us exactly what happened so we are now reading excuse me we are reading this detail as to what's going on it looked at the action install action and it did nothing for that because it is already installed the start action was also already started so there was no need to no need to do anything different the enable action already already up to date so it did not do nothing so these three actions were already up to date and there was no need to do anything as a consequence of that it is looking at your current state of the machine and comparing it with the desired state of the machine and taking corrective actions if necessary the current state already installed so nothing to do action nothing to do here as well and as well as here but now comes the opportunity to do something which is there is this new item called file resource but the action is to create a resource this action matches what we just did here we have to create a file resource on the computer on the target machine where we are running and there we saw it did what we we thought it would do and that is to create a file containing a content block that we are providing the content block is provided in the code itself and what it finds that there is this need to update the content in that file so there was a file already in that location and that file was will be changed so that's why you see that this this block of code is getting updating in that file in the file already contains this segment so you can see that this particular highlighted segment contains the item that the file had before you can see very carefully that these minus minus minus and a plus plus plus are an indicator of what is getting removed and what is getting added you can see that these minus signs here are an indicator of the fact that these lines will be deleted from that file so you see that these lines are deleted and then a plus sign comes along this plus sign is adding html block welcome trither and then end of html so this new block or rather one line was added to this file in the exact location that we had and you will also notice one thing which is the ownership of that particular file is now with www data earlier the owner was root now the owner is www data and this change happened because we said so we said that we wanted to make it www data the owner of the group group owner sorry group and owner both to be www data and initially the mode was 0644 we wanted the mode to be 0755 the mode changed the ownership changed the content changed in that existing file that was already there so now you see what you see here welcome trither is because that is exactly what we desired 0755 www data and the content in that file to be different it's now trivially simple to modify it we can just say here welcome cloud genius save it come back here and run chef again right now you will see it is still trither but when you come back here and say run chef solo it was going to go change it and you will see the change exactly very systematically item by item line by line it will report what was the thing different and what I'm going to change so it sees that this content in the file needs to be changed because earlier it was something different now it needs to be something different earlier it was with a minus sign welcome trither now it is a plus sign welcome cloud genius that changed was it was tracked the change is actually keeping track of line item level changes in the file that exists in the system so it knows what is changing at a very granular level inside a file in a particular line what are we changing at a line by line by line level detail about how is the state of that machine before and after what was different and how things are need to be matching with the desire that you have for yourself so back here we refresh we see cloud genius now we'll go deliberately modify just for the fun of it we'll go to that file called vi or rather sudo vi etsy no not etsy var www html index html that file I want to modify instead of welcome cloud genius we'll say hello and save it as a consequence of the save you will find the browser suddenly says hello and that is not what we desire right our desire was very different what was it it was welcome it was to say welcome cloud genius so when I see chef runs again it will fix it for us and that is expected it's trivial so you already understand it I'm going to now move on to a slightly more complex example you see that this change is showing up here the minus sign shows that hello cloud genius will be removed and it will be corrected back to the way it was supposed to be in the design in your desire welcome cloud genius that's what you have in your repository in the folder in git repo that's what you have welcome cloud genius so it will be doing that it will be maintaining that desire no matter what happens to the actual machine it will fix it in the context of this discussion like it will not fix other things it will just focus and only on these things called out in here these actions apache action install apache action start enable file resource it will focus on these items and make sure that these things are exactly what you tell them to be and it will maintain that for you so it is not a general purpose tool is a focus tool to implement your desire in the context of these recipes that you write with that I will now expand our scope and make it slightly more interesting you know in the context so now here comes a question the question is what if I design and change my desire to something different and not apache so today I am using apache server I don't want to use apache anymore so what do I do I want to use something else like nginx for example what do you need what do you think needs to happen so in our go ahead yeah let's destroy that and get a new new chef which has a different yes yes yes yes exactly that's exactly what you think first of all you it's a great thing that you thought about is to destroy the old machine bingo that's a good way to think about now the second thing you said is is you know this apache thing that we have apache to not use that anymore but instead get nginx so this nginx is what again it's a cookbook right this is also a cookbook and these cookbooks are available in the supermarket so we have to delete the soup this cookbook pretty much like like here we'll go and delete the apache to cookbook like that and move to trash and it's gone okay now this guy has no idea what apache means right this apache is unknown variable it has no idea what apache is so what it will do is give me a second phone calls okay so if you now run this this tool as is it will not understand what apache is so we have to tell what is apache or if you just change this apache to nginx like i'll say nginx and save this guy has no idea what nginx is either so we're to communicate with this tool as to what is nginx it has no no idea so we're to first of all go to the supermarket like there supermarket and in that supermarket we'll look for nginx cookbook and download it put it in the right in the right location and then we can proceed using it so i am going to now search for a nginx cookbook like that and see what comes up so i see that there is this nginx cookbook available and i like it i like it because there are lots and lots of people that are contributing to it i see that there are 727 followers i see the source code i see that the source code is actually built by a bunch of people not just few guys so if i dismiss this there are how many contributors 127 contributors that sounds like a good product with lots and lots of people actually contributing to that product so this is a good cookbook apparently because there are people who are building and maintaining it that's one way to look at so now this cookbook is available can i download it yeah absolutely we should download it how do i download it we just go look at the download button right there somewhere up on top and scroll a little bit download cookbook right there and then we will download it and save it into our folder that we would like to have which is the downloads folder i think it will go straight to excuse me and so that folder we now have in our in our computer somewhere so we'll go to that in the explorer window we'll go find out where it is in the download section there is this this folder called tar gz it's a tar ball i want to expand it and open the archive and extract it here like that so the archive is expanded into the cookbook itself this is the cookbook called nginx i would like to use it now and so what i'll do is i'll drag it from here and go to my home location in the home location i will find that i have this using chef solo folder so i'll go in there and then i go to the cookbooks folder i go in there and then i drop it right there i think i i went a couple step deeper for no reason so i should go to the cookbooks folder and then drop it right next to the main cookbook now i have it in the right location what i will now see is in the atom editor i will see that this nginx cookbook has showed up in the right location alongside the main cookbook so now i have again two cookbooks the cookbook i have are the main cookbook which i already know about i'm editing it right there here and i have the nginx cookbook to support this same idea install nginx and start enable nginx so i know that now already however this file location is not the right location for nginx doesn't store it in that folder it stores in user and again this you can find out through documentation and experience user share nginx html that's where i think it stores so we're going to save it in that location and then we should be ready to run but before that you should be aware of the fact that you said you start a new blank new machine and then run it and that's going to be working out okay but our exercise we are running right in our workstation right which means this machine i cannot just destroy and get a new one in a in a moment's notice i had to wait like five minutes for the machine to get destroyed and import a new one so it takes a small time small amount of time but in the cloud you can very easily quickly do it to get rid of this machine you know i don't want to get rid of it so what i'm going to do is instead work around the problem the problem workaround would be something similar to this sudo apt remove apache 2 i'm removing and uninstalling that thing because currently currently apache is occupying port number 80 i don't want it to take on that port because i want to make room for the nginx server to take over and so there's one more step here sudo apt to get auto remove to remove all the dependencies and all the things that i don't need anymore i'm getting rid of those things also so i'm basically completely getting rid of apache as as is it's going away and it's gone so now there is nothing that is taking on my port number 80 currently and there is no nginx either having done this part what i would like to now do is i'm ready to run my run chef solo command and as a consequence it will run the main main cookbook default recipe in which you will see that this segment i would like to bring it on later right later on so i'm going to cut it for right now leave only these three sections just for a demonstration so you can do all these things one shot not a problem but i'm just keeping that aside for now once once i run it first time i'll bring this one back in back in business and run it with that file resource also in place i'll do that but for right now i'm cutting this segment out saving it back here in the terminal and i'll say same thing sh and let it run and you will see exactly what you would expect to see is it will then find that the machine doesn't have nginx so it will install nginx it will find that the machine doesn't have nginx running and enabled so it will enable and run nginx and it did and so now we should see what do we see now here in the browser we go to the local host and we refresh and what do we see we see the nginx default page as opposed to the apache page we were dealing with before we'll go see where exactly this page sits the page is currently i think sits in this location called user share nginx and html and that is the file that shows you that file that i had in mind so that's the file we want to modify this is the same file which you see thank you for nginx that file shows up in the browser right here this is the same file i'm going to modify this file again through automation i'll go back here and put that piece of code that i had before and i would like to double check the file location is correct and it is so i'd like to save it now go back go here and go to my using chef solo folder like that and run chef again and you expect to see your custom file placed in the right location and nginx serving it you will see that happening just now you will see that it is updating the content in that file and this minus minus minus all these minus signs are an indicator of the old content getting deleted and then this plus sign indicates welcome cloud genius html segment inserted and permission structure changed to 0755 and ownership changed to www data and you have the new page showing up right in place of this page we should see expected and that's what we did but now we'll make it further complex a little bit more interesting in in fact let's see how we can extend the same discussion topic to a different exercise different exercise idea so we just we just did a couple of things what i want to do this is take this to the cloud same exact idea just run it in the cloud so we will get rid of this machine get rid of the we keep the folder that we have here we can be we'll be able to run it right from here in apply this idea apply this desire to a cloud machine very very simply just pick your cloud of choice pick your machine of choice make sure that you can connect to that machine through ssh and then off it goes so that's the goal we'll be we'll be having so let's go see the website i think we'll have some written exercise steps written down already in that website location somewhere in here so we ran this exercise we go to the next section see what we have written down i think there are a bunch of steps i have the engine x written down already for you let's see the expected outcome next section yeah i think what i want to do now is the the extended extension of this concept of using chef in a little bit more depth because we just started very simply so i want to understand all the capabilities that this tool provides you saw that if this tool has this thing called cookbook you saw this the idea of a dna json which basically focuses on the idea of run list this run list idea you saw that there are this new folder created called nodes which has a folder called cgjson which has a file called cgjson and this has some details about something this is a file that got automatically created as a ran chef it has some local mode cache files bunch of caches stored locally so we like to you know make sure that we understand each of these details as we go along by the way there are there are more such details in a chef repository that i want to talk about and so this model of usage is what i want to discuss and describe but i want to make sure that i don't bore you with the stuff so the best way to read about these things is right here on this website where you here you will find that this documentation website has a variety of different things described for you so i would suggest that you you should read it their ways of doing things are slightly different than my ways of doing things i do it the the way that i like and of course this is open source so people are free to do whatever they like and you are also free to use the way the tool that works for you according to what you think is right as long as it works it's okay yeah question so uh the chef solo whatever we are using this is only uh customized for a web service or it customized for anything we can do it anything absolutely because because except that a cookbook whatever the folder cookbook is there where where we are placing the recipes but other other folders like uh json all those nodes those are related to not related to uh our web server they are related to your chef installation chef install okay yeah yeah they are related to chef installation this cgjson you saw is a representation of the target machine where this chef ran let's go read that so here is your cgjson let's see what it contains so this these files contain some information of a kernel uh which is a linux kernel what is it which machine they're talking about the cloud genius workstation the name is cg the temporary directory is tmp at virtualization is vbox virtual box the kernel is linux it is gnu this this machine version that you see release version let's go see where this came from this came from the fact that uh you go you name dash a and it is the same name called cg and the version number they picked up from here through a query when chef ran so it is keeping track of what it is running on and that is how it is able to decide what operating system it is so you go scrolling it further down you have a ton of details about the operating system itself how many network adapters what basically framework so we we installed the framework on top of the framework we are installing the software like a website yes yes yes indeed this is a local file and this particular file that you're looking at is directly mapping to so the the nodes folder in there we have cg dot json this maps to the target that we ran chef on this target is the machine where we run it on like you know we ran this chef directly on the cloud genius workstation and therefore it was able to generate this json information based on where it is running it is running right here locally and this model of usage is not the ordinary common one mostly people will use chef to run run chef on a cloud some cloud we are we are running it running chef right here on our own machine that's not how most people use chef although you can use it like this you know there's nothing preventing you from running chef right here on your laptop like we did but dominant usage is not like that dominant usage is that you run chef on a target machine in the cloud you will have machines m1 machines m2 like that hundreds and thousands of machines any number it will run chef on all these machines for you by itself and maintain the state of the cloud for you that's the usage model that is more interesting we are getting to that soon running it on a machine like your laptop it's okay i mean nothing wrong with using it but it's just one machine you can do it by hand what's the problem on a cloud with thousands of machines that's a problem to do it by hand so you just use the tooling for that and that's where the usage will come now when it comes to using chef tooling chef by the way it comes into two aspects to rather actually i would say two ways of using it the one we just used was the solo version or solo usage model the second type of usage we will get to it pretty soon is called the client server model now the in the client server model i should i should be very very particular in calling out that this is not just client server model but it is chef client chef server model there is going to be a chef server and yes the chef client is what we will use with the chef server and you will have a cloud you will have a chef server somewhere and you will be here with your laptop and you will run let me be specific you will run the knife command here on the laptop and this knife command what it does is is it remotely execute chef here in the cloud so when you run knife on your laptop it will run chef on hundreds and thousands of machines in the cloud and then these machines will interact with the chef server and collect whatever is necessary to make the desire state that's how most chef tooling will work most of it in our example we just ran we were like not even using the chef server we are using it solo and we are you know just running chef directly on the laptop and we are implementing some desire like right here locally and that's not a cloud way of using like this is there is local implementation of something that you wanted and it will do it and we just chose the two web server examples but that's not really the idea you can do anything you like so the core scenario of usage is not this that was just an easy start now the other usage here is the solo usage as I mentioned which doesn't require a chef server now we did not use any that doesn't mean that this is any less powerful no it doesn't mean anything like that it is not less powerful it actually is you know in a if you think about it it's more capable that it doesn't even require a chef server it is able to run by itself is how I would think but practically speaking the binary that runs your chef solo and the binary that runs your chef client here all these machines hundreds and thousands of them that binary is the client binary that runs on your chef target nodes in the cloud this binary and this binary are pretty much similar in the nature they're not exactly identical they're different but from an implementation and execution perspective they operate pretty much the same way except in case of chef solo there is no need to connect with the chef server there is no such thing needed that's the only difference in the two now what happens is this company that makes the chef server software and the chef product and lots of other people from the open source contribute to it the way for this company to make money is from the chef server licensing so that's where the most of the money comes from so that's why you will see that the company website talks about this product quite a bit and they kind of sort of don't mention this at all you will see that in here if you see chefs all the pages at the top level you try to search for solo and there is some solo docs some pages because it's there but it's not highlighted there is like couple of lines one two or three three pages probably dedicated to chef solo but not much and rest of it is focused on how the company is interested in making money and so that's what they're doing and nothing wrong with that approach but I'm all I'm saying is the two things are pretty much equally capable the usage model of chef solo is independent standalone the user model of chef client chef server is a combination so that's the that we will begin we'll use both I have no you know I'm not one versus the other I've used both you choose whatever you are you think is right for your case so with that understood what I would like to now do is go into details about what does that chef capabilities that we talk about that are available in this documentation you will see lots of lots of references their concepts of these things like what is a node what is a server what is secrets what are cookbooks and a bunch of other details I want to understand these things right so there are a couple of ways of handling this one thing is to read it yourself which is you know kind of boring to say the least if you just give somebody to read the docs people find it boring is what I have found so what I found is that I will illustrate these concepts to you in form of small exercises that you will relate to much easier than go reading these documentation which I find personally boring to read I must admit you know give me a piece of documentation to read what do I do with it now that's the problem so that's why I have in this segment that is coming up is this chef solo segment where I actually go through item by item concept by concept in form of small exercises so there are these things that we will go look at you know all these characteristics and then do them in small exercises so now let us begin with first concept which is a kitchen like what is a kitchen a kitchen is what you're looking at right now this is the kitchen the repository where you work where you keep your stuff all these things inside a folder that you keep in a git control so that it is nicely version controlled that's what essentially is the kitchen so now I'm going to actually close this kitchen out and say you go remove the project so I'm not dealing with that project anymore I'm going to move on with another exercise so this project that I have called using chef solo I am moving on with another one so I'll create another one practically and so here what I'm saying is I will create a new folder called my cloud and then go into that folder and in there I will prepare a new kitchen now first time I gave you this exercise called using chef solo I prepared the kitchen for you and I just gave you ready made okay hey take this but that's not going to be in real life you have to begin from scratch so you begin here you begin with this like empty folder and then go inside so here we go mkdir my cloud and then go into that folder and then open atom editor right there so now atom is open and it has nothing expected now in there what I want to do is put a very disciplined way of dealing with what am I going to deal with so I'm going to use this idea that I've been using before in in a platform exercise the platform as a service exercise this idea that we I think a little bit used was the idea of a gem file the same idea we will apply to chef and so what I will like to now do is grab these five lines and put that in a file called gem file and this is our way to put discipline around whatever we are doing and I'll tell you what it means in just a minute so this idea of discipline is essentially making sure that we enumerate exactly which version numbers of various products we will be using and so I'm going to call that out very black and white simply like this and I'm going to save it in a file called capital G gem file that is the list of things that I would be using and so I have enumerated that I'm specifically saying that I will be using this knife solo gem zero six zero version the Berkshelf gem four three five version the chef gem twelve ten z twenty twenty four version the rack gem one six four version and these versions should be downloaded from this location the Ruby gems website as you know this is just a notepad right it's a just a document file correct every file in Linux is just a notepad document everything in Linux is a plain text file there is this Windows type thing and Apple type thing those complexities are gone out there so you just simply think of it as plain text and there's no extension to it it's a gem file right and so that's this gem file now we will use this file as a regulation or a tool to regulate our behavior this is the discipline we are putting on our own project in the context of my cloud project that I'm starting to build so I'm putting this gem file in place and as soon as I create the file I save it I close that window out because the moment you leave this window by accident I have seen through prior practice people in the program just leave these files open and they accidentally hit space or some other thing or some fat finger by mistake and they save it and they forget that they made a modification and then things don't work for the exercises they're doing and so it is very very important for us to be diligent in terms of what we are writing what we are saving where we are saving it and when we are done saving that you press ctrl W to close it that is the most important thing to do because then you you cannot touch that file by mistake the file is there but then you close it keep it closed now back to here we are going to use that gem file in our location to install the gems that we have called out that file contains these things I would like to have these one two three four gems installed gem as you know is a ruby package a packaged ruby application that I want to use so I'm going to go to that location and have these four gems installed in my computer for use with the my cloud project that's what I will be doing and so what I want to now do is use the bundler software to bundle up these gems from the ruby gems website and make it available here so for that if you remember from the last time it was bundle install and I will just bundle install it will look at the file currently and what the four things that we need will be downloaded and installed and will report that it has done it and so we'll be happy about that so it will soon report that it is installed for dependencies for gem file dependencies installed and total of 76 gems installed where this 76 come from they are dependencies of the four things we called out we wanted these four it obtained 76 everything is now ready and now if you look at our folder you will see that it contains a new file called gem file dot lock this file is a dependency lock you will see that file showed up here and it contains a graph a directed graph of all the dependencies of other things that are required to make our project complete and successful there are 76 different gems needed to make our four gems work the 76 are listed out here in form of a cross dependency map and it's also a text file plain text files called gem file dot lock this is something that you could also keep in your version control so that you can actually enforce discipline now we have done this step I would like to now illustrate a subtle point for us to understand how do we actually enforce a common shared disciplined approach to working with a bunch of team members in our company if you now look at this particular folder called my cloud and say chef client can you please tell me your version number and it tells us the version number is 12 12 13 now I look at my gem pile and I say the version number of chef was what again 12 10 24 so I had this idea crazy idea of making sure that everybody in my team will be using 12 10 24 version of chef but for some reason my machine is giving me this version 12 12 13 what is going on what is wrong with me what is wrong with my machine nothing is nothing is wrong but just look at my gems if I if I list up all the gems and find out if I have a chef gem and how many of them are there you will see that I have oh my god versions of gems installed I have this 12 12 13 installed I have that 12 11 18 installed I have that 12 10 24 installed so three versions of chef the one that actually kicked in was 12 12 13 which is the latest one right there and the one I wanted to use was slightly older which is this called out in the gem file so what do I need to do to make sure that everybody in my team actually uses the one commonly agreed upon gem version 12 10 24 this is what we agreed and this is the item that I have put down in my list and so although I have these three versions of chef that I don't need I need only this version but I have these for other projects that I'm working on in my computer and so I have multiple projects I'm working with and I have multiple versions of chef installed but for this project I would like to have only 12 10 24 so what do I do and how do I deal with this complexity with other team members having the same problem some of others are working on even more versions of chef or more versions of something else how do I ensure that everybody when they are dealing with this particular project called my cloud they will all be using this version there are a couple of difficult ways one of them is to ask everybody else to uninstall these things which you don't need which is not a good idea what if they need it for other projects and then the easy way out the easy way out is to actually force discipline and that forcing is a function of behavior and I'll show you how you can very easily enforce it I'll demonstrate that so back here you saw that I run chef client and ask for its version it tells me 12 12 13 it can conveniently ignores the gem file dependencies that I have listed out conveniently ignores it and I don't like that so I want to make sure that my computer will listen to me and follow the discipline provided to it through the gem file so I have to run the same command under the control of bundle execution same exact command chef client version same command but running in the control of bundle look at my gem file and execute it according to the versions called out in the gem file and then run this whatever it may be just run this in the context of the gem execution so look at my gem file and run it so I run it and I get exactly what I want the key thing was to remember that I can get everybody on the team to use the exact version that I have specified in my gem file which is 12 10 24 provided everybody on the team agrees that we prefix our execution with bundle exec by doing that we are very easily able to enforce discipline across multiple computers in multiple people working on different projects different versions but all operating with a known set of variables like version numbers and these are critical you know sometimes things will work with one version and not with the other and sometimes you need the other version for other projects and not this version and so there will be complexity everywhere and you want to reduce that for the project that you're dealing with and you want to put everybody on the same page and make sure that everybody on the team operates collectively in a collaborative fashion not I am using this and you are using that or it works on my computer what's wrong with you kind of things happened you know it may so happen that you know you might find that your people in your team it runs for your computer in your laptop because you apparently have a version that works and they don't so or they might but they may not be using the enforced discipline method of operation so this is the key to remember bundle exec having said let's proceed so back here I have a gem file that I call out and then I bundle and then I see a bunch of gems installed gems are basically packages of Ruby language you use the bundler software to install it then you sharpen your knife this is yeah this is yeah just ignore it the sharpening part you can ignore but knife is a tool and so knife is a tool that we actually wanted we wanted to install the knife solo tool and what this knife tool does is basically gives you this idea that I mentioned before is you run this knife tool on your laptop and it will run chef for you in the cloud so it's a remote execution method so you run something like knife solo and you you pass some parameters and you do this on your laptop and it will go magically run chef solo on the cloud if you run knife on the cloud it will go run chef client in the cloud if you run knife solo it will run chef solo if you run knife if you learn chef client and the chef client will then interface with the chef server and do its thing so it knows what to do with and we'll go through those exercises as well we'll do those things but for right now this knife tool is kind of interesting is what we need to actually go in and sharpen our knife so sharpen part is kind of a bad kind of a joke so you can ignore that but focus on the four things that it does for us this knife solo plugin that we just added through our gem file it gives you four different things so knife solo in it knife solo prepare knife solo cook knife solo bootstrap knife solo clean five things I don't actually use the fifth thing much the knife solo clean step because what I rather do is I destroy the computer and move on with another one I don't bother about cleaning up I just shoot the cow and move on the cowboy approach so we focus primarily on these steps initialization preparation cooking and bootstrap let's go understand what these things are so in our in our concept of using the knife tool we talked about four different things initialize step repair step cook step bootstrap step and these commands apply to the solo knife so you will basically say knife solo in it knife solo prepare knife solo cook like that but what do they do so when you initialize it prepares a kitchen it creates a new kitchen for you when you say knife solo prepare it it will install chef in the cloud this is the install action when you say knife cook action it is the run action when chef client or chef solo will run this is the solo action so knife solo cook will actually run chef in the cloud knife in it knife solo in it will initialize a kitchen in your laptop when you say knife solo prepare it will install chef right here in every machine you have in the cloud all the machines big big large collections it will do the whole thing now what is this bootstrap thing this is basically prepare plus cook it's a shortcut for these two things so first you prepare and then you cook that's called bootstrap so prepare and cook together in one shot shortcut one step bootstrap it will install and run chef that's what bootstrap is install and run chef so these are the four critical things that we need to know and use in using knife solo there is also the clean step that i just ignore i don't bother clean up we are just move on do another machine so we'll begin using it we'll begin with initialization and when you initialize it will actually create a kitchen for us and there is this other thing called stock your bookshelf so we'll see what it means so coming up soon what i want to do next is go to the next page and stock the kitchen so what i'm doing right now here is bundle exec knife solo in it dot this dot is indicator of the fact that i want to like to have i would like to have my kitchen created right here bundle exec knife solo initialize my kitchen right here in this location in the my cloud folder so now right now i have two files the gem file file and the gem file dot lock file those two files are there in the my cloud folder you can see them in the atom editor what i want to do next is initialize a kitchen and remember i'm running in the bundle execute control which will force every single execution to follow the discipline provided by the gem file which is basically these gem versions so just stick to those versions please and run run what run knife solo in a dot and when i run it you will see a bunch of files and folders created inside the my cloud folder like that like that there we go so a bunch of things created and that's nice now i will list them out you see them that now i have a cookbooks folder data bags folder environments folder nodes folder roles folder site cookbooks folder and a bunch of files these files in this file they were already created a little bit before the other files but other files they just they just showed up because i ran this command knife solo in it in the current location the dot location so now again all these files are basically flat files and folders and most of them are empty like containing nothing because we just initialized right so you will see that most of these files and folders are kind of empty you'll see them like right now and there is this special folder here called dot chef that folder contains some secret information i would like you to be aware of so we will see what that thing is and you will use it very very carefully so be paying careful attention to this particular folder that you can see in the atom editor also and i see in my command line because i'm using the a option in listing so you see that the dot chef folder as a hidden folder shows up and now we will see what these things are one by one so beginning with the dot chef folder the most critical folder in a kitchen that one it has a very important file called the knife rb let's go see what that file is this file is kind of sort of similar to your solo rb the first exercise solo rb it defines what is stored where is defining you right now that my cookbooks are stored in a folder called cookbooks and i have other folders also where i store some of my cookbooks and they are stored in a folder called site cookbooks i'm going to store my nodes details the nodes are the machines where chef will run so whenever chef runs i need to know where it runs so i need to store that detail about the machines i want to run chef on in a folder so i will know and that is stored in a folder called nodes so this cookbooks folder currently is empty there is an empty file in there it is also empty there is a nodes folder in there that folder is also empty and that folder contains or rather will contain a list of nodes that i will deal with this cookbook folder will contain the books i need later on and that's what the function of these cookbook folder and the nodes folder is going to store my nodes and cookbooks folder will store my cookbooks that's how i'm telling my chef client or other chef solo binary that i am going to store my cookbooks in this location and in this location which is another empty folder right there empty and it is something that i would like chef to know so i'm just telling chef that i will store my nodes in the node paths and my roles in the roles path which is the roles folder but what is a role and how is it different from a node and what are these things called environments and data banks and nodes and you know all this is so confusing so i like to simplify this for you very very simply and i'll go one at a time so i'll not just dump it on you but we'll that's what we are really doing to understand what kitchen contains and how do we make use of these things so this file contains the details about where things are environments stored in the environments folder data bags are stored in the data bags folder and there is a hidden rather commented item here called encrypted data bag secret we'll talk about that later there is this thing called knife bookshelf path which is another tool we'll be using to grab our cookbooks you can you know you will know that you will use a bunch of cookbooks in the program as we go along and so you don't want to be manually going to the supermarket and downloading books from there and putting it in that folder which is kind of cumbersome so i don't want to do this by hand i would rather do it through automation so that's why we use this tool called bookshelf which is a play on the word bookshelf which is which is if you know what a mean is is uh bookshelf is the name of the tool and so if you if you use the name of a new product and call it bookshelf and try to search it you know you will probably not find something of relevance to you so you want to create a product and you want to name it very very distinctly different like bookshelf for example and it is so hard to make that name pop up on the top of your search results when you go to google and try to search for bookshelf it is not going to pop up so the creators of the product deliberately chose a bad spelling and by doing that see what happens the very first link that pops up is the correct link for the thing and that is bookshelf this product is now a part of chef they have purchased a company basically acquired the team from the other team that created bookshelf used to be a separate product now it is included with chef the person who was rebuilding this thing early on is now a part of the chef team and they just acquired it basically but the key thing to remember is that they deliberately chose a broken spelling and this idea comes from the ArmaGuard this you may have seen this thing before like this idea ArmaGuard and so this idea is a deliberate misspelling of a word called oh my god and so that's where this phrase of bookshelf comes from is that if you know what a meme is you will understand what I'm talking about I'm not too much into this so I don't really I'm not able to explain to you what this thing is but this is what it is bookshelf ArmaGuard oh my god that's where it comes from well enough of ArmaGuard so closing it and back to here bookshelf every time I say that I just remember that ArmaGuard oh my god and so that's where the tool will come and put the books in the bookshelf sorry I cannot resist and so that's what this knife rb file is right there now let's go into the website here and see what next steps that we have for us so we initialize our kitchen like that and then we will examine the content which we kind of did we'll further go and understand what these things are each and every item of that and then we will start looking at a specific thing that we want to do which is very important is the function of this folder called site cookbooks and you see that in the knife rb there are these two things called out here I will store my books in two different locations like here is one the cookbooks location and then there is another one called site cookbooks now the question I have is why do we need to have two different locations for cookbooks why can't we just put all the books in one location and just get rid of this thing called site cookbooks why do we even need that like that and that's totally fine it works it is totally okay however you shouldn't and here is why instead of writing the way they wrote I suggest writing it this way and save and I'll tell you what I did and why I did that the reason why we have these two different folders called cookbooks and site cookbooks is to distinguish and differentiate between the work you do versus the work that other people have done for you for example here is the NGINX cookbook now did you write the cookbook no did I write the cookbook no who wrote it these guys these guys like this person and this person this person is the Mike the man I think he started writing that book his name is Mike Fiedler Mike the man and he began writing this NGINX cookbook and he is generous and he's giving that to open source and he has given it to all of us everybody can use this book we just used it like few minutes ago and so there are people like him and others who have collectively contributed that piece of work called the NGINX cookbook and as you know there are tons and tons of books available like thousands of them so we want to use these books and there are 3650 of them for right as of right now so I want to use these books but I want to keep them separate from whatever I will create for myself and so what I want to do is put the cookbooks I obtain in this location in the cookbooks folder right there and the ones that I create for myself as I am could basically begin writing a fresh cookbook I don't want to keep them in the commonly distributed folder called the cookbooks folder instead I want to keep them separate in this location in that folder right there and thereby I am distinguishing these two folders because one of them is my creation and the other one is not and by doing that what I'm doing is reducing the amount of trouble that other people on the team need to worry about because what I can now do is just like I can get NGINX cookbook they can also get NGINX cookbook so when I'm sharing my project my kitchen my repository with my team members I don't need to give them the entire NGINX cookbook or the entire Apache cookbook or the entire MySQL cookbook or another bunch of cookbooks that are downloaded from the community I don't need to pass them around because they are big so I can just tell them oh I'm using NGINX version number that or I'm using Apache version number this and just focus on that number and they can also download I can also download they are publicly available and so we just use those references in a separate location in the cookbooks folder and whatever we create in our team you create I create our creation in the team we will keep that in these site cookbooks folder and thereby I want to give that one priority over and above the other creations I want to make sure that my creations take priority take precedence over the other creations so that is the reason why I moved this item before the cookbooks location before it was not here I think it was kept in this location so I just put it oops yeah go back go back like that yeah that's what I would like to have inside cookbooks first and then cookbooks by doing it that way I'm giving precedence to my creation and closing that back here so we gave precedence to this because the default cookbook path and give precedence to the site cookbooks over that because we will keep our custom cookbooks that you create in that site cookbooks location now we will use the tool like we had in mind before to ask that tool the Berkshelf tool to go to the supermarket and get me a version of NGINX cookbook for me so what I would like to do is instead of going to that website called supermarket.chef.io I would rather go here in this kitchen and in this file called the Berkshelf Berks file I will ask my Berkshelf tool to go to the supermarket and get the NGINX cookbook for me like this book that version from the supermarket would you please go to the supermarket and get that version for me please that's exactly what I'm asking I'm asking my Berkshelf so when I close this file I am ready to ask my Berkshelf tool to go to the supermarket and get me the book now the key thing to remember is when you run the tool the book will as you would expect should appear in this cookbooks folder magically should appear right there but it would not it will come there eventually when you need it but right now it is not going to magically show up as soon as you run the tool it is there is a reason why it doesn't do it that way and we'll see what it does we will run the Berkshelf tool right here in our command line and you will see that the book NGINX version 276 the one that I wanted what I wanted is written down in this Berkshelf file just like gem file like Berkshelf file like gem file what it does it gives you a list of list of gems that you need from the Ruby gems website I can get a particular version of this gem a particular version of the chef gem similarly I can ask my Berkshelf tool to look at the Berkshelf file and go to the supermarket to grab the version 276 for the NGINX cookbook for me very very similar concept here back to here so in in case of a gem file as compared to the Berkshelf file what you're looking at is these tools operate conceptually the same way this guy is going to get us gems this guy is going to get us books Berks and books cookbooks basically and so they go to the Ruby gems website this guy goes to the supermarket this guy will bring down a version of gem that you require in the gem file so you have say chef version 10 10 whatever number you call out yeah that's the version you will get and get installed in your machine you specify that I would like to get NGINX cookbook version 276 it will go to the supermarket download this particular version of NGINX cookbook and make it available to you when you need it pretty similar concept this one deals with gems this one deals with Berks I mean cookbooks so that's what you are going to run now we are going to basically run this command called Berks install but we'll not run it alone in isolation we'll run it like this bundle exec Berks install what this command will do is look at the Berks file that we have and do what it says what does it say get me NGINX 276 so that's what you will see boom run it is going to go to the supermarket and get me the version 276 of the book that I need right there and it's there it's available to me but is it available in my cookbooks folder not yet so if I now examine the cookbooks folder it contains nothing that's expected because where it actually stores these books is not in your kitchen but in a different folder in a hidden folder called dot Berks shelf there you will find that it has a folder called cookbooks and there you will find a bunch of books and this location is a central location central to your computer where it will keep track of every book every version it ever downloaded like package cloud zero three zero package cloud zero one zero zero NGINX 276 or my sequel five six three yum five zero one yum five one zero like that every version that you ever downloaded using Berks help in the computer is stored in this location dot Berks shelf cookbooks that is to optimize the download of various cookbooks that you might need across a various set of projects that you might have each one with a different Berks file you might require a vertical version of a book in one project and another version of a book in another project and so you don't want to like bundle and package up various versions of cookbooks in multiple projects and bloat your computer hard drive there's no point in doing that so you want to centralize the storage of these books in a central location that is what Berks shelf is doing for you and we will come out of there we go to our my cloud folder and you will you will find that in them in the my cloud folder you will find that this folder contains nothing in the cookbook folder quite yet expected because it is storing in that location now when you need it it will automatically show up as you as you need it not right now but you installed it so it's available that's what we did we use the Berks help tool to install Berks install and it did exactly what you thought it would do and it did now there are some examples written here that are not necessary you can run it doesn't hurt for example Berks vendor is the command you would run when you actually need those books right here right now when you need the books right here and in your laptop right now and you want make that available in your computer you should be able to get that I'm pretty much I think the call I got was a emergency call so I will need to take that so standby I'll finish that quickly we have a give me second I will I will answer that call a mic . Sorry about that I'm back what I would like to now do is this Berks vendor what we have is a situation going on in which people need to come and fix we have a big leak in our home water flooding that has happened and that's the reason why these people are coming in to help out and so the insurance company and the people who are trying to come to help fix that's what's going on right now in the in the other area of my home so that's why I have to you know handle that call otherwise they will not come in so it's just a short call letting them come in because they just want permission to come so they will come and do their thing but they'll do it so let's focus on this Berks vendor so when you want to go in our the books you want to get this locally in your computer you want to run this command called bundle bundle exec and then Berks vendor again this command is not strictly necessary to run it's more of an illustration if you want to receive the books right in your kitchen not in the cookbooks folder but in your kitchen you will run it like this and what this will do is bring down the books and store them in a folder the folder is called Berks cookbook and they just are locally available to you you can see what files came down right there so all the all the books that you wanted are now locally available in a separate folder called Berks cookbook in case you want to vendor them rendering is basically a method of receiving something from the vendor a vendor is somebody who sells or gives so that's what we did we rendered these books in our folder we have them here but we are not going to use it like that so I'm going to remove that folder so RM Berks cookbooks getting rid of that and they're gone now we just have these things so you will see that this Berks file that we had also has generated a lock file just like this gem file created a gem file lock when we ran bundle but Berks install created this Berks file lock file let's go see what that lock file contains and you will notice that this Berks file contains only one entry and that is the cookbook version 276 of nginx and this book requires other books to be as a dependency so that dependency map is structured in this Berks file lock file that looks like this it requires other books blue pill and rsyslog and apt and blue pill and build essentials and ohai and bunch of other books that are required for nginx to properly correctly function in all possible scenarios and so the additional dependency dependent cookbooks are also called out and they get installed in your machine when you run Berks install or you render the books like we did vendor like right there now what is missing in this exercise is I should have bundle exec prefix which I don't see and so I need to fix that in the exercise it needs to be corrected if I don't have the bundle exec step something different than what is expected will happen so you should have the words bundle exec as a prefix to any command that you want to use in the context of running ruby and chef is a ruby product so bundle exec berks vendor is a necessary prefix that was missing in the exercise so I just added that and I'm going to save that and that change will now update live so it is going to pop up soon and then I view the lesson and then I see that it is already up to date so now we have the the berks vendor with a prefix already included and that that step was mostly a way to describe that you can get Berkshell to download a thing that you don't need don't directly need but you can examine you can bring the books directly into your kitchen if you like this step is not required and we can proceed further so next what do we have we have stocked our kitchen now we want to define at least one machine that we will target our chef to run on let's go back to the picture what I want to do is I have my my laptop in there I have my kitchen and in my kitchen I have created a bunch of things but I have don't have any nodes these nodes are the machines are references to the machine in the cloud where I have this machine m1 m2 and like that I can a have any number of them where chef will run and chef runs there here and here and here when I run knife here when I run knife on my laptop in the kitchen chef runs here in the cloud it doesn't run any random location it needs to run exactly on those known machines m1 m2 this machine that machine and all these machines are nodes you need to have an entry for every machine in form of a JSON file in that nodes folder so that node folder we are talking about right here this one currently empty we need to fill it up with a JSON file the JSON file will describe exactly which node are we talking about otherwise it doesn't know where to go run this chef right it needs to know where to go where to connect how to connect and do what the do what part is described in other portions in the run list and the cookbooks but where should I run chef that piece is the node definition and that is what we need to provide in our kitchen so we will do that we'll go back here and define a node a node definition will include a run list pretty much like this so we'll create a file and we'll put some things to do like you know when you you're basically providing the DNA sample you're saying when you grow up the embryo the just enough OS you want it to grow up to become this and here you can say whatever that this is you can say you grow up to become a tiger or whatever or database server Davis DB server you can define your run list right there and that is what your row your function in life is going to be when you grow up when chef runs you will magically become a database server so you define that in your run list currently I have no idea what I want to do with my machine so I'll just leave it empty but then this item itself needs to be saved in the nodes folder and there I need to save and give it a name to that file the name has to be very very specific as to what name am I talking about it needs to be a very specific fully qualified DNS name ideally ideally something like this if I'm dealing with this cloud machine here this M1 I need to have that file that I'm creating in the nodes folder the file should be named m1.cloudgenie.us that's what I would like to have so that I can just have chef connect to this file I mean this name this DNS name and do whatever it says like it might have some run list for it and so I need to specify this file in form of a fully qualified DNS name dot JSON that's how I would like to structure it now this may not be feasible every single time so you can alternatively use the IP address that is also acceptable just create the file with a name that has the fully qualified name or you can use the IP address both methods are valid what does that translate to we need to have a machine say for example web1.cloudgenie.us so this machine that I'm saying m1 I'm going to now name it web1 and that machine doesn't exist right now so we will you know in very very simple terms go to some cloud and you pick your cloud name right now you pick me a cloud I'll go there and we'll create a machine there I'll just go to the easiest one right there it's my favorite because it's so quick and just log in and get a machine that's how quick it is so here we go log in and then put in my verification code and off I go to create a new machine like that and I'm going to make sure that I have my keys in good shape and so I check my security key pair and I think I have my workstation key in the good place right there that's good and now I'll create a droplet a new machine a new machine right here running Ubuntu and I get a $5 machine and I will put that in San Francisco and I will select the workstation key pair and I'll give it a name the name I would like to use is web1.cloudgenie.us now this name has no nothing to do with DNS like nothing I can call it some machine I like yeah I can basically call it whatever I like it doesn't even matter this is irrelevant so and then I create don't confuse the name I just put with the DNS name that I want the name I want here is actually a DNS name like that now that can be possible provided you have a DNS entry for it then it becomes a fully qualified DNS name we will get that very soon provided we get an IP address from this machine and then give it to our DNS control panel this IP address so we'll go to our cloud player DNS like in my example I am on cloud player so I go login and there I will go to the domain cloudgenie.us go to the DNS then find out if I have a web1 already and I don't so I create a new entry called web1 and then paste the IP address and add the record now I have managed to create a fully qualified domain name for the machine I just created the machine name is different from the fully qualified domain name machine name I just typed in some random so now with that understood what I would like to do is go back to my command line and save this run list as web1.cloudgenie.us.json I'm going to save that in the node folder and save so now my nodes file contains a file node folder contains a file the file is called web1.cloudgenie.us.json and the file contains this and this is basically empty run list is to do nothing like when I have a run list that indicates nothing it will gladly do nothing for us that's exactly what I want so I'm beginning with nothing but the file name is very clear web1.cloudgenie.us.json so what this will do for us is when we go and look at our terminal and find that can we connect to our sorry cancel cancel can we connect to our web1 machine on a SSH prompt like that we need to provide our login name the login name in case of digital ocean is root that is important and then we provide our fully qualified domain name and remember the file name was.json this is a DNS FQDN fully qualified domain name it doesn't need the JSON here so I am entering this it will let me in when I say yes so I'm connected so now I'm connected to this whatever this name machine name is so some blah blah blah whatever that is but that is the host name it is not the FQDN see the host name is whatever this thing is you can change it it is irrelevant from our perspective that machine host name has nothing to do with what we are doing that's why I don't confuse that host name or whatever you choose with the digital ocean control panel or any cloud control panel whatever name you choose here you can change it to some monkey it doesn't matter it's absolutely irrelevant a droplet is renamed but who cares I don't even bother so once I am done with this login test which is actually not necessary you don't even have to test this login whether you can connect to SSH or not you don't have to worry about this verification I did because I'm just demonstrating but I can come out of that all I really want is this file called web1.cloudgini.us.json that file is a unique method for Chef to find out which machine needs to be connected to and then do its thing what is its thing that thing is called out in the file itself the thing to do is the run list the run list is currently empty so it will connect to the machine and do nothing it will connect to the machine it will it will prepare it will cook and when you say prepare it means install Chef it will install Chef on that machine when you say cook it will run Chef on that machine and then when it runs it takes a look at the run list and decides that the run list is empty and then it decides to do nothing and then report success that's what's going to go happen right now when I run Chef when I run Chef it will do its thing it will go remotely install Chef on the target machine in DigitalOcean then it will run Chef and do nothing and come back and say I do nothing fine that's exactly what I want so let's go do its nothing thing let's go figure out what it does or in fact you know if nothing is quite boring let's ask it to do something here so we'll go in in the JSON description in the run list we'll say you know what why don't you do something for example let's see run the recipe called nginx like that so do something I think we are typing mistake here we need quotes we need double quotes so you can see that this atom editor is smart it tells you by highlighting that there is an issue so now there's the issue is gone it was expecting quotes around it and so recipe nginx colon colon default is what I just added as you can see here recipe nginx you can write it like that or you can write it like this the default recipe in the nginx cookbook is what I would like you to be doing when you grow up when the just enough operating system that I now have in this machine is just enough operating system I would like that machine to grow up to become a web server and that's what I'm trying to accomplish so I'm saying you run the default recipe in the nginx cookbook and my cookbook folder is empty and that's okay that's okay the okay is because you know this this this tool will know that there is a need for the cookbook and so it will automatically place the cookbook in the right spot and then invoke and do its thing the step that you will see happen are first establish a connection to the machine second install chef on the machine third install the nginx run list that we now have expanded initially it was nothing we expanded it to include something that something is the default recipe in nginx cookbook so we included that as our extended expanded run list so as a consequence of the run you should see nginx getting installed and functional that's what I'm expecting so now I will run chef how do I run chef I don't run chef I run knife locally right here and as a consequence of me running life on my laptop you will see that chef gets installed on the remote computer and then runs over there so now let's go see what happens next I am I think I already did this create and connect to a node in the cloud so I already did that and yeah we should go to the next step okay so what I want to do now is to understand how do you invoke this knife thing like how do you do invoke this knife command so there we go you have to grab it and grab this this this whole syntax and discuss what it means to us so we go here in the editor and see what it says right there so there are there are two ways of running this thing one way is when you have a fully qualified domain name when you have a fqdn ready for example in the setup here when I have a fully qualified domain name ready I am going to bundle exit run knife solo and I want to prepare my machine which machine I can say this machine just go to that machine web one dot cloud genie dot us and prepare it using the knife command and when I say prepare action this basically installs chef you can you know if you don't have an fqdn method you can use the IP address method also if you're going through the IP address method the thing the other method was like right here when you don't have a fqdn in place and you want to run it with just the IP address that is also possible but you have to add an extra flag extra flag is line number eight and so we have to run bundle exec knife solo prepare root at then whatever IP address you have and as in an extra parameter and this parameter will tell chef to pick up the right JSON file like that one and that's what it will know to associate this IP address with the name that you have in mind and it will associate the two together for my example since I have a fully qualified domain name already ready with me I want to run it like this bundle exec knife solo and then I have the name already so I don't need to have these things in my command line I just go straight like this like that bundle exec knife solo prepare and this prepare action will you will see chef getting installed on the remote computer it will connect to that machine web1.cloudgene.us like that and then you will see it is actually installing chef let's go run it right here in the command line clear run and you will see item by item every single step as we go along and I'll run it right now invoked so it's going to bootstrap chef what's going to happen right now establish a connection install chef and then come back report success that's what's happening on the remote machine out there in the cloud we are currently installing chef that's what it reads and let's go see what it's doing it is downloading the version of chef we wanted 12 10 24 the same version that we wanted not arbitrarily anything but whatever we specified and it then installs it and then it sets it up and says thank you for installing chef and finished that's what we wanted the prepare step basically means install chef the next logical step is to actually run it to run chef on a remote machine we always use the knife command locally and we want to now cook the machine over there in the cloud cooking basically is the chef way or the cooking way or the kitchen way or whatever the food way the chef may have telling to cook is to just run chef that's what you will see when I run chef or other I should say that's what you will see when I run knife solo cook under the control of bundle execution this bundle exec is super important it enforces discipline in the project across your team so you run it like this and it will go connect to the remote machine finds out what version of chef it is there already and then okay here are my cookbooks please take it is sending the cookbooks right now to the remote machine it is uploading the kitchen the whole kitchen is going there you will see the whole kitchen showing up in here is getting uploaded you will now see the cookbook showing up in your folder the books that you wanted the engine X cookbook with the metadata showing version that you wanted 276 you should see that version somewhere here maybe not here but there should be a motion number in the readme file that the version number is written down I think 275 is mentioned but 276 should be the version of the book what is correct don't worry about that it is going to be somewhere defined in the in the file itself now that that cookbook version and all these other dependent cookbooks are locally available in your cookbook folder your site cookbook is still empty because you did not create any cookbook yet and this command line shows you that something happened let's go see what happened let's go see so I'm going to go all the way to the top and see what happens it is sending books sending the kitchen uploading the kitchen and then it's installing the cookbook gems compiling cookbooks over there and then it starts to figure out what needs to happen it is going to understand what needs to happen so it is first of all begins collecting data about the machine itself like which operating system are you and for that it runs this oh hi gem this oh hi cookbook this oh hi is a cookbook which is a dependent cookbook that will go check the remote computer as to what do you what are you are you are you debian linux are you ubuntu linux are you redhat linux are you windows or what and then figure out and then do appropriate actions it needs to find out where it is running so it goes does that resource definition it dynamically defined a resource and that detail is available to chef next one you will find further it needs to create a oh hi plugin for nginx it does that then we have I think the nginx plugin getting installed and for furthermore we will see the actual action of installing our nginx cookbook will happen which is somewhere here so nginx package is getting enabled nginx action common directory is getting created you have a new site definition enabled sites available sites enabled and then the interactive mode as you will see in this is actually enabling and the correct set of sites and I think at the end we should have a 404 page showing up because I what I saw is it disabled the default site so let's go see whether we should be happy to see our 404 page error page should show up at the end of this run it should have nginx correctly functional but nothing to show and therefore you will see that the site here is actually going to show you web one dot cloud genie dot us should show you a 404 page let's go see it and it does it shows you 404 not found page because it doesn't know what to show you but it is given to you by nginx it is showing you that nginx server got installed and nginx server is up and running it doesn't know what to show you it cannot find what you expect and therefore it is a 404 this is probably the only time when we are super excited to see a 404 page because it succeeded in showing us whatever we wanted we said you know just get the nginx role in our in our sorry nginx hold on the nginx what did we say here was the run list definition this get us the default recipe for nginx installed so it did exactly that you can see that verification by going to the ssh connection to that computer like that and here on that remote computer you will find it has nginx running right there there is a worker process and a master process running nginx on that target machine and it is functional it does doesn't know what to show you so it doesn't show you anything it shows you a 404 which is expected now further go further ahead is I think we yeah go ahead so you ran the command the knife command exclusively you specified the fqdn correct right so even though you have provided the fqdn do we need to have the node already specified in the node folder yes and also can we do like a in the node you might have keep like a three nodes instead of one one fqdn add a three nodes and take out this in the knife command don't give any default so it's going to install on the three servers yes there are there are various ways to do this but yes you're thinking in the right direction okay okay if you want to run it on a multiple number of machines you can handle it absolutely like that there are various ways to handle that piece in terms of how you want to do it on a multiple servers not just one we are beginning with just one machine okay and here in the node definition we need to have that file very very clearly called out because it contains our recipe in the run list section that's where you communicate as to what you want that machine to become when it grows up right yeah other ways to handle you can pass this run list idea directly in the command line like that directly that like right here you can say run list like that you can pass on run list right here in the command as you invoke in that case you don't need to have this node file okay various ways to do that but we'll just begin build build complexity step by step okay okay so now let's go further we we ran this step we tested it installed nginx for us it showed us a nice and happy 404 and now what are we doing next 404 not found that is expected and then we go further next session idempotence okay so yeah basically the same idea we ran before we can go here and we can we can destroy our nginx like like that 3298 3394 and kill so it killed nginx is gone sorry nginx is gone now if you go to the window and open up web one you find unable to connect because nginx is dead now you come out of this machine and run chef one more time how do you run it you say cook cook command like right there and you cook it again it will go to the machine fix whatever is needed and tell you that it did simple same process again and then it will identify what is missing what is different between the actual and the desired state and bridge the two match the two bring the actual state in line with the desire and so that desire and actual are equal to each other when that happens you have success and that convergence is what you will see happening right now so the chef is about to start it finds that nginx is not running so it runs it and that's what you see as a consequence you have this site up and running again with a 404 which is again expected so that's the idea of idempotence it's keeping things in good shape for you you can further expand the exercise very very simply and expand our run list according to whatever your desire is and you can make the run list more and more complex as you go along now we'll see the next idea define roles for your nodes so this is this next idea what i want to do now is you know expand a collection of machines together and put them in a common role so this role idea that we are talking about essentially is this folder that is in our kitchen there is this folder called roles which is currently empty and so what i want to do is make use of that roles folder before that i want to understand what the roles folder does for us so in the idea of a role the role is basically a way of classification a classification of various machines i have m1 m2 m3 three machines two of them are web servers and one of them is a database server so i can potentially classify these two together and put them in the web role so that i can just deal with these two together in one shot and say both of you take on this role called web role and you you take on this role called the database role so i'm defining two different roles here called the role called web and a role called database and i'm assigning this machine a role called database and these two machines a role called web and so when i do that i am classifying these three machines into two different types and therefore i can then simplify how i deal with a collection of machines in a in a in a classified in a in a manner which is easy to handle so say for example i have a big cloud and i have a bunch of machines that are my web front end web form i can put all these machines together in the web role and some other machines in another role and other machines in some other roles i can basically classify and divide that's what this role definition comes into play and i can create a role pretty much similar to what i did here in the nodes file i have a file and i define something in there in form of a json structure like that i will do pretty similar to that and i'll put a new file in the roles folder and i'll call it my role that's what this section is so this section is how i define my web role let's see what it contains so i'm going to put that in our atom editor in a new file paste it save it and call it in the roles folder call it my web role and then give it a name called web dot json and save it in the roles folder and save you will notice that this role web dot json file in the roles folder has a name called web it has a description the description is kind of irrelevant it's only for people chef doesn't care you can put whatever you like here it doesn't matter the type is a role the json class is chef colon colon role there are attributes that i'm using here to override some extra attributes i'm defining for engine x to listen on and i'm telling engine x to listen not only on port number 80 but also 443 and here i'm interfacing with a i'm including a run list directly i'm saying that in the same role definition i am going to include a default recipe of the nginx cookbook as a run list item i'm including the run list action right in the role file itself earlier this node file had the definition here of the run list what i did was i moved that and put that in line number 11 here and thereby i can classify and use a role as a way of describing a collection of machines together right that's what i'm doing and by doing that what i am going to change is this recipe i don't need to include by its name but i would rather have it refer to the role itself i'm basically assigning a role called web to the machine node called web one that role definition i'm changing what i'm going to do that is basically replacing this idea here by this line here so what it was before and now this so earlier it was directly referencing a recipe now it references a role and thereby it is able to invoke the role i just defined for it the role in turn has a run list which was there here before but now it has this and the role has a run list so the run list will still be there executed but the actual invocation will begin as we start through the run list here which has a role definition and that role will then invoke a recipe but before the recipe is invoked i'm extra defining some extra attributes and in there i'm saying that nginx i would like you to listen on this 443 port also so that's an expanded definition expanding the attributes so there are two things we are really talking about here we're not only talking about a role definition we're also talking about attributes override and i'm when i'm saying i'm saying override i'm not saying overwrite i know my accent can sometimes come into play i'm not saying this i'm saying override and what we are really trying to do is to override the default attribute behavior i'm trying to do this and override and by override i'm what i'm really doing is changing the attributes of listen ports not only 80 but also 443 and these overrides that are being included in the role definition called web json right in this file is actually going to override the nginx cookbook attribute default attributes listen ports there are some listen ports defined maybe not in this file but these ports are there we can search for it and find it out exactly where these ports will kick in basically in the nginx default collection of attributes we are overriding it with this one additional port not overwriting and i hope that is clear to you so i'm closing this file not saving and closing this role definition closing the new web one node definition which points to a role and then run chef client rather run chef solo so back in here we are going to basically invoke the command just like we did before back here in the command line we'll say okay go cook it again and it will do its thing this time the invocation procedure will be slightly different it will first take on the role called web and then in the web role it will have an override attribute and then it will have a run list the run list will then run the same recipe called the default recipe in nginx cookbook and then it will show us a 404 page by making sure that nginx still runs so we change our desire in a basically restructure how we wrote our invocation we first time went through directly to the node asking the node to do a default recipe in nginx cookbook but this time we modified we created a role that role had an invocation to the recipe and it did the same thing basically and you see that it practically did nothing because nothing actually changed we just restructured it so back here what do we have next next so we define a role we can assign multiple roles to a node for example this is not an exercise you're not going to do this particular step so this is not a part of the exercise this is just an illustration to tell to let you know how you can potentially do this when you have multiple roles and you want to assign it to a common node for example a case in which you might have multiple roles like web role and a database role and you want to run this machine will become a web role as well as a database role which is kind of you know not don't don't most people don't do it that way but you can run a web server on the same machine that is your database server that's not how most people run it but you could in practice in in using chef you can assign this web role and db role both to a machine like m3 for example and if you want that you could go do this way web role and database role both assigned to the run list for that m3 machine if that's the desire so you chef allows you to do whatever you want whether it is appropriate or not it's for you to decide so you can assign multiple roles like this to a machine like m3 again this is not what i would suggest so it's not a part of the exercise but possible you can do it similarly you can also mix and match roles and recipes so web role and a database recipe mixing and matching you're installing a PostgreSQL database default recipe and also at the same time assigning a web role and this is nearly an illustration don't do it that way it's not a good idea good idea because you're mixing two different things on the same box not a good idea do one thing do it right that's the principle so this idea itself is a good one but the illustrations that we have web and db together not a good idea so don't do it next one this is also another not a part of exercise but merely an illustration so if you now look at a given cookbook for example this one this is the apache cookbook opening up in github you will see that the apache cookbook has a bunch of different attributes this is the attributes file in the apache cookbook if you go to the apache cookbook the attribute section you will find that the attribute has a bunch of default attributes and here you will find all of these attributes for example you will see some of them that are interesting to us say for example the timeout value the apache default timeout value is 300 seconds per case of development purposes you may want to increase the timeout value to a larger amount of time as you debug and test something you could potentially do that in the context of modifying this particular timeout value if you are setting up this apache server for a development purpose and that situation will allow you to actually override the apache default timeout value from 300 to something larger say a 1500 whatever you choose it will set the timeout value appropriately for a test case or for a demonstration whatever you need it will handle so back in this example what we are looking at is you can potentially modify it a particular attribute say for example the default timeout value you could edit this number and override it to be something different from what the default values are the way to do that will be something like this so you keep it out this is a different timeout value but the idea is to assign an attribute right in your role or your node definition as an override attribute and that will basically override the default value for example you are going to expand listen ports to not just this and this but also one more and you could do that in a given situation if you are setting up something for debugging testing developing or for some subcustom deployment purposes it is totally fine to handle attribute overrides in the default collection by choosing to assign specific value that you want in an override fashion you can read more about these overrides in the documentation in the main Chef documentation about what attributes are and how do you make use of them that's what this thing is about and so let's go see I think I closed the page by accident so go back to our DevOps section where we are dealing with the understanding of Chef concepts and we will soon get to an exercise we have 15 minutes left let's see how much is the concepts left so let's get to that where is that thing I think we were doing attributes like that that's what we are focused on and so after understanding the attribute section briefly what I would like to do now is to tell you the idea where you can use a particular collection of machines for a different type of usage I think that was very cryptic in the way I described so I'll simplify that say this is your cloud and you have these machines you know bunch of machines out here and this segment is where your production is and so other segment you have here are the machines that are for development more machines your developers come in and play here this is your development area this segment you have some machines here there are your testing area and some other machines here are your staging area things like that you can define various areas or different actually label these machines as your as your machines that are dedicated for production usage as machine dedicated for testing machine dedicated for development and you can classify your collection of machines according to what your usage is and that can be accomplished by the idea of environment so in the context of chef we can define environments pretty pretty simply by creating a file for example here is a development environment definition so let me go see that development environment definition in a file here and you will see that it allows you to set default timeout attribute as an override if you are in the development environment you may want to have a slightly higher timeout value to test something out but if it is production environment like that you may want to keep the timeout value as low as say 30 seconds or something like that that's more appropriate to your production scenario and so that kind of a definition file you need to save in the environment folder right here so go back here you will find that there is this folder called environments and that's where you will store this file and you will store this detail which will automatically get flagged because every machine that you will have in your cloud will have a label designating the environment it is running in and so this machine is a staging environment this machine is in production environment so it will automatically pick up the appropriate flags for the appropriate attributes and select the correct selected attribute that you might have this is a timeout example but I'm just saying that you can select any attribute for any given environment for a given deployment scenario test stage dev and you can come up with new names so I have a new thing here called xyz environment and that's where I use it for some xyz purpose and I have these machines here and that's totally fine you can come up with any name you like staging dev test prod xyz but chef server comes with a default environment called underscore default and that is just a name that is assigned by default if you don't have a name chosen for environment that's the name that they choose and give it to every machine that's given to machines by default but for most of our purposes you will actually create separate areas where you will define separate environments for different scenarios of usage and you will use it that way that's what I would expect you to do in our exercises so now that was a brief discussion on your environment definition we'll further proceed ahead and see what do we have next we have very interesting things coming up right now and I think I should demonstrate that we still have time so we should be able to do that we are talking about this thing called data bag and data bag items now this idea is a very interesting one I have for a variety of reasons I'll tell you why a data bag is not data database so just just be very clear about it a data bag is a place where you store what would you store in here you will store data bag items it sounds silly but that's exactly how it is structured and you will store data bag item number two data bag item number three like that you will store these items in the bag but what is an item an item is basically a file think of it as a file so this is a file it's typically a JSON file and that in that file you will store stuff making up making up stuff it's not stuff it's not called stuff it is you know you can store things like usernames passwords secret that kind of stuff some detail that you want to protect and you want to keep it secret and you want to use it in the context of your applications and that detail which is things like username passwords access keys access credentials database connection strings or any SSH keys public keys think of that thing think of that nature you could potentially store those things in a JSON file and put that file as an item in the bag what do I do with it okay I put the JSON in a file and the file is in a folder called data bags practically that's a folder and this is a file it's a JSON file containing these things like that username passwords bunch of other random refraff secrets in general for an application you want to keep things secret you will store them in a data bag that's what the usage scenario of this thing is and that's what a data bag is by the way there's a folder for that in here called data bags right there and so in that data bags folder it's it's plural here data bags and so in that bags folder you will keep data bag items one two three any number of them you can name them whatever you like and there will be just some files and that's what you will see as a live example coming up now the the problem that we have in solo in knife solo is that it doesn't support data bag natively but that's not an issue it's not an issue is because it is open source so what has happened is people who began developing solo decided to not develop data bag capabilities for solo and so that's fine don't have it no worries other people came along in open source and said we will create this capability for use with chef solo and so now there is this capability available now we don't have it in our in our kitchen currently so we'll have to add that capability that capability as soon as we add the data bag capability into our kitchen our knife solo plugin will be capable of managing data bags because chef solo doesn't handle data bags by itself so we'll expand our data bag capability by expanding our gem file and so we have our gem file that looks like this we'll add one more line at the end that make it look like this as we grab it go to the kitchen and look at our gem file that looks like this and now i'm going to add one line what i currently have it looks like this and i'm going to add line number 13 to it like that added line number six save now what this thing is is knife solo data bag gem this gem adds the data bag handling capability to knife solo in my kitchen so i added new gem in the gem file and i saved it and close it and now since i added a new gem to the gem file i need to bundle so i will bundle and the bundling action will bring that particular missing gem and add that to my machine and now my bundle is updated and i'm ready to move forward as a consequence of this action what i will notice is that bundle exec now has the knife has a capability to handle solo and so with solo and the capability to handle data bag you will see that capability popping up right here you will see that bundle exec knife help will have extra capabilities for solo command to deal with data bag creation data bag edits data bag listing data bag show those capabilities came along because we added one more gem that adds the data bag capability to our knife solo setup and now what we see is that we can actually make use of these commands knife solo data bag create edit list and show now that we have these in our hand we'd like to make use of it so let's use it let's see how we can use it what i want to now do is create a bag and create an item in that bag how do i create that you can do it multiple ways and i'll show you the the chef way and i'll show you the manual way and both are valid let's go begin what i'm doing now is running this command and i'll help you understand what that command does for us and so before i run it i want to make sure that i put these windows on a side by side so it's easier to see so that window is side by side like right there and you go this way there we go so adjusting the font cleaning it up running this command and what you will see now is currently i don't have any items in the bags folder it's empty i'm going to run bundle exec knife solo data bag create create what create a bag called pass with an item called my sequel that's what i'm really doing i'm creating a place to store my my sequel login name of user password database password that's what i'm really creating i'm creating this data bag called pass with an item called my sequel so i create that and it sounds convoluted it's actually really simple well let's go create it and it gets created you see on the right side there is a folder called pass right there it is currently empty but there is this editor opened up this editor is allowing me to store my details about the login name i have for my sequel and the password i want to save for my sequel so i want to expand it and say that my password will be something really super secret so i will then paste that password right here like that and so this password is a super secret password and i'm going to save this editor that opened up so save and then i close this window close and then i close this g-edit window close and i see a new item showing up right there this item you know yes it was convoluted to create this file you could have created this file directly as a my sequel json in the pass folder in the data bags folder why this convoluted method create data bag item called my sequel in the bag called pass why there is a reason for that as to why does this command invoke an editor for us in our in our view why can i not just create a new file like that and paste whatever i want in there and save in the json file in the data bags folder in the bag called pass and call it boom yes you can it is technically okay technically it's okay as long as you have the json correct this one is not this one is correct as long as the json correct you can save a file in a plain text and save it in an item and that item becomes your data bag item and this item is sitting in a bag called pass and the bag itself is sitting in the bags folder so bags bag item password that's the sequence you can create these things manually no problem but then why this convolution this this complexity and then the editor pops up and all that why is that so the reason is that when you use the chef client chef server method the data bag itself is not stored locally it is stored in the chef server the chef server doesn't give you an access like your laptop gives you in atom you know in atom editor in laptop you can do whatever you like but that atom editor will not interface with your chef server just like that and therefore when you are dealing with chef server in a more complex exercise later on you will see the same exact situation where this command bundle exec knife without the solo there will be no solo in that time because right now it is solo but at that time will be just knife no solo data bag create and yes you will create bags and you will create the bag in the remote location in the chef server and you will store your mysql username passwords god knows whatever details you want to store in there for the secrecy of your database access credentials you will store them in a bag and you will create that bag called pass and you will do that in a remote location this is local to you so you can do it locally in a text editor that's fine but in a chef server you don't have access like that in an editor so that's the only method you will have at that time is to access using the editor that pops up which did in this case so now that was the case of your your use of using the command called knife solo data bag create pass and it pops up an editor and you you save the super secret password and all that good stuff now similarly you want to see what your password is and again you don't have this flexibility to access a file system in the chef server so you don't have this luxury of using atom when you are using chef in chef server so what you have in that case is this command line which will show you your password pretty much like this it will go to the chef server location bring your password and show it to you like that it says your password is super secret password you know that in your local system you have it stored like this and that's what is shown to you like this now you know this is json and you know this is not json but if you want to receive json you can just pass a flag json and get in json format so now you receive json pretty much what you have in there this was the show command now this is exactly what you will see when you're dealing with the chef server which you will do soon what i'm now going to show you is the edit command pretty similar to the show command is the idea of edit the item that you want to edit is the mysql item so you want to edit the password edit the data bear called pass and the item called mysql in the bag so we edit and the editor pops up again and here you can change this super secret password to not so secret and save and close and close and what happens to atom it also changes because this nice solo thing is actually dealing locally right now it is the solo mode of operation that was the edit process let's go further is that you were able to modify your secret password but this is not done yet we still have to make sure that the password that we are storing is encrypted encryption right now it is stored in a clear text format so anybody and everybody can access not so secret super secret but that's not what we want to leave it like that because this chef server is going to be remote from us and we don't even directly control it sometimes sometimes we do sometimes we don't if you are renting it from somebody else and they control it if you are making your own chef server it's open source product you then control it yourself but those cases of encryption are a good use case which we will cover the next time when we get together which is going to be on wednesday at wednesday is what date is 18th 18th of april at that time at four o'clock we'll get together again and with that i will stop recording now so we'll get to the encryption part next time and we'll go further into into more complex ideas and then deal with live exercises as we go along so i am now stopping recording