Hey guys, we are recording this, so please verify that you see the red blinking dot on your computer which shows you that the session is being recorded. We record this so that you get a file at the end of the video, end of the session so that you can rewind and repeat if you like. And that's the whole purpose behind this recording thing. So now you see me on two cameras, one desktop being projected and a couple of news items. One of them is Bellevue College has agreed to run another bootcamp so that this website, which is this one, has been updated with new dates. And so that means your dates that you are currently going through that date may not be visible to you, so don't get panicked. The dates are still valid, I just don't have it. The next one is scheduled. So the site has been updated, which is already indicated here, so your dates have disappeared from there. So it shows Monday, Wednesday, Friday, but that should not bother you, right? I hope it doesn't. Your sessions are still Tuesday, Thursday, Saturday as we are doing it, so there's no change for you as such. I'm just pointing it out that the next one is scheduled, site has been updated with new information. You don't get to, you know, don't have to worry about these changes because they don't do not impact you. New dates are available. These do not apply to you because these are future dates like April, May, June timeframe. Your dates that I have done, what I have done is I have pasted those dates for you and your dates are at the chat session. The dates are called out here. So today plus eight more days. They are already there, no change there as such, but just that the site has been updated. I just wanted you to know, otherwise you will get surprised what's happened to our dates. Nothing happened. It's still just like it was and no change. Okay. Having discussed that part, the idea, next idea would be to continue what we did the last time. We are doing a lot of new stuff, which has never been done before. So at least in the program. The reason for that is that we are incorporating new content, Docker is improving in its abilities to create new feature sets. So I'm learning them and I'm providing content to you according to what I find. So what we did the last two times, two exercises for Docker, where we created what I would like to describe that as, build a Docker file, build an image and then run that image. You did that. We also did the last time we created a compose file, which looked sort of like this. You remember we did this last time. We created a service called DB, another service called WordPress, and then we ran something. So that was what we did the last time. Today, we want to do something different, slightly more complex. So step-by-step increasing in complexity, we'll do one exercise, which is more about Docker files, a lot more Docker files today. So we begin with opening a terminal and cloning that repository already. There is an exercise already available on GitHub, you can go there by the way directly. I will paste it in Slack chat. Here you will have it. And this is the exercise I'm talking about. You see that coming in Slack chat. We'll bring that whole exercise file content locally down in here by saying git clone. And when I say git clone, the thing comes down and I think I did not type properly. So git clone and then paste, oh, did not paste properly. So let me get there again, poppy and paste. So it clones the whole thing. The folder has been named as more Docker files. That's the name. So I'm going in that folder, more Docker files, and then opening up atom editor to see what is there inside. We will see a slightly different way of putting a lot more content than we did in the past. In the past, we had a Docker file. Now we had some file called, you know, in this file and that file, we are adding content to a Docker file to make it at least give it a semblance of some changes being made to an image that we have to begin with. So what I'm going to do now is at least put in a proper application and incorporate that in this Docker file. So we have an index HTML. We also have some CSS styling here and also index HTML. So these three files I want to add into the image that gets built. So I'll begin with the nginx image in the Docker file and then put my name as a maintainer and copy these three files, one, two, and three, and place them in the appropriate location inside that container image where nginx expects it to be. And by doing that, what we are doing essentially is making this nginx container, which is provided by the community, by the way. So Docker hub nginx is a community provided image right here. And so in that, this is the nginx team that, come on, nginx. So this nginx official image is already available to us. We can just use it. And so this is provided by the nginx people, which are these people. You can see them on Wikipedia. So these people create that nginx product in open source. And they have their website that looks like this. And the product source code is available in, if you go and browse it, this is the source code for nginx. What they have done is created a readymade image for us to use in this location. They call it the nginx image. It is available for us to use in line number one here. So we are basically using that image and putting our content in there. So what we'll do now is using that collection of files we have, which is these files. Okay, I'm not typing properly. These files we have index, reset, style, three files. We want to put them inside that image that we will create our own. And in our own image, what we will do is use this Dockerfile to construct that image. And there is a readme instructions provided here with it, so that basically you build an image for your own like this. So we'll say, you know what, use the Dockerfile stored in this folder, I mean, this folder as in dot, in that location, dot location, meaning this location, and build an image and tag it Dockerfile, more Dockerfile. You can tag whatever you like. Tag is not important at this moment. The important aspect here that we are trying to accomplish is to build a new image. Currently we have, we will base ourselves as the foundation will be this nginx image that we have, this one. That image is what we will use. And on top, we will build our own image and tag it as this name. So that is a very simple process. You will see it happening. If nginx image is locally not available, it will be pulled down. It will come down from this location, this official location. And then once you have that image locally available, the other steps will be followed, which is what Dockerfile says, which is, you know what, assign a maintainer and copy three files. That's it. So when we do that, you will see that happen very quickly. It happens very quickly because I have the image already available and it's a question of basically copying this file, copying this file and copying this file. So three files copied over into the image and a resulting image is built. That image is this number 288. Now you will see that Docker images shows me that image. It's the 288 image with the name more Dockerfile. That's the name I chose because I chose a tag here in my command. So that's how you tag it. But essentially if you don't tag, it will not get a name. So you remove that command, this part you can remove and just say docker build dot. It will still build the same image except will not have a tag. But since we already have a tag, there's no change as such. And so you see that it already had that tag, so it is not going to remove that tag. Now having seen that thing, I want to remove that image just to prove that the tagging doesn't, tagging is not relevant. So I'm going to say docker rmi, sorry, rmi 288. So that thing goes away and now Docker images shows you that we don't have that image anymore. So now we will say docker build dot, docker build without the tag and you will see that this thing will build and it builds, it copies all those three files again. You will see that it uses the image already available to us, assigns a maintainer, copies one, two, three files inside a new image and assigns it a new identifier that the ID. You will see that this one does not have a tag. See that tag is none, the same ID, the tag is none. The name is also none. It was created 21 seconds ago. We can now assign it a tag pretty much like this and tag it. So if we tag it, it gets a name. Now you can see the name there, Docker images shows you a name, more Docker file. Now that we have an image with a tag and a name and an ID, we can use it. How do you use it? You run it. How do you run like this? If you want to expose the machine port 8080 and associate it with the internal container port 80, you run it exactly like this. If you want to change it to say, maybe I want to show 80 on the machine map to the container port 80 inside, I will go like this. I want to do something different. I can like 81, for example, or something else, or maybe 8080 like I had before. So it's up to us as to what we want to do. I want to choose this style now. And by doing that, what I will do is run that thing. So simply I will take that command construct, paste it here and understand that it is going to be run interactively and assign the ports. The ports will be assigned from the machine 80 to the container 8080 colon 80 and run this image more Docker file. When we run it, it should run and it is running. And you can see that here, open the browser, go to port number 80. You should see a new nginx container, which shows our files. They look better than what you have seen before. You have seen http colon slash plus some simple examples. This is a slightly better written example to show you somewhat realistic real life scenario. So localhost 80 basically means localhost. The same thing. So I'll say, show me what you got. And so it shows you there's a proper logo, links and Slack chat and pictures. And if you click on the link to prove it works, it's a full-fledged site you're beginning to build. It's still not comprehensive, but at least somewhat different. That shows you something useful than just showing an index page. So it has some styling and all that. That was just another example of how you begin to build something in more realistic fashion about what do you have in mind in building something. Like you want to copy over your applications inside, put that in an image, construct an image, and then use that image in the context of some cloud. We are running this one locally here, but we can run it in the cloud. And we will do that. That will be one of the two exercises from now will be that exercise. So we'll push it to the cloud. For right now, let us see what do we have here. So like we ran this at port 80 and we saw it running. You can see that this browser is trying to get. So the browser is hitting, trying to put a get request. It's a Firefox browser. It is trying to get this file reset.css. It is also trying to get this file style.css. Those files are getting pulled from the server and shown in the browser as you open it up, as you open localhost. That action of localhost enter actually pulls that content. So you can see more logs showing up right there. As you will have people visiting your site, your logs will generate, they'll show up here. That's the scenario of how do you see what is going on under the hood in the container if you want to see that. Suppose, for example, I break that container, so control C, and I want to run that container this time in form of a daemon. When I run it like a daemon, it will run just like that except will give us the control back. So we have the control back in our command line. It is still running. You can see it running and it shows you and all that. However, you are not able to see the logs. So you want to see the logs. You can say docker ps to see the process running. You get the container ID there. You get the container ID is charlie148. So we'll say docker logs charlie148. Show me the logs. You can see the logs like that. So pretty much any container, you can go and examine the logs. If you want to tail the log, like keep seeing the logs, looking at it as new actions happen, you can say docker logs and then dash f to keep looking at the logs as people will visit. So if you want to interactively like tailing, this is called tailing a log. You can keep watching its tail as to how it moves and you can keep hitting and you can see that new log entries pop up as people hit. As I hit the browser, I refresh it again and you see the new log entries pop up. You can break that. So that was a simple example of examining logs if you like. Now what you want to do is switch over to another exercise. So I'm not saving it, just closing it out completely. The exercise is available to you in somewhere in here. I may paste a link for you. So I'll write that up in detail as time progresses. But now let us do a little bit more involved example with a collection of Docker files right there. So I will close this window, close this window also, break this one, come out of it and close this browser. Then clone that new repository that I have in mind for the example exercise here. So git clone and paste. This is a different example, slightly more involved and let us see that. So it's coming down. So it comes. Its name is Docker files, plural. So I'll go inside there, Docker files. Now we'll see what it has in there. And it opens. Now what have we got? What do you got here? A lot more things. There are these examples that actually I would like to walk you through. And so what I will do now is make this full screen. So you can see a whole lot more detail and adjust this window a little bit like that. And here you will see that there is this thing called Docker compose YAML, which is basically blank and it is telling us something that copy YAML snippets from the example files paste into this file and then compose your stack by running this command. Docker compose up with a dash D daemon option. So it runs in the background. That's what we want to do is copy example snippets from these example files that we have three of them. One is this, one is another and another. So lots, lots more. So we'll begin with the main example. Is it too big? I think we'll begin with the first example. I think this is where we should begin. So there are three different segments to this file. So let's go understand them. So I'll copy the whole thing as is and put this in the empty file here like that. And let's understand. In the first segment, we have line somewhere eight, nine, it's a very, very simple thing. It says there is going to be a new container that builds a folder called data and the folder called data is here. It has a Docker file in there. So we'll go into that later on. Next one, we have a DB container that uses, does not build, but uses a ready-made image provided to us by the MySQL people. So here, if you go and look at this library and find out MySQL, the people behind MySQL, which is Oracle company, by the way, and a bunch of people from Oracle and open source developers produce this thing called MySQL image ready to use. And it is open source. There are certain licenses that, certain license conditions that people do not like about this MySQL application, specifically the Oracle components of that license. So if you can go to MySQL license, you can read more about it. I'm not talking about legalities, but people dislike somewhat, people do not like this much too much if they're purest in open source domain. So what people have done is created this thing called MariaDB. And it is totally open source. There is nothing to worry about what is this and how is that licensing or legalities do not come into play. And it is binary compatible with MySQL. So you can just use the MariaDB in place of MySQL. And it works exactly like that. There is no other code change that you need to do except that you have to use a different image called Maria. And you use this image. It is basically identical, but license is different. So I should not say identical from a source code perspective, but functionally, it is a community developed fork of MySQL. So this is intended to remain GNU GPL. So it's public license. So that's the reason why many people love it and they can still get the benefit of MySQL and remain open source, true to it, as opposed to some miracle conditions on top. But that's legality. I'm not talking legal here. So don't take my advice as legal advice. Having said... Yes. Is this validation from Microsoft SQL? Yeah. Microsoft SQL is not open source at all. True. So it's not about the function or something. A Microsoft SQL is a different animal. It's a different product. Let's think of it this way. So if you do not know these details, let me point out. So there are traditionally two types of databases, I should say. The classic SQL style and the other one is called NoSQL style. This No doesn't mean negative. Does not mean negative. So don't take No as negative. No means not only. So not only, that's what it means, this No. So there are two types of databases, SQL, NoSQL. Your Microsoft SQL, MySQL, Maria, PostgresDB and a bunch of others, OracleDB, you name it. They all belong in this segment called the SQL segment. There are others. By the way, among themselves, they are different. Their style is similar in that there are relational databases. And the other guys are not only SQL. So they're not negative. They're not NoSQL, but not only SQL. They're called NoSQL, but it is not negative. This PG is a unique animal. It also belongs here because it supports the SQL style as well as the other style. It is one of the oldest, most stable, and most reliable solution available in the open source domain. It's better than these guys, Oracle and Microsoft and others. If you think about usage, it is extremely powerful and useful across the board. Many large deployments are actually using Postgres SQL. It is available and a foundation of many of cloud services available from many companies. In the NoSQL side, you have another classic example is MongoDB, Cassandra, and your Google tables, Google database. There is a Google database. I'm forgetting its name. Amazon has DynamoDB, Azure has DocumentDB, so these guys are all the NoSQL style. By the way, in here, you will also find Hadoop is another NoSQL storage. And all of these databases are also functionally different. They're not identical. They are conceptually similar in the way they operate. This is relational versus not only SQL style, but also other ways. We'll talk more about this if you're interested in later. But that's the big picture distinction between the two types of databases. Back here, Microsoft SQL Server is like MySQL, but different. It's in the same family, but different animal. We are going to use this image, MySQL. You can practically replace and use the word, delete that and say Maria. That will work just fine. That's the only change you need to basically make. And now you're using MariaDB because it's actually functionally identical to MySQL. And that's what I just wanted to point out. So you will stick to MySQL. It doesn't hurt us. So in MySQL, we are choosing a specific version tag and that image will be used. Whereas in here, we don't have an image reference because we want to build something like Docker build. So you build what? Build this. What is that? It's a folder which has a Docker file. Now in here, WP, we are saying, you know, use the image called WordPress. Who's giving you that image? These guys. You go here, search WordPress. Who's providing that image? WP image provided by the WordPress people like these guys, the people who develop WordPress software. They provide that image. They keep it up to date. It was published a day ago. So it is probably fresh. And the question I have is why did they, what did they change a day ago? Meaning why don't I have it yet? So I'll investigate that later as to what they changed. But essentially we are using that specific image. We are also making a port assignment like this port mapping to that internal port. So this is basically machine container. That's the machine first, container next. That's the mapping relationship. Whatever you have in the machine level, port number 80 is mapped to the container level port number 80. That's the mapping. We are providing a link connection. This link connection 27 basically is a reference to this DB name, which is here. And the way to read this line 28 is like this. In English language, I would read this line 28 as, you know, refer to the DB container I have in line number 11, but call it MySQL. That's how I would read this line 28 in English. So I'm referring to the DB, DB basically means refer to line number 11, but call it MySQL. What does that translate to? I can use the name instead of DB. I can decide to choose some other name, like for example, instead of this DB and this DB, I will use the word and that is okay. That's what I'm saying. I'm using the word ASD, ASD, ASD as the name, but I'm actually going to call it MySQL in line number 28. And that is just a figurative way of saying something different for the thing that I want to actually call it MySQL inside the WordPress context, but outside its name is DB. And so that's how you read this line 28, which is basically a conceptual link between this container WP and the other container 11, line 11, DB. That's the link you want to establish a link. It's conceptually like this. You have a stack layer where your application is running and another layer where your database is running. You want to associate that connection. You want to make a connection on this port 3306 so that this application can actually access the MySQL database running here. You want to be able to enable that. It needs to know what that IP address is internally for this location where the DB is running and that linkage needs to be provided to this container where the WordPress application is running and that link establishment is what happens here, 27, 28. You also want to expose and override the layered file system in some fashion. In specifically, you will read line number 30 like this. Line number 30 is somewhat complex to read, but if you read carefully, it has two parts. This is the one part followed by colon, followed by the remainder. And you read line 30 like this. You're talking to the WP container and which is line 23. That's where it begins. And you tell that container that, you know what, hey, container, you don't want to use your internal layered file system for that folder, but use the folder that I have highlighted and it is on the real machine, not in the container. So on the left side, you have something in the real machine. On the right side, you have something inside the container. You're overriding that. Always in the context of these files, especially Docker files and Docker related things, the colon is the demarcator, like this colon here or this colon here. And on the left side is the actual machine. On the right side is the container. And same thing here, the colon. On the left side, you have a folder. On the right side, you have another folder. The folder on the left is a real folder in a real machine, folder on the right is inside the container. And you're basically saying, you know what, skip the one inside, use the one I have outside for the real folder. I will provide you that folder, by the way, here. And I want you, container, to please not use that location, instead use my location. I'm giving you a real machine, real folder, like that. And same here, real machine, real folder, like that. Now that folder doesn't exist yet. And we see that there is this folder called home user Docker files, it exists. But this WP content doesn't exist yet. It will be created. You will see it create right here on the left side when it runs. You will also see that this folder called etc mysqlconvd is actually available. The etc mysqlconvd contains a configuration, the custom configuration, which we are providing so that it knows to allow connection from any location. You will remember in our Chef example, we had to expose, we had to go do this, by the way, if you remember, we did this something like, where is that? We did this. And in that exercise, what we did was a separate database and app server, for which we had to modify privileges. And we allowed access from instead of localhost, we said percent, create username at percent and username at percent. This percent basically says allow connections from any other box, even if that user is not on this box. So basically it modifies and lets connections from any other box. So you know, like for example, by default, if you create a container like this, let me draw a different picture. So if you create a mysqlcontainer like this, by default, the rules are, if you have a user, the user has to be on that box, and then it can connect the database. And that is okay. But in our case, what we want to be able to do is have another container where I will run my application, it has a different IP address and this IP address is different. They are not the same. So I want to allow a user sitting here to connect to this database. And I want to allow that. So I'm saying that my IP address will be different from your IP address. So I'm basically saying allow it from any IP address that I like. So that is the configuration I want to do a modification in the my.cnf file for mysqlserver. And this allow from any other IP address is what I'm really doing here. Changing my bind address to star 0000 basically means anywhere, any IPv4 address. That's a custom modification I did to my mysql configuration. And this is the reason why it will allow a remote container to connect to this database, even though the user is sitting on a different location. That override of configuration you're passing through a reference to a file that you provide in a real machine like this in line 16 on the left side is the real example, which is this file here. And you're mapping it to the actual place where the configuration is supposed to be sitting inside the container. The configuration for mysql sits in that location. You're telling that mysql container to stop using your internal thing, use my thing from the outside. The outside thing is on the left side. The inside thing is a reference to what you have inside the container, which is supposed to be overridden by what you have on the left. Same example here. This is the place where WordPress will store its files, user generated content, plugins, whatever others, others people will upload, whatever people will upload that will go and sit in that folder WP content. You want to not use the container folder, but use a real folder on a real machine. So you're giving it a location where you want the machine to actually store content. So this is the location which currently doesn't exist. And that's okay. There is a question from Shiv. I can see your, I can read your mind sitting right here, I guess. Yeah, you do that because these are all internally internal to the container environment where the whole containers are inside a private area, so you don't bother. You cannot even go there, it's on separate subnets. So you will have independence. So when you say anything anywhere on the internet, it basically means anywhere on the subnet and it's all self contained. So you don't actually, random people cannot even visit that. It's blocked out in a VPC in practice, in most properly constructed clouds, like, you know, Amazon, Azure, Oracle, private cloud you have. So they will not be able to like a random visitors come here, visit your database. Not possible. So that's where this opens up. If you want to really be, you know, clinical, you can actually put that IP address where you will expect to see that WordPress container pop up. So you have to get the IP address of this container and then put that in that location. It's not a work for no benefit. And you can, but the point is we want to run it inside containers. So why worry? Why bother? So one more question I had was, I don't see the data mapping out there. Line number, please. Line number, please. I mean, be linked containers, but we don't have the actual data map. We don't define the data maps that done internally. Did not understand you. Please repeat your question. Sure. So even though your audio is not clean, I'm not able to hear your question. Okay. Let me try again. Yeah. So the actual data mapping is what I heard you say. The data will flow because of database calls. You will have this application will actually make a database call with a database connection string. Something like this. Let me show you an example. So example of a server connection string from MySQL would look something like this. Yeah. An example is coming up. So it looks something like these are all windows examples, apparently. So essentially what you're doing is you provide a server address, your database name, user ID, password, and that's the connector string for.net application example. You will have for every different type of application, you will provide basically these things, these couple of things you need for any kind of an application to connect to any kind of database. You need a bunch of things. First one, for the app to connect to some database, you need to know what type of database. So here if it is Postgres SQL, you need to have a PG client, PG client software. This PG client for Postgres server is here. You need to have that PG client running. You need to know the IP address of this location. You need to know the username. You need to know the password of the user. You need to know the database name. And all these things will allow this client to connect to the Postgres server. And this is what this application will make an API call using the PG client to connect to this database server. So these are the parameters you need to pass, user, password, database name, and IP address. This IP address belongs to this database. And this is a separate box, a separate container. That's just one example. We'll change it now. Another new application and a new database. This database can be MySQL, for example. And that will mean we have an IP address for this. We have to have a MySQL client library. This client will be the application that will actually connect to the MySQL server here. You will have a database name. You will have a username, or rather I should say DB user, DB password, and what else we are missing? IP address. Yeah. So one, two, three, and four things are needed so that this client can actually connect to the server. And this application will use this client to interact with the database after a connection is established. So this application needs to be provided with this detail. And we do that when we connect. So you will see in an example that we are about to run that I will be providing these values specifically one, two, three, and four. These values I will be providing that to this application. Now let us talk about this one. Where do I provide that number one? It is that link reference in the Docker Compose file here, which is 2728. That's where we are providing a link. So it is implied and that is where you provide the IP address. That's the item number one taken care of. Now we will talk about DB name. This DB name is provided already in the Docker Compose file. And that name is line 21, right? Next user, DB user. That name is provided in line number 19. Next DB password. That detail is provided in line number 20. By the way, 18 is not used. It is provided but not used. The user for MySQL and password for MySQL is used. This is the MySQL root password, which is for administration purposes. The application is not going to use that. It is for administration purposes. So these are the details that you need to provide. So one, two, three, and four, like you see here, are already there in the Docker Compose file. The link is the IP address. The user is the user. User is the password. DB is the name. This is how you pass the details so that the connection string establishes and the application can actually connect to the database here. And you can change this MySQL server and this MySQL client, make it something like Mongo, for example. Let's call it MongoDB. MongoDB server and Mongo client. And you will have pretty similar scenario again repeated. And your application will now be using the Mongo client, which will in turn connect to the Mongo database server, which happens to be a NoSQL server, different. And since it is different, it doesn't traditionally use the well-known practice of what a classic relational database is supposed to be. It is structured differently. This is not your, not your, how do I say that, in a politically correct fashion. I would say, I would say it is not your, not your father's database. That's how people say, right? Sorry? Yeah. It deals with unstructured data, so it's not your father's database. Like you know, in your relational databases, the discipline is very well structured, which is, let's say, it's very structured, like you have a name. So you have a first name, last name, and then your street address and your city, state, zip. It's well-defined, right? That's how a contract record will be. So you will have, you know, put Shiva and Ganesan and your street address and your Ellicott, whatever, wherever you live in Ellicott city, in the state of Baltimore, what do you say, BT? Is it BT or? MD. MD. Oh, you live in Maryland or Baltimore? No, no, no. Okay. So I'm sorry, I got confused with geography. So Maryland is the name of the state. Baltimore is a city. It says Ellicott city is where you live. And then zip code is some number, right? One, two, three, four, five. So that's structured data you have, you have a place to store your stuff in a specific location inside a relational database. It is highly structured. But one thing that is missing is that I figured out recently that Shiva Ganesan is a poet and he has written 1023 different poems and I have no place to store it. So where will I store? And that is where these NoSQL databases come into play, is what you can do is not only store these things, very structured fashion, but also along with Shiva and his record, you store his beautiful poems, one, zero, two, three of them, lots of them. And you can all collectively put all that structure in form of what I will call not so structured data. In that NoSQL database, it allows you to expand your schema on fly and let you accommodate to store your poems if you like. And so Raj may be a photographer and he has like a million pictures. You can put them there, although it will be a bad idea to put a million picture in a database like that, but just saying, you could. So it is not structured to begin with. So the functional way of dealing with this versus this, there is a simple way to describe these things. One of them here is you have to know the schema upfront, at the writing time, at the time when you save something, you need to know the schema and you need to follow that schema. Whereas here, you don't have to worry about schema, you just write, it just shut up and write whatever you got, poems, bring it on, photographs, bring it on, save it. But is schema relevant? Yes. When does it apply? At the time of reading from the database. When you read something, you will put a filter on it and then read whatever you want to read and you can then read poem number 352 and grab it along with the name Shiv and poem number 352 and zip code. You can read, construct a filter according to what you need and you will extract that information from the database at the time of reading, schema comes into play. Schema on read versus schema at writing, that's the fundamental difference in the two databases. So back here, I think these are good questions that you're asking because it clarifies a lot more in terms of understanding a detail behind what we are really dealing with. And so what you're now looking at is this compose file, which is easy to understand. It is basically three blocks of code. One of them begins with line number eight and nine. The other one is 11 through 21 and the rest is at the bottom, three blocks. We talked about every single line except line number nine. What does it contain? It contains a Docker file in that folder called data. And that Docker file can be anything as you like, which is going to be built on the fly when you have something there, this Docker file will be built. In our example, the Docker file is very minimal. So it is basically one line or two lines maybe. And so here is using some Ubuntu image and just assigning and exporting a volume. I want to use this volume. I want to use this specific structure. Because I can use that volume for backup and restore of my data. That's the whole purpose of constructing this specific data image, which is where line number eight and nine come into play. What I want to do is not store the data in this database, but instead store my data outside in a separate location. And that location happens to be line two, where lib MySQL. That's where data sits in a MySQL database. That's where data is supposed to be sitting. And you can see that in MySQL documentation will provide you that detail. So our idea is to run this by constructing three containers on the fly. One of them will be built on the fly. Process will be using existing images like this one, existing image, this one, also existing image and it will be run. So when we invoke that, you will see the build process kick, which in turn will look at this Docker file, download this Ubuntu image and create it with these three lines in the Docker file very quickly. And that container will execute. It has nothing to do. Practically it has a command called true. So when you run true, it does practically nothing and exits. You will see the container finish off its work and exit. You will see that this container will actually run MySQL database and this container will actually run WordPress. So that's what we will see when we run. So did that change something? I probably did not change, but it looks like I did. Oh yeah, I did change. So I want to save it, close it. And then terminal. Here in our terminal, we have this file called Docker Compose YAML, which is what I want to really build. So I want to bring up my stack that is defined in that file, which you saw here already. This is the stack. I want to build it. So I'll basically say Docker Compose up and it should build. Bring my stack up. So it begins building three containers, db and data and WordPress. So it builds, first thing, builds to data container. It doesn't have Ubuntu 14.04, so it pulls it down. And it is doing that Docker file right now as we speak. It should not take too long. Just this download takes a little time. And so this image got built. It is now pulling the MySQL image. And then it will pull the third image called WordPress. And then we are basically done. It is running at this moment after this pull happens. The WordPress pull happens. It is running a live container, a three container service for us. So we'll see how it works out. So let's let it finish. And it did. Almost. And it encountered errors. So what was that error? Cannot start service program already allocated. So probably this is because I was running something already in my machine before the class started. So I need to go and kill that. And I have lots of things running. Oh, the previous exercise, we were doing the nginx exercise that is running. So I need to kill those guys, Docker rm-f, Docker ps-aq. I kill everybody. So everybody goes away. And then Docker compose up one more time. And it creates three containers and it starts and running and all that good stuff is happening right now as we speak. And it is active. And you're seeing the logs because we ran it without the dash d option. Dash d was the option to run it like a daemon. I did not do it like that. I wanted to see the logs. So I'm seeing them. And I will now open another terminal. In that terminal, I will see what I have running. I'll have two containers running and one not running. So Docker ps-a will show you the data container exited because it had nothing to do. All it does is creates a volume reference in that location because we can then store our content in there, use it for a place to store our data from the database. We have two more containers. One of them is this, which is the WordPress container. It uses the WordPress image. It is mapping port number 80 from the real machine to the port 80 inside. You have another container called MySQL. It uses the MySQL 5.6 image, which is based on what you have in the compose file. It is also up and running. It is not exposing a port to the real machine, but it is actually opening up a port in the container 3306 so that this other guy can talk to the MySQL guy. The other container can talk to the MySQL container, I should say, not guys. And so that's what we have right now. What does that boil down to? It basically boils down to a running WordPress. So let's go open it up. We should see that running WordPress in port number 80 on this local host. So here we go. And there we have it. I mean, I just said before it came, but it's coming, I know. And so it came. And so you know the next step. So I'm not doing the next, I'm moving on. And so that other container we have here is showing you all the logs about what's happening on these containers that you saw. What you will see is that this is where we are storing the data for the MySQL container. And this WordPress container is using the database provided by the MySQL container. Pretty much like the picture that we had drawn before. And I'll draw it again, WordPress container connects to MySQL container. And MySQL container stores its data in the data container. So three containers, this exited as soon as it was created. It is showing you where lib MySQL as the place where you store the data from this. The data actually gets stored in this location in that container, which is exited successfully. This is running as MySQL server. And this is some application called WordPress running. And it is exposing port number 80 from a machine to the connecting to the port number 80 inside the container. This is the machine level. And that's why you see this thing running when you visit this IP address, you're basically presented with WordPress. Now we'll deal with that WordPress thing that is running right now. And we'll actually try to persist some data. So for example, I will go to WordPress, and you can see logs popping up in the other corner in the log in that container, we'll give it some title and give it some name, just doing it quickly and choosing a complex password and giving it an IP address or email address. And then proceeding to install. Now when I install this, you will see that I see WordPress and that is expected, right totally. And so we are now on the front page, we see the reference to the name we have chosen, which is cloud genius. So it's good. Now if you kill these containers that we have, two of them are running. This PS versus PS-A is the difference in, I will answer your question David, the difference between PS and PS-A is that PS-A shows you all the containers, even those that are exited. PS does not show you the exited containers. Now the question from David is, it is localhost because the question is, since you are not providing an IP, it is defaulting to localhost. That is not the reason why it is defaulting to localhost. The reason it is on localhost is because we are running these three containers on the machine right here. And that machine obviously is localhost. Any machine's name is localhost. Every machine on the internet, or even if it is not on the internet, is called localhost. Its IP address is always 127.0.0.1, always, no matter what. And you can see that here. This machine also will have the same exact IP address like I said before, and its IP address is 127.0.0.1. That has to be the exact IP address for every machine on the planet, including some machines in the space if you have them. They will have this IP address that is called the localhost IP address for every box. Yes, if you started the containers on a virtual machine running, that would still go within itself, right? It would all be the DB and the WordPress and the bicycle would be on that one machine talking to itself. It will go to that IP address, which is going to expose on that box. So for example, the public IP, if you run the same exercise on a digital ocean droplet, you will see that it will get you the droplet IP address. The reason is like this, see this 0000 slash 0, it automatically picks up the public facing IP address if you happen to have one. If you don't have any, it defaults to the localhost. So if you run the same exact exercise on a digital ocean droplet, which I will do right now if you like. So here we go to digital ocean, login, get a droplet, and we'll get one pre-configured with Docker. So there should be one pre-configured I think, and we'll just quickly run this exercise. You will see it picks up the same IP address that you will have. So instead of clicking an Ubuntu OS, it is not going to give us Docker. So I would rather get Docker for which there is this button called one click apps, and there should be a Docker example somewhere here, the Docker, that one is a good one. And is there a newer Docker available on, we don't really care. So just select that Docker ready-made and choose a $5 droplet, select San Francisco, select your IP address, I don't have, so I want a new IP address, new private key. And this is a new machine by the way. So SSH keygen, and then cat the public IP, and then grab that, provide it. And I'm going to run the same exercise in context of a cloud. And so I'm providing that public IP and calling it my new OVA and new OVA and add the key. Now we have a key placed. This is a Docker container. So you know that it is Docker and in there I will create. So I'm ready to create a box containing Docker pre-configured. This $5 droplet, San Francisco, my key pair provided and create. The machine starts in a minute, we'll connect to it, place the same exact compose file and go Docker compose up. And you will see the IP address will pick up, will be the public IP address automatically. You will see that exact thing happen. As soon as we are able to get this IP address from this location, which I will now receive waiting for it, and it should come in 30 seconds. In the meanwhile, I'll get ready with my Docker compose file, which should be here, right there. So we have it. So we'll copy it, go back here and go connect to the SSH root at IP address, which is this IP address. It will be brought back to the machine, paste it here, connect. Oh, it is not ready yet, not ready yet. Come on, get ready. Go change the world. It said, so I want to change the world, connect. And now I connected. Now I'm going to provide this Docker compose file to that box. So I'll basically grab this compose file as is, copy and go to the machine in there, say edit Docker compose.yml and paste it and save it. And then Docker compose up, that's it. And Docker compose not found. So what do we do? I guess they should have installed Docker compose, but they don't seem to have installed. Pip not installed. Okay. I have to get install Python pip. And then, yeah, they should give complete images with ready solutions, but they don't do that sometimes, which is fine. So we are installing Python and then get pip to get Docker compose. And then here is pip install Docker compose. And now that we have Docker compose installed, we should be able to say Docker compose up and build your stack. So it says what data doesn't exist. Oh yeah. We have to get the whole folder, not just that one file, we got to get the whole exercise loaded. I am being silly. We have to get this whole thing loaded, this whole thing. So we load that whole thing, not just one file. So back here, we'll say git clone, paste, and then cd Docker files. And then modify this Docker file a little bit because our home user doesn't exist. So this home user we have here doesn't exist. We have the folder called root. That's where it is sitting right now. So we modify the Docker file accordingly. Copy now and go in that location, edit that Docker compose file that we have, which is empty and paste it and then say Docker compose up. Now it should work. So it is bringing the one to image for the data container. Then it will bring the MySQL and then we'll bring WordPress and we'll have a whole thing running in a matter of one minute. This is actually pretty fast. Access to the network is very fast on DigitalOcean, probably a gigabit connection. So your downloads are very, very quick. And so it should not take too long. But when this thing runs, you should be able to see live in DigitalOcean on that IP address, a new WordPress popping up because of the same exact reason like I mentioned is it picks up. And so it is running WordPress and that's what it is doing right now. So WordPress should be ready for us as soon as this thing is ready for listening. You will see the same thing run on, yeah, so it is getting ready now, three, two, one, init process completed and init process is in progress. Something else is going on right now. Connection refused on line one, line 22. Something else is bad. I don't know what that thing bad is, but probably it's not connecting to the Docker, to the MySQL database. That's the error that we are looking at. And it exited with code one, it erred out. It did not successfully let WordPress connect with Docker container for some reason. I don't want to really investigate too much right now, but you get the point. We can get the root, get to the root of it, but not right now. So back here, kill this guy, destroy and say, you go bye bye, where are you? Droplet has been destroyed successfully. Okay. So it is dead. Now next example, we will now quit this and say, cut, close, close. And this other terminal we have running already, we'll close that also. And say docker ps-a, we have the three containers in our listing, we want to kill them all. So we'll say kill everybody. So all of them are dead. So now docker ps should be clean. So we're beginning again with a clean state, drawing more examples. So this example was three containers, data, DB and WP. So now we will erase this and save, make an empty slate, and then take a slightly different example. This is going to be a little complex example for us to understand. So I'm going to describe it to you. What I'm doing here is that you will see that it's a large file and lots of them, lots of things here about like say 86 lines. So what does it contain and why it is written like the way it is? That's what I would like to explain to you. So I'll keep it open, but I'll go back to a whiteboard to draw the idea that we are trying to accomplish. So where is my whiteboard here? So what we are trying to run in this next example is we will have containers, multiple of them, C1, C2, C3, like that. Multiple containers running a variety of different applications. Like we run WordPress here, run Joomla here, run Drupal here, some other different types of example applications we will run and run another WordPress here maybe. So multiple containers and sometimes we will run a separate container to address the need for this WordPress and a data container to address the need of this data storage of MySQL. Sometimes we will use a container that runs WordPress and MySQL together, Joomla and another database in a separate container, Drupal and another database in a separate container. So we'll run a bunch of containers in that exercise that is coming up. Now the question that remains is that this and this and these are independent applications. You have this application called WordPress, you have this application called Joomla, this application called Drupal and maybe a C5 running RedMine and other applications. So lots of different applications that we are putting together and putting them all in multiple containers, all of them sitting on one box. Ouch. Now how do we access them? Each one of them will be conflicting and taking on the port number 80 and we want to be able to not cause conflict. So you want to be able to access them individually, independently. And so there is a question from Shiv, I don't understand how the data model is defined in inserts and updates to happen from website to database table. I'll give you an example to help you understand Shiv, stand by on that question. And so what are you talking about is this multiple different applications like I mentioned here is what is described here. And we have an example that runs the Tutum WordPress. And I think now that we have Sami here, Sami can explain what Tutum actually means. I think it is a Turkish word. And so he'll tell us what that means, but it's the name of the company acquired by Docker. The name was Tutum. So this is an image Tutum WordPress. We have another image called WordPress that links to database and another image called let's see, Joomla, there's a Joomla image. And then another image called Redmine image and another image available. No, that's it, these are the images we have. So let's quickly look at what those applications are. So here is Redmine. Redmine is an open source application for project management. So this is what Redmine looks like when you start a new Redmine application. That's what it looks like. You can use it for project management. It's an open source product. This is Joomla, another open source product, just similar to WordPress, but different. Many people use this. It is also open source. And we have used in our exercise, we're using some other examples. Let's see what are we using. We're using Joomla and two different types of my WordPress. And that I think is it in this example. So back here, what I'm referring to is that we will run into a conflicting situation where each one of these applications will want to occupy this port, port number 80. Everybody wants to run on port 80, right? That's the design. And you have heard me talk about this concept before called load balancer, which is a traffic cop, which is what you do is you have officers number one, officer number two, officer number so many in a DMV. And each of these officers pretty much have a similar function. All they do is one thing, give you driver licenses. Their function is pretty similar. You can actually practically replace officer number two and put officer number 35 in there and they will be all functioning the same way. That is how you scale out. That's the scenario we have discussed already in context of a load balancer or traffic cop. A classic example is DMV. You put more officers. If there is a lot more people coming to request a driver license, if there's a queuing up queue being formed, you will see that they will add more officers, officer number 37, officer number 39, like that they will add and more officers will serve you. However, in our example here, we have different applications. We don't want to necessarily scale right now, but actually we want to what I call mix and match different applications and give them uniquely different names. So let's say you want to run something like cnn.com, newyorktimes.com. And these are independent applications. I want to run them in their own containers, all of them exposing to port number 80. And I want to put all of them in the same box. I can do that provided I use one more container. And that container should function somewhat in concept like the load balancer, but the purpose is slightly different. The purpose will be a reverse proxy. In that, what it will do is somebody is looking from outside, comes and visits New York Times colon 80, I mean port number 80. People don't type this 80, but it's implied. If somebody is visiting newyorktimes.com, this port number 80 is implied. It's not written down, but you can write. You can actually go to newyorktimes.com and type, did I type, no, newyorktimes.com colon 80, enter and it will go to the right location that is expected. So you see that it went to New York Times and it expanded to www and all that. You can go to port number 80 if you like, but nobody types like that. But people will visit that site and same people can also visit cnn.com on port number 80. And so this reverse proxy, what it does, it deciphers the fact that you want to access this site, will automatically route your traffic to this container. And you want to access this cnn.com, it will automatically route this inbound traffic to this location, to this container. It is smart enough to understand and do that according to what the user wants to do. So if the user says, I want to visit CNN, okay, bring it on. Reverse proxy will decide and let you go this way. And that will allow you to mix and match multiple distinctly different services on the same box or the same collection of boxes by providing the same entry point with different people visiting the same IP address for different names, different functions, different services that you can access. And this is just an example. And that's what we are building. We are using these names cnn.com and newyorktimes.com fictitiously, but that's the core idea behind a reverse proxy is it lets you use a variety of service types, service type number one, service type number two, mix and match different types of services behind a reverse proxy. And there is a container available already for that. So we want to use that container. That container is available somewhere in this location called J Wilder NGINX proxy. So this gentleman has prepared that container already and his name is Jason Wilder. He has prepared his NGINX reverse proxy, which is a very popular NGINX based proxy for Docker containers. And we will use that image in our example. And what this container will do is basically do exactly what we described, is to allow traffic to go to the appropriately chosen container by its name, by its name, DNS name. So if you assign a name xyz.com to this service, and if the user asks for xyz.com, the traffic will be routed to this. And that's the purpose of this reverse proxy by J Wilder. It's a very popular proxy. It has like thousand forks and all that. So it's decent. I use it also myself in cloud genius sites and other services. So we are running basically a bunch of different services under the hood. Different people come and visit different things. For example, you can visit the cloud genius website like this, or it will open up like this. You can also visit this site called chat dot. I don't know if you have visited this or not, but probably not. Probably you did. If you visit this site, it will go to a different container in the same box. If you visit this site, analytics dot analytics dot cloud genie dot us, you will visit the same box or same collection of boxes with a different application. Now what am I talking about? Again, let me explain. What I'm talking about is this, when I say dig, what do you see? The IP address is this one, 252.7. When I say dig, what do you see? The same IP address. When I say dig, what do you see? The same IP address again. What's going on under the hood? This is the IP address where the front end reverse proxy sits. And it looks at the incoming traffic. What do you want to see, sir? Do you want to see? I gave you three examples. The three examples were this service, another service, and another service. They all point to the same IP. This IP address belongs to the front end, which is where my reverse proxy sits. That reverse proxy is the code that I just showed you here. And what that reverse proxy is doing is depending on your inbound request, do you want to see here? Most of you go here, by the way, most of you. Some people who want to establish a new Slack channel connection can go here. And when they land on that page, they'll just enter your email address and receive an invitation to join Slack on Cloud Genius. That's the purpose of this site. It runs inside in a separate container. The container is called chat. It's a chat container. Under the hood, there is a separate container. So let's draw it like this. Inside here, there is a chat container that runs by itself. Under the hood, there is a varnish container, which is a cache. Varnish cache is a separate container that I run under the hood to expedite speed up traffic served to you. And behind this varnish cache is the WordPress container, which processes the code that actually you see when you see whatever you see when you visit this site. And behind that, there can be multiple of these guys, by the way, more or less, depending on, it'll scale automatically, grow and shrink. And behind them is a database container. And behind that is a data container. And this is where I actually store the data for the database, which in turn gets connected from multiple WordPress servers, one, two, three, more like that. And they will all be served by a varnish cache. In addition to these, I have another container called Redis, which speeds up access to database queries from here. So whenever this WordPress will query something, it will actually query this and ask Redis to see if you have a fresh stored... This is stored all the way in memory, by the way, memory-based service. The Redis is a fully RAM-based service, so no access to disk at all. So it has a large RAM available. And when WordPress looks for a database query, it will first, instead of going here, it will actually ask Redis, hey, do you got something? Do you got my query already in your RAM? If it is there, give it to me. And so it gives it immediately. And then it serves right to you. If it doesn't have that in memory, it will go and actually query the database and then give it to WordPress. And when it gives, it will store a copy in RAM. So that next time we ask the same thing, it doesn't have to go to the database at all because database is actually stored on this data container, which actually sits on var, lib, MySQL folder on a real machine. So it's a disk here, SSD. So you have a disk to deal with. Disk is slow, RAM is fast. Therefore I keep it like this to expedite the performance of the delivery of content to you. So this is the most complex service we have running, which is this Cloud Genius site that you see. We have a third site running that none of you visit, only I visit. It is specifically for me. And that site is this analytic site. And that is a separate container, which has its own database. And what it does, it monitors every activity on the Cloud Genius site. And you will see that thing in real live example, if you go here and sign in and you don't have to log in there, only I have it, but here you can see every activity. And so I'm going to log in and come on, here it goes. So here you see internal activity of all of you and people I don't know that come and visit and I see what's going on in the aggregate level. Like for example, last week, what happened? We had visitors from these places in multiple countries and multiple continents in real time. You can see a map as to who's hitting the site. So a lot more activity information in terms of analytics available to me. And this PIVIC is also open source. And there is a container for that available in the library. That is what we are running here in the PIVIC analytics, which basically monitors activity in this location. So multiple containers running behind the scene, behind this reverse proxy. And that is a real example I just showed you. And in our academic example, you will be able to run it. Similarly, similar to this, but not exactly all these complexities we have in like war nation redis. They're hard to configure. And so we are not doing that right away, but dealing with a slightly simpler example with only 86 lines of code. And in there, there are several containers. In fact, there are precisely, let's count them. So at the very top, you have access to the main example file here, this one. You probably have cloned it. You have it in here, in that location. You have it already in this location, if you don't have it, by the way. So in there, you can just click on it and read the examples. The main examples is what we are reading. And this file has 87 lines. Let us see what the details are. The very first one is this container based on NGINX reverse proxy by Jay Wilder. That's the first container at the very top. And it is going to do that routing for us, like traffic routing. So inbound traffic, you see who's looking to get what site, what service. And accordingly, the particular, this NGINX Jay Wilder proxy, reverse proxy, will run on port 80 at the front end. And the very front, first thing you deal with is this container. And that is what we are invoking in our example, beginning from line number four through 10, this segment. The components that we have underneath, we'd use a port number 80 on the real machine, map it to the port 80 inside this Jay Wilder container. We'll use a volume mapping also, meaning we'll connect the docker.soc on the real machine to a temporary location inside the container. So it can actually understand what docker containers are running under the hood. So that this NGINX proxy needs to know which other containers are running so that it can actually route traffic to them. So that is what is done through this volume mapping. We'll use the Jay Wilder NGINX proxy image. And when it dies, you restart it. If it accidentally dies, restart it again. That's the whole purpose of this reverse proxy container, line number four through 10. Let's see more examples. So it's a collectively one big example with 87 lines, we are breaking it down step by step, part by part. Next one begins at line number 14 through 21. It's a self-contained WordPress container, includes a database. It is not good for production. It's all in one. It's a bad idea, but it's good for demo. And that is this two-to-one WordPress container. In that container, we are exposing port 80 at the container level, not mapping it to a real machine, just exposing it. This is mapping line six. This is exposing, just opening up a port at the container level. Mapping will happen something like this. You map a physical port 80 to the container port 80, whereas this is just exposing the port 80 at that container level so that this NGINX reverse proxy can actually detect and communicate with this port 80. And we are assigning a specific name for it. We are going to call this a service called Charlie-1, Container-1, C-1. That's the name I chose. You can choose any name you like. In our example here, C-1, this is the C-1 when it dies, you restart it. So very simple C-1 container will show you WordPress. How will a person visit C-1? They cannot unless they define a record in their host file that assigns a name C-1 to an IP address, which is going to be localhost for our example. So we'll do that assignment later on. But for right now, we want to finish this understanding of what this thing actually does. So this is a C-1 image containing two-to-one WordPress, self-contained, all-in-one, includes MySQL, everything fully contained package. If you run that container, you get a WordPress immediately. And that is not good for production, as you know. But that is what we are using in 14 through 20. Next we have a proper container using a library image called WordPress. And this is how production deployments are recommended, is that you will use the library image for WordPress, expose port number 80, and then call it C-2, Charlie-2. This is the name that I'm using for this container, so that JBuilder EngineXProxy can then identify if somebody wants to visit Charlie-2, they will go to this expose port on that container. And it will be called C-2 by that name. It will link up to another database container called DB. But internally, it will be called MySQL. We are going to use the current location WP content folder in the actual machine and map it to that folder where WordPress stores files. This we have seen in a previous example, where the actual files inside the container are stored outside in a real machine. And when it dies, you restart it. And that is this C-2 container. To support this container, there is line number 32, 31, 32 link. It needs to link to a database. And that link container is the next one, which is a container based on another library image called MySQL. This is properly structured MySQL image, which is referred here. And it is very similar to what we saw in a previous example. It is lines 39 through 50. And what it is doing is it is going to store its data in another location called data, which is the next container, by the way. And its configuration is getting picked from this location. This is the custom configuration you saw before and is mapped to the internal configuration on the container. We are assigning a root password, a user, and a password, and a database name for connections. If it dies, you restart it. And this is your database container. To store the data in line 42 here, we have to have another container that actually holds the data in var lib MySQL. And that is the next container. It is an empty container, basically holds the storage for this location. For the container to store the data for this database is where it will go here. And you have seen this example before. Next we are creating another service called Joomla. We are going to call it C3, just a name, and exposing it on port 80 and calling it a virtual host by the name C3. And this is going to allow your proxy, reverse proxy, to detect the name C3 and route traffic to Joomla accordingly. If somebody visits C3, you will go to Joomla. Next another container, running red mine. This container requires another database called Postgres SQL. So we will get to that Postgres part later, but first call it Charlie 4. So red mine is called Charlie 4 in our example. We are using a library image called red mine and exposing a different port. That's what Ruby on Rails runs at. So it is exposing port 3000. So practically what this boils down to is this reverse proxy that we are talking about. We can then connect to port number 80 on WordPress, port number 80 on C2, port number 3000 on C3. Whatever you define, you get to define which ports you want to expose. You want to call them C1, C2, C3, you assign the names, you assign the ports. Whatever you expose, this reverse proxy will automatically connect a user who wants to visit C1 will go here, a user who wants to go to C2 will go here, and user who wants to go to C3 will automatically go here. Even though the port is different, whatever you expose is what will get picked up. So 3000 is what red mine exposes. And that is what you will see when somebody visits C4, you will see red mine up and running. It links to another database called PG, which is the name Postgres inside the container. It will be called Postgres. And we are calling it PG, and that PG is defined like this, which uses the library image called Postgres and exposes two environment variables, Postgres password, Postgres user. The username is red mine, password is secret. Now if you run this Docker compose up in the main example file section, which is where we have this main example, we'll take whole thing, copy it and put that in this Docker compose YAML, make it delete everything first, and then paste it. Now that we have 87 lines in this example, we should be able to run Docker compose up just like that in here, Docker compose up and dash D. So let it run in the background. So when you see that it doesn't have JBuilder engine exproxy, it will download it. And then all the other missing images will be downloaded pretty much, Postgres getting downloaded. And then your other missing things like MySQL, red mine, they will all, Tutum WordPress getting downloaded right now here, Tutum WordPress. It's a separate image, different from the regular library WordPress image. It is just an example, not for production. Yes. As soon as this run finishes, I'll do a break. Running it is very easy once it finishes downloading. The run part is trivial. You will see the magic that you expect to see is when somebody visits C1, it goes to one location, C2 goes to another service, C3 will go another service. And that will happen provided you have defined these names, C1, C2, C3 separately. You can do that very easily in this thing called hchost file. So there is a file called hchost. In every machine, you will have this file. Every Linux machine will have this file where DNS, local DNS is kept. In Windows, you have the same file in a different weird location called c colon slash windows slash system32 slash drivers slash hc slash something like that hosts. That's where you will find in Windows. I may be wrong. I may be wrong about this. So it is too complex for me to remember. But we just have these bunch of containers, C1, C2, C3, all of them running apparently. So what does that mean? Can we see a list of our containers? Yes, we can. You see them all here. But what about visiting them one by one in a browser? What happens if you just visit local host? Nothing. Why is that so? Because it doesn't know local host, the name. Your people will expect to type the name C1 here and expect to see what C1 is. But C1 is not defined. So it doesn't open anything. So we need to define what C1, C2 is. C1, C2, C3, C4, four of these things are. And we can do that in our hchost file simply like this. In that file, oh, I did not type properly. Password I type, and then I go here to my local host and say, your machine is also known as C1, C2, C3, C4. From now on, you will also be known as C1, C2, C3, C4. I just defined that, declared it. Starting from right now, you will be called not only local host, but also C1, C2, C3, C4. Save. Now I can go here. In my local machine, I'll type HTTP colon slash slash C1. And what do you think I'll see? I'm expecting to see WordPress, which I did. This is the Tutum WordPress. It is Tutum, as we can tell by example, our name C1 is the Tutum WordPress. So we will make sure to call it that. We will call that you are the simple demo Tutum WordPress. Yes. It's C1 and local host are exactly the same. Yes. What is the website for different? Yeah, exactly. It's a good question. I'll answer that. Just a minute. It's a very good question. Why does it look different? It is because of your reverse proxy. It knows that the user wants to visit C1, so it directs the traffic to C1. And so now here you see the user will be shown Tutum WordPress. You see that right there, Tutum WordPress. Now let us see if you open up a new tab and say HTTP colon slash slash Charlie number two. Show me what you got. C2. What do you think we'll get? So we go to C2. We are expecting to see Charlie two is WordPress again, right? But don't confuse yourself. This is going to be a different WordPress, not the Tutum WordPress. And here it is. This is the library WordPress image. And we have to set it up again differently from what we had done before. Before we had only the Tutum WordPress image already configured on the first tab in here. What we are dealing with is a library image of WordPress with a MySQL database container and another container to store the data. And this WordPress looks different. It is the latest version of the library WordPress image. It is the C2 image that you're looking at. Similarly, HTTP colon slash slash C3. What do you see? What do you expect to see? Click here and read. What does C3 do? C3 is Joomla. So when we type C3, we are expecting to see Joomla right there. And what do we see? We see Joomla. Next HTTP colon slash slash C4. And visit. What do you expect to see? Red mine. Here comes red mine. So now ask your question again, Raj. Just by the name, it looks at this name and is able to decipher that the user wants to go to this container versus that container versus that container, that name distinction this reverse proxy is doing. We are all visiting the reverse proxy. That is the first thing that you hit. The only thing that you hit, because as you see here in the example that you saw, where is that link? Slack chat. Where is Slack chat? Here. Here is Slack chat and there is this Docker file that we are looking at, main example. And here you see that this reverse proxy is the only one which has a connection to a physical machine port. Nobody else, none of these other containers have actually have a port mapping. So not C1, it has exposed, not a port mapping, not C2. It has exposed, not a port mapping, DB doesn't even expose anything, data doesn't expose anything. Joomla has an expose, not a port mapping, C4 has an expose of 3,000, not a port mapping, EG doesn't expose anything. You don't want to expose anything. It's a database. So you want to expose only the user end points, which are C1, C2, C3, C4. So here is C1, here is C2, C3, and C4. You see them all different. But what you actually are looking at from the first time a visitor comes like C4, it is visiting your IP address and it is greeted by the reverse proxy. That's what you really are hitting, the first thing. The reverse proxy understands that, oh, you typed C4, I see, oh, you want to see C4. Let me show you C4. Here you go. I see that you want to see C3. Let me show you C3, and there you go. So the idea behind this is to consolidate microservices to be specific in a bunch of containers that you may have across multiple machines that you might have somewhere. And you run these microservices across the board and then the services will dynamically allocate. Sometimes you will see a scenario where you have this one box and another box and another box. These are boxes, physical machines, and you have containers C1, C2 running here and C3 here and C5 here and C4 is dead. So it was actually here, but it died, so it creates here and then it comes live. Now to address that scenario of a machine dying from here and showing up here, you need that kind of a proxy. So you need some kind of a JBuilder reverse proxy example, which is the only thing that outside world knows about. So outside world will hit here with the name C4, and when this container dies, you have to report back to reverse proxy that this guy is dead, so that new one is created here, so that reverse proxy will then transport the traffic of this user, that you will go meet the reverse proxy and the reverse proxy will say, you know what, your C4, that old one is dead, you better go here. And that's where it drives. Now imagine if this dies for some reason, a new one will pop up here and reverse proxy will automatically detect the fact that you have a new container here. And so it will stop diverting your traffic to this direction and instead go here. And so this migrating movement of live running code in different containers, you can just take them from A to B, no problem. And if accidentally something goes bad, like this guy dies, no issues, create one more. And this creation also happens automatically. You will see in more examples that we come along is this high availability scenario is accomplished because you can actually now afford to have a user come in and say, I would like to see C5, please. So the reverse proxy says, okay, C5 is right here. So you go and see that, that's good. But imagine now the whole box dies. And now you have an outage, but not really because what Docker Swarm does or Kubernetes will do or any of those services will do essentially is when you have an entire box die, no problem, move these guys here. And you say, bye bye. Basically say, I want you to go away from my mind. So just erase your existence in my mind. So you don't exist anymore and now create a new box, make capacity and have it configured as a part of the cluster and then balance the load. So we can then start rebalancing a little bit and put C3 here if you like, and maybe C1. Anyway, you feel like this automatic allocation happens in a systematic manner, not randomly like I'm drawing, but I'll talk more about how it allocates. There are many ways to allocate, spread the love or spread the load among multiple boxes as you decide. And right now in a picture form, I'm just randomly drawing whatever I feel like. But that's what will happen is now this user comes along, new user says, show me C1. So it says reverse proxy, C1 is now here and not here because we have migrated it over there. So you go there. And that's the concept behind this concept of service consolidation or traffic discovery of microservices where you can associate unique DNS names to a bunch of services in your cloud and have them automatically route traffic to wherever the container may be at that point in time. And you can migrate them A to B to this location, to that location, to this box, to that box and kill the boxes, bring new boxes, who cares? And that level of what I call abstraction is provided by services like this. Like in our example, we are using, where is that guy? This, this tool like J welder engine X proxy we are using. Like this, there are a couple more, there is this thing Trafic, it's a difficult, difficult spelling, but the pronunciation is Trafic and it is on this location and it is also open source and they just got funded for a million dollars for a, for a new, new company. It's an open source, the user interface is nice, I'm going to use it and show you an example of this. This is just, just, you know, yesterday I got a million dollar grant from somebody, I don't know who. The idea is you want to visit the users on the left side. Users will visit API.domain.com, domain.com slash web, backoffice.domain.com, different people trying to access the different things. And this traffic controller will then automatically route people on the internet on the left side to the containers inside. So those who want to go to API.domain.com will get through the traffic controller and go to the API box here and people who want to go to domain.com slash web will automatically get the traffic routed to the web container and people who want to go to the back office will go to the back office, one, two, three, multiple. And that's the concept that you're looking at from a picture perspective, that's what you saw. This is a different product. We are not using this one right now, but we are using this one. It functions similarly, it's the same idea, except this traffic is a little bit evolved and I'll talk more about what it means and what the evolution is and how it impacts us and it is somewhat elaborate. So we'll talk more about that when we do that exercise, I have built that one also for you. It's fairly new. So we'll at some point discuss that. But these are the tools that as it is described here, a modern reverse proxy and it works with Docker, Docker Swarm, Kubernetes, Mesos, Consul, etc., a bunch of other things it will run with and do things that your users expect to use and at the same time giving you the ability to move stuff around so that you're not tied to a particular box or associating or tolerating downtime. You don't have to tolerate downtime. You can just create a new box, move your service there and let your users not even notice. It's like a blink of an eye. You see that things move from here to there and people don't notice. And I have moved this cloud genius site and services in your face and you have not noticed unless I told David that I'm moving it, then he knows, okay, you did move apparently because I told him that the IP address changed. And so he said, yeah, he did. And then he saw the migration, I showed him live migration. And so we'll show you that also when at appropriate time. But I think it's a good time to break to digest this. So we'll start our timer. Questions are welcome. Let me just start a timer break. So timer 10 minutes. There we go. Yeah. Questions are welcome. So this setup that I have in our example is not resilient. It doesn't allow you to create and move containers. We are running all these things on one box. Right. So it doesn't play the scenario of live migrations. It just an illustration of how you route traffic. So coming in from the top end, users visit with the name, the traffic will be directed to the appropriate container running appropriate service. It's not as easy as chat.gini.us, as simple as that. So instead of C1, just use a different DNS name. And so that's it. That's pretty much it. So C1 will be a container name, a full flesh container, C1.cloudgini.us, C2.cloudgini.us, something like that. Totally do it. Totally works like that. Let me read your question. You have a question there. It says, I still do not understand where and how the data model is defined for inserts and updates to happen from the website to the database table. Okay. So it has to be discussed in the context of a given application. Then you will understand. So let us take an application example. So let's see. Where is my pen, there. So I'll go to an application example, which let me ask you, which application are you familiar with? Like some example, open source application that you know has a user as a database. So pick an example. So I'll give you the example in context of what you're thinking as opposed to some random example. No, we need to take a certain example, WordPress. Okay. WordPress. Here. Let's go to the source code for that exact thing that you're talking about. So here is, it's a good, good example. So WordPress, this is a source code for WordPress. You're looking at a file inside the WordPress repository. The file is called WP config sample. It's an example, right? The actual file inside in a functional database and a functional website using WordPress will be without the word dash sample. So you have to create a file that is somewhat similar to this file, except the dash sample has to be removed. And then it will become WP-config.php. This is where configuration for database connection is stored. So remove the dash sample, that becomes a real example. Now inside that file, let's go see, there are 90 lines. Some of the important things here are line number 23. What is the name of database? You have to provide that database name here. What is the name of the user? You have to provide that name here. What is the name of the database password? Provide that value here. What is the name of the DB host? Provide the IP address here of the MySQL server. And these are the four key things that are necessary. Others you can change, but for database purposes, these are the settings you need to connect to so that the WordPress instance can actually properly connect to the database on the other end in another location, in another database service, or in another container you might have. So this is the code, the exact code that you will use to connect a WordPress instance with a database. Does that answer your question? So now when you run the code, for example, so let's go see some more, this WordPress is huge. I have to go and identify the exact piece of change you will make to edit something. So I'm going to modify, I'm going to quick search on the source code to give you another example, hold on. I think this is a good one. So when you edit a user, so in code you have to go refer to multiple files to get to the point you want to get to. So this is just a reference to editing a user. So you have a user, WP admin user, user.edit. And what this does is allows you this to load this file called useredit.php. So this file will be loaded. So you have to go read corresponding admin.php to understand. So understanding source code can be difficult if you're new to it. It is there inside the Docker container, you will have the source code right there. So for example, here, it is not on GitHub, it is locally stored on the container itself. So for example, here is the Docker WordPress image. And this image is based on, you can pull it like this, and this image is based on this Docker file. The Docker file looks like this. So you begin with Apache PHP container that has Apache already. And then at the end, you will find that it is downloading WordPress from WordPress.org. It is expanding the WordPress zip tar gzip tarball and placing that in this location. It is then removing the tarball and the source code is actually placed in this location. And the chown will modify its user to be www.data colon www.data. That's the user and group that will be the owner for this source code. The source code was obtained in line number 36 from WordPress.org. So you download it from the source, put it inside the container, and then start the process with the Docker entry point and run Apache in the foreground. Apache is where the container already contains Apache right there. And that is how you will actually execute the source code that you will download. So the Docker image contains WordPress source code, which is obtained from the source. The source is this site. The site is where the source code is sitting. This is where WordPress can be downloaded WordPress.org. It is different from WordPress.com by the way, just so you know, WordPress.com is not the source code. This is a service and the WordPress.org is the source code. The source code is also available on GitHub that you saw that I showed you here. It is also available on GitHub. But primary location that WordPress source code is available is WordPress.org. So as you see, that is what the LAMP stack is helping with, so WordPress, PHP, and ICC both are in hand. I did not get your question. Your audio is not clear. Yes, I know why. So basically that is why LAMP stack is very highly integrated. Correct. So correct. So the idea behind LAMP stack is you have got the source code, you have got the source code, you have got the source code, you have got the source code, you have got the source code. You have got the source code. You have got the source code. You have got the source code. So the idea behind LAMP stack is you have got these components and then you put WordPress on top. Right, you already have PHP. You already have MySQL database. you have Apache on Linux box and then you get source code on top and then it runs so do we have the audio is not clean I don't know why yeah so do we have you can customize you should customize to meet your need if you don't customize you will get very bland implementation like that looks like this but this is not what you want to use you want to customize it heavily totally not not the source code you customize WordPress by clicking something like this customize button and so it gives you it's a very nice software it gives you very easy to use tools to customize customize the look and feel without having to modify source code so visually you can just change something like for example if I don't like this style I'll go the 2016 style and I go click and I go live preview and I like it and I say okay this is good so I activate it so now I get a different style and so I activated it and now you will see my Charlie number two looks different C2 looks different I can go back to my previous style by customizing the 2017 style there are four or five styles already preloaded so I want to now customize it again back to the 2017 style which is somewhere here so I say yeah I like that style so put that in live preview and then activate so now my styling will completely change without me having to write any code I just change the look and feel of my site very easily click and there it goes different look so changing WordPress customizing WordPress has been made tremendously easy that's the whole reason why we use WordPress and that's the whole reason why 25% of top 10,000 sites on Alexa use WordPress some statistic like that and maybe approximate but you know most of the top side 25% of them use this application why it is easy to customize that's the whole reason so we have timed out let's proceed so we were doing that exercise right which is where we had C1 C2 C3 and C4 four different services in bunch of containers more than four by the way and all defined in a compose file written like that we were able to construct it primarily because that's what we decided is that we would like our bunch of services to be constructed and connected the way we want and choose the name C1 C2 C3 C4 you don't have to stick to the name C1 C2 C3 C4 in fact you can modify the name C1 wherever we have so we have it in two places line 14 and 19 we'll make it New York Times and save and similarly there is C2 right so in C2 I want to call it C2 I would like to call it CNN calm so I just renamed it and then we have C3 right so C3 we will call it say B dot a dot cloud genie dot us and C4 C4 is going to be called fake news dot com say for example fake news right so C4 that's C4 so we have four four sites configured so closing this back here in our containers they're running there right so I want to kill all of them and so here we go deleted everything and now I will edit my host file and put my password in there and then change my C1 C2 C3 C4 and say you know what instead of these guys I want to use something very useful for example New York Times oops okay New York Times calm CNN calm be a cloud genius and fake news calm so four sites I chose these names this time saving it I can do it locally you can hack it because it's my machine I can do whatever I like but you cannot really actually hack New York Times or other sites that exist because you have to go and modify their DNS records and you don't have the privilege to do that so you will not be actually be able to hack it like this only for your view you can change it like that so when I have that modified I can then go docker compose up again with modifications what does that boil down to it means I can open my browser and instead of these guys I will now say I want to open the browser again and open New York Times and CNN and fake news calm I don't even know such a site exists but I want to see that I want to see fake news calm how it looks so fake news calm see what it shows bad gateway is because our process probably did not pick up maybe a typing mistake somewhere did I type fake news fake news calm fake news calm there is no typo so maybe refresh will do it yeah yeah it will do so yeah refreshed I think NGX the reverse proxy did not pick it up it was too short a time for it to pick up so now it picked up so you see red mine there this was our world c4 it is now fake news calm we'll see new tab CNN calm what do you expect to see you expect to see WordPress and then next one this is going to be New York Times calm what do you expect to see either Joomla or one of those right so here you see WordPress and what about the cloud genius site you would see Joomla and here it comes so how did it work like the way it did because that's the name we chose in our virtual host references we chose that name like this nice CNN calm and so whatever you choose is a name that NGX proxy will look for from the users point of view and then appropriately divert traffic to that particular container which you have labeled as the virtual host for CNN calm but this is not CNN calm and it's it's WordPress so this is not how you will actually do in real life by the way so don't get me wrong it's it's purely fictitious exercise but it works to illustrate a point having done that back in terminal I want to break all these guys so say bye bye to them and they all went away there are more examples for us to try if you want but I think the point is clear and there is more here so we have we have Drupal and red mine and more examples if you like so you can get as more complex as you feel like but most of this is fairly pointless in that you know you never actually run it on one box you know run multiple services you can but you don't in large production system you don't run it like this instead you would rather run multiple containers on multiple boxes across spread out and then have a method of dynamically shifting your workload from one to one to the other or in case of you know for load balancing purposes or for accidents or disasters that you want to be able to have that flexibility and so that's the user scenario this scenario of running everything inside a virtual machine as a test case is merely a test case merely a point to illustrate that it is possible in real life you will do something like here you will have multiple boxes these are boxes and these boxes will run containers C1 C2 C3 C4 like that box 3 C5 C6 like that and then you will assign names virtual host names to these guys and run actual microservices in here and have a reverse proxy of some sorts whether it is jwill.Nginx proxy or whether it is traffic or some other solution you might have that will actually divert traffic to these services these containers are just the front end containers so behind the scenes you can have more and you can have a larger complex layout these are just the front end points of these services so the question from Shiv when a machine goes down how does the log gets moved in other words how do we preserve it the best way to preserve something is to not store it on these boxes so where do you store you store it outside in another stack say for example this this is a stack called elastic stack elk stack it has it stands for elastic search log stash and kibana so it's a open source implementation of a large commercial company which I think I will remember the name for it you may have you you may know the name that does log analysis for large companies and that is where you store so let's see log log entries is a smaller company there is a larger company for log analytics I'm trying to remember its name the SP LUNK is a commercial company that does what you just said is to store the logs outside in a service and that's where you do analysis so this is a large entity that does log analysis for your infrastructure and so scenario the question you're asking is where do you store the logs you don't store them locally you move it out to some service outside like Splunk in a commercial setup if you don't want to spend money you go this elk stack store it here and this is equally good it looks actually much more beautiful than Splunk if you play with a elk there by the way there is a container for that elk stack containers are available you can run E L and K it's a stack the three things elastic cache sorry elastic search log stash and kibana so three things these are independent products they're all acquired by the company called elastic search now elastic co is the owner they have these things together in form of products that they sell but they are also open source so you can just use the open source versions if you like here is elastic cache elastic search it is kibana kibana is a visualization engine this log stash is like moustache and the idea is to store your logs and elastic search is this search capability to let you search your logs and that is the elk stack they have added more capabilities to it these days and they will keep building and provide services on top of these open source products to compete effectively with Splunk Splunk I think we spent like five million dollars a year contract when we hired them by when I was at Microsoft a long time ago and this I think is a good solution elk stack so we think about it if you want to store logs that answers your question about you know how do we preserve data and the simple answer to that was don't store it there we know it will die don't throw it there store it outside store it in an s3 bucket very highly durable right you know durability of s3 you remember okay so let's first check this durability answer you know it is 99 so it's very highly durable this is our 11 or 13 nines or something like that now how do you store your logs to s3 here how you do it how to store logs in AWS s3 that's the answer so managing but these are hold on how do I export logs to s3 using console so here is what you will do is you will create a bucket and then choose a bucket policy like this and then create a cloud watch task so we haven't talked about cloud watch specifically but this is a service to monitor it's a logging service available as a part of Amazon subscription that you get so when you when you use AWS you automatically get this service called cloud watch which will gather your logs on your machines that you have and you can then consolidate those logs and put them in a bucket and this is how Amazon will suggest that you do so that's one way and bucket storage is cheap so you can just store your data log data inside in a bucket separate from your boxes you keep destroying your boxes nobody cares about those boxes anymore because your logs are outside and you can do your log analysis separately by running elk stack against your bucket so you have a belt stacks whenever you want to analyze you you're in the mood of analysis so what you do you start your elk stack and connect it to your log storage your logs are stored in bucket so you connect to your bucket and then you do the analysis you're done with the analysis you shoot your elk stack in the foot the elk stack dies your logs are still there you got your analysis out that's what you wanted you studied you got the idea and you shoot your stack the stack is dead your the logs are stored and you got your result so you move on that's how I would use it in fact that's how we do this in this location we are actually doing analysis on the logs and this is a the two to three containers we will do the analysis for us and we can do all visits on bunch of analytics available that we can basically detect and do this is conceptually similar to Google Analytics by the way except that you have to pay for Google Analytics to Google this one is free I free as in an open source free not just zero dollar free so the difference between a zero dollar free and open source free you know what that means right I hope that that's clear a zero dollar free is just zero dollars it's like the that it's like free beer whereas open source free basically means liberate liberty so that's what I like about being open source is let me change whatever I want so this is pivot and they give you a container and it is liberating so it is freedom as opposed to zero dollars so let you choose customize and do whatever you feel like and control your destiny so that's what we are using in analysis so that's this back to our exercises yes can you type your question I cannot really cannot I decipher what you're saying for some reason okay so while you type I'll gather my thoughts and continue forward and so let me close a bunch of open windows here okay so I am ready with my next exercise and the question that you have is that she asks when a machine goes down how long how does the log gets moved every we answer that whatever how do we preserve it we get did that so step it's a two-step process question mark map docker volume to VM volume yes and then connect that storage to a bucket you don't connect you copy over your logs you have to physically copy them over to the bucket and the command to copy would be something like this it will look something like this let's see you will run your AWS command line interface so AWS CLI AWS s3 CP some file some log to a bucket say something like s3 colons slash slash my bucket something like this this is how you copy a log from your computer to the bucket so it will copy for that to happen you need to install that library which is the AWS CLI so it will be something like pip install AWS CLI and then you can install you can run line number five to copy your log some log to a bucket and it will copy so it will basically copy so you have to actively copy like lift forklift your log from here to there then it will copy you cannot just connect to something like that you have to actually actively copy and that is how you post or export your logs out from your box your container to something outside so this was a s3 example okay next idea it's a it's a different idea which is what I'm going to introduce in that idea is this concept of what this company used to be there used to be a company called to tomb it got acquired this new thing is called Docker cloud and let me surprise you by telling you that Docker cloud is not a cloud so what is it so it is a service that lets you manage and connect to any cloud you like and they have some built in connectors to connect to any cloud of your choice but some of them are predefined so let's go see in Docker cloud if you go and log in by the way account is free I think for some limitations are there but the account is free so you can use your Docker hub login and go to cloud.docker.com use the same login that will work so when you go here the first thing you will notice is let me refresh let me actually go to the front page so so I'm on the front page now and what do you see here is this idea on the left side called nodes and here you can either bring your own node like for example here it says bring your own node what does that mean it means any cloud even if it is not officially listed you can bring any cloud you like any cloud any node in anywhere in your garage in your grandmother's home or any just about anything just as long as you can connect to it SSH any cloud can be controlled by this service that's the idea it's a service orchestration method it's not a cloud it's called Docker cloud but it is not a cloud so you can bring a node if you like as long as the node is running Ubuntu or Debian or CentOS or Red Hat or Fedora it is okay so you have to have these operating systems you can bring your node in the management of Docker cloud otherwise you can create a node and you can choose to provide it a label this is my preferred name I used Chrome box for some reason that's my name I like to give my boxes and then choose a provider here you can choose Amazon digital ocean as your soft layer packet so five short list five names already provided and you can go to your settings and configure your connections to Amazon your connection to digital ocean your connection to Azure soft layer and packet so five cloud providers they gave you ready-made connections so I have connected some of them not all of them like I have not connected soft layer not connected Amazon but it's easy to connect all you need to do is go edit like somewhere yeah connect the provider and here you follow these instructions and you are able to connect to the cloud and what this Docker cloud will do is basically manage your cloud for you your cloud in terms of it will actually automatically create boxes put containers inside those boxes scale it out all these things you will do for you provided you add credentials so you connect the provider to click and then you follow these instructions that's the way to that's the way to deal with this cloud in digital ocean when you click this blue button it's already connected but when you do click it is asking you for digital ocean token so you give it the token and then you are connected to digital ocean cloud digital ocean by the way just today started a load balancer service so now you can get a load balancer from digital ocean it was not there until yesterday but now you have this idea of a load balancer you don't have any load balancers of course I don't you can create one and they will give you a ready-made load balancer managed for you that's a new thing back here in Docker cloud you can also connect with Microsoft and software and packet but this is not limited to only these services because you always have the option of bring your own node which is what I did here so I have a node connected already and I want to delete it to begin fresh so I will terminate that node it's a terminate so the node dies it is terminated as soon as it is terminated I will get a notification in slack that says you know your node died terminated so I have connected these things together so that I get notified as to what's happening back in in here this node terminated the actual machine that have I have created is sitting here I will also delete that so that I'll give you a complete perspective on what am I doing so this is a separate new cloud I've been playing with it I'm just deleting that machine right now and say you can go bye bye so the machine will die now what I want to do is with this Docker cloud now that I have nothing connected I have nothing under management I want to bring a node in management of this Docker cloud so I will create a machine and I again I can either go to this cloud which is this scaleway cloud or I can go to digital ocean either way I choose I get to choose what I want so I decide that I want to create a new server and I want to choose this server here VC 1s size it's a small size machine and I want to select Ubuntu's Daniel and off we go so we will just create the box and that box becomes ready what we need to do is in order for us to bring some random cloud in the control of Docker cloud is we need to connect to that newly created machine which has to be one of these operating systems and then run this command it's a custom command created for you just for you this one command will actually make a connection between your doctor clouds Docker cloud service and your machine that you might have in some random cloud location like this one for example so this machine is getting started is getting created as soon as it starts to run I will see a number one here and I will be able to hook up to it through an SSH method which is basically this machine so I'll type this and connect as soon as it is ready I should be able to connect to it it is not ready yet so I'll wait for it but until that time let me describe that I what I intend to do is connect to that box and run this one command then that box will be under control of Docker cloud then I can you know do so a lot of basically start using Docker cloud at that in that stage so from a from a picture perspective what we are doing is we are using this service called Docker cloud it is again not a cloud it is a service what it does though it allows you to connect to any cloud of your choice so for right now I am using scaleway cloud putting a new box in there and the box gets started and then I'll connect to it and then once I connect I will run this one custom command here custom command and then as soon as I run the command this connection completes and then it becomes under control of Docker cloud if I want to choose those specifically named clouds like Amazon digital ocean and packet and bunch of others then I have to provide credentials and then it will automatically manage them but I'm not choosing that route I'm using the custom route for our demonstration so we are not using predefined methods but using totally you know customized methods where we will choose our own machine out in some random cloud here for example and connect to that so what I will do now is see if I can connect to that my public keys outside not in this machine so I'll have to use a different terminal which is this terminal so I'll use this terminal instead and I will erase my known host file and get ready so here I will see whether my machine is able to connect now so SSH root at IP address see if it is ready yes it is ready so the machine got connected now what I need to do I'm connected to that new random cloud called scaleway in that location I need to run this unique command which is provided to me by the Docker cloud console and that command happens to be right here it is unique to every instance every your account will be a different command but it will pretty much similar with a different token this token will be different so you come here and run that command and let it run takes a minute and your machine will be commandeered literally it will take control of that box from a Docker cloud perspective will become a part of Docker cloud at that stage so it is doing that thing right now and as soon as it finishes it will report back to Docker cloud that I am ready for you service so then you can install containers inside that instance that you just hooked up that hooking up is happening right now what it is doing is running that sample code that it provided back to us and it is it tells us that you know Docker cloud agent installed successfully so it's a secret agent the Docker cloud agent is running on that target in scaleway cloud and as a consequence this machine is commandeered by Docker cloud I think commandeered is the right word I would say so you will soon report that this Docker cloud reports that if I refresh that a new node is getting deployed right now that's what's happening you can see that this new node has the same IP address that you see here in Docker cloud that you see here in scaleway cloud 130 147 same thing showed up here and that is deploying Docker cloud elements that is also the same that you see here in this machine if you just say if config you see the same IP address right here no it's not exposing from outside it's not showing you the internal but here you see it the outside IP address public IP address and so that's what we are looking at we don't need to actually operate on the SSH command line at all we can just exit and even in fact close the terminal window we don't need it anymore practically speaking and what I want to be able to do now is continue along in the exercise which we have defined in this location specifically here so I have a new exercise that I will paste a link for you here and what this says as you can see is in your virtual machine you have to install this element to manage and have the virtual machine operate using Docker cloud so that's a one-time setup pseudo pip install Docker cloud one time and then Docker login so that you know what you're dealing with I will do the same steps install Docker cloud on my machine somewhere here in the terminal or maybe I can do it there let me see if I can do it there correctly I think I should be able to yeah so I will what I will do is I'll go to the virtual machine now and run it right there inside the VM these commands so here is the virtual machine coming up and I'll unmaximize it put it on the left on the right half and close out these browser windows and open terminal and we should be able to now easily play with our Docker cloud yes so what we want to do is install the Docker cloud binary using pip and then Docker login which we have I think I have already Docker logged in so there is no reason for me to log in again you should log in by the way so remember to log in and then what I want to do is create a new application on the fly like very simple application we'll create one and load it on to Docker cloud which will effectively load it on the scaleway actual cloud this machine this machine is where it will have run this IP address this box that you see here in Paris is where it will actually run the application that I'm going to create right now so new application I'm going to create in a new folder so create that go in there and then open item in the item editor I will create a simple example using go language so this is the go programming language produced by Google and Docker is written in go by the way you probably know that and so what I'm doing right now is adjusting the screen a little bit so I can see the whole segment of code there's a very simple program written in go I'm creating a new file saving it there and as the file name happens to be main dot go that's the main program all it does is says hello world so very simple program and that's what it says hello world when you open a site that's all it says so simple short program may it should be main dot go not mail don't go so renaming it main dot go now that we have that file there I need to create a Docker file so I can put this program that I wrote inside a container and then send it to the Docker cloud pretty much you know that's what I will be demonstrating so what I'm doing is creating a new file called Docker file and putting in this content and there is a new Docker file that contains this and save it as Docker file save what this file contains let's read that it uses a ready-made image called go language the go lang image which is a library image provided by the go language people and then assigns a maintainer name adds this directory which has the dot reference here meaning this entire folder you put that in that location inside the container then you run the go program with the code that is provided in that folder which is this folder and then the command to run is that in that location and you expose this port number 80 so now we have a new Docker file which means we need to build it so we'll build it like this build it is basically Docker build dot with a tag dash t this dash t is a tag the tag I have chosen is my own name my name in Docker cloud is this for my company name in Docker cloud is that cloud genius so that's the name I have chosen for the company and the command complete command is Docker build dot with a tag reference your tag will be different paste so it will build that new image and it built it built so quickly because the go language image was already available locally I had done this exercise before this morning so that image was already available so it did not download it again and it assigned a maintainer added the folder run this and commands are specified and exposing a port and the container 37 echo 35 33 that that number is ready that image is ready now so when that image is ready I want to push it push it to where to Docker hub with that name that I have chosen for Docker hub so I push it it is going there I will go log into Docker hub and see what we have there and in my Docker login I have this user by that is me and this company that I have here called cloud genius and in there I search for the name called CG dash go and so there is that repository that I just pushed and so it was pushed a few seconds ago it says that now that I have pushed that new image the image is available in the Docker hub all I need to do now from here is I want to run this for real in a real cloud which is this one in that instance but using Docker cloud service so let me adjust the browser so you can see it a little better and then I will actually invoke the command provided here the very last one is to just run it and that's the command let's go read that command itself before we run it the command is slightly different from Docker commands but similar the similarity is you will notice but the command itself is the new command called Docker cloud command this is going to be creating a new service in the Docker cloud and we'll run it using this image that I have just uploaded calling it the name web and assigning port number 80 on the physical machine that we have in that cloud here in Paris mapping that port number 80 to the port 80 inside the image that I want to run with this source that I just pushed so what I want to do is invoke this command in my command line locally and what you will see is that it creates a new service in Docker cloud and pushes the code to that scale a cloud in Paris and gives me a ready-made website that says hello world because that's how it is written you saw the main.go program this is what is going to run in the Docker cloud through the Docker cloud in Paris that instance we have happens to be in Paris so we will now take this command and go the terminal and run it now before I execute I want to actually show you the view of Docker cloud what it shows is slightly different so you will notice a couple of things that machine is deployed by the way the cloud provider it doesn't have any idea what the cloud provider is because we never told it that all we said is this IP address and we ran the code it told us to run and we are running that code so it installed Docker and took control of the box the box is here in this cloud is running here in Paris and command here by this guy the nodes under management are only one and that is this node this number in this hash assigned now let's see what stacks we have that are actually active none of them are running right now so a bunch of things installed but none of them are running so this bunch of stacks are there already and they are not running as you can see services there are some services but they are none of them are running so I want to make sure that I don't have a service by the name web because that's the new service name as you can see here the service name will be web so I want to remove that name and delete that particular name so I'm deleting that terminate and just to cut any confusion with that naming of that web service I will rename my name of this web service to be something different like something different so you rename my service as something different that is the name of my service now it is going to be running in the Docker cloud as a service expose port number 80 on the physical machine to the port 80 inside the container call it something different and the machine image will be this image that I just created and pushed now when I run this you will see a new service pop up in this list none of them actually are running right now which is fine we leave them like this this is some other examples that I have been playing with but we are going to create a new service by the name something different that will show up here you will also see that there are containers actually there are none there is nothing no container running in any node there are nodes available only one which is deployed containers there are none services defined a lot of them but none of them running and stacks a lot of stacks available but none of them active there are other examples you don't have to worry about these things specifically because they're not related to what we are doing so we should focus on the services segment and we will see that a new service starts when I invoke that command this command creates a new service by the name something different users the image that I just pushed and let it go so as soon as I run it you will see a new service pop up here in this listing and also see a slack message coming to me in that slack chat say it says something different service created so service was created finished successfully the notification I received already here it came just now and then back in our back in our visual display of services let's refresh and what do we see now we see that something different is starting now what is this going to happen what is this going to do in the under the hood it is going to connect to the box that we have in Paris in here in that box because it has command here that box and then instantiate a new container called something different dash one and that container you can see that is running a service called something different it is using an image that I just pushed a few minutes ago the tag is cloud genius he go you will see that it is still starting meaning what is going on right now is that image that I pushed to Docker Hub is getting downloaded to the Paris location and then once that downloaded finished once that this image gets pushed out to the Paris location the service starts to run so you can see the service is running which means there will be another notification in slack chat that shows that something different service after it created it started so a new service started finished successfully start action finished successfully so we can now see our service status we can see that something different service is running and it is using this image and it is exposing an endpoint the endpoint exposed is this location so we'll just open that location in a browser and what do you see hello world so we complete a very simple thing just wrote hello world but in a convoluted way like really convoluted way and what we did it this way is to demonstrate the usage of constructing any service that you like in any bunch of nodes that you want to manage using Docker's Docker cloud as a service let's recap what we just did by the way this link that you have here you will go to dig and find out which you expect this is nothing unusual here but you will see when you go dig that it will be the same IP address that you expect to see but it is a generated domain name just for our use by Docker cloud and the IP address is that address which is what that Paris machine has it's the same IP address they gave us a friendly name something different dot some number dot service dot Docker app dot IO is a name generated for this IP address because that's what Docker cloud as a service did for us and you can go and match it with this location and the IP address for this box happens to be the same one 130 147 match it here there's the same box running the same thing except we did not go to the box directly to run a service we were orchestrating that service from Docker cloud actually we don't even go here we ran our service command from here Docker cloud service run some name of the service and some image that I want to use and map the ports according to what I want so now recap what did we do where is that whiteboard here so we began with we began with this idea of using a service called Docker cloud which is not a cloud but it lets you manage bunch of clouds C1 or say CL1 CL2 a variety of clouds that you have in mind except that gives you some ready-made connectors for these specific clouds and there is one more I forgetting its name I think it's IBM's IBM's cloud so those guys have a special treatment for others that do not have a special treatment you need to actually create boxes manually or some other way manually create a box in some cloud and then run this custom code that these guys provide to you you run that code here and here and as soon as you run it these boxes get commandeered by the Docker cloud if you are using any of these these cloud services then you don't even need to do this step at all all you will do here is ask for a node hey give me a node and it will just give you that node or give me a node here it will give you a node Docker in that cloud of your choice I did not show you that because it is trivial instead how do you do the difficult ways what I demonstrated just choose any cloud and run it manually to get it commandeered by the Docker cloud so that's the commandeering step so that you have some machine in your control at least one now that you have Docker cloud in control of one box somewhere happens to be in my example in Paris and that box runs that code that you had Docker cloud make you run so it is controlled by that cloud now in our virtual machine in our cloud genius workstation we wrote a new program in go language we wrote a Docker file and then build an image you push that image Docker hub this hub is a central service available from Docker so that images becomes available here that's where you pushed and then you ran a service when you run a service here you basically are communicating with Docker cloud and asking it to run a service that happens to use this image so it image when you refer to a service a service refers to an image available on Docker hub and you instantiate the service it will use the available machines you have in your control and pull the image down the image becomes locally available and then instantiate that service here and then expose port number 80 and map it to the port number 80 in the container and then anybody visits that site will see the program site visitor will see your program that you have provided here and that is used in the Docker file that you built and you pushed so it get pulled and you get the result so complex to think about it but once you have this in place what you now do is create more programs and put it in Docker file build them push them and run a service and do more of it in variety of different new applications that you build all you do is just repeat these steps you basically you become a new developer in effect you create new programs you create Docker file corresponding to that so you have an image for that and then you push you run and all of these things will run remotely not on your box there by the way you can run locally if you like but nobody wants to run it in a VM you want to run it on real cloud like this example I just showed you from this company called scaleway just a cloud provider just gives you boxes so you use that cloud command here in the control of Docker cloud and run a service if you are doing it through command line this is the approach if you want to run it visually totally visually you will go do something different you will not use the VM in that case you will like to use complete user friendly tools like a browser for example so what you will do is you go here in the browser and create a stack a stack is another YAML file and in there I'm going to edit the stack to illustrate a ready-made stack for you and this is this tag that I want to run so visually purely without using terminal what I'm going to do is describe you a new product that I have used in the past and it is an open source product this thing that I'm showing you is very similar to slack chat very very similar to what you already use here this slack chat interface and what is that thing though that's what I'm going to explain to you briefly so the code that you're looking at here which I will post in slack chat by the way is actually a YAML for what is it called rocket chat so I just posted that entire YAML for you so here it looks like and in there we are going to use another open source project called rocket chat it is a competitor to slack chat it is called rocket chat and it is open source and what you need to do is run backend services using Docker so you will run some service in some cloud running the rocket service and then you will have a local client application for rocket client that will hook up to the rocket service and you want to run this rocket service in the cloud you will have to define a stack the stack looks like the YAML I just posted and I have it locally here it has a bunch of containers one of them is line number one through 15 another one is 16 through 20 this second container is the Mongo container Mongo database the first container line number one through 15 is the Hubert container which is giving you interactivity and the third container is the chat container which is line number 21 through 32 and those lines have a reference to a connection back to the Mongo container in line number 23 you're providing a connector that connects to the Mongo container in line 16 so this stack if you just go back here in your browser and say Docker cloud can you please run me a chat service and so all you need to do is create a new stack like this stack and create and then paste the YAML that I just gave you in slack chat and then you get a new entry here like this rocket and all you can do is just say rocket container is not running so you just start it and you get to see that this service will invoke this new stack will get deployed did I click the button or not let's see rocket not running so start oh I have to tag it so it's actually generating error the reason that is generating an error is that I have in my YAML file tagged it it gave me four errors by the way it has failed failed failed because the rocket has no note to run on it it has to be told that you need to run on a box so in my example I need to give the name Chrome box that's the name I have chosen in the example so I need to assign a name called Chrome box to my machine here and I will like to say that you will also be known as Chrome box you dear box you will be known as Chrome box so now that I have assigned a tag for it it knows where to go and run it knows which box to go and target the stack so this stack is not running but now I will play it so it should now start and it is starting it knows where to go and run now that I have assigned a name Chrome box which I have used in the YAML file it knows where to go target that run so you can now see this starting the starting action will actually go and invoke these services called Mongo and Hubot and rocket chat so three services defined and at the end you will have a new endpoint here and this will be the back end of your rocket chat when I say back end what I mean is this service back end on the front end there will be an application that you will be using in your phone or your Mac there is an app for that called rocket chat and so I'm going to download that rocket chat I think I already have but there's an app for that called rocket so I'm looking up and rocket chat is available I'm going to open it and connect it with the server that I just created based on the endpoint that this rocket service is going to make it available for me so this rocket service endpoint will become available soon and as soon as that endpoint becomes available I need to provide that to my rocket client and then implement my own slack like service with a client application called rocket client that looks like this and it is not running apparently so opening up again let's let's go to the back end and see what that thing does so is it running now is still starting let me also finish that container which we have this something different I want to delete that service because I am not interested in that anymore so this something different service that I have I want to terminate it because that example we are done with right so I'm killing that service just to reduce the load on the box so just terminating the stack that we created is starting and as soon as this thing finishes we should be able to see a new service available at that endpoint which is what I need to provide to my rocket chat application in the client side in here where I need to provide my URL so I think this I need to I need to kill this app and install it again apparently it's not behaving properly so I will go to my applications folder look at my rocket chat app here and go delete it it is running so let me quit it and then delete it so I deleted that app now I'm going to install the client app again which is in the App Store I'll look up search for the client application called rocket chat and this is basically the client interface to the back end server which I am going to install so this is the client getting installed it will install on the back end we have this service getting ready is still starting I'm losing patience on this why should it take that long it should not take that long but if it if it takes longer I think we are already beyond time and so I will kill it and do it again but let's see if the client application opens if it doesn't open I'll break this exercise and proceed forward and at least explain to you what the concept is the concept that I'm trying to accomplish is that I will install a new app for the clients to use this is the client application yet you get from the App Store or the iTunes App Store one of those places you can get and you have this client connect to a back-end service that you just started using the stack that you uploaded in the browser and then you click the start button that's all you need to do to have a new service ready in a bunch of containers three specifically you have a huebot container you have a Mongo container which runs the database and the rocket chat container which runs the application on the back end rocket chat and these three containers together will provide the back-end support for this app to run correctly and that is what I'm expecting to see if this thing is running now yes okay so we have an endpoint we'll grab that endpoint right there and then go to our rocket chat engine see if that rocket chat behaves properly apparently no it doesn't want to behave properly it is not showing me anything okay add new server and here we add the enter the URL for the server we just had so notice what I'm doing is grabbing the URL from the service discovery endpoint on Docker cloud and providing that URL to the rocket chat client application and then clicking the connect button that's what I will do essentially so I grab the URL from the service endpoint on the rocket chat which is running now and providing it here and say connect so it is trying to connect validating am I typing it wrong or what no let me see the URL copy what what's the error here okay the rocket's action started successfully and the URL endpoint I have from the website is let's see the service service endpoint I should grab service I have running is the rocket chat service and that endpoint is rocket chat endpoint is this endpoint so we grab that copy and it should connect to this location no it doesn't want to connect no okay I'm not debugging it further but that the idea I have I have seen it success succeed earlier this morning I tried it and it has connected successfully maybe it's a transient issue that I had to figure out but bottom line remains that you can create new services that you can use and configure visually using Docker cloud and have the front-end client facing applications like this connect to your back-end service and create new scenarios for you users to use with absolute ease once you have it running one time scaling it is easy all you need to go do is provide more nodes like here and bring more nodes if you like if you have a properly configured recommended services available you can ask it to scale the number of nodes automatically it will grow and that's the user scenario of Docker cloud so what we did today will we'll continue to do a little bit different going forward and we'll put our perspective of an end user a end user company that uses a cloud as a solution for their business problem so we'll slightly put our business hats on and think of it next time going forward from a application orchestration as well as design blueprint perspective what does it take to construct a complete solution in the context of using a cloud service focusing primarily on architectural discussions we'll talk through case studies and examples as we go forward and in the mean as we discussed I will throw in examples of how we will construct something that is of relevance to a discussion so we'll bring those examples live again interactive pretty much like this but our discussion will focus slightly differently on use cases in terms of business cases of actual companies trying to attempt in to use cloud how do they use it how it is structured for a variety of use case scenarios and that is what we will do starting our next time which is where we will begin our architecture segment and that happens on 23rd of February at 4 o'clock Pacific time 7 o'clock Eastern time and I will see you at that time thank you guys thank you you're welcome this rocket this rocket chat did not work but I will I will fix it right now if you are interested in watching all right bye bye