you you you you you you you you you you you you you you you you you you you you you you you you you you you you you you Hello. How are you doing? Good. Good. Good. Good. Good. Good. How are you doing? Hey, guys. Hello. How are you doing? Good evening. I'm going to try something different today. I don't know if it succeeds or not, but we'll figure it out. What I have in mind is using an iPad to run the class, which is kind of a scary thing to do, but I might give it a shot, but just for a few minutes. If it doesn't work out, we have this other option already. But there is a new feature set that has come into Zoom, which allows me to exclusively use iPad and even share my screen. That is something I wanted to try. So maybe today is a good day. I don't know. We'll find out. I've used my iPad for my video and my watching the class before. So right. Right. That is OK. But what I'm doing is as a presenter, can I project my iPad screen to you and can you see it properly? Can you make sense out of it? And is it actually any more productive than regularly using a full-fledged desktop with a keyboard or a track or a pad like this? That's the question I have for myself. So this thing that I use is the extra device that I need to usually use so I can write on the screen and all that. But I'm a nerd. You probably know that. I don't need to tell you. But I like to use writing instruments of a variety of different types. And so this is an evolution of technology where I can potentially start Zoom right here on that Zoom thing on the iPad. Let me see if I can actually let me project. I know what you want to do. You want to be out by the beach in Hawaii. Exactly. Exactly. You read my mind. That's exactly what I want to do. And so what I'm doing is how do you read my mind? I mean, that's amazing. Great minds think alike because I want to do the same thing. I know. Pull deck, beautiful sunset, doing a class. Why not? Why not? Exactly. That is my goal. So what I'm what I'm going to give now right now is a shot at see if I can actually project my Zoom desktop through the iPad. So let's go, let it go, let it broadcast and let's see what happens. So it should three, two, one, there should be, yeah, it should be there. You should see my desktop coming up in the other other window. There should be another nilesh and that nilesh should be showing you some kind of a display on the on the iPad. Is that visible to you right here? Yes. Yes. And is this legible? Can I can I write something here and is it legible to you? We cannot read the text, the bottom, but everything else is visible. Yep, that's better. Okay. So yeah, this gives me the ability to like zoom at comfort so I can go use my fingers to zoom and just circle around and you know, maybe talk about this, focus on that. That is the better pitch, better part of this experience for me at least, but I don't know if it actually works out for you. So we'll give this a shot. So let's let's get ready. So I think Rob and I discussed briefly and he had something of interest that I think I would like you to bring it up, Rob. So if you can please mention what you had in mind and I'll stop projecting and just focus on what you had in mind. So why don't you discuss that? Okay. So before I share my screen, I just want to set the stage. A friend of mine who I helped mentor, he's a young man, is working on a project for DevOps, developers and operations. And we would like to show it to you guys and get your feedback. Let us know. It culminates really what we've been talking about in the class and puts a UI, a visual aspect on top of what we've been. Hey Rob. I think we lost your sound, Rob. I think we lost him. Yeah. No. Okay. I'm back. I think I keep losing because I have the other machine. Let me go disconnect the other machine and then I'll come right back. Okay. So what Rob was talking about, he and his friend that he mentors has developed something interesting. So he wants to show it to us. And briefly, I think it's kind of interesting in terms of how it does is that he has, I think I'll let him describe what he would like to say about what the project is. And so he'll show it to us. And so, yeah, let me go ahead and share my screen. Okay. And I hope it's not going to take long, like maybe a few minutes and let me know if you can see my screen. Yes. Okay. So what we have here is on the right-hand side, we have the application. On the left-hand side, we actually have a system that is running and the application, the server is actually running right here and it's a Node.js server. So let me go in and do a Docker PS and show that we have two different containers running. We have an Alpine node, which Alpine is a very small container, Linux container, running the actual data server, the server, and then we have a Mongo database entry point. So let me go ahead and log in. As you can see, it's called IcePanel. And what we have is we have engines, we have your projects, we have plugins. And so the plugins are what wrap the different containers that we allow other developers and other people to be able to add their containers and their applications specifically to this. So I will show you how each of these containers work and accept Docker. Docker is kind of the underlying plugin for allowing to manage Docker images. One of the cool things is we have our own IDE and development for those plugins in here. So if you come into Apache, you can see that we're using our model view controller state. So the plugin main TS, we're using TypeScript, is your model on controller. And so it's a very short amount of code to do this plugin. And then your view allows you to actually input your view. So this is the individual component here. So let me make sure that I change this IP address to my current IP address, which is on this machine, I'm using this 3.55. So actually, what I'm going to do is I'm going to change this to just make it local host. And then I'm going to and what's really interesting about this is I just made a change and I can install this Apache. Now that Apache is now updated over here automatically. I just did an install. So let's go ahead and start an engine over here on this machine. The way you do that is you generate your engine. We have a couple different helper functions in here for count and dates and things like that for expiration. But I'm going to call this one rob demo one so it shows up, I'm going to generate the command, I couldn't paste this command. So I have a little copy over here, I'm going to bring it over here. And I'm going to paste it. And so this is just the the running of the slave machine. And you should see it come up over here. So that's the slave. That's a one to one relationship between your Docker slave, which is a proxy on a single machine. So you can have it one to one to any of your VMs in the cloud. So we would be able to you would be able to start up your cloud run that that command and start that proxy server on that cloud. Now how do we use that we come over here and we create a project. So I'm going to name that project Rob demo one if I can type properly. And so now we have this demo. Now how do we how do we use this so we have this selection tool up here we can add a network we can add an engine let's add that engine that we just created. So the engine is an is a container that contains other plugins. So we create a first engine we double click on that and we select the engine that we just created notice that we give an IP and a port that that engines running on. Now we have a couple of different things over here that allow us. So we have two of the patches. That's something that we'll have to fix. But the hardware we have a volume. So we remember what no no less was teaching us about volumes. Well we can actually create that volume here. And then I give it a volume name. I'm going to call it data. What's interesting about this is this actually creates a volume over here. So if we were to come over here and do a Docker volume LS. Let me spell volume correctly. You'll see master data and master config. So these are these are two volumes that this is created in that our server is using. And then also if we were to create a terminal session. So I want to create this Debian session because I want to demonstrate the terminal session. So we do terminal. And now if I come back over here and I do Docker PS you'll see that we have now a Debian and we have our slave and we have our two servers. Now let's say I want to connect these this data. Now I can connect the data up to restarted the server. Now I have another plugin is called a tools plugin. So each of these plugins can actually have tools that you add to it. So we're going to add a terminal tool to this. And I'm going to go into that terminal and I like using LL. So I'm going to use I'm going to alias that to LS minus LA just so it's easier for you guys to see what we're doing. So notice that the data directory right here has been created. So if I go into see CD data, notice that there's nothing here yet. Haven't really connected anything to that. So let's go ahead and change that. Let's go ahead and come over here to the Apache server. Now this is going to be interesting because I haven't I don't know which Apache server I'm running here. But let's just use web server and say OK. Now the Apache server I can also have a terminal. OK so the terminal will allow me to have the Apache server and I can actually view the site. Let me do 80 80 on this and say save. And then I'm going to view this site and it's it's still oh I haven't restarted it yet. I have to let it restart. Let me restart it. OK and you say 80 80 it works. OK so now we we want to do is we want to save our data we want to be able to control the data on that. So what we do is we'll connect the web server up to data. And it's downloading some packages and it's doing its work. Now it's connected. So I want to be able to change things on that. So let me go over here add the files object to this and notice that if I come back over here and I go to the terminal and now I look at data notice that index HTML is there. If I cat it it's the it's work index. So we just have a running web server within minutes I actually could do it in seconds but I'm talking so it's minutes. So let's go back and let's take a look at the files. Let's change that file. So one easy way to do that is just to upload a new index HTML file. So that is pretty much what it did. I came down here. I uploaded. I selected the index HTML file and it automatically opens it because that dialogue automatically does that with this KDE. So now notice this change. Now if I come over here and I now it works. And if I come back over to the web server and I look at the terminal you can see the terminal output. If I come back over to terminal data I can see that the index has has changed. If I cat the index HTML it's changed. So within minutes I now have a running jump box terminal could be your jump box your data volume and your web server. That's it. Any questions. I think this is good and questions are of course welcome. Well that was really cool Rob I'll tell you that. It's a real world kind of exam. That's cool. Yeah. He simplified Rob has simplified a whole lot of what I call terminal work and just bring up a particular thing that you would like to have as the project defines the concept of a project and an engine and a plug-in and you can construct your own place to drag and drop and put things together without having to go through the grunt. I think that's a good value. Is that on GitHub Rob? It is not on GitHub it is not a public product yet. We're still doing quite a bit of work on it. There's a lot of there's a lot of rough edges. So he's he's still programming a lot of this but he wanted to get feedback to ask are we hitting are we even hitting hitting are we even in the ballpark of something useful for an ops. Yeah. Don't say you guys saw yourself short. That's really good stuff. Oh thank you. I mean it's you know the really fun thing about it is that it's you know GUIs are great because they're helpful for learning and conceptualizing you know maybe you can't get exactly the same control you get with you know typing letters in to a terminal and a complex YAML files whatever. But you know a lot of what we do is repetitive and sometimes it's fun to experiment with things quickly. So that's where I see this coming in. Oh good. Okay. Experimentation. Yeah. Or I don't know. I mean I'm not saying that you couldn't put it in production maybe you know you could find a way to do like a testing pipeline with this or something like that. Yeah. Yeah. That would be good. Good to do. Yeah. Integration with some kind of a CI CD would be nice when think of. Yeah. So if you guys you know that's that's where this developer tools comes in. If you guys could create a plugin to do whatever you want it to do we're just providing the platform. I see Minecraft in there. Yeah that was one of the that's one of the ideas that we had was game servers are really popular and stuff like that. So really looking at taking you know for 13 14 15 year olds and they want to play game servers. Right. There's a way to manage that right in the cloud Rob I missed the first part of that where where are you developing this. No it is a young young man that I help mentor in software development over in the UK. He's the developer. He just wanted me to get some feedback from real life dev opera developers and operations engineers that have real experience with these types of tools and wanted to know what you guys thought. Yeah. Cool. Yeah. I think it's good. Okay. Party parrot. Give him the party parrot. The pony. Oh the party parrot. Yes. Yes. Okay. Okay. I'm going to log off. If there's is there any other questions did you guys want to if there wasn't I'll go ahead and stop sharing and then I'll hand it back over to Nilesh. Okay. Great. Thank you very much. I appreciate the time. We do. Let me see if I can project and if this iPad thing causes too much trouble please tell me. I'll switch back to the old ways of doing things and I am projecting about to project now and here we go. So my screen should be coming up in 3 2 1 there we go and it should pop up and what I'm doing here is focusing on our architectural diagrams and I have a lot of them and I think what I will do is I'll paste links to these things on to Slack chat as we go along or maybe if Travis you have them can you dump them in Slack if you have them handy if not I have them here and I'll dump them but let me just focus on the ideas first. So in here if you find them there are lots of these pictures that you will find on the internet and this is just a small subset collection that I seem to like some of them. So I have picked and chosen these guys for our discussion but I will cover other things also we are not limiting ourselves to just these but I want to begin somewhere. So this is our beginning point. One of the things that I would like to begin with is to understand by the way these pictures as you see are most all of them almost all of them are Amazon pictures. Amazon Web Services are the leaders in this domain in cloud computing as such so you may see that they have been doing this business for I think about close to nine or so years. So they have the first mover advantage they prepared their own architecture center and then they created all these illustrations as to what does it take to build something out and they first began with creating this picture which is kind of sort of a representation of how Amazon Web Services thinks retail e-commerce websites should be constructed or in other words how is amazon.com constructed. So that's an illustration of what they think this picture is an illustration of what they think a typical retail service should be functioned and designed and laid out. They have done this work actually in three parts. This piece of puzzle is the website front end part. There are two more pictures to go along with this and those two are the marketing pieces and recommendations engine and the third one being checkout. So the actual purchase access and the shopping cart and other details regarding so the it's a three part picture by the way and so what I would like to now do is begin discussing this from the point of view of what Amazon thinks or other Amazon Web Services things is how a retail e-commerce operation should be set up in the cloud from their perspective. So that's their recommendations most of these guys are recommendations coming from Amazon. The recommendations keep evolving the good thing about this thing is that you know if you look at the Amazon's retail website which is what you're looking at here and I occasionally go there to buy stuff but that's what I'm talking about in terms of what Amazon thinks is should be the implementation of a retail website. Now let's go see other things AWS architecture is something that I would like you to go to this location called architecture center and this is the place on their website which is where they host a whole bunch of information whole host of information and it has evolved quite a bit over the years and I have seen its evolution. And here you will find interesting things for example you know a variety of AWS solutions like here in the middle section AWS solutions if you go there you will find solution sets by industry or solution sets by application type. So if you're looking at a particular type of application they will give you what their recommendations are in terms of how a particular application should be constructed or set up in the context of yes Amazon's cloud because this is Amazon focused now you can look at by industry and so you can go and understand what would it take to you know solve the problem of you know your enterprise IT for example and what problems enterprise IT have and you know what are the typical use cases and how do you go about implementing those things in the context of Amazon's cloud. So that is this idea which I'm talking about is the architecture center AWS architecture center that is a keyword I think you should go and search on Google and that is a good source of information for general purpose reference material as to what Amazon Web Services thinks is how businesses should be doing or rather how companies should use Amazon's cloud. That's how they're it's basically a marketing website but with a lot of good information. So I actually take this I consume it but of course I take it with a pinch of salt sometimes they put in a little bit of spin and make it believe make it make it appear that it's possible to do these things only in Amazon's cloud. That may not be the case by the way and so what has happened over the years is just like this AWS application architecture center other companies have caught up for example here is Microsoft Azure architecture is there it is. So this is the set of solutions that basically in competitive response to Azure sorry from Azure to Amazon is this site where they have also done similar effort and their pictures and illustrations and architectural diagrams all talk towards addressing the need of either a given industry or a type of application or how you would do things if you were doing those things in the Azure cloud. So most of these pictures again coming from Amazon Microsoft are designed in a fashion that will make the reader believe that this is possible only in Microsoft's Azure cloud and the last person or rather the last company which has also come along as you would expect to do the same thing is the Google cloud architecture website. So Google cloud architecture also have no architecture there it is so that should bring up the right location I think oh they have too much too much on certification yeah that's not none of these links are right so I'll get the link for you but there is a link in which they describe how things should be done in the context of Google cloud and so it is focused on marketing spin towards getting a solution getting a product getting a solution to a problem as implemented in these clouds Amazon Microsoft and Google so that their perspective on how to do things what I would like to do today is remove that piece that is marketing in here and give you the actual story about how you can do it in a choice location of your choice like how would you do these things where you think is the right place and how would you go about making that decision what to take here what to keep what to retain what to maybe take it with a pinch of salt how do you go about thinking these things that's what I would like to focus on so we have three three big companies giving their recommendations trying to push their solutions and it's okay I mean let them push but we don't need to seek the whole thing we need to take we can choose very judiciously and methodically is what exactly makes sense for our case in our business context for our customer our employer and go forward with that so as a first example discussion I would like to have is again back to the concept of e-commerce websites how do we construct a classic e-commerce and amazon.com itself is a e-commerce site as you know already and so in this picture I think let me just dump these pictures out to you or let me see if Travis has already done it I don't know we have it already I don't really have many okay okay hold on let me then grab it and I just start dumping them so you have access to it otherwise all these pictures on the AWS website the problem is it is kind of hard to find oh I was able to find them pretty quick then then dump it please please dump these dump links if you're able to find them that's awesome I was not able to find I tried I could not find them like for example the e-commerce ones there are three pictures like that if you find them please post a link in slack chat thank you and let's go discuss this thing so from the point of view of a customer visiting an e-commerce store there are basic things that are expected like you know what do we expect a customer who comes on a website lands on a site typically looks for something looks for a product or a service so there has to be a list of some sorts that contains a product or a service so a list of things could be list of product list of services list of something you know basically this list is stored in a catalog of some sorts the product catalog and when it comes to cataloging and it's a large collection of things that you will find you will need to have some kind of a category you know category categorical listing so you should be able to basically do search and sort and go identify a particular pinpoint a product that you know I think you know these things already you know if you if you if you compare what I'm talking about with the actual Amazon site it will look something like let's go to Amazon and find what I'm talking about is the product category so you see it right there here in the all section all departments and Alexa skills and this and that and appliances and baby and you know clothing and courses they have courses these days huh interesting I don't know what courses are you selling here but apparently they are selling some courses and so here is the access to the great or the great courses course the creative blog the creative live edX courses interesting edX is now on Amazon okay as a introduction to Linux I mean they're selling a course from that for ninety nine dollars from edX so that's a catalog that you looked at here so you go here on and you select the department of your interest and then you have a way to sort that further you can select by price select by reviews and other types like books and courses related items so that's the concept that I'm describing briefly basically about about this picture with my picture here is this categorical listing of items products or services of a variety of nature is what the customer is looking for typically and so from point of view of the customer who is hitting some kind of a service busy with a website for e-commerce they will look up the DNS query the first thing and then hit some kind of a front end and that front end typically is where is the front end here this is the checkout application am I on the right site I think okay I think I need to go back to that there are some pieces missing in the iPad discussion that I don't like it here so that is something that I'm abandoning my iPad effort and going back to my usual way of doing things this did not like my did not meet my condition my my taste of doing things does not fit with this I'm stopping my screen sharing here and going back to the old ways and here it comes so the web front end that you look at here is what I was discussing and I'm going to project it from this computer which is what I'm projecting right now off it goes share my screen and there it is projecting the same thing that I think Rob just pasted and that is the picture I'm talking about so the point here is if you look at from the perspective of an e-commerce website there are a couple of things that we need to have in hand for it to succeed from an end user's perspective and if you look at that perspective from the point of view of let me just get let me get my bearing straight I think I just change of screen did not did not work out very well so let me get it right okay where did it go e-commerce checkout service e-commerce website and e-commerce marketing okay so the the three things that I'm really talking about why is the font size not correct and this should be adjusted okay so from the first picture perspective the front end perspective the web front end perspective what we need to be able to do is have a place where we actually store the catalog and that catalog is what we just discussed briefly in terms of you know product listing the categories the tags pricing availability ratings things of that nature and that should allow the customer here to be able to look at the choice of the product that they're looking that they are pinpointing at and an internal search engine that actually cat you know helps them search a particular item within the product catalog which is they are showing as stored in a database of this database doesn't need to be dynamo DB it can be any database of your choice that you like so there are lots of choices of databases available you need to select what your application actually is able to handle the best and use your choice of databases your choice of search engines so the typical choices in the open source domains will be elastic search for a search engine elastic search as a product available in the open source that is typically deployed together with some database like MongoDB or maybe Postgres SQL Postgres or Maria MySQL those guys that's the typical deployment you will see in most cases which are using open source products and it is completely up to you to select the data store once you have a data store you need to have a method of letting the customer search for a particular pinpointed item I mean that one item that I'm interested in I want to grab that item out and that is what I want to be able to have this e-commerce application present to the end user customer as when they look for a particular item from a search perspective and so that that entire experience of looking at a particular item so specifically for example if you open a browser and go to amazon.com that's what we are really doing I pinpoint let's say what is that some product so I'm going to buy maybe this quick end product how about that so I want to go do my taxes so I select that product and I'll I am focusing on that particular item this item has a unique index number that index number in the case of Amazon is called it's called ASIN right there you can see it here this number is the index number in the catalog ASIN is the Amazon specific index number or item number and that number is unique across the entire e-commerce site so this ASIN number is what you will see identifies that one item entry that I am interested in currently which happens to be this tax software that I was looking at and searching so the search engine will help me bring it up that item upfront and present to me through the e-commerce application that I am looking to looking to explore and maybe potentially buy maybe I want to do another thing which is find out what other people may be doing instead of buying this product the one that I have in mind which is quick and deluxe 2018 maybe there are certain other products that suit my need better so Amazon will or rather the e-commerce application on the back end will already understand that the customer this time the visitor this time is interested in potentially interested in solving the tax problem which is to be able to file the taxes before the deadline if that's the idea for the customer the customer really trying to actually file taxes right so the depending on that situation that Amazon or the e-commerce site needs to understand that they need to be able to be presented with other possible choices or ways to solve the same problem and there is one thing that I would like to point out here which is a important thing to remember is when people go to say Home Depot Home Depot and they buy a drill what are they really buying is not a drill and what are they really paying for is not really a drill but a hole in the wall a hole in something that's what they're really paying money for so they buy a drill they buy bits and other things to go along with from Home Depot what they're really trying to pay for is this hole in the wall that will actually solve their problem of maybe hanging some scenery a beautiful picture or something like I don't know what they're doing but that's the goal that these customers have so you need to understand this underlying logic under the hood like why are people paying money why are people buying a drill what is that thing that they're trying to solve so that is the hard that's a hard problem to understand from an e-commerce application perspective but that is what Amazon or rather this website will attempt to solve and they will attempt to suggest alternatives available right there so here there are this is I think advertising so this segment is advertising segment but here is the other idea what other items do customers buy after viewing this item they're trying to give you a hint as to what alternatives are to this product which might actually solve your problem maybe in a different fashion or maybe better at a lower price or something else and they give you this recommendation set which is customers who view this item also view these other items apparently it's all full of the same company but there should be something else other than quicken but apparently not I think quicken is the king of the jungle here looks like it because I don't see anything else people who use quicken apparently only by quicken there is other products available that they're not showing it for some reason they should but they don't so this idea of showing a recommendation like I mentioned here is when you look for a product you should be given a set of recommendations to go along its recommendation is to be run as a separate service to go along with the particular item that you search for from the catalog that you need to actually generate that recommendation through a separate entity which is here which is the third picture that you see in that list that we have is the third picture which will actually generate a recommendation and also present that recommendation right to the user in form of either a separate frame or within the same page that the customer is looking at and so the customer not only looks at this particular product that they're interested in but also given a list of things that they may also be interested as a recommendation you're to generate that recommendation almost instantaneously and that is what this piece of puzzle which is the third picture in this part three picture that you will look at as we go along as we as we describe this now in here you will see that there is the elastic cache item which is basically storing catalog and cache and transient session store what this elastic cache does is basically it is a in memory storage or a cache of whatever queries have happened in the past so that whatever whatever transactions either this customer or other customers that have had in the past in the recent past with the data store this store that is kept in a memory location using this particular product available from Amazon if you look at that particular product it is not unique to Amazon it might sound like unique to Amazon but it is not so if you go to elastic cache that is the product from Amazon and so if you go and click on that link this is the product that they're talking about if you go under the hood and actually go sign in you will find that these are this elastic cache is actually comprises of two products both of them are open source and so those two products happen to be Redis and memcached so if you go to elastic cache which is in memory cache and you try to get started with it it will actually give you that option whether you want a Redis cache which is a in memory cache available from this location Redis.io and running this is as simple as running Docker Redis so there's a ready-made image available you can just go Docker run Redis and Docker pull Redis Docker run Redis that's how it you can stand it up very very easily effortlessly and it just does its work totally totally no issues that's one implementation of the same thing under the hood the Redis implementation you can also select this checkbox to have it cluster enabled so you will you will see that this thing is actually structured like a multiple machine large memory machines cluster together to form a big Redis cache the idea behind this product Redis or elastic cache cluster or also implemented in another open source product called memcached which is the second choice also does very very similar things but it's up to you as to what your application needs to select between Redis and memcached there are two choices available in the elastic cache cluster and memcached also happens to be memcached is a product available in the open source right here on this location which is very very similar to Redis and it is also available on Docker and so you can just say you know Docker run memcached or something like that to that effect what these things actually do under the hood is let's say ec2 machine type large memory so if you look at this machine type here instance types large memory instances so there are memory optimized instances if you look at here x1e is one type and x1 is another and there are other ones which are I don't know if rtype is also memory optimized or not I think it is rtype r4 is also memory optimized so these these three or four different types which are memory optimized machines are specifically focused on very high amount of memory you can see that the number of memory 3904 GB memory the disk may be less but the memory size is huge that's structurally how it is defined as optimized for memory consumption so these are the underlying machines that you would typically deploy for a large scale deployment that requires a design which needs to cache cache what cache queries that this search engine is actually caching from the store of the data store or the catalog or product catalog whatever you have in the product catalog people come people come and search they use elastic search and that that query that you are running from your from the query from the customer's point of view it needs to be cached simply and this is a necessary thing to do because rather if I say can you tell me why is it important to cache those queries that people come and search for products why is it necessary to cache that thing in a large high memory high memory cluster of machines running redis or memcached why is it necessary to cache these queries that you do here in the store and keep them in a a session in a transient store in a memory store basically high memory why is it necessary anybody so this is basically like oracle pinned queries right yes yes that's one good example beautiful the core idea here is that you don't want to bother what kind of query so oracle allows you to pin a query along with the results into memory performance yeah yeah it's speed because disks are slower than memory yes so this is basically slow this is disk and this is memory so you're talking about high memory machines running redis running mcached or in amazon's language running elastic cache and you don't want to hit the disk too much you maybe just hit the disk as only when needed and so that is the idea of not hitting the disk over and over if you have a query that is already executed at least once by a given customer that he sent fast you cache it and then you use your next time the query will be served directly from the memory as opposed to from a result from a from a database so you're doing things so the first time query has to run against the database right yeah first time it has to go to the database no doubt about it okay so it's not like it's sucking the data out of the database and storing it in the no not by itself that doesn't make sense you know because there are there is you know petabytes of data here there is gigabytes of data here so it doesn't make sense to fit it won't fit okay so you pull the queries that are first executed somebody executed like you know 10 days ago five days ago maybe two two seconds ago something like that you query that and you know copy that query which is the result right here in the transient session store which is in gigs of memory so you saw like some gigabytes of memory right here right how much 3904 it is still gigabyte not terabytes disks are typically in terabytes and this database dynamo is a big is a is a database I mean come on it's a disk so it won't fit in the memory by the way the whole thing it may not but certain applications like SAP HANA as you will see as an example for HANA in a future example you will see that this entire processing of the SAP pipeline especially in the HANA product is done in memory not in elastic cache but in a high memory machine not in this exact product but that's another way of thinking about what needs to be stored in memory versus in memory versus on the disk you don't want to hit the disk too much ideally you want to not hit the disk at all but that's not technically possible so you minimize minimize the hits to the disk and serve from here as opposed to from here and the search engine will actually act you do the query part and get the results and results will be stored here and then sometimes things change and what have the what if the product availability changes the product price changes some things about it change then some other application has to actually update the catalog and when they update the catalog with a new product or a change of the product or change the product's picture product detail or price or availability or something else that product data stored in this database has to change when that thing changes you need to remember to invalidate this query corresponding to that product this has to be invalidated otherwise the customer will receive all data which is not what you want so anytime you have a cached session you want to delete the transients this transient session that are stored here you want to kill them out that's something to remember now from a usage perspective of an application that actually uses cache so as you look at this picture there are tons and tons of caching going along every step of the way so let's go see the very foundation here is your disk which is inside the disk there is some memory in the in the disk itself has some cache if you notice this some disks will mention that they have cache in there so there is caching going on between the disk within the body of the disk itself and the the ssd portion and the cache portion of the disk that's one level of caching right in the disk itself now comes the data store and the search engine will search and query and put the query results back in in the in memory cache of some sorts there's the second database level queries in oracle's language they call it pinned queries stored sitting right here and that will be served when a request for the same product from a different customer comes along and this customer asks for the same item that query will come from here that's the second level of cache right here this is the database cache let's see further along along the way how many caches are involved in the picture so here is this e-commerce application that application is typically some kind of a you know smart application that actually is able to help you search help you present the resulted picture that looks like where is that like here for example this is a result of an application generated html it painted the picture like this now what i want to talk about next is that entire application has to send this html out which requires some kind of a web server like apache or nginx so that apache or nginx needs to also identify that the html constructed by this application that looks like this picture for example which has sorry this screen for example which has this picture included here as you can see that there is a image of a quick end product there are some details about it some title and some html and some javascript and some css included in this entire page so that component of html css and javascript some of it is repeated over and over again for example for different products there are certain pieces of here that are repeated and let's go see what is repeated so we'll go here and let's go see just some other product so i'll go to say you know today's deals and we'll find out there is this product called whatever this thing is laptop yeah so see the the end and the deal is not available apparently for some reason but whatever so this this picture of the logo amazon prime logo so you see that image here the the section of this the the black colored bar on top this segment the the footer portions this segment back to top and all the all the way to the bottom this segment the entire bottom portion is repeated on every single page the certain components of this page are repeated over and over and certain others have not so what does that translate to from the application's perspective here is that it can identify certain constructs of the html css and javascript and make sure that those pieces that are repeated over and over in every page are not asked by the customer over and over again so what do they do or rather what should you do is to make sure that those components should be sticking here sticking here stick right there in the browser of the user i don't know did we discuss this idea of browser level caching before or not did we i don't remember it so let's go pinpoint all the possible places where people can cache and i will i will begin from the company on one end and on the other end this poor user the user has a browser typically most cases if this is repetitive please tell me i don't remember having discussed this but i think this is important to understand the caching along the way from a from a source to the destination so from source to destination there are multiple levels of caching and and let's go divide that into two parts so the company side which is the sending part and the user part is the receiving side so they receive content right so let's go see here on the database aspect so there is a database here there is typically some kind of a disk the disk typically has some cache this database is typically the database queries are stored in some kind of a redis like cache that is another caching so that repeated queries in in oracles language they will call it pinned queries repeated queries over and overdone that are stored in some kind of a redis or memcached type location in addition to this this company that is serving a page needs to also produce some kind of an imagery some jpg or some png files some css some javascript some html has to be constructed and that will constitute your web page that the customer looks at like this for example so there is this item and it has some detail like this and some pricing and all that so the the idea here is this picture this png picture the css some of it is most of the time reused over and over not the picture part but some css in javascript is repeatedly used the same css applies to the entire site sometimes the whole styling of the site the whole set of javascript files that this site is currently serving to the user those files css and javascripts are typically given out to the browser at one time and they just stick right there they stick in the browser cache now before they reach the browser there are other places that you may already know this is called the content delivery network which is another cache and the cdn as you know already you have done exercises with the idea here is it caches it caches until there's a user who needs something that they will actually go and ask for the origin hey give me the thing that the user wants and they will and the cdn will cache it locally in the cdn location and when the user asks for it another user asks for it second or third user they will give that content directly from the cache as opposed to going all the way to the source of the javascript file or source of the css file or source of a particular picture in fact if you look at this jpg in any jpg or the easiest way to look at is actually in this site so i can actually give you examples which you see literally this guy if you'll copy the image address is coming from a different location it is not the cloud genie location it is not the cloud genius website see that this is a c dot some other location it's a cdn location that's what i'm talking about this picture came from cdn even though the actual origin is this location b dot a dot cloud genius that's where it's coming from but to me here i see it is coming from c dot cloud genie yes so that's the idea of cdn caching even before that there are there is one more thing that i want to talk about the we talked about database caching here db caching in your redis or memcached we talked about these things you want to be able to compress minify or in in other words of compress the idea of ugly file is also used sometimes ugly file to make it look ugly to a human so humans don't find to find it easy to read it is still possible to read but it is just ugly to look at and i'll tell you what uglyfication means and uglyfication reduces the size of the item minification is is also reduce the size of the item also reduces the size of the item so reduce the size of the javascript css files and there is also another idea which is called concatenation concat concatenation basically means put multiple files together if possible so instead of sending one file at a time so this javascript and that javascript and this css and that css you know too many queries going straight to the browser the end user's point of view will slow down the site delivery so from a performance point of view you're looking at multiple levels of caching one is right in the browser itself that you talk about all the time you clear browser cache to clean up the cookies or other other state that may be sitting in your browser that's one cache cdn has another cache now on the other side this machine that is sending you these items like jpegs and pngs and css and javascript are compressed and aglified and minified and concatenated and are then stored here in cdn on demand they're actually stored at the origin to begin with but let me show you what this thing from another example point of view what i mean by uh minification concatenation aglification so those ideas are important to understand because we have never discussed these things and that is a necessary thing for performance so i'm logging into the website here and this is by the way a place where i think somewhere in the caching pipeline is where i have a bug in the cloud genius website and i'm not able to pinpoint where exactly that bug is so it raises its ugly head occasionally on some of you you have seen it in the past maybe one or two is that you have to log out and log in back again otherwise you see this image that shows hey you need to buy the course that is a cached in your browser situation it can be cleaned by the browser cache getting cleaned or cleaned by the the cdn cache getting cleaned or cleaned by removing this compression aglification minification concatenation either it is a bug somewhere here or some somewhere along the way in the pipe but those are the things that i i will describe i'm implementing these things to the extent possible but now you will see the idea which is minification or aglification concatenation all together handled by a plugin that plugin is auto-optimized auto-optimized i'm not spelling it wrong but it will pick up correctly i think no i did not auto-optimized so that is a plugin what this does is you can see the source code of it it's available somewhere in github where is that no not showing you the github link oh how interesting so auto-optimize github and so here it is here's the source code of it you will find in here that it is actually doing what these four things that i mentioned here compressing aglification minification and concatenation let's go see it's a view as to what it actually see what it what it actually does when i use it because that's the easiest part aglification is that obfuscation or is that different sorry say again did you when you wrote down ugly of vacation is that basically ob obfuscation or i believe uh really secation obfuscation is that basically obfuscation i think the obfuscation actually changes some of the calls around and adds in kind of like you know circumlocution essentially yeah it helps make it so you can't d decode it so the decoding so the aglification doesn't involve code changing yeah obfuscation at the root level shouldn't either but it it basically makes it just harder to decode there is no code in aglification by the way so there's no coding as such so let's go see an aglifier now there is a there is a library called aglify which actually does the aglification and uh it is the library which uh you know you should use it your project that you showed us today actually uses gulp i noticed uh so it is node js so you can actually include this library in there to aglify your javascript to make it a little bit faster for the end user so this is aglify js yeah we do something like that but we do it so that it's it's not as easily so it's some protection in so that nobody can just read it they they have to kind of like decode it and make it you know yeah decoding it okay my my way of thinking in this direction is that it's okay to use it but don't waste time in unnecessarily coding decoding because it's kind of pointless at the end of the day people can who want to steal your content will steal your content when you don't want to prevent them because more along the code not the content but yes i understand yes we just want to make it a little more difficult for them to just out and outright steal our stuff yeah it makes sense but my way of thinking is slightly different uh in that what i what i worry about is those who matter and i don't worry about those who don't matter to me because there are people who will steal your content and you'll don't worry about them i mean they'll always be there right people will steal and so they are not your target focus so i focus on those who matter to me and i ignore the rest that's how i think maybe it's actually it's a it's a it's a choice of a personal way of thinking so i'm not suggesting that you think like me but my way of thinking is very simple i just ignore the rest and i focus only on what matters to me or rather to my customers and that's how i operate i ignore the at the cost of letting somebody steal i let them steal and put it on pirate bay and that's okay with me i just said that because it's irrelevant to talk about other things because people who steal will never actually become your customers so why bother that's how i think but that's that's just business now back to jugglification is this idea there is minification which is simply simply think of minification like this let me just write some some html here oh no not this here so let's go write some html so i write html and this is very crude right now but just bear with it head and this is the end of the head and then this is body and this is hello and this is the end of the body and this is the end of html so this is html for example so some simple html that's my expanded form if i want to minify this what will i do is simply remove the white spaces like that and that is the idea behind minification so by minifying your what you're doing is improving the download performance of this particular file here download performance of this particular file set like that so this is classic minification and i save this file as my html file it's hard to read but of course you can beautify it just like you know there's beautifiers html beautifiers html beautification and there are beauty files available online like there for example and you can just copy paste some ugly html and say format and it will make it clean for you and so this is the beautification thing which is the reverse of ugly vacation or minification what these things actually boil down to is performance so that's something that you want to make sure that you compress ugly file minify concatenate and there are tools available for you to do it in a sense you're compressing the whole thing in one shot ideally one if not more than one but let the cdn grab it and keep it for you keep it for the customer so that the customer can then receive whatever you're sending to them so realistically it matters only in the high volume transaction environment right like amazon yes yes boeing does not have near as much traffic as amazon not even close yes true true and you know cloud genus has not even as traffic as boeing i mean no comparison i i got like 25 customers so yeah so but but you know what i still do caching you see me do it right all the time why do i do that and that is something to discuss and understand caching is not just for the the purpose of you know uh so let me let me just show you a real live example where i will disable caching right now and show you some metrics of how it is important i actually disable on the live website so there is justin saying something minification in action beautiful thank you justin yeah you should click those links and you can see that here is the full jQuery expanded form and the other link that justin has given us is the minified version of the same jQuery that is looking like this so this is the minified version of the same jQuery library available from code.jquery.com and this is hard to read this one is somewhat easy to read if you can read javascript so the same idea beautiful example justin thank you and see the same file is compressed or rather minified to make it easy to transmit or on a large network you are saving some tiny little bytes to help you with something let's see how it actually boils down to from a performance perspective so here is the here is the site that we use and yes i am a very very small company as you know already tiny tiny small but let me just understand what is what does it do if i just disable my caching and see how performance metrics actually take a hit and it is important even for small people and i'll tell you why and you will see it i hope i can demonstrate a live example by disabling certain activities like i'm disabling my optimization i'll also disable my cdn right now so remove those pieces and so i'll completely unoptimized site is what i am going to show you right now i'm disabling the content delivery network also there we go and it is also disabling right now my site will not suck it will it will not suck but it will be slightly slower tidy tiny bit slower and let's go examine whether the site actually functions correctly so i will go now to the different browser different browser and because this will probably be not cached so i'm going to clean up the uh what is the thing here preferences and privacy and security and clean caching completely out and cookies completely out and that's clean clean browser empty browser there we go and now i will start the browser again and what you will see now is i'll open that website again the cloud the cloud gns website it's open opening up and you will you will inspect what it how long it takes it i just hit the enter hit enter for that website it took me like seven seconds to load typically it loads in like a second or two if i have repeat visitor this visitor was the first time visitor visiting the site if i if i reload it it will still be slow see i reloaded i'm waiting counting seven eight seconds or something like that and still not properly loaded yet now it loaded like 10 seconds of reload so that is the amount of time that i am wasting and every transaction every person hitting maybe this link or that link or something else will waste about 10 seconds on a click basis what if i'm able to optimize this performance and you will suddenly notice that the site is somewhat slow it will be hard to diagnose this unless you actually test it so let us go test this site performance we'll go to a testing tool and here it is opening up in that tool what i'll do is i'll grab this url and say you please test the website from say san jose so go test this website from the san jose location and give me a report we'll start the testing it should take like 10 seconds 30 seconds or so and finish it off you'll see that the test is proceeding forward and it has disabled caching and disabled content delivery network and every other every other performance caching has been completely turned off so the site load time is currently at 9.25 and the site is performance grade 67 which is a degrade performance it is faster than 14 percent of tested sites which kind of sucks and here we are not combining javascripts there are 32 javascripts served from the cloud genus website wow why so many because that's how the application is written that's how we have so there are lots of javascripts that means there are 32 queries going back and forth between the client and the website and that is slowing down somewhat because there are so many queries involved in making sure that they actually serve properly so the combination of javascript files into one or two files will be an ideal situation then the other idea here is the 103 parallelizable requests omg and all these pictures that you see on the front page that these are the pictures so lots of little things mostly pictures and some css and some javascript are total of 103 that can be parallelized parallelized not parallelized and so the other other suggestion here is leverage browser caching the idea here is these assets or these html css files that i am looking at there are lots of them and pictures i am not leveraging browser caching what is browser caching is recall is the same idea here browser cache which means i would like to make use of the resources the user is providing to himself the browser it has a cache and i am not using it why not so let us enable browser caching how do you do that you set let the browser know that hey i would like you to please keep a copy of this thing until the next time i change it until i tell you so so yeah i will basically notify the browser by doing certain things for example let me show you an example of something that will give you an idea about how you talk to the browser to tell the browser hey keep this or no don't keep that you notify the browser by letting the browser know of certain things so i'll pull up some things that are currently have browser cache and the others that do not and let me see if i can find an example so here is a picture let's see if it is already yeah i think it's a good one so curl dash i get the header for this and you will see that it has a cache control max age entry and an expiry date this expiry date is march 13 you see that i just queried some picture and found out its header it's a different site by the way and actually i'm going to the cloud genius site but from the cdn portion so this is the c dot so the cdn representation of the picture actually if you go through the cdn it is advising you to cache advising the advising the browser to cache this object until this time that's the way the cdn is communicating to the browser to please cache this picture and keep it with you the origin of this picture is coming from here so that's the idea underneath under the hood now if i look at this picture itself and go straight to the website you will find something that is not currently enabled for any kind of optimization and no kind of no browser caching i think let's go confirm that piece so i will confirm the word employer and this file is currently being reported as as leveraged browser caching they have a very short lifespan which is a complaint it's a complaint as in as if i am not leveraging browser caching leveraging browser caching and that is correct because i disabled browser caching right so i'll go back here and now curl this link that i have grabbed from that website the complaint that you don't seem to browser cache and so i say okay let's go see what it says and you will see that something is missing from here can you tell what is missing or maybe not so let me let me go side and show you what i think you can tell what is missing so let me just adjust the font a little bit and run the other query right there you'll see certain portions are missing in the first segment currently the site is showing you this whereas it should be showing you this and that is these pieces are missing that piece is setting the expiration of this object in the browser's cache because that picture or the employers.jpg picture doesn't change so often right it should not change so often this picture should not say change so often on a daily basis i keep a seven day expiration period and i keep this picture in the browser's cache it's a great thing to do it like that because i am actually saving money you know tiny little bit money but yes there is saving and i'll tell you where it comes from it comes from the fact that this item picture which originates from the source is actually stored in a cdn and it doesn't get sent to the user over and over and over again with every click because the user will come and click and they will actually you know go on this website they will hit left and right wherever they want to go hit there hit here every time that picture might pop up in some some location like this picture pops up if you like right click on this picture copy the image address it will not have the cdn entry right there at the top you can see it doesn't have cdn because i've disabled it and so if you now look at that same picture again here from the command line perspective and examine its header properties you will see that it is missing the cache control directive to the browser whereas the other representation which is the content delivery network representation actually of the same picture actually has that directive right here and these directives are generated by your web server so this server the source has to originate and put this directive in a header for every object that is eventually compressed magnified minified and concatenated and served to the user through a content delivery network and that directive created here needs to be replicated here so that it is actually sticking to the browser cache that's what this picture is really trying to communicate which it doesn't go into great level of detail about what it is done and how it is done but there is one caching which is the database level database level caching the other caching goes around right here in the web html css javascript layer and this content delivery network you're looking at also helps the browser stick the item that you don't actually go and ask for the same jpg over and over and over again because there is the saving right here you know what cdns are expensive because they are global and they take your picture wherever you want it to go depending on wherever your users are and so an example of that a practical example of this will be something like me let me just see if i can find a bill to show you so where the saving comes from and so let me see if i can find my bill uh bill what is it called invoice yes so here you can see that this is a monthly operation report and you can see that we have a person coming from singapore as you recall and so there is there is some bill corresponding to singapore let's go find out where it is and why is it a singapore specific bill because the user is there i mean you know we know ramesh is there so he's hitting it and there's a consequence there is a reference where is my detail of the bill the pdf file i was expecting to see in the bill but it's not there where is my pdf file okay let's go see the google cloud build the bill should show you exactly which which regions are hitting your content and it will actually show you some extra charge for that particular region which otherwise there are no customers apart from apart from ramesh but because of him there is a cost to it i'm not not complaining about the cost but i'm just pointing out that there is a cost and so uh don't don't get me wrong ramesh i'm not saying that you are causing me an extra bill that's not what i'm saying but just want to point out that there are some charges that accumulate if you go look at where is the last month not this month the previous month go to billing and where the previous month charges transactions maybe okay last previous month that should be a good indicator and so here you should see a singapore charge coming up right there so there it is see compute engine network inter region egress from america's to singapore of 0.549 gb bytes source from my production server from the production region whatever this project that i have and it costed me seven cents that's a cost i mean it's a tiny cost but it's a cost and it's actually visible there is one cent going to virginia apparently and some into china i don't know why there is nobody in china maybe there are people visiting from china and uh that's what you're looking at so this this this seven cent actually is a small number figuratively speaking but in the overall context of what i'm dealing with the total bill 213 dollars for the month of february this is not zero i mean it is still non-true as you scale the business the reason i'm showing this is not to save you seven cents but you know comparing cloud genius with a larger company you will have these numbers accumulate and grow and so the core idea to remember is to optimize to the extent you can because if you don't cash it is going to be a cost to you and not only that it is going to be slower for your customers so they will not only hate you but you will also end up paying more if you don't cash that's the key thing to remember that's the key thing to remember so caching is not just good for your performance of the end user but is also actually helpful to save money because you are unnecessarily not sending traffic which doesn't need to be sent so then you use the laptop this guy has a laptop right this user use the machine that they have already and so just keep your asset that you have here the picture the png picture the jpg picture this person visited that website just store it right there keep it here so the next time the user visits and you will see that the performance greatly improves now you saw some performance metrics showing up on this result sheet where it tells me that i am a degrade site i mean that's what you see now let us go see that site again and i'll enable the the the tools that i use for caching and see the difference in performance so i will go back to the website and toggle that button to enable caching enable optimization and let it stabilize just a minute and then we'll go and test and you will see the difference in ranking and rating so here is the list that you are looking at i'm activating this plugin which enables the concatenation concatenation optimize you know minification aglification and all that thing together and then the second plugin is my wp rocket plugin which enables the content delivery network for me so that also starts up soon in three two one come on there we go and i had to clear the cache cache and also clear the auto-optimize cache so there's one more cache that i want to clear on that side delete cache and one more time i will delete this cache and let the let the cache rebuild so i'm clearing the cache again and let it rebuild in a minute or two and when it rebuilds you will see that the performance metric from the outside perspective will actually improve and it will be measurably different it will not probably not be noticeable to a common user you like you may not see it visually you may see that what is nilesh talking about you know it doesn't even make sense the performance is same yes it might appear to be the same but let us go compare numbers as to what it actually says and so this is the site i will leave it up in this window i'll open another window and i'll now test that website again same site one more time and before i test this site which i did i will test it again in the second and second window here and i'll also test it again from san jose location and before i hit the test i'll actually warm up the site just a minute little bit warming up means actually exercise the site so i'll go visit that site and check it out like common users do visit i'll do the same simulation for like a couple of clicks so it loads up properly and caches the whole thing across the way across the way towards the end users so i'm hitting this website in multiple places and just making sure that it loads properly even after this caching has changed we have now turned on caching and so that's what i'm visiting the website from the first time user perspective hitting these pages and you will suddenly see that the performance of these pages have magically improved i don't know if you notice this or not but i do notice that suddenly now every load is quick you saw that one two three front page bootcamp it's fairly fairly quick relatively speaking to me i can notice i don't know if you notice or not but i think it has warmed up the cache by now so i will go to that browser and hit speed test one more time and here we go starting the test and we have a comparison between before and after we'll see how the load time improves and how the performance grade changes the load time seems to have cut in half the performance grade is now b c and i am so let's go to side by side so here we have before and after this is before and the other one is come on this is before and this is after so let's go then side by side review you and see what it says so this is visibly some noticeably i find it different but i don't believe a person like myself i would rather believe the test so i can see that the performance grade has improved from d to b the load time has approximately cut in half it is site is apparently faster than 30 websites tested which is nice the page size is 2.1 before 2.1 now is the same same site right the page size did not change but the number of requests suddenly changed 160 before and now is 105 let's go see what has changed in case of this parallelized download there are still 51 parallelizable requests now here it's the the the thing that the first error that we had before was combine external javascript and here the first error was combine external javascript let's go compare them two at two at the same time these two before the grade was f as in failing and there were 32 javascripts served from this domain they should be combined furthermore it says three javascripts served from this domain and three from this domain and three from two from that and two from that so that's the notice that i had now i have you know one of the things that is completely disappeared is these 32 files they have been optimized already you can see that it is actually one file that file has been compressed minified uglyfied and served as one shot so it doesn't complain now i have some other files that i'm loading from other sites like for example this site which i will talk about in the in a few minutes which is the marketing focused aspect which is the third picture from the e-commerce implementation perspective we'll talk about that in a minute the idea here is there are three javascripts served from youtube i'm using some youtube imagery or videos in the front page you will see that i am actually using youtube those videos are i think alex speaks about them so there is a video from alex right there that's a youtube video one two and three three videos they come up and those are the files corresponding to that youtube player there are some google apis there are requesting jquery and webfont there are a couple of files from google ads they're coming not from me but from vimeo which i have a plugin so vimeo wants to track how i am using their service so that's what these two files are from and there is video vimeo zen cdn which is a vimeo site i think it's coming from vimeo those two javascripts so that's the optimization it has already done you will see further optimizations also happen for example in the other ideas of parallelized download across host names and here it was before parallelized downloads across host names the number has shrunk it was 103 now it's 51 but still it is complaining that you have a f grade because you need to parallelize downloads and pages across multiple domains and this i think is a error in the tool that i'm using the testing tool itself is bad how do i know i know that for a fact because i'll show you how the testing tool itself is wrong in reporting that this is not parallelizable request what does that mean it means that it means like this you have 51 files coming from some location to this location and they come one at a time and that is okay and that is true in case of http 1.1 but when http 2 is in play which is what i have enabled in my site it is a moot point this parallelization parallelization is moot because http 2 protocol already includes and handles multiple parallel downloads by itself and testing for this is moot and now let's go to that site and let's go look at the site itself in a different browser tab in full screen and open it up you will see that this site actually is a complete http 2 enabled site and you can convert it to a website and confirm that by hitting this button and it tells me this button is a http 2 is active at top level document the entire website is http 2 already you can also cross-check this in curl curl http 2 it will tell you it is http 2 right there so the whole site is already http 2 which by design of http 2 this parallelization doesn't even you know the error complaint about your site is not parallelizing resources itself is a bad error the tool is wrong in reporting that error because this is already a moot point we have moved on to http 2 already which speeds up the delivery of content to end users already by its protocol so i don't have to worry about it so that tool is kind of broken in some sense some sense let's finish off this discussion moving on to the other so browser caching we talked about briefly what else do i get a negative mark on so combine javascripts there are a couple of things that i still have to do work on this port this portion also should be combined in one possibly one in that case if that happens my performance grade will improve suddenly too close to about 90 or so percent and my site will probably be faster a little bit if i buy a slightly more expensive machine from the kubernetes cluster that i use currently they're slightly cheaper machines because i i can afford to go slower because you don't seem to notice or rather i assume that you don't seem to notice so may i ask you this question do you notice that the site is slow compared to what uh between the before and after when i did you get a chance to test uh before and after when i enabled disabled testing this caching did you notice or did you even bother caching definitely helped i noticed it when i was testing the caching had yeah okay okay your caching does help but sometimes it causes issues like you know you know you have causes issues like you know like i'm running into i know of a bug that i have in my caching algorithm somewhere that actually results in a occasional case where one of you will say that you know society is telling you to buy the product which is basically wrong caching bottom line is it's bad caching so i still have to figure out where the bug is and fix it eventually i will do that but caching seems to generally help it improves the score improves performance and also saves me money seven cents to be specific so small money but i'm doing this because it's it's uh necessary for me to do to be able to give you an example of what it makes in a larger situation if i don't implement it myself it doesn't make sense for me to talk about it because it it doesn't become credible so that's the reason why i do caching it is actually you know esoteric if you will but now justin reports that he noticed his performance difference that's a good thing but i do this primarily to be able to show you the concept because this applies to a larger case especially when it's companies like you know boeing and microsoft and other big guns you know big people they need to do this they do this already by the way you do it i can check that check your website and show you where you're doing it i i'm pretty confident you already do these things let's go check it out so let's go see one of your companies are not microsoft but boeing so let's go see what you do so we'll go check take a look at inspect element and find out how you're doing it and also take the same test that i just did on boeing.com the same test before now i'm testing boeing.com and from san jose location start the test let us see what it comes out with the load time is pretty damn pretty darn nice 860 milliseconds the site is faster than 91 percent of the sites tested which is totally expected and you are given a performance grade c why is that one of them is these reasons combine external javascripts apparently you are not combining them maybe you should let's go see something else leverage browser caching you're absolutely not doing it why are you not doing it i don't understand you should do it there is no reason for you to not use the browser cache it suggesting you to use leverage browser caching and if you don't trust this particular tool there are other tools better than this by the way like web page test dot org that's a slightly better open source tool that you can use if you don't trust this particular pingdom tool you can go www.boeing.com and test it from any location that you like there are a bunch of choices available this is all an open source tool by the way and you can choose from any device any location let's go choose the oregn location any location let's go choose the oregn location where's oregn north america okay there's oregn location from a firefox browser and a chrome browser a firefox browser let's go test it and start the test so now oh my god okay one two three street signs next how many more one two verify okay test so it is testing it is another test so it gives you a different perspective on testing the performance of the site so this is the source code of this tool itself is available on on on github so you can see the the actual usage of this tool as to how it does so it is testing in that 14 15 seconds should be finished off and run the same test three times and gives you a comprehensive understanding of how the user is seeing and how the load times are actually happening apparently you can see this timeline view and overall the load test finished in how many seconds three second is the first time oops what did i click back yeah first time three seconds waiting for the second test and third test so the second test finished in 4.8 seconds the third test finished in 3 2.9 seconds and you can see the water flow sorry waterfall of these assets loading up all the way to the browser in this picture you can see the waterfall view and item by item and go into detail about which particular asset is actually slowing down so after this blue line here three second line after anything after that is typically slowing you down so this picture which is this boeing.com whatever it relative by 720 jpg picture is like taking so long this item this resource is taking way too long to load and you know that is something that should be optimized so we can go further in there and identify by the way one thing i notice is curl on that website let's go see see you're not using it they're not using those you know cache control and cache expires those errors will help you well i don't need to tell boeing to save money but yeah you can save money and improve performance of your site simply by storing this picture whatever that picture is this picture yeah this beautiful picture you can use the browser's cache the user's computer browser's cache to store this beautiful picture the first time they visit they take a little more time but the subsequent times this picture is not going to change over and over and over again on a daily basis so why not leverage that is the the thing that i would suggest that you please do leverage browser cache like this what is that like this like here you can specify item by item how much duration when is the expiration date and set the cache control parameters like that and you can very easily do this in apache and in nginx and any other thing that you currently run which i think you do run apache right there and so it's easy to do it's just a question of enabling it enabling it and another thing i notice is this thing right here why are you not using sgt ps i don't understand it because we're brave okay let me salute you for that but you know you should i mean sgt ps is a given you are brave and that's that's awesome but it's a given that you have to use it i mean there is no other there is no reason why you should not use sgt remember boeing sells planes they don't do websites i know but the website is not what you sell right website is a channel for your customers to visit right so if you somehow have a in other words it wasn't it's not a priority for boeing to spend you should i'll tell you why it should be a priority i'll tell you black and white just listen you don't have sgt ps which basically means somebody can run into a man-of-the-middle attack between say american airlines and the boeing.com website those two points american airlines has your customer boeing.com is another as you and so in between if there is a man-in-the-middle attack of some sorts somebody can show up a picture of another plane not not this but what's the other company in france airbus airbus picture shows up here what would you what would you do do you want that to happen is that not a priority that would be funny i know it will be funny it'll be bad funny and bad for all of us but it will be it'll be bad and that's what the more important point is it's easy to do a man-of-the-middle attack hey dave easy to do a man-of-the-middle attack hey dave aren't you infosec shouldn't this be in your area of expertise i mean yes you should not pinpoint people and that's not the point the point is to think about how these are important critical issues you're leaving a a endpoint that represents your company as not sgt ps that means your your services that you are interfacing with your customers like american airlines for example is is looking at from a non-secure point of view that is your sales channel i mean it's not it is a critical thing it is your sales channel to your customers your website this is not a sales channel this is just a marketing website the sales channels are actually secured you're right i get it i was asking dave more of who would we even is this something you would be involved in who do we talk to about this yes exactly it's uh you know and and that's true that this is website is just uh it's not the it's not the high priority ones like that so yeah but yeah i agree the point of view there's nothing wrong in thinking like that to begin with but it is so trivially simple to convert in the entire infrastructure into sgt ps it doesn't cost you a single penny today this thing is a service available for open source and free which will i'll demonstrate as another exercise it lets you create sgt ps certificates on the fly don't even have to think twice it automatically gives you a certificate and puts in the right spot automatically renews it for you you don't even have to think twice just enable it once and enabling is easy and it is open and here's the documentation and we have an exercise coming up at some point it's already written down somewhere on the website you can go and find it or bad guys do that too sorry bad guys do that too yeah let's encrypt you mean let's encrypt uh is what bad guys do they can use security too to their advantage oh yes correct and they will they will but the idea is uh what you can do is what you should be doing as opposed to worrying about what other people can do because there will be adversaries all the time and they just exist everywhere so what we need to focus on is what we can do so what we need to focus on is what we can do because there is no point in worrying about what they can or cannot or may not or may or do because that's not in our control so we control what we do in our domains in our purview in in our marketing in our series in our operations whatever we have and we focus on our thing and we do our thing the right way as what we think is right for our customers and let them worry about other people and you know yeah they will exist adversaries will exist they're not going to go away from here okay so enough of this uh can we take a short break yes please short break so we'll just start a break timer and really that timer uh notification and disable this do not disturb okay there we go and the timer is running okay I will come back and continue let me get the cable will the cable okay Thank you. Thank you, guys. started beeping on me. I was late. Sorry about that. Let's resume what I want to talk about. I'll just check if you're here. Yes, most of you here. Back to the picture. In this front-end representation of what this image is trying to communicate. I think we discussed it briefly to the extent it allows us to discuss this. But there are two aspects to it, which is this aspect and this aspect which are separate and not included in this illustration because the picture itself is quite big and they want to put focus on one thing at a time. So the second important aspect which is about checkout is the part number two of the same picture. And that's what I would like to open next and discuss that from our point of view as to what it takes to enable that complete cycle of checking out a product and actually purchasing it. So that I think is maybe this picture. Yeah, this is it. So this checkout picture actually is available in your Slack chat. And what I'll talk about this is the experience that you get when you go to something like amazon.com and then you actually purchase something. And so here we go put an item in the cart and I would like to buy this onion powder for example, or maybe I think that's the Amazon person delivering something that was a doorbell ringing and my dog barking. That is what just happened. I don't know why he rang the bell. He left the item and gone. But if I want to buy this item, so the chili oil from China or something else, I'll put them in the cart and then this process is actually a separate service, completely different from the first service that you saw, which is what you see when you look at the product catalog and you find the banana or some other thing that you want to buy. That's a completely different checkout experience. It's a different service. That is what you're looking at that I'm interested in buying this organic banana, but organic whatever this thing is and this chili oil, the two items in my cart. Great. How does it translate to the end user? This guy, this end user expects a certain experience in terms of very, very similar to putting an item in a physical cart, taking it out to the location where the person can actually pay for it and then have the item delivered to them. So that's the kind of experience that most of you already know it. So it's nothing new, but the person basically does that thing in the front end portion of the website, which is where the product select product person gets to select a product, puts that in the cart. And when you reach that experience of checking out, that is this piece of puzzle that this picture is trying to communicate to us in terms of what needs to happen and what pieces we need to enable for that particular item in question. Like I have in my shopping cart, I have those items in the e-commerce application, selected them for me. So now what I'm doing is basically interfacing with the checkout application. What this application is going to do is make sure that I give them my money in form of a guarantee or some commitment to pay on a credit card or some other sorts, some other type of payment. And as a consequence, they will create what is called an order, an order with an ID number. And you know this already. I think most of it is trivially simple, but the concept here is you have a bunch of different orders coming in all along the way and those orders need to be able to put in some form of a workflow. That workflow needs to happen is basically what happens on the backend of amazon.com or what happens in the backend of a physical goods and shipment of delivery of a product to your door kind of a setup. E-commerce, basically physical e-commerce, an object reaches your home, an object reaches your office, that kind of a setup. It requires physical workflow. And so that has to be kept track of and you need to fulfill the order. Only one fill is it, one L here apparently. So fulfill the order. You need to do that and you need to keep track of the order in some kind of a data store. Whatever orders that you see this person has assigned and given so you can see orders issued in a different location here. You can go to the orders created by this customer and here are the orders that I have purchased in the past. And the same idea is basically pulled from the data store, this data store, which is keeping track of my customers and orders that I kept in this data store database, which is a RDS representation. And here is a replica of the standby replica of the same database. So these two are basically operating hand in hand. If one goes down, the other one takes over kind of a scenario. But it keeps track of all the orders that I have eventually placed and currently placed or an item that I want to buy, which happens to be this chili oil in my shopping cart right now. So as soon as I put that order, an entry will be created in this database. So it will keep track of that data about the order itself and also assign a workflow to happen along with it. It will also make sure that I collect my payment, that order ID will include a payment that is also processed through a third party service, typically like the very commonly used service these days. Extremely popular is this service. It's amazingly popular. It might, you know, this is the one I currently use today. So this is the new standard in online payments. It is a service that is available and you can, you know, basically hook up through an API call to your, through your web application or any kind of application. You use the, use the API available from here to interface with your checkout applications. As soon as a person gives you a credit card number, you invoke that Stripe as a solution, by the way. And but there are other solutions also available. Stripe is just one of the best ones that I have found and I like it and I use it and it costs me 3% for transaction, whatever transaction is and I'll just pay them 3% and collect the balance money. Money goes to my bank account directly. So that's how the Stripe application works. It is integrated into the checkout process. That's how most newer applications are written. Almost exclusively these days people write on Stripe because it is trivially simple to use it and it's very developer friendly. That's the reason why people seem to like this very much. And it's a service, meaning it also supports, it also supports a lot of different types of services that like subscriptions and all kinds of things and it handles all that for you. Yes, yes. FYI, the Stripe guys, the guys that invented Stripe, actually are ex PayPal founders. Okay. So PayPal is also another one, but I don't know why, to me, I don't like it. I think it's a business policy thing that I don't like them. From their policies, I don't like. Functionally, they're good, but this is better, functionally better and policies are great and the timeliness of delivery of money in the bank account is immediate. Like next day you get cash in the bank. That's so nice. And, you know, you have Apple pay support, Google pay, Google checkout support, Amazon pay support, bunch of other things, Bitcoin payment. It's going to disable Bitcoin apparently for some logistic issues, but Bitcoin is getting disabled from Stripe, but that's a beautiful service. I just love it. No doubt about it. So that's the checkout application. If you are a big company like Amazon, you will build your own Stripe. They have their own Stripe by the way, internally Amazon doesn't need Stripe. Boeing, I don't know if you, I don't know if anybody buys a plane on a checkout shopping cart, but I'm just saying, maybe there are people from, you know, Middle East who will just bring cash and buy a plane, but they may not have a credit card, I guess. So they might just give you a boatload of cash and just get a couple of planes, but Boeing uses banks to do financing. Boeing uses banks to do financing. Nobody pays in cash or bring Stripe to credit cards. I'm just kidding. Just kidding. I know. I'm just saying that, you know, people from the Middle East who have oil money can just bring boatload of cash and give you cash upfront. That's not how it works, but I'm just joking about it. So imagine Emirates, United Arab Emirates, those guys come in with a tank full of cash, just give you cash and get a couple of planes, maybe buy more than a couple of planes. They can do it. I mean, totally. Not that you would accept cash like that, but possible in theory. Okay. So orders come along. You need to now fulfill the order. And those are what are essentially is a workflow service where you have these worker machines in an auto scaling fashion scaling to handle your work. And when the work essentially boils down to a couple of things, for example, first thing you do is you notify the customer, I got your order. So there's an email going out. Then second thing you do is you let your warehouse workers, factory workers know that I have this order. So you need to prepare and package and send. That's the second piece of workflow. And the third workflow is delivered, let them delivery notification. The delivery notice has to also go out in some fashion, depending on what the customer wants. So those are the three essential puzzle pieces of puzzle in the order delivery fulfillment pipeline. You need to notify the person immediately that I got your order. That's a critical thing. If you don't do that, people get anxious. Therefore that these workers do is basically interface with an email service that sends out emails to the customer saying, Hey, I got your order. So just hold on. I will send it. That's the first thing that these worker machines do in an auto scaling group is interface with a sending service and send emails. You can see that this experience actually is integrated into, where did I do this? The experience of this email service that I use actually is integrated somewhere. Which website uses that? Let me just go there and find out. No, not easy to email service is a separate service somewhere here, here, simple email service. So separate service available from Amazon. And I will just understand my sending statistics, very, very low numbers. I have a humongous quota of sending like a million emails in every 24 hours. But I send one very low send rates because I am not using it actively. But that is the delivery charts. You can see the reputation dashboard and the dedicated IPs that I don't have and configuration sets and SMTP settings. And I use these settings to send out an email and I can create a credential. And occasionally I'm building starting to use this service to send email notifications of sorts. For example, somebody lost their passwords, they get notification. You may have seen those notifications coming to you from the CloudGIS website. Do you see such notifications occasionally pop up? Like you did something or you forgot your password emails, those emails come from here. Do you receive such emails at all ever? Occasionally? Yes or no? Anybody? Never? I think I've changed my password once. Okay. So there is this thing that seems to send emails, right? Okay. I hope it does. I hope you're receiving those password notification, password forgot, lost password email notifications because that's what this service will do is this worker. One of these guys will actually interface with the email service and have that email sent out to you saying, Hey, here is your link to reset your password. That link is generated on the fly and is sent to you in an email. So in this context of a checkout, the notification needs to go out to the user saying, Hey, I got your order. So that's the first thing. The next thing is about the actual decision about a package and send, right? That's the second most important action, which is to actually send that needs to be sent in a notice to the warehouse worker of some sorts. So these warehouse can be mobile workers, can be robots depending on how big the application or the company is. But somebody needs to get a notice that you please package and ship. And here is the address you need to send it to. So the address is right there. And so that package and ship action has to be either done by a manual worker or some robot that has to do it. That is a second action. It's a very, very important action for an e-commerce perspective. And so somebody has to receive that notification from the workflow. That's what this workflow service is also handling is to let the mobile worker or the robot handle the transaction, the physical shipment of goods and services or whatever you might have. Then comes this third idea, which is here, which is on the front end side, we looked at the e-commerce checkout application portion. And the third idea is the marketing and recommendations engine, which is here. And let's go understand what it does and what it will actually roll out from a usage perspective. So the idea of generating an experience that produces recommendations is a data science based concept. So if you go here on Amazon's website, it is generating those recommendations on the fly. So I go buy this, whatever that thing is Roku, and I get a recommendation about people who buy this also buy that. The recommendation generated on the fly dynamically given to me. Sometimes I go random things like exotic items from say Sichuan, Chinese Sichuan peppercorns. So I buy Sichuan peppercorns. So I seem to like this. So I buy them. And so I buy this Sichuan red pepper chili powder. Beautiful. So I buy that. And then I get some recommendations. Like people who buy this also get that. And so here you will find that McCormick whole black pepper. That's not really Sichuanese, but whatever. It's a pepper. And so here is another one, red pepper from Korea or more other red peppers. And there is probably a bunch of more recommendations you will see, but they are at least related to the concept of hot pepper because I am, I seem to be interested in chili powder. So it seems to be, this guy seems to be interested in hot spices, at least at this point in time. So they are dynamically generating recommendations and they are presented directly here. Like people who buy this also bought that. And so that idea of generation of a recommendation is in data science is known as market basket algorithm. Market basket. You should read that actually. It's an interesting read on its own. Market basket analysis. And so here is a gentle introduction to market basket analysis. So you should maybe read that. Is that the right one? Let's go see. Are they talking sense? Yeah, seems like it. Seems like it. So you read that later on, but that gives you an idea about what a recommendation engine will do when it is trying to recommend you something. In effect, what they're trying to find out is what is known in statistical science as the confidence level of a similarity. A similarity based on a priori probability. Some of these are statistical probability related terms that I am just throwing at you. If you're not familiar with it, don't worry, but just know that read this will help you understand what this thing is about market basket analysis and give you some idea about what's going on under the hood. So that is what this thing is coming from. This customer who bought this also bought that. And you will find most of the times it is fairly accurate and somewhat related to the topic at hand. Occasionally, you will find a mistake. For example, no, that is also actually some kind of chili. So apparently I'm not able to point a mistake somewhere in the recommendation engine. And most of these results seem to be okay, except not this one after like six or seven of the 10th page. On the 10th page, I find that this is something that is not directly related to the chili, but it is kind of sort of related to kitchen cooking and those categories, but not chili, right? So this item is something else. It is a Detroit style pizza pan, whatever. It has nothing to do with chili whatsoever, but apparently it came up, came up as a recommendation for some reason. I don't know what that reason would be, but you can find out these things and you need, when you have like millions of products in your, in your market, in your, in your catalog, you need to run statistical analysis on market basket to understand the similarity and make recommendations. That is a complex and CPU intensive and also time intensive algorithm, by the way. So what does that translate to is that you need to do data science, map reduce. And this map reduce is an algorithm created by Google guys in 2004, by these guys, what are their names? Geymawat and Dean. Those two guys created this paper on map reduce, which is what is the foundation of what you these days know as Hadoop is the product that has evolved out of it. So the product that we use or the industry uses left and right is this product known as Hadoop. It is created based on the work that the Google search based on that also. Yes, correct. Originally Google search was the one that picked it up. Yes. The Google search was constructed based on this concept and then eventually came out with this paper later on. Map reduce I thought was done by, or not map reduce. The other tool was done by Facebook. Who? Facebook. Which one? Hadoop. Cassandra? I don't know about Cassandra, but when we were using Hadoop, that was associated with Facebook when we started using it. Maybe it's not Cassandra. Maybe there is something else, but these days even this Hadoop thing doesn't have the favor of the industry anymore. The thing that has taken on is this thing. Let's go pick it up. That thing has taken over. Kind of taken over on top of Hadoop. This is the new thing. It is the lightning fast cluster computing. That's the tagline. The thing is known as Spark. It is also an Apache product and this has taken up quite a bit from adoption and it works like Hadoop, but also works way faster than Hadoop could. Let's go talk about the speed. It runs programs up to 100 times faster than Hadoop and it does map reduce in memory and 10 times faster it does on the disk. It is doing it on both memory and disk, but it's basically 100 times faster if you do it on memory, 10 times faster if you do it on the disk comparing with Hadoop. It's a successor if you will think about it that way. That's what you look at in this illustration is this map reduce concept running. This elastic map reduce is a product name by Amazon. That is not necessarily what you need to run. You can run another type of cluster which will do the map reduce operation for you. Either you use Spark or you use Hadoop or you use something else depending on what your flavor of choice is, but I suggest that you use Spark and not Hadoop. Spark is open source and available and you run it on a cluster and you can run it in Kubernetes by the way and there is a module for that. All these things are available in Kubernetes already. Basically, this is your Kubernetes cluster running Spark. In there, you will program it to do market basket analysis and you are looking at user profiles. You're looking at a ton of e-commerce application logs that are generated based on what people are browsing, what they are visiting, what they are looking, what they are adding to the cart, what they are deferring, what they are marking as their list for future or save it for my... What's the name for it? Something in there that you will see that Amazon does is if I look at this product and then I will say save for later or something like that. I think that's the button. What the button is called? I think it's save for later. This button called save for later is a way for me to keep track of what I may be interested in the future. If not right now, I can either delete it or save this for later, for example. Maybe I don't want this chili chow, jiu chow, whatever oil, so delete that. Now, I have a bunch of items saved for later. I will eventually buy these things, but not right now. This is how Amazon is keeping track of my preferences. Those are kept in the application itself. Then random people like me who are visiting this website will eventually click on the onion powder again for some reason. I don't know why, but I seem to be clicking on this powder for some reason. What Amazon is keeping track of is in our area, in our geography, in our zip code, for example. If there are cases of people like me who are apparently clicking on this organic onion powder over and over and are hovering their mouse on this page for some reason, and some of them act to the cart. Some of them actually happen to buy it right away and get it delivered and all that. Amazon is gathering those things about these events that people are doing, generating about a particular item like the organic onion powder, for example. Those logs get generated. You will take that logs from the bucket that you will store it in and include it in your analysis as an input. You will begin to analyze that detail about what people's behavior are from the logs that are coming in, because logs are getting generated. People are visiting your sites, which generates logs. You need to consume those logs and use spark-based analysis. An example of that analysis was the market basket analysis of what people are buying and what else they are buying as a consequence. That is the study that you should read more before we talk about that if you're interested, is to understand and identify what is considered as a recommendation. This generation of recommendation is happening not only by the logs that are coming in from the website, but also from customer's orders and databases. In the database that we know, we have stored our customer orders from the checkout site, that same data store. There is a read replica that you can have allocated to the cluster here running your spark that will conduct this analysis that is necessary for you to identify recommendations and you need to do them in almost real time. Almost real time analysis. That is what is possible today with spark is you can actually take that read the database of customer orders and incoming log files and appropriately adjust your recommendations that you generate and send that out to the customer through your recommendation service, which basically injects this portion of the recommendation right in the e-commerce site itself. In the other picture where you see this, the very first picture we saw which was, was it this? Yeah. Right in this front end, the marketing recommendations are already injected along into the e-commerce application itself, which is what is presented to the end user. The customer sees the recommendation as soon as they go and search and it's almost instantaneous as soon as I hit this item and you see the recommendations coming up. For some reason, I see that people who buy onion also seem to be interested in garlic powder and so that's what you're looking at. Where are these things coming from? Coming basically from the analysis that you just saw and also another thing to think about in the same context of marketing recommendations is the idea of this recommendation engine that is suggesting us to include that recommendation right on the page on which the customer is landing right this. People who buy this also buy that recommendations. In addition to these items showing up on the page where I'm looking at a particular item like the garlic powder, you will also see sometimes Amazon will email out to you saying, thank you for buying this item and when you buy this item, you might also see recommendations right in the email which is given to you as a receipt or as a order in your inbox. In that email also, you will find such recommendations popping up and that is what is here, marketing emails being sent out through the email service from the marketing application and this you can implement either yourself or use an open source product which I use which is here. This is the world's largest open source marketing automation project. That's the claim. I mean, it's just a claim, but let's go to GitHub and find out what Motic actually is and what it does. This Motic is the source code. It has 800 forks, 2000 stars and lots of people watch it. 119 contributors and it's stable version currently is 2.12.2 and it was refreshed as late as seven days ago. It's a fairly active project. I use this project to send out emails and I have used it to send it to you. You may have received it. You may have seen those emails and this is the other side that I've said that I will talk about which is what I'm opening up right now and you will see how I use it. It is to help me market. This is my marketing site, by the way. Only I get to see it. You never see it, but I'm showing it to you because I think you need to understand what's going on. In here, you will see that my customers are globally distributed as shown in the depth of this color. Most of these people happen to be in the US and some in Canada, some in Russia and in other parts of the world, but that's where people who visit CloudGenius website are in these locations. I see some statistics, aggregate statistics that show up to me in form of a report, but more importantly, the primary way I use this is actually let you know that a program is about to start again. That is my way of marketing, is my way of doing things. Simply speaking, that's how I do things. What I do is actually send out to you and I classify that email to Boeing different from other people. You can see that those two types of emails categorized and so I'm classifying this. Some of these emails are actually automated. You will suddenly see any email pop up into your inbox depending on a context has to trigger. Something has to trigger to make it happen. Something has triggered in the past and some of you, at least 28.92% of you have read this email that seems to have been sent to you. Let me just see if I can open it up and you will probably recognize it. Some of you may have received it and it looks like this. Did you receive that email? Yes. Okay. This email was sent to you automatically based on certain triggers that happened. The trigger that happened was actually me scheduling this bootcamp. It picked up that trigger and automatically sent that email to two kinds of classification I do in my customer base and I handle Boeing people slightly differently than everybody else. I think you know that already because Boeing has this LTP program, whereas other people don't have it. Other people have their other methods like they pay out of a check or a invoice that I had to create with these companies. Whereas Boeing operates through LTP program. That message is given to Boeing employees. That is tuned to you who already have a method of LTP. For other companies, the message is slightly different. Pretty much similar, but I classify and I segregate my existing customer base who have participated with me to let them know that the next bootcamp is going to run. That's the only thing that I currently use it for, the automated emails that I send out. Back to this picture. That is my marketing engine. My marketing management app that sends out notifications in form of an email that sends and you receive it because you are the customers who I think may find it interesting. If you don't find it interesting, I also give you a way to unsubscribe. At the bottom, there is a way to remove yourself from that. If you don't like it, I give you that option to unsubscribe. That's a requirement by law. This is the law that requires us to do this is to let people have an option to get out of this marketing messaging that I'm doing right now. That law is called Can Spam Act. That's the law we have to follow. This is our US government law signed by Joy W. Bush. You can read the law, but that's the law that has certain requirements that anytime you send a marketing message, you need to include a bottom footer that says a clear method of unsubscribing. You should see that unsubscribe button and you can actually click on that button and the machine will stop sending you any future emails. That's it. No questions, no nothing else. Just click, boom, done. You're removed. That's how this marketing system works. At least for me in this example that I use Motic, but there are various ways to run the same thing. There are various and sometimes nefarious ways of operating the same thing. One of the big ones in this domain is not open source. It is called Salesforce.com. Where is that? Salesforce.com. You may have heard of it. They have a service called Marketing and the Salesforce Marketing Cloud is a very expensive solution set. I think three to $4,000 is to start and that is available and you can make use of it if you like. I don't know if you use it already or your company uses it already. I don't think you are in that business. At least I'm speaking to Boeing. I don't think you want to use these tools because you don't actually go out and market your planes, but Microsoft uses it. Microsoft has their own solution set. Let's go see what Microsoft does. Microsoft has its own solution. It is called Dynamics CRM. Microsoft internally uses this for their own purposes, which is to market to people and that is this application suite of marketing, Dynamics for Marketing preview and you try the preview. This runs in the Office Cloud and this basically does the same thing like what Motic does in a much more elaborate aspect and it is expensive. You can look at the pricing as to what it means to your cases. The beginning plans are like $210 per month. My entire cloud bill is $210 per month. This is just one service by the way. It's not right for me, but it's probably right for your companies, other companies other than Boeing. Maybe it's a good vehicle for you to consume for your own use. Either you use this solution or you use this Salesforce solution or you use another one called Eloqua from Oracle. Eloqua has been blocked by my piehole because it's a marketing website apparently and it sends out advertisements, it seems like that's apparently was black-holed or rather pie-holed by my machine. I cannot go there but I just want you to know that there is this Eloqua service from Oracle, maybe Oracle Marketing Cloud that might open. Here is the new branding of Eloqua I think. Eloqua got acquired by Oracle and they now have this marketing cloud which is conceptually similar to what we're talking about here. The concepts here are identify your customer base, divide them into segments and you send them only relevant messages that they opt to receive. They want to receive. If they don't want to receive, let them get out very, very simply. We use Salesforce at Boeing. There we go. We have an example. I don't know why you send out an email, hey, buy one, get one plane for free. Just kidding. We're not using it for marketing. We're using it for return. Yeah. Fair enough. There are tons and tons of applications that Salesforce has. I just give you an example of marketing as just one example. By the way, there are huge sets of solutions for sales and service and marketing, e-commerce, sorry, commerce and communities, collaboration, industry, blah, blah, blah, goes on. It's a big company. Same with Oracle. This is just the marketing piece, but then there are other pieces. Go to oracle.com, you will find all of them. Or in the menu, you will find more of them. So, products. And here is Eloqua, Oracle Eloqua marketing automation, Oracle responses, Oracle Blue Chi. I don't know what Blue Chi is. Data management, content marketing, social marketing, industry solutions. Blue Chi is, it's funny because I used to work with the founder and creator of Blue Chi. They do basically what we call behavioral targeting. So, whenever you have like the Amazon stuff, all those things that you checked on, now if you go to your mobile app or you start playing a video game or you start doing other websites, it will retarget you for those onion powder and the pepper, things like that. That's what's behavioral targeting and that's what Blue Chi does. Yes. Yes. It's crazy. And it is, at least the industry thinks that it is a way to get the customer's attention. So, they do that, track that all the time. And in fact, the next time I will disable that pie hole to show you the ideas of how remarketing or Google tag-based remarketing works. So, Google remarketing, and I think it will not let me open up, but here is the way to, oh, it did. It did open at least. Otherwise, I thought I would be pie-holed again. But remarketing is the Google way of doing what Rob just told you about. And you can do this on your own businesses, large or small, doesn't matter. And what this does is basically tracks people on one platform and then identify them again on other platforms, other devices, other locations and other properties like Facebook or LinkedIn and target them and show them the same thing. For example, if I search for that spice, they will show me spice ads everywhere, wherever I go. And similar thing done by other companies, Facebook pixel, this pixel that you can take and put on your website. It will track using that one pixel that you inject into your web browser on the back end of your website. That's exactly what BluKai does. And the audience science, the company I used to work for, pioneered that concept. So yeah, it was cool. So these are the ways that modern advertising works, but this is not marketing. Just remember these pixels and ads and these are advertising, which is different from marketing. I just want you to understand when you spend money, that becomes an advertising. When you're not spending money actively and buying other people's attention, it is marketing. My way of describing this thing is the difference between push and pull. If you're trying to push something, that is advertising. If people are pulling it themselves, it is marketing. If people come on their own and want to buy your thing, that is good marketing. If you're trying to push things to the customer, you're basically harassing them in some sense. That is advertising. So that's what people hate. People don't hate marketing. People hate advertising because it comes in the way. And so you put things like pie hole in the picture. So you go to Facebook and you don't see ads and sometimes you do. And so these things people hate. And there is a sponsored fans, I don't know what this thing is. Some injected feed came along, a suggested post. That is what people don't like. Because I don't want to see this Marvel thing, whatever that thing is. That suggested post is equivalent to the retargeting or retract advertisement. Exactly. So this is Wakanda. Apparently I seem to have somewhere liked the movie Wakanda or another movie. That is Black Panther. That's what happened apparently. Because you did all those Black Panther searches and everything. So here we go. Yeah, exactly. So it popped up right there in the front page. So that's what they're doing is targeting through Facebook Pixel. So that is the idea here. I want to see my friend's pictures, but apparently they don't want me to see it first. They want me to see this because they want me to buy it apparently. So if I click this picture for no good reason, I just cost them $10. I just want you to know this. If you're getting harassed by advertisements, you can punish them by clicking on their links for no good reason and closing it. And every time you click, they are paying Facebook $10 for no reason, for every click. You can punish your advertisers if you don't like them too much by not buying from them, but clicking on them. The cost is expensive to get on a feed like this. It's really expensive. About $10 to $15 on a per click basis. Yes. In all fairness, I think that hoodie would look good on you. I will buy it then. I will buy it. Don't worry. So that might actually be interesting. So I'll buy it the next time. But yeah, the idea that I think you get the idea is to be able to identify, track your visitors, and then don't push. That's the right way to do marketing. If you push, you're doing the wrong way. This is how I think. It's not black and white. These are very subjective decisions and discussions, but I'm just giving you how I think. Push is bad. Pull is good. That's a simple way to think about. And so this is remarketing and also the idea. One more, which is the last one is LinkedIn also has a similar thing. LinkedIn also has a pixel. They don't call it pixel though. They call it something else. It's called conversion tracking. So the same idea. You basically set up a tag like this in your browser, in your website that looks like adding LinkedIn insight to your website. Come on, open up and you go inject a piece of code in your campaign manager, which will look like, and I have done this in the past, like in 2014 and 2013, something like that. And so I haven't visited this quite a while ago, but I tried this and you can see that I did back in 2013, which basically generates a bit of code that you inject on the backend of your servers to let in LinkedIn track your visitors. And then you can place advertisements in the LinkedIn feed. And that is what you're looking at in. It typically is just a single JavaScript call that has, that's very, very tight JavaScript call that. And the reason they call it a pixel is it used to be in the old HTML, it used to be so that it wouldn't show up on the page. It would be an image that was a single pixel that you would put on your page that then had a bunch of query strings after that pixel. And then the system would pull those query strings and log them to the system. And then the database would pick those query string items out. And then that's how it would process and know that it came from your page. But now they do it all in JavaScript and there is no actual pixel. Yeah. So in what Bob just said is this kind of a string that you can inject. Let me give you an example of what he's talking about from a specific example, an example string that you will inject would read something like this. Now it has been disabled on my site. As you see, pause Google tag manager has been paused. This is the code. This is the live code you're looking at. And this is the actual Google tag manager tag, which includes this segment. And there is a little tag manager tag right there. This tag GTM nine five number, whatever that number is. This is how Google will track visitors to cloud genus website and keep track of who's visiting from which location, which time, which browser, what size, what font, what browser, sorry, what operating system, what device type, what resolution and location, geo position, date, time, every single thing that you can keep track of, keep that data in Google analytics and use that information to then help me advertise on Google if I want to. That's how they would track this piece of code, Google tag manager. And if you go look at that particular tag in the tag manager, you will find it has, you know, a highly sophisticated, highly sophisticated layering in there. I'll show you what I mean by layering this piece of code that I'm injecting that I'm not currently using because it has been paused. Let's go see what this actually goes under the hood and does on the Google's website. It is able to interface with a bunch of other services. For example, let's go see Google tag manager. I hope it let me open it. Google tag manager. There we go. Sign in. Tag manager. And here is the tag. And underneath that tag, I have a bunch of services. What services do I have? Latest version. And I have five tags. In there, these five tags are the first two from Google. The third one is from LinkedIn. The one I just mentioned, LinkedIn insight. The third one is pixel based code tracking custom HTML, which is I think Facebook pixel, but I'm not sure. I haven't gone to this site for quite some time. And universal analytics is another Google tracking. Let's go see what it actually does in the LinkedIn insight. You will find that it tracks using the partner ID 35223, which is the LinkedIn partnership with Google. And they're tracking all pages on which a visitor who is a LinkedIn customer also happens to visit cloud genius site will be marked as a LinkedIn visitor. And it will notify LinkedIn that this guy visited this site and also keep track of that in the Google analytics database. And that's how both Google and LinkedIn will be able to get notifications of a particular web person visiting this site. Because I will have this tag in here on the backend of the core on the backend of the JavaScript in the website, which is currently disabled right there. So it is not active. It will not show you if you go to that site, but the code is there too. I can, I can come back here and enable it anytime. I feel like I can just come here and say, you activate pretty much like talk about the other side of this. Go ahead, describe that. So the, the, the other side of this, so this is the, what we call the publisher side. So the publisher adds this pixel. And so what they can do then is every time I come to cloud geniuses website, he can now create an advertisement campaign on the other side. And when I'm on apple.com or I'm reading my email and Microsoft, I'm going to get his advertising campaign show up in the advertising section because they know that this pixel they now are what's called retargeting. They're targeting me for interest that I've shown interest in his product. And so that's the loop is the app, the publishers you can actually pay, he could pay to put this pixel on other people's websites in, in a co-op and then he could retarget people on other publishers and other sites for his product. It's a whole ecosystem out there and there's literally billions of dollars and hundreds of companies doing this. Yep. Yep. And it is a viable thing to do. It is not how I like, so I have disabled it. I tried it and disabled it and it is currently disabled for I think quite some time. I haven't actually activated it. Activating takes just two clicks and now it is active, but I will undo that and leave it alone as is not want to advertise, not doing anything. And this is a live code and I came out of it. The plumbing is there and I just don't want to use it. So I don't use it. The plumbing I have, I did experiment with the whole thing like publishing, putting an ad, see what it does. But I have, what I found is in the business that I am in, it doesn't make sense to advertise because the moment I advertise, I don't think anybody is going to pop off like, you know, thousands of dollars on attending a two month bootcamp. That's not how it works. Nobody would spend that kind of money on seeing an online ad of something and say, Oh, this guy looks really awesome. I mean, just go look at him and I'm going to spend $10,000 on this guy. Are you kidding me? It doesn't work out like that. It doesn't work. What I have found is advertising does not work. So what makes it work is you talking about it to other people. That's the only way I think it succeeds. There is no other way that I have found that it works. So all I need to do is let you know, and you will talk about it. I hope. And so I hope you need to like the program to be able to have you good things about it in your memory. So when you do these, you know, discuss with your friends, you mention it. That is how the business has been running for so many years, but not because of advertising, because advertising, if you see this guy, this orange shirt guy, you know, I don't think anybody will pay. Even I would not pay $10,000 just by looking at this guy. No, no, it doesn't work. So, okay. Let's keep, keep forward. How much time we have left? 15 minutes. How do we get a refund? Sorry. How do we get a refund? You want a refund? I will give you a refund. I'll process the refund in the original form of payment. How about that? Okay. Okay. Yeah. Yeah. I do offer a refund guarantee, by the way. So yes, it's a good question. It's a valid question. And if you don't like the program, you should actually ask for a refund because that's how it should be. You know, it's good business. That's how it should be. So the only challenge is I'll be refunding you the in the original form of payment. So that's how it goes. Okay. Let's see what else we can talk about right now. The thing that I really wanted to talk about is the other idea, which is about HTTPS. So that's what I want to talk about. I will cover other charts next time. We have a bunch of charts to discuss and go through to understand how other industries, other businesses actually make use of cloud services. We'll do that sometimes in the next session, which we have in 7th of March at that time, unless you want to change the agenda, change the topic you're welcome to please remember that you can change the topic on me and tell me to cover something else. I will do that. No problem. But my current thinking is to discuss the other remainder charts and do more exercises. Unless you want us to do something different, like go more detail with something else. For example, there is a case that I want to be able to discuss with you about handling high availability storage, cluster based exercise. We have that exercise handy and functional. We can try that if you like. Or if you like to have discussions like this with other cases, not just e-commerce, we have pictures and I'll paste those links for you. You take a look at that and tell me if you want to go in that direction and more discussions in terms of how use cases of other industries go along. We take that route. That is the current way that I am thinking of unless you want to change it. I'm giving you the option. We have two more days left, seven and nine. Hey, Nilesh. You just remind me, did you see that Digital Ocean came out with a storage offer? Yes, they do. Yeah. It's like initial, like a free, you get free storage and stuff. An email went out. I thought that was interesting. Yeah. There's a service available now from Vulture Cloud and Digital Ocean Cloud. There are two different types of storage available. One is called Spaces and the other one is Block Storage. Both of them are available now. The Spaces service, which is like S3 bucket, if you will. Yes. This is S3. In the droplet side, you will find that there is a service called Volumes service. That's a storage service, basically giving you an externally attached disk. Yeah. They've had that for a long time or what? This Block service has been there for a long time. This one, I think must have come out of beta or something. Yeah, because I got an email, so I wasn't sure if it was like an advertisement or they were, yeah, it says trial right there by Spaces. Yeah, two month free trial. You are currently using, I am on a trial already. Okay. I just got the email yesterday. Aha. Okay. That means this thing came out of beta because it was in beta and I seem to be on a trial and a subscription for $5 will begin on 13 days. I want to cancel it. I don't want to use it. Yeah. Okay. Great. I don't have Spaces anymore. Nice. And so that's a new service. What I'm talking about is not this, but Gluster. That's the exercise I want to be able to let you play with, which is a high end storage system. I think a couple of you are interested in storage ideas. So if that's your focus, I will play an Ansible exercise using Gluster if you're interested in that direction. That is functional currently. Or otherwise we'll go along this route. But for the remainder time we have right now, I think we have like 10 minutes. What I want to show you is an example of HTTPS. How easy it is. And I hope I can run that example in 10 minutes. Let's go see it. Let's go see what we can do. So I'll go to the cloud genius site to see a cheat sheet. See if I have a cheat sheet ready. It should be there somewhere. So we'll go HTTPS. Not found. How come not found? There it is. No. Where is the exercise? I think this is it. Okay. So this is the exercise we have. Let me go paste a link for you so you can also play along. It's a very easy one to do. And so the link is coming up right there. What this exercise will do is to help us put an SSL certificate in a completely automated way. What this requires is a machine running some application somewhere. So we need to get that. And then install a piece of binary available from this company called Let's Encrypt. This thing called Let's Encrypt is available from the Let's Encrypt website. Okay. Come on. Where are you? Let's Encrypt. No. Copy. Open. Let's Encrypt. That website has this getting started documentation. You will follow along and you can go that route or you can go in easier route. Even easier than that is this exercise, which is what I'm going to describe to you is this concept, which is you need to have some kind of a machine running some service. Running something here. And it is currently HTTP. You need to have a name of some sort, domain name. And that name has to match this IP address. If that matches. And then if you go to this Let's Encrypt website as a service that we run a service for, it's an open service and you run this Let's Encrypt binary right here. And what happens as a consequence is it will present a challenge to the service. The service will come back and validate you, but it doesn't know where to go. So it goes to your site by the domain name you have chosen. And if the domain name correctly points to the IP address you have at hand, then it issues you a certificate. And then you can use that certificate right here. And then once you have that certificate, your site becomes HTTPS. As simple as that. That's how we will see it happen in an example that I'm going to build right now. And so the idea is we will first thing, get a machine and then install Docker and run a bunch of things in there. That's what I'm going to do right now. So we will first thing is get a machine. So here we go to DigitalOcean. As an example, get a machine from them. So we'll say DigitalOcean, please give us a machine. And we have five minutes left, oh man. Let's see if we can speed up. Let me log in. Where's the login button? Login. And then I am already logged in. So I grab a machine there. And I'll say, give me some machine somewhere. So this is the machine I get. And I need to put a SSH key in there. So I grab this machine. And I say, give it to me in San Francisco and put my new key. My new key pair is this key pair I would like to paste. So I grab my key pair. I cat my IDRSA.PUB and pipe it to copy buffer and then paste it here. And I say, this is my IDRSA dot whatever. That's the name. Add. The key is already on my account. What? How come? How come the key is already on my account? Are you sure? There is another key in DigitalOcean that probably matches. No SSH key on the left or Nilesh. This Nilesh I think matches for some reason. So that's a good thing if it matches nice. But I like to cross check because I will be surprised if it actually matches because that's not what I expect. But maybe I did see the news. That's a good thing. Okay. I'm using the key and I create the machine. The machine gets ready and I'll connect to it. So that's what I'm waiting for. The machine gets ready and I get the IP address and I connect to the machine right here. And I'll say terminal please. And in there I will say SSH. SSH and then root at that IP address. I'm waiting for it to show up. And I will configure that machine to run Docker and then run some application and get SSL certificate loaded up. So copy that thing. Connect to it. I'm surprised Nilesh that this is in a Terraform file. What is not in Terraform file? This whole setup. We just got done doing Terraform. It should have been a Terraform file. I know but then you have to understand the concept. I know. I'm joking. I know. If you're so much into Terraform, let me give you an exercise. I'll give you an exercise by the way. The exercise will be to extend Terraform hard way. The hard way project that we did. This one. You do this thing through Terraform. So I give you some starting point. Terraform hard way. So this exercise has done some of the work already by the way. This guy has done it. But he did it like a year old ago. So it's about 2016, roughly about more than a year ago. He did something. That is useful. So grab that and extend it to make it current. That's the exercise. If you're into Terraform, extend this to match the current state of the hard way of doing things. That's the exercise. You can do it if you like. It's a good challenge. Okay. Let's move. What I want to do is in here in this machine, I want to set up Docker. So we'll go and set up Docker. Right there. So it becomes a Docker machine in a few. Once that machine becomes a Docker machine, I'll start a reverse proxy and also set up a companion to handle the let's encrypt certificate generation and automation part. So that currently is setting up with Docker. And then once it finishes up, I will then set up the reverse proxy, which you already know what it is. And from a picture perspective, what I'm really doing is I'm grabbing a machine, making it Docker. In that machine, I will be putting a reverse proxy and then a companion for let's encrypt. This companion application will run in a container. This reverse proxy also runs in a container. Then I'll actually put some application in another container. So three things. This reverse proxy proxies. This looks at the reverse proxy and interfaces with let's encrypt the external service. And this is running on port number 80. This is running on 443. And this generates certificates for you. So it uses the HTTPS certs. And the certs will be used by the reverse proxy and render everything outside in HTTPS. And so the user who visits this box will need to visit this box by a name, not the IP address. So this IP address that I have here, this address that I, sorry, this address that I logged into, I need to make sure that it currently runs my reverse proxy. So I'm running this reverse proxy right there. And then also running the companion, which is the next step. And in the meanwhile, I'll open my CloudFlare account and set up an A record for this setup that I have here in the CloudGenius domain and the DNS servers. And I will create a new DNS entry in the DNS section. I will say, get it some domain name like some other, some other domain. And so I'll create that domain someother.cloudgenius.us and give it an IP address that I have for that machine, which is available to me from DigitalOcean. So I copy that IP address and go here and say, this is your IP address. And so please set a new record for some other domain, sub domain and add it. So now my website should be someother.whatever that name. And so that is what we just did. We finished our reverse proxy set correctly and running. You can see it running right now. I will now set up a companion container to look at the proxy and actually generate certificates. That's this companion running. And then we start some application. So the application itself goes like this. So the third thing that I'm running is an application that will basically this application itself. I first ran reverse proxy, second I ran a companion container and third an application. This is the third one I'm going to run and all three of them in the Docker containers. And this is the application running now. So those three are running. And when you see that this loads up, it will automatically be HTTPS enabled because I've chosen that name, some other, and it already points to the correct IP address. So as soon as this thing runs, you will find that it's already enabled for HTTPS. So let's go visit that site and you will see that the HTTPS is already active. I'll go to some other right now. You will find that nothing is working because the NGINX setup is functional. The reverse proxy is responding, but the actual application is not in the background. Right now it just finished. So Docker PS-A should show us the application, the companion container to generate certificates and the reverse proxy. Three of them are running. Now, if I refresh the page, I see the site. If I visit the HTTPS site, it will not yet give me because the certificate is currently getting generated. If I visit and say dig some other.cloudgenie.us, do I actually reach the same address? Evidently so. So if I SSH to that location, what do I see? I see the same IP address and that tells me that the site should be generating our certificates right now. So we'll see Docker logs for the companion. So let's go first of all check the processes and find the logs for this container, which is the companion container, which will give us whether the certificate generation actually completed or not. So Docker logs and that should show us generating new certificate, retrieving, let's say in terms of service, creating an account key, saving account key JSON and so that thing is now HTTPS already. It just finished. So now let us go grab this site URL and curl it. So go curl-i and it shows you that it is currently HTTPS successful. So let's go open up a browser and in that browser we'll open up this site. Firefox browser, please open some other site and there it is HTTPS correctly functioning. Click and click and view the information about the certificate itself. You see that this certificate was issued by Let's Encrypt. It will expire in three months. The certificate itself shows that it was issued like just now on March 5th to this site called someother.cloudgenie.us by this authority and that's the quick exercise. Now it is a very, very quick one, but you saw that it worked end to end without any issues and one thing you note that this expires on June 3rd. Today is what March 5th. So in a matter of three months the certificate will expire. Ouch. Is there an ouch? Not really. It is not an ouch because we have automation. This thing is running a companion container right here and it is going to keep track of that automation. This automation companion will keep track of certificates about to expire in the next month or so and it will automatically renew certificates and get new ones every so often. So typically a certificate issued by Let's Encrypt is valid for 90 days, but this application container running the companion container will automatically renew after 60 days of issuance or in other words 30 days before expiration. It will automatically go ahead and renew the certificate for the next 90 days. That's how the system works and it is trivially dead simple and with that I will