Categories
general

Happy New Decade

The 2000s

With the turn of the new year it got me thinking a bit about the change in the decade. Hard to believe it has been over 20 years since I was celebrating the change of the century with my friends in Minneapolis. Things have changed drastically since then. At that time I was 6 months into my career. I had joined a telecom startup and moved to Texas after graduating from the University of Minnesota. The 2000s were a decade of change and growth for me. It had me working really hard to get established in my career. Going through a couple of acquisitions and finally moving into my second startup. I met my wife then and got married, and we transitioned from apartment life to having a house. We traveled as much as we could. Mexico was much safer back then and we explored a lot of central Mexico as well as trips to Europe every year.

The 2010s

Along came the 2010s and it was another big shift for us. The 2010s were the decade of having children with 4 children to join our family. It is also when I got really career focused as well and decided that I didn’t just want to be a coder, but I also wanted to really move up and make an impact at work. I started focusing on what I could do to drive more value individually which lead me to moving into architect roles, team leads and technical leadership type opportunities. I also left my 2nd startup and took a corporate role which taught me many things that I needed to be a more effective technical lead and a better software development process. I then left that corporate role for startup 3. This was the first real turnaround startup that I did and we successfully turned the company around and were able to bring it to an exit. I also worked on the highest performing software team of my career at that point there and really saw how to build an engineering machine that could consistently crank out software and value for a business.

After we sold that company I needed to land a job as we were expecting our 4th child. That resulted in me taking another corporate job. That role had me with a great team, but in a business that had all sorts of issues. My main learning there was in many ways things not to do. I left there with some really good friends, and really happy to be out of an unhealthy work environment for me to take on startup 4.

Startup 4 was the first time I was the first engineer, and really got to build a complex system from a founder vision into reality. It has been extremely rewarding and I have learned a ton in the process. We took a vision and have driven it to an early access release that we are evolving into a full featured product.

The 2020s

That leads me into where we are now. I have started to think about what the focus of this decade will be. This next year is going to be crazy and we are going to accomplish big things as we hit product market fit and our startup takes off. In many ways this is going to be a hyper focused decade when it comes to career. I am at a point where I can make a huge impact and I need to really dig deep in my growth areas and help bring us to the next level. I think your 40s are really your prime earning years as you have so much experience you can really drive huge value to a business. Ideally we will have a huge success in my current company and maybe another startup by the end of the decade. Whatever happens I am sure it will be interesting.

Categories
Uncategorized

2019 Year in Review

Wow what a year. It has flown by and as you can see I haven’t gotten to the blog since I upgraded the site earlier this year. Looking in my drafts it looks like I was intending to write a post about travel, but I didn’t get back to it. Anyway given that the year is rapidly ending I figured it was time to start looking back on it.

I never imagined how busy this year would be. We went from the start of the year where we created a plan on what we were going to build for the year to drive to an MVP. We were building out what we thought would be our alpha release, but discovered in user testing that we needed to pivot. Along the way one of my employees left. We then drove the new concept to an early access release grew the engineering team and it has been an amazing ride. 2020 is going to be focused on product market fit and hopefully upon achieving that, rocketing into a growth phase. Given how crazy it was to build what we have built on such a lean team a lot of my themes for the year fell to the wayside especially this blog. When you are super busy and tired this is an easy thing to drop.

I feel like I have gotten much stronger in Go this year which has been nice growth. I no longer feel slow in it as compared to Java like I did for so long. I appreciate many aspects of the language, but also find a few areas lacking. I have gotten very comfortable with Kubernetes and Bitbucket pipelines and built out a nice CI/CD pipeline this year to allow us to run faster at work.

We also spent June traveling which was nice. I took advantage of working for a remote company by working from Mexico for a month and for the family taking a 2600 mile road trip in June. It was great to spend time in places I haven’t been for 10-15 years as well as spending time at my in-laws so the kids could hang out with their grand parents and some of their cousins. I would like to incorporate more remote work in the future.

The big challenge I see going forward in 2020 is moving myself out of the operator mode and into a more strategic role. Most of my past roles have been about execution getting on the ground and grinding out the product to drive us forward. For 2020 I need to be more much strategic in the business and not just on the day to day to really help drive the vision forward. Without further delay let’s review my themes for the year.

Theme 1 – Health

I did fast through Easter and lost about 20 pounds. Some of that has slowly crept back but it was a good start to the year. I think for 2020 I need to go broader and think more along fitness goals. This needs to be some amount of walking and maybe some amount of strength training. I need to think on it yet over the next few days. I have many times made better choices food wise this year. I eat more salads that I used to, but I probably need to continue this and extend it more.

Theme 2 – Learning

There has been a lot of learning in 2019, but not in the way I envisioned. I did most of the Coursera Machine Learning course, but got busy near the end of it and didn’t finish it. I thought the course seemed too much like a CS course in that you are building everything from the ground up. While most of my degree was done that way that isn’t what I am looking for anymore. In reality I am not every going to build all these ML techniques from scratch, I am going to use a library or tool that someone else built. I think a more valuable class would be an overview of what the different available techniques are and when you would try different approaches to get different results. Also more time needs to be spent on teaching people data exploration and sort of things you can try on a given set of data.

I also didn’t get all the reading done that I had set out to do. I found that I was so tired in my evenings it was difficult to do a ton of reading though I did get some things read.

I think most of my learning in 2019 was really focused on learning how to build a team, and build a product from 0, and really learning how businesses work. I really appreciate the view I am getting and what I am learning about building a company and getting to see the broader view of how sales, marketing, finance, and customer all really come together to drive a business forward. We also focused on leadership training this year and I was fortunate to go through that program.

Theme 3 – Security

This theme was a resounding success, I have gone all in on the password managers with really long complex passwords that I don’t know and 2 factor authentication. It was a good change and I can’t imagine going back to the old way of doing things.

Theme 4 – Public Speaking

This theme didn’t happen and was an epic failure. I started the year out noodling around an idea for a talk and putting some stuff together, but ended up getting way to busy again to devote time to it. I think this is something I will need to revisit in 2020 as it will be important to get our name out there in the community as I think this could really lead to us getting a nice hiring pipeline. The perfect scenario would be there are so many people that want to come and work with us, that when we have an opening there are tons of people competing for the role and we can continue building out a strong team of really talented people quickly without some of the hiring lag I faced this year on getting the right candidates. The only way we are going to get to that is if people around here no who we are and what we do and want to be a part of it.

Conclusion

All in all it was a bit of a mixed bag on my plan for the year. It was a good year though, and now I need to start thinking about my themes for 2020. I am hoping to get some more rest in before I get back at it in January unfortunately it has been a pretty busy holiday so far so I am hoping I can slow down for a bit for a couple of days.

Categories
general

Site Upgrade

My site was down for much of today as I finally took the plunge and rebuilt my image from ground up. This was something I had considered doing for some time now. The old site had been through a couple of ubuntu upgrades and was chugging along, but I always thought it would be good to do a fresh install. However I never really wanted to dump the time into it. Then I saw this article: https://www.anandtech.com/show/14284/amazon-offers-another-amd-epyc-powered-instance-t3a

I have been interested in AMD’s latest processors as they seem to be killing it with their Ryzen line. I checked the pricing and saw that the price for running my site was about half of the cost of what I was paying for my old t2 instance. Additionally at the same price level I was getting more CPU cores with the new instances. I hadn’t even realized that Amazon had rolled out their regular T3s or these new T3a instances. I figured it would be an easy switch, stop my instance, change the instance type and start up the new instance.

I attempted that with no luck. It seems that since I setup my site Amazon had come out with a new Network card driver, which my Linux kernel didn’t have. I switched back to my T2, and decided I would start building my new image on the side. When I had time I would spin up my new image and start working on the Amazon setup. I was happy to see there is now IPv6 support available. All in all there was just lots of new stuff to play with. I finally reached the point yesterday where I decided to backup my site and finally go for it. I had hit a stopping point where I brought my nginx config over (which had all my SSL settings), but I didn’t have my DNS pointed to the new machine in order to pull down new certificates. Without the certificates nginx wouldn’t launch, and at the same time because of HSTS preloading I can’t test an unencrypted version of my site. I pointed to the new IP this morning in my DNS settings and then proceeded to start working on restoring the backup of my site while I waited for the old DNS cache entries to expire so that I would pull down my new certificates.

Right around that time I had to depart as we had mother’s day plans and thus the site needed to remain down. When I got back on tonight I finally finished restoring my backup of wordpress and we are back. Now that I am back up I realized that my SSL Labs score dropped so I have been tweaking my configuration to get back up to my high A+ score.

Future projects now will include figuring out IPv6 routing. I was able to assign an IPv6 address to my instance and VPC, but when I try to SSH to that address it seems to time out. When I get that working I would like to setup DNS to allow getting to my site solely with IPv6. Anyway that is all for now as I need to go to bed, but more updates coming soon hopefully.

Categories
Uncategorized

The Chug

So far this year I have not been keeping to my blogging schedule. Since posting my themes for the year I have been AWOL too busy working on launching a product in my current role. That being said a friend of mine just launched a new show on YouTube called The Chug. If you are into craft beer like I am check it out at: http://thechug.life

Categories
general

Themes for 2019

I am late to the party. Normally I tried to take off until Epiphany before I return to work, but I needed to start working on the 2nd so I wasn’t able to get to this post until now. As is my tradition I set my themes for the year that I want to focus on. I find doing so helps me stay on track and make sure that I am growing and improving myself.

Theme 1 – Health

I am carrying this one over from last year but it is always a good one for every year. In my feasting for the holidays I put on a little weight so I am going to attempt to lose 20 pounds before Easter. I hadn’t decided what I would do at the start of the year, but on Epiphany I decided I would do intermittent fasting and cut back on beer (and try to eat fewer carbs when I am eating). So far I am down 9 pounds 2 weeks in, so I should be able to exceed my minimum by Easter since the fasting will continue until then.

In addition to the health benefits of losing weight, I also would like to look better. Later on this year I will be involved in pitching for a series A and I want to look my best in those meetings. Ideally I would like to be more active this year too. When I was testing the Apple Watch that Sofi had I must admit the gamification aspect of closing rings was getting my to walk more and when I was walking was pushing me to walk faster to keep the heart rate up and close the exercise circle. So maybe I will take the plunge this year on one. Though at this point if I do that I would probably hold out for the series 5 in the fall.

Theme 2 – Learning

I have a couple of things in this bucket for the year. First is the machine learning course that I mentioned a few weeks ago. I am just about finished with the week 2 work and I can already see the benefits. It is already kicking my brain into gear thinking about the problems that we are trying to solve at work. Additionally I feel like the base knowledge will enable me to ask the right questions and help keep the ball rolling on that part of our system.

The second area of learning is going to be focused on is reading. Traditionally this is something I really push myself on, but I have fallen away from it a bit in 2018 just with interrupted sleep due to the ages of my children. Now that I am past that stage 2019 is going to be refocusing on getting through a bunch of books. Currently I am reading Leaders Eat Last which was a Christmas gift from my boss. I want to get back into my old pattern where I would try to read 10% of a book a day so I can get through a book in 10 days

Theme 3 – Security

Given all the sites that are constantly in the news each year with getting data stolen, I have decided it is time to take my password security to the next level. Instead of hard passwords I memorize I have switched to using a password manager and generating really long and random passwords that I don’t know. This makes is possible now to have no repeated passwords anywhere and I have found that it has already simplified my life.

I am also considering going the hardware key route instead of just using an authenticator app for 2 factor passwords, but I haven’t yet decided if I want to do that.

Theme 4 – Public Speaking

I have never been very comfortable speaking in public. Anyone who knows me knows that I am very talkative after I get to know a person, but something about being on a stage with all that attention directed at me has always made me uncomfortable. That being said it is a skill that I need to develop to go where I want to go in my career. Beginning this month I will be speaking at our board meeting, and then with needing to fund raise later on it is time for me to conquer this.

Fortunately the timing couldn’t be better in that I was also invited to speak at a developer meetup this year. I can’t think of a better way to practice this skill than to be speaking to a friendly audience of developers discussing technical topics. As that is an area in which I always enjoy speaking. Over my holiday break I began work on a talk, but when I realized it was going to take too much time I pivoted over to the Machine Learning class first as that is a greater need for me. Once I finish that class I intend to focus my time on putting this talk together and then I will get myself on the Calendar as a speaker (hopefully by Q2).

Conclusion

Those are my areas of focus for the year. As always I will do the run down at the end of the year and see how they turned out, as well as what popped up that I didn’t predict for the year. May everyone have a happy and productive year.

Categories
general

Coursera Stanford Machine Learning Course

I just signed up to take the Stanford Machine Learning Course for free on Coursera. Anyone who wants to take the course with me is welcome to join me over here: https://www.coursera.org/learn/machine-learning. They are only accepting enrollments until January 12th I think it said so not much time left to sign up.

Categories
general

2018 Year End Review

Recap for 2018

As is my tradition it is time to review 2018 and see how my year unfolded. The first thing that I always do is review my themes for the year and see how many of them I hit.

In general 2018 was an interesting year. The year started out with me on vacation. When I was on vacation I realized that I would have to change jobs in 2018 and luckily it all worked out for the better. I can say making that move was one of the best decisions I have made in my career. The difference between the first half of 2018 and the last half was a study in extreme opposites on the career front. Probably the biggest takeaway from that whole situation is that company culture may be the most important thing when it comes to choosing a position.

Without further delay here is how I did for 2018:

Health

I didn’t drop as much weight as I had hoped to at the start of the year. I think I lost 20 pounds in the first quarter and kind of slacked off after that. There will be another push for that again this year as I always like to start Q1 as a quarter to fast since Q4 for me is a quarter to feast. I haven’t figured out what approach I Am going to take? In the past I have done the Primal Prescription, Slow Carb, and Intermittent Fasting / Time Restricted Feeding. I will consider what approach I am going to take over the upcoming days. I also did DNAFit this year so it would be good to try to incorporate their diet ideas for my specific genes in whatever I decide to do. I should also push for more regular activity, so maybe in the end I will need to break down and get an apple watch just to use the data to drive accountability for myself. Not sure yet I have been holding out for it to do more than it currently does before I take the plunge. I do like that the series 4 at least has a larger display.

I suspect 2019 will involve more public speaking for me based on how things are looking so I also want get healthier for that as it is nice to look good if you are on a stage or the center of attention.

More Blogging

This theme was an epic fail. As I check my posts not counting this post there were a paltry 13 blog posts this year. Barely an average of 1 per month (vs my goal of 52 in general). This one I don’t even mind missing this year because it was for a good reason. I had so much stuff to learn for my new job this was one of the easiest things to cut to get more time. As I have settled in I hope to do better this year, but in general if I am pushing for a sprint deadline this is an easy thing to cut so I can get more code across the line. Once we get our product to market I hope to be able to plan more reasonable sprints.

Learning

I nailed this theme. It has been a great year for learning. Instead of the learning being books I read it has been practical things. First I learned Go Lang and implemented a micro serviced base architecture in it. I am still doing a lot of learning in Go. I haven’t yet figured out really the ideal patterns yet. One thing that I miss from Spring is how opinionated the framework is. There is a remarkable amount of consistency in code when you have the strongly defined stereotypes like that. When you see Go on the internet it is all over the place. Sometimes it feels like node code, sometimes it feels like C code, but it never really feels like people are using the same consistent patterns. I have played around with sort of the controller / service / repository layers and not sure that is the correct idiom. I have played around with just general event handlers and middleware patterns. I think this year will still be me trying to figure out what is the best patterns to use for backend services.

On the business front there was also a lot of learning for me. For the first time in my career I built an agile process that our team uses, and got a brand new team that didn’t know each other at the start off to a very good working cadence. Our process has worked so well that other parts of the organization have adopted it as their baseline and started iterating on it to adapt it to their teams. I have also applied some of the things I learned from domain driven development to build a model that allows the business and the technology teams to communicate with the same language.

Architectural Changes

This was underway when I left my previous company. We managed to convert some of our apps to Java from Groovy. We were doing this on a piece by piece basis at first going after the easy parts. We were also doing some restructuring to address some performance problems in the architecture around the time of my departure. This I would probably only give 50% to. Had I remained in that role I am sure we would have made major changes by the end of the year, but the organization had become too political for me and I got the opportunity of a lifetime at the same time.

Conclusion

All in all 2018 was another great year. The year ended much stronger than it started and I feel like I had a lot of personal growth this year. Looking forward to 2019 it is going to be a huge year for our company and for me personally.

Categories
general Security

Site Upgrade

I decided to upgrade my site to the new version of Ubuntu as I haven’t done that for a couple of years. It is always a nice thing to work on when I am on vacation as it is the sort of thing that I don’t really get around to normally when I am busy. What a pain that ended up being.

The Upgrade for the OS itself went very smoothly as it seems to normally do so for Ubuntu. But the upgrade to the newer version of PHP broke everything with my site. As I think back actually I think this happened last time when I went from Ubuntu 14.04 to 16.04 as well and it jumped from php5 to php7. I ended up with about a 3 hour outage trying to sort everything out.

The big issue I saw was the default user that php was using was different than the nginx user so it couldn’t write to the Unix Domain Socket. I also noticed all the configuration advice for nginx was very different than when I set up this site. It seems like things are laid out better with the whole snippets of different configurations instead of sort of everything going in the default.conf. At some point I may want to start over again from a blank AMI image and import my content into it. Then I could setup nginx a little bit more modern. Seems like there are lets encrypt plugins for it too, so I am wondering if I could have it auto-renew my certificates.

Another thing I could do if I redid the site, would be to switch to mariadb. I have heard it is supposed to be faster than mysql it might be fun to mess with something different. That being said I probably won’t get to that this winter as I am currently working on some content for a talk and I also want to spend a little time doing some machine learning classes on Coursera before I get back to work.

I did take advantage of the time in the config files to figure out how to tighten up my SSL Labs score. I found that I was missing just 1 item to push my key exchange test from a 90 to a 100 so I implemented that. I was hoping to be able to turn on TLSv1.3 as well, but unfortunately Ubuntu 18.04.1 ships with a version of OpenSSL that is too old to support it. I saw on a mailing list that it is supposed to be coming though so hopefully soon I will be able to update to that.

Categories
Java

Reactive Streams Talk

Of all the regular conference speakers on the Java circuit Venkat is my favorite. He has the ability to break down complex topics and make them very simple to understand. In addition to all of that he also shows you why you should care, and how whatever he is presenting can make your development life better. I always hope that when I am explaining something that I can do it 1/10th as good as he does as then it will probably come across pretty fairly understandable to the person that I am talking to.

Venkat has done it again, I was looking at some news in my blog feed and one of the stories had a link to a talk he did at Devoxx Belgium (a conference I have always dreamed of attending) on Reactive streams. In 2014 I was at Venkat’s talk at SpringOne and it completely blew my mind and showed me why I needed to embrace Java 8 Lambda’s and Streams. With this talk on Reactive he is taking the same approach. Breaking reactive down in a way that you can relate to the Java 8 streams features and showing you why it matters. If you have a spare 2.5 hours this talk is well worth it:

Categories
Uncategorized

Google Kubernetes Engine

Introduction

I have been messing around with Google Kubernetes Engine for the last few weeks now (as we are deploying my new app to it) and I have to say overall I am impressed. There has been a lot of talk about Kubernetes for a while and at first I was wondering if it was just the next piece of tech being over-hyped like so many things. Having used it now for a month I can say I understand why people are so excited about it. The learning curve is steep, but once you climb it, you will really appreciate the power of the platform.

As stated in my previous post, I have been building a Micro-service architecture in Golang and for deployment we decided to go with GKE. Go seems to be extremely friendly for docker containers, I have been using Alpine as my base and the container size of each service has been really tiny (around 10MB or less). This is quite a difference compared to Java containers that end up very large when you think of all the jar files that go into a typical spring boot app. There are a few things that you need to do to build your Go app for docker. You need to disable CGO so and tell the build to use Go’s networking for DNS resolution and not to rely on GLIBC’s as Alpine is built on musl libc. The other great thing about Go is the cross compiler is built into it, so you are an environment variable away from being able to compile your app for Linux even when running on a Mac as I do. The only other thing I do is add ca-certificates to my alpine base for SSL connections.

History

Initially when I started building my backend I was using Spring Cloud Gateway and Consul to handle load balancing and service discovery. When I went to bring our services to the cloud I discovered those would no longer be needed as Kubernetes has built in load balancing and service discovery. The integration between Spring Cloud and Consul is great, and what happens is you can just register your app name in consul and spring cloud will automatically route to it based on name. I have an Auth service named auth and I would hit spring cloud at http://localhost:8080/auth and it would look up the auth service in consul and route the request there automatically to my auth service which was running on 9999 at the /. I wanted to keep the same sort of approach in my Ingress and service discovery in Kubernetes.

Kubernetes the beginning…

 Initially I started using the default Google Ingress which behind the scenes provisions a Google Cloud Load Balancer (HTTP/HTTPS) load balancer and then should route the different requests to the different services based on your kubernetes service and Ingress. I was having issues getting the Google Ingress to work correctly with the URL rewriting that I wanted to maintain from my previous design and I discovered a second issue with it, which was it doesn’t support Websockets. We are planning on doing a lot of app communication between the backend and front end via a  websocket and this was going to be a painful limitation for us. We considered using Server Sent Events to push events to the client and rest calls for the messages from the client, but this wasn’t ideal.

I did some digging around and discovered that you can install NGINX as your ingress and behind the scenes it provisions a Google Cloud Load Balancer that is a TCP Loadbalancer. The advantage of this is that we can now send websockets into our cluster. Instead of our SSL terminating at the GCLB, it would now terminate at our Ingress when we were ready for it. As soon as I switched to NGINX all the URL rewriting issues I was seeing went away.

TLS Configuration

Once I had reached a point of having traffic into my cluster from the outside, I decided it was time to provision TLS. I discovered Certmanager which allowed me to configure my certificates through Let’s Encrypt my preferred certificate issuer. When I ran the SSL Labs test against my cluster I discovered that SSL was configured really well and I scored an A+ on their test. The only issue I am having is trying to figure out how to enable support for TLSv1.3 I haven’t been able to get the Ingress to support that even after messing with the NGINX configmap. It is supposed in the current version of the NGINX Ingress, but it is disabled by default and I am still fighting that part of the config.

Configuration examples

The final challenge that I faced was supporting multiple URLs for my cluster and just routing the Ingress based on what URL was being requested. I created a fanout Ingress which worked great but I struggled to find a config that would allow me to have multiple secrets for TLS depending on the URL. I finally found that config and thought I would share it here (which sensitive details changed).

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/rewrite-target
: "/"
nginx.org/websocket-services
: haskovec-api-service
name: nginx-haskovec-api-ingress
spec:
rules:
- host: api.haskovec.com
http:
paths:
- backend:
serviceName: haskovec-api-service
servicePort: 9999
path: /auth
- backend:
serviceName: haskovec-api-service
servicePort: 9998
path: /service2
- host: service2.haskovec.com
http:
paths:
- backend:
serviceName: service2-service
servicePort: 9999
path: /auth
- backend:
serviceName: service2-service
servicePort: 9998
path: /service2
# This section is only required if TLS is to be enabled for the Ingress
tls:
- secretName: api-certificate-secret
hosts:
- api.haskovec.com
- secretName: service2-certificate-secret
hosts:
- service2.haskovec.com


The above Ingress shows using multiple hosts connecting to multiple pods behind a service and routing based on the URL and domain name. It also shows what multiple TLS certificate secret stores look like.

To create your certificates with Certmanger you will need to configure an issuer  as below (again details changed):

apiVersion: certmanager.k8s.io/v1alpha1
kind: Issuer
metadata:
name: letsencrypt-production
spec:
acme:
# The ACME server URL
server: https://acme-v02.api.letsencrypt.org/directory
#Email address for acme registration
email: jeff@haskovec.com
#Name of a secret used to store the ACME account private key
privateKeySecretRef:
name: letsencrypt-production
#Enable the http-01 challenge provider
http01: {}

And a certificate:

apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
name: api.haskovec.com
spec:
secretName: api-certificate-secret
commonName: api.haskovec.com
dnsNames:
- api.haskovec.com
issuerRef:
name: letsencrypt-production
kind: Issuer
acme:
config:
- http01:
ingressClass: nginx
domains:
- api.haskovec.com

You configure an additional certificate as above for each domain name that you are getting certificates for. And just like that certmanager automagically goes out to let’s encrypt gets certs for that domain name and stores it in the referenced secret.

Conclusion

All things considered I am blown away by working in a Kubernetes environment. Google takes away the pain of actually provisioning your cluster so you can focus on your app. Deployments are a breeze, I just push all my new docker containers, updated my deployment yaml file and apply it, and kubernetes does a rolling update of all my services. This is allowing us to basically have infrastructure like we are a company with a huge devops team when in reality we have no devops engineers. I will definitely be using this on my projects going forward!