Spring Boot 2.0

Spring Boot 2.0 has finally arrived. Unfortunately we aren’t yet in a position at the office to be able to begin the upgrade so I decided to start playing around with it on one of my projects at home as I didn’t want to wait until we were ready to update our app.

The first thing I noticed about it they removed findOne() from CrudRepository. This to me is a bad change for all the users of the framework. It is one of the most commonly used methods in Spring Data, and not people either have to refactor their code to use findById() which returns an Optional<> of the type and deal with the optional instead of a null or you have to add a findOne with an @Query to all of your repositories. I have worked on projects in the past that have hundreds of tables and Repositories. This change is forcing them to update hundreds of classes just to upgrade to Spring Boot 2.0. It seems to me a better choice would have been to @Deprecated on findOne and encourage people to upgrade to the new findById method.

The next breaking change I found was also in Spring Data. They removed the delete() method from CrudRepository that takes an ID as the type. It has been replaced by a deleteById() call. There is no good reason for this change. In Java we have method overloading so the language can determine whether to invoke delete() with the Entity or delete() with an ID. It seems like they were trying to be consistent with findById() but they just end up breaking a bunch of code in the process. The only benefit that I can see with this change is groovy will get happier as it seems to be very poor at understand overloaded methods.

The next change I noticed was when calling new PageRequest(page, size) it was telling me that constructor was deprecated. To me this is the right way to handle the situation. I clicked into the method and it suggested to use PageRequest.of() instead. That was an easy fix that I made and seems more consistent with Effective Java about using static factory methods instead of constructors.

Finally I got all my unit tests updated to replace the findOne calls with the findById changes I had made and I went to launch the application and Hibernate fell over. It looks like the new version of Hibernate requires me to specify a GenerationType of Identity with MySQL where before it didn’t force me to specify it.

All in all it wasn’t too bad to upgrade my toy application. I am really looking forward to using Spring Boot 2.0 going forward, but a little disappointed by some of the Spring Data breaking changes. I feel like this could have been transitioned in to make it easier to upgrade existing Spring Boot 1.5.x applications. It is interesting that 2.0 is the first release to support running under Java 9, and it is finally released a month before Java 9 goes away and Java 10 is released. Up next for my test application will be switching it to Java 9 to see if I see any issues with that.

MacOS Sierra Slowdown update

I have an update on my slowdown issues on Sierra. It appears the real problem lies in the AWS Java SDK. After talking to the spring boot people via github they were able to narrow it down to an Amazon issue. I opened an issue on github with Amazon and they responded that the version of the SDK that ships in the current spring cloud has this issue in it, and it has been fixed in a newer version of the SDK. One of the big value propositions of Spring Boot to me and the release train concept of Spring Cloud or Spring Data is that it is a collection of dependencies that have all been tested together, which lowers my risk of using them together. So I opened a request with Spring Cloud AWS to upgrade their SDK. Unfortunately they don’t seem very timely in responding to issues as I notice it looks like there are no responses on any of the issues raised in the last 2 weeks.

In the meantime I have a work around that doesn’t involve having to manually bump up your AWS SDK version and that is to set the following property in your application.properties file of your Spring Boot App:

cloud.aws.credentials.instanceProfile=false

Obviously setting this flag for your app running on AWS that uses the IAM profiles isn’t good, but it is a good local workaround on your development machine until the SDK gets updated in Spring Cloud.

A Two Month Recap

It has been a crazy couple of months. Since I last posted I made a trip out to San Francisco to meet the rest of the team I work with (and had a great time). If you live in Texas there is no better time to visit San Francisco then at the end of July. It was a welcome break from the heat. I had a great time and realized when I was out there that I hadn’t been out there since 2005, so I was over due for a trip. I had forgotten how much I love that city it is a really fun place to hang out (though not a place I would really want to live).

We are also about to launch our second Spring Boot Microservice. This one I am really excited about it is even more ambitious than our first one and should provide huge value to the business. The interesting new technology we used in this one is Amazon’s SQS (Simple Queue Service). I am going to say off the bat that the documentation for Spring Cloud is kind of crap, so I spent a lot of time debugging the framework to figure out how to integrate this thing. But now that we have it working it seems to work great. It definitely is pretty light on features compared to something like RabbitMQ, but it seems to get the job done and the Spring Integration is great. I am hoping to put out a blog post on how to configure it as we got hung up a few places with it.

One of the nice things about going to SQS is it forced me to upgrade our Monolith to Spring 4.2 from 4.0 as the current Spring Cloud Brixton release train won’t run on Spring 4.0. I am hoping to roll out Java 8 there as well in the next couple of weeks so that we will be sitting on a fairly current framework on our oldest app and our Microservices running the latest and greatest of everything.

We already have 2 more microservices planned out, it is just a matter of figuring out how they will fit into the product roadmap. I think once we get that delivered we will have cut our monolith down quite a bit and I hope at that point it will be small enough for me to convert it over to boot and simplify it. As I see lot of opportunities to really get everything we are doing lean which should let us turn around features for the business even faster.

Anyway that is the latest update, hopefully I can carve out some time to work on a SQS configuration post. Until next time…

We are live and MySQL settings

We’ll do it live..

The big news is our new Spring Boot micro service went into production! It has been running in production for just over a week and we have had 0 issues with it, everything just works! It ended up being two crazy sprints to get it done, but we shipped it this week with no production issues after it went live. My first big project at the new company couldn’t have gone any better. The team really pulled together to get it done which makes it even more rewarding.

MySQL Database Connections…

We did have one near miss though. It turns out that MySQL drops database connections that have been idle for 8 hours. We discovered that the morning we were about to go live. I really think this should be more widely publicized. It seems like the sensible opinionated default of Spring Boot should be to have the connection pool test the connections so that people don’t end up in this state. Anyway luckily additional testing the morning of uncovered this and I was able to find out how to fix it with some digging around. Just so I remember for the future the magic settings are:

The settings…

spring.datasource.remove-abandoned=true
spring.datasource.remove-abandoned-timeout=600
spring.datasource.testOnBorrow=true
spring.datasource.testWhileIdle=true
spring.datasource.timeBetweenEvictionRunsMillis=60000
spring.datasource.validationQuery=SELECT 1

What it all means…

So breaking it down, testOnBorrow checks the connection before the connection pool gives it to you. This is my first fix that I started with as it it has the connection pool test the connection with the validation query prior to returning it back to the application. This is good enough to fix the issue, but if all connections are killed like if the database had gone down and came back up it is a little slow the first time the app goes for a connection as it has to run through everything in the pool before it allocates a new connection for you.

The second thing I found was the test while idle and the time between eviction runs. With that every minute we test idle connections. This is great for 2 reasons, it keeps the connections alive with MySQL and if someone gets messed up with one of the connections we can be proactive about cleaning it up.

The third set of options I discovered in all this is the remove-abandoned option. This protects you from leaking connections in the application. Above it basically says if we haven’t returned a connection within 600 seconds consider that connection to be abandoned and remove it from the pool. I think with all of these options you are going to make your application much more resilient to database connection issues.

Gotta love open source and github

The Problem

I have been working on this project to make our app run in any Java Container. Currently we run in JBoss, but ideally I would like the app to work in JBoss or Tomcat, or TomEE or Wildfly. One of the challenges in making this change is to remove JBoss specific dependencies from our app and pull those libs into the webapp as part of our project. We did the first piece of this a couple of years ago when we stopped using JBoss’ version of Hibernate and pulled a newer version into our app. We have since upgraded JBoss versions so this is somewhat moot since the bundled version and our version are the same, but it is one less thing that I will have to deal with as part of this project.

The first part that I have decided to tackle is not using JBoss’ built in transaction manager and instead bundling one. Looking around it seems like my choices are Atomikos, Bitronix, or JBossTM in standalone mode. Our situation is somewhat more complicated based on the fact that we are not running an XA jdbc driver. Originally the company was using one, but at some point ripped it out because Microsoft SQLServer wouldn’t work correctly or scale with it in there. We are aware of the limitations of not running in XA mode.

Bitronix

That alone rules out Bitronix as they pretty much require XA to do their thing. I had also sort of ruled them out given that the commercial company behind them ceased operating some time ago, and ideally we want to be in a position to pay a company for support. I will say one nice thing about Bitronix is it does have good Spring Boot support.

Atomikos

Atomikos does have a paid supported version if we found we needed it down the line and they have support for a Non XA datasource. Another thing Atomikos has going for it, is that we are already using it for our persistence unit tests, though not with the NonXADataSourceBean but rather with Apache Commons DBCP. I switched over to the NonXADataSourceBean version and most of the unit tests seemed to work which was promising.

JBossTM aka Narayana

At the end of the day though my thinking was the lowest risk solution would be to use JBossTM in standalone mode. Given that we are already using JBoss’ built in version of it this seems to be the least amount of change we can do. The first challenge is actually finding anything out about the project. Just searching JbossTM or JBoss Transactions doesn’t find much for you. I eventually discovered it is called Narayana. Ignore the horribly designed website, I just found that tonight, I originally found it on the JBoss home pages and on github. JBoss has a quickstarts example github repository showing you how to use their stuff as well. The Spring example was sort of what I wanted, but not exactly so I figured why not just do a quick Spring Boot app as a proof of concept. So I forked the repository and added a Spring Boot app. I hit a problem though, I created a unit test to try out reverts my insert wasn’t reverting on a Runtime exception. Just to make sure I wasn’t completely crazy I created a quick Spring Boot app doing the same thing using Atomikos. That test app worked perfectly.

At this point I feared I would be forced into switching to Atomikos as that was all that I could prove was working with a trivial test. I went over to the JBoss forums as a last ditch effort and posted a question about it. Back in the day when I would post things to forums I would often get very little response or things would be vague cause it is hard sometimes on the other side of a forum to be clear what a person is asking about. In this case I could just include the links to the two github repositories as examples of what I was trying to do. I posted it and didn’t hear anything back on the day. The next morning I woke up and saw that at 3am a Redhat Engineer in China had forked my project and provided a fix. Immediately I could see what I was missing and he made my test pass right away. Back in the day I probably would have given up and just gone Atomikos, but with github it is so easy to share sample code of what you are trying to do and Spring boot makes it so fast to stand up a sample app there is really no reason not to do so anymore.

 One more note while I was working on this experiment in testing how to configure different JTA managers the Narayana guys submitted code to Spring Boot 1.4 Milestone 2 to include a Narayana Quickstarter so now it will be possible to easily wire that into a Spring Boot app. I know I talk about the greatness of Spring Boot all the time, but once again it saved me a ton of time by allowing me to prototype out a few options before I go too far down the road of solving the problem, and with github allowing me to collaborate with people around the world, we truly live in some amazing times.

Spring Boot and Security using Spring Data JPA for authentication

Recently one of my friends was working on a Spring Boot project and he was having trouble finding an example of how to configure user login for his site with Spring Boot using JPA. I had mentioned that there is some mention of configuring security in Greg Turnquist’s book Learning Spring Boot. He had just purchased Spring Boot in Action and I don’t think he was rushing to grab another book, but he hadn’t been able to find a good online tutorial.

Today I was browsing Twitter and I came across exactly what my friend was looking for. Priyadarshini B wrote a great online tutorial on doing just that on her blog. The post was so great I wanted to share it. So head over there and check it out, you won’t be disappointed as it will walk you through the whole process.

JavaMug Spring Boot Discussion

I attended JavaMug last Wednesday as the speaker was Craig Walls author of Spring Boot in Action. When I heard about the book I had planned on purchasing it, but was disappointed there was no kindle version on Amazon. It does state if you purchase the print edition they will give you the kindle one for free, but I am trying to move away from paper books in general.

Overall the talk was pretty good. It is nice that there is a Pivotal employee local to the area so we can get a talk like this done. For most of the talk Craig just sort of demonstrated examples of what you can do with Spring Boot since there were people of varying degrees of experience with it. It was held at Improving in Addison which I had never been to, but they had some nice beer on tap (Kentucky Bourbon Barrel Ale). In a talk like this where you are just trying to introduce the concept to people it is hard to get as deep of a dive as I would like. But I did enjoy the part of the demo playing around with the metrics. That is something I haven’t really played around with, but of course got me immediately thinking about how much I would like to use that at work. I think maybe this year I will attempt to convert our legacy app to Spring Boot. It will be painful, but it just seems like more and more the benefits are so good that is what we should be doing. Hopefully I can find the time at work.

Then at the end of the talk they did a raffle and lo and behold I won the digital copy of Spring Boot in Action. I was pretty stoked about that given that I wanted that originally to begin with. When they get it to me and I get a chance to read it I will post a review up here.

Spring Boot Actuator Guide

One of the most interesting Spring Boot features to me is the Spring Boot Actuator. I just love the concept of having all these restful endpoints to get useful operational data right out of the box. One issue I have had is that I don’t actually know all the endpoints just a few that I have used here and there. Today I saw a link to the Complete Guide for Spring Boot Actuator which is a pretty amazing post that covers everything and anything that you would want to know. The purpose of this post is just so I can go back and find this data later. If you are playing around with Actuators go and check out this post.

Spring Boot for prototyping

I am on a new project at work that looks to be very interesting. I am redesigning our Cassandra layer. Currently we have a beautifully done layer that was designed and implemented by our former architect. It ends up making Cassandra look just like a JPA entity and we have Cassandra Repositories that look just like Spring Data JPA Repositories. After this was in place we discovered the Spring Data Cassandra project. We went to the talk on Spring Data Cassandra and it turns out they had implemented pretty much the system that our architect implemented.

Now my goal for this project is to create a higher level Cassandra abstraction for our system. Often in Cassandra we create multiple tables to represent the same data. The reason is depending on how you structure the data in CQL determines what queries you can run. We have the need to query some tables in many different ways so we need multiple tables to be able to answer the questions that we could in the SQL world. In our architecture we don’t want the developer to worry about which table they need to query for a given question, we would like to present this more like a standard JPA entity where the developer doesn’t need to worry about it and we abstract away which particular table is being queried or which tables are being saved to.

One of the initial thoughts of this project was to use Spring Data Cassandra. The advantage of that is we would have a third party library we could use so we wouldn’t have to maintain the low level Cassandra code. At one point I was considering ripping out our custom Cassandra Data layer and just using the Spring Data one (prior to this project) as ours is so close to what they are doing anyway why should we maintain that code when there is a perfectly good library we could use and we are already using Spring Data JPA. Right now I am not sure if I want to do that anymore though. If I go with Spring Data Cassandra each of the underlying tables would need to be modeled as a separate entity like our current framework. In that case then I would need a second layer on top if I am trying to present 1 domain object backing multiple tables so this seems like it isn’t ideal for what we are thinking. Another thing I am not 100% on with Spring Data Cassandra is that it isn’t backed by Pivotal. It is a community plugin. That isn’t necessarily a problem but I do have concerns about how up to date it will be. One way to look at this is maybe I should be involved in the plugin since clearly I could be working on that as well, but I get a little concerned about if it will be around otherwise. I noticed that the Spring Data Cassandra project is built off of the Datastax 2.0 driver. So if we are running off of Cassandra 2.1 (which we probably will be by the time this project goes live) we aren’t necessarily taking advantage of the new features. Again I could submit a pull request to update it, but given that I am not sure this is the right approach anyway at this point I haven’t decided to go this route (though that may change). Another thing I noticed is Spring’s initializr doesn’t list Cassandra as an option out of the box. Though I suspect if you checked JPA and then added the Spring Data Cassandra to your maven pom or Gradle file it would probably work.

So how does this tie into Spring Boot? Well I am not a big design everything up front kind of person. I tend to like to play around in code to come up with my design ideas. So I decided I want to prototype against what we are doing so I can see how this design would play out in code. I downloaded a new Spring project from the initializr with AOP, JAX-RS, Actuator, and Shell support. I then pulled our code for our Cassandra layer into there and added the Cassandra driver to the maven pom. I updated the application.properties file with my Cassandra database properties and low and behold I was up and running with a framework to test all of my changes in. Even though I have played around with boot, seeing myself able to build a sandbox to play around with this design in so quickly was very impressive to me. So now I can evolve this design without messing up our current code base and when I get to something I am happy with I can just bring those changes into back into our branch and we will be ready to go with our new design. So even though I am stuck with a monolith and I couldn’t easily bring boot into our main product the design of our project is such that it was very easy to pull that whole layer into this project and just be up and running with almost no configuration (aside from setting up the Cassandra connection properties).

So as usual I think Spring Boot is the bomb and everyone should be using it. I look forward to seeing where this design takes me. Once I have my high level sorted out I may try playing with dropping Spring Data Cassandra in the low level to see if we want to use that as our base, but my current guess is we won’t end up going that route.

JHipster webinar

I saw this come across the Spring blog this week. They are going to be doing a webinar for JHipster. As I mentioned in a previous post I am very interested in JHipster as it combines 2 things I am interested in learning Spring Boot and Angular. If you are interested in checking it out sign up here.

Also as a completely unrelated side note, why doesn’t projects.spring.io support HTTPS? This is 2015 and all sites should really support secure access.