Friday, June 17, 2016

Microservices with Spring Boot, Spring Cloud, and Spring Cloud Services


Lately I've been helping a lot of teams accelerate their journey into developing 
microservices with Spring Cloud Services and Pivotal Cloud Foundry.   Most of the teams already have a good foundation in Java programming, a good background in CI/CD, and many of them have some level of devops experience from cobbling together some variant of a container management strategy.   However most of them haven't experienced the productivity boost they get from working with Pivotal Cloud Foundry and Spring Cloud Services.  In the process of helping these teams I've amassed quite a collection of material to help people learn how to use the higher level patterns in Spring Boot, Spring Cloud, and Spring Cloud Services to build microservices based systems.  In this post I'll try to share this material in a consumable "playlist".  This isn't just an exhaustive list of every resource available, instead I've hand picked content to help accelerate your understanding of developing microservices using modern Spring components.

Microservices with Spring Boot, Spring Cloud and Spring Cloud Services Playlist


Great Background Material:

These background materials are not pre-requisites for developing microservices, but they are very helpful for setting the context and laying the foundation for modern development practices.  I strongly recommend consuming all of this material somewhere on your journey to building modern projects with microservices.

The Phoenix Project
Great book that puts the best practices and anti-patterns of modern development into a narrative story form.  Highly recommended background for everyone near software development projects.

Release It!
Great book on modern practices to deliver successful software projects.

Continuous Delivery
Foundation book that provides great coverage of the entire topic of CI/CD.

Continuous Delivery Live Lessons
Live video lessons from Jez Humble.  Hearing the stories direct from Jez provides even more insight into CI/CD.  Great refresher if it's been a while since you've read Continuous Delivery.

Domain Driven Design
The seminal work on domain driven design

Implementing Domain Driven Design
A more practical book on how to put domain driven design to work.

Building Microservices
Great book on the foundation concepts of microservices

Beyond the Twelve-Factor App
This book takes the idea of Twelve Factor apps to the next level by adding additional criteria for a well build cloud native app including API First approach, adding telemetry, and security.

Conceptual Material:

This material starts with the concepts of microservices and the Netflix OSS components

Gene Kim at London DevOpsDays 2016 - 35 minutes
Excellent short course from the master on why we do DevOps and Microservices

Adrian Cockroft - Microservices State of the Union - 50 minutes
Adrian Cockroft Discussion the State of the Art of Microservices.  It provides a great overview of what's important in the microservices world.

Java World Article - How to build self-healing distributed system with Spring Cloud - 10 minutes
Great short overview article by Matt Stine on what Spring Cloud Services is an how it relates to the Netflix OSS components.

Developing Microservices for PaaS with Spring and Cloud Foundry - Matt Stine - 90 minutes
Matt Stine at SpringOne - Part 1 - Great for Architects.  This session talks about architectural patterns for at-scale cloud native architectures and how Spring Cloud and the Netflix OSS components help you implement these patterns.  This is part 1 of a 2 part series from SpringOne 2015

Apps + Data + Cloud: What Does It All Mean? - Matt Stine - 1:20
Matt Stine at SpringOne - Part 2 - Great for Architects.  This section continues from Matt’s take about architectural patters and goes into microservices architect, lambda architecture, and big data and why Pivotal Cloud Foundry helps.  Then he walks through code examples to illustrate his points.  The code from this talk is available at:  https://github.com/cf-platform-eng/springbox-datacloud

More Technical:

This group of material dives deeper into the technology behind Spring Cloud and the Netflix OSS components.

Webinar: Smarter Service-to-Service Invocations with Spring Cloud video - 90 minutes
Excellent Josh Long Bootifiul Microservices video, I just love Josh's code-centric approach to describing the concepts.

Spring Cloud at Netflix  -90 minutes
Fairly deep dive talk from Netflix developers

Simplify Cloud Applications using Spring Cloud Connectors video - 90 minutes
PCF and Spring Team members talk about Spring Cloud on PCF
https://www.youtube.com/watch?v=T8SPV4rnxjg&feature=youtu.be

Practical Microservices eBook by Rohit Kelapure
Excelent technical treatment of the concepts from one of Pivotal's top field consultants.


Hands on Tutorials:

This group of material is for people who want to get their hands on the code.

Building Microservices with  Spring Boot Live Lessons Video - on O’Reilly Safari
Excellent 6+ hour course hosted by Josh Long and Phillip Webb from Pivotal includes lots of hands on labs.
https://www.safaribooksonline.com/library/view/building-microservices-with/9780134192468/

Micro services with Spring Cloud - on Udemy 4 hours
Highly recommended because of the excellent coverage of Spring Boot, and Spring Cloud Services with lots of hands on labs.
https://www.udemy.com/microservices-with-spring-cloud/

Cloud Native Java: Designing Resilient Systems with Spring Boot, Spring Cloud, and Cloud Foundry 
Josh Long's newest book which includes lots of technical depth and code examples

Microservices with Spring Tutorial
A simple example of setting up a microservices system using Spring, Spring Boot and Spring Cloud.

Putting Microservices Into Action:

In the study of martial arts students learn classical techniques, practical application of those techniques, and how to put it together in tactical situations.  Material in this section is about applying microservices techniques to real-world situations.

Migrating the Monolith - Rohit Kelapure  - 1:20
Excellent session featuring one of Pivotal's top consultants discusses his teams agile test driven technique and the tools they use for migrating monolithic applications to Cloud Foundry.

Documentation Pages:

Spring Cloud Services for Pivotal Cloud Foundry
The definitive reference for Spring Cloud Services on Pivotal Cloud Foundry

Spring Cloud Project
The Spring Cloud project page.  This is the overarching project page for all the sub projects.

Spring Cloud Cloud Foundry
The Spring Cloud Cloud Foundry project page - includes a quickstart tutorial and a sample project.

Conclusion:

I hope this playlist helps you on your journey to developing your next microservices based systems with Spring Cloud Services and Pivotal Cloud Foundry.  And I hope you agree that this new wave of tools and technology brings an amazing new level of productivity!   If you find other great material that want me to include in my list please give me feedback.  Never stop learning!

Thursday, May 26, 2016

Connecting a GUI tool to the MySQL Service in Pivotal Cloud Foundry



As a follow-up to my post about Managing MySQL Databases on Pivotal Web Services I want to address a common follow-on question which is "How do I use a GUI tool to access the MySQL service in Pivotal Cloud Foundry with a GUI tool such as MySQL Workbench?"  This is simple enough it doesn't require a video.  

First I recommend reading the Pivotal documentation on Accessing Services with SSH.  It show how to execute raw SQL via the command line.

To connect one of the MySQL GUI clients to the MySQL service in Pivotal Cloud Foundry you need to need to establish an SSH tunnel into the MySQL Service because it does not provide an interface on the publicly accessible network.   Fortunately most of the modern GUI clients have this capability built right in.

Prerequisites:
Before you get too far you'll have to gather a few pieces of information.  The first is the Ops Manager IP Address.  This is the address you use to access ops manager as shown in the following screen shot.



You will also need the Ops Manager Credentials.  You need both the username and password for vSphere and on other platforms you need the username and SSH key file (.pem file).  In most cases the OpsManager username is “ubuntu".

You also need the MySQL Server Proxy IP Address.  You obtain this from the status tab on MySQL Service Tile.   Make sure to use the Proxy IP Address rather than any of the individual MySQL Server addresses.



Next you need the Credentials for the MySQL database.  You obtain this from the Credentials page of the MySQL Service tile the value you need is the "Mysql Admin Password” under the “MySQL Server” job on the credentials tab.



Once you've gathered all this information you're ready to configure a connection in MySQL Workbench.  The following screenshot provides a guide of what goes where on the connection panel:


Alternatively you can also pick your favorite SQL tool.   Here's how to configure Sequel Pro:


With this you should be all set to take advantage of your GUI SQL tool's features to manipulate data inside your PCF MySQL service.  

By the way if you've made it this far, I know you are using PCF and a relational database, so you should also go read the excellent article by Marcin Grzejszczak on Zero Downtime with a Database using blue-green deployments and flyway.  It's a great description of the right way to update databases using PCF.

Thursday, February 4, 2016

Security for Pivotal Cloud Foundry


Security is a multi-faceted topic as highlighted in a recent talk and article by Pivotal's Justin Smith who's strategy is to Rotate, Repair, and Repave the entire stack to keep ahead of security threats.  You can learn more about this approach from his keynote presentation at the 2016 Cloud Foundry Summit at: https://youtu.be/NUXpz0Dni50

In addition to this systemic approach to staying ahead of advanced persistent threats and credential leaks Pivotal Cloud Foundry and Spring Cloud Security also help your application development teams build secure applications by simplifying the problem to something the average developers can manage.  Here are several resources that can help you better understand building secure micro-services applications and operating them on Pivotal Cloud Foundry.

First for the developers here are two excellent security presentations from this the SpringOne2GX 2015 conference that discuss building secure applications in PCF:
Securing Micro-services with Spring Cloud Security - by Will Tran 1:29
https://youtu.be/USMl2GNg2r0This session provides a how to guide to use Spring Cloud to build secure applications with Spring Cloud Security and Pivotal Cloud Foundry.
A how to guide to security in the PAAS Cloud - by John Field and Shawn McKinney 1:32https://www.youtube.com/watch?v=FIyJBoq8cBcThis session is a broader architecture discussion about building secure cloud native applications that integrate with legacy apps.

Both operations teams and developers should also learn more about the basic PCF security concepts in the following resources:
Pivotal's documentation page that discusses PCF Security Concepts is fundamental to understanding the security mechanisms Pivotal Cloud Foundry provides:
https://docs.cloudfoundry.org/concepts/security.html
There are also two articles that the design of the Cloud Foundry User Account and Authentication Service UAA.
https://blog.pivotal.io/pivotal-cloud-foundry/products/high-level-features-of-the-uaahttps://blog.pivotal.io/pivotal-cloud-foundry/products/introducing-the-uaa-and-security-for-cloud-foundry
Aliaksandr Prysmakou from Altoros has also created a blog post describing the security of Cloud Foundry Containers and the isolation that they provide:
http://blog.altoros.com/cloud-foundry-security-do-containers-contain.html
James Masson from Cloud Credo has also written an article that talks about the network security of Cloud Foundry:
http://www.cloudcredo.com/cloud-paranoia-cloud-foundry-confidential-data-public-clouds/

With this under your belt, you should be well prepared to secure the spring micro-services based applications you build on Pivotal Cloud Foundry.

Wednesday, October 14, 2015

Managing MySQL Databases on Pivotal Web Services


Pivotal Web Services is a hosted instance of Pivotal Cloud Foundry that is great environment for getting started pushing cloud native apps to the cloud without standing up a server.   This is great for developing projects, for learning, or even for hosting your personal projects.  And you can get it for free for the first 60 days.

One of the things that makes Pivotal Web Services great is that it provides a ton of great services in the marketplace that your application can use for free or for a nominal fee.  One of my favorites is the ClearDB MySQL database which provides a free instance of MySQL as long as you can live within their limits.  I love having MySQL available because so many of the developers I work with know MySQL.

However, one of the common questions that people ask is "How do I manage the database inside of Pivotal Web Services?".   The answer is pretty easy, you can use your favorite GUI database tool such as MySQL Workbench, Sequel Pro, whatever you use.  All you need to do is get the connection information for the ClearDB MySQL service.   Here is a little 4 minute video that shows you how to setup a very basic configuration.




Note: For the sake of showing you the core concepts quickly, this video skips the SSL configuration steps.  In real life, I highly recommending the security of SSL.  Adding SSL is really easy, you simply downloading the key files from the ClearDB dashboard and providing them in your GUI tools configuration properties.  Those easy additional steps just complicate the core concept I wanted to show in this video.


Tuesday, October 13, 2015

Pivotal Cloud Foundry



An important step along the journey of becoming an expert with Pivotal's technology learning how to build applications that run on Pivotal Cloud Foundry.  PCF provides application developers the ability to run applications in the cloud so you can scale it up as far as you may need. This could be any kind of app from a typical web app to a graphical front end for a data science project to a series of micro services that serve as the backend for a mobile app.  The interesting thing about PCF is that you aren't limited to one specific language or development framework, and you aren't limited to running it on any specific cloud infrastructure.  PCF also lets you as a developer focus on the code that adds value to your clients, not on all that ancillary stuff you have to do to move your code from dev to test to prod.

In it's simplest form Pivotal Cloud Foundry is very easy to use.  You simply push apps bind them to the necessary services and scale them up to meet your needs.  But it has all the sophistication needed to support modern cloud centric businesses.

To learn about Pivotal Cloud Foundry I suggest the following resources.  I've arranged this list in the order you should consume it from the most fundamental to the most sophisticated.

Cloud Native Custom Software by John Funk - 40 minutes
This introductory video introduces you to the concepts of Cloud Native software and the problem domain that Pivotal Cloud Foundry is built to address.

15 Minute PCF Tutorial - 15 minutes
This great tutorial gets you up and running with your first PCF application in just 15 minutes.  It's great to help you understand what Pivotal Cloud Foundry does.

Running Your Spring Apps in the Cloud  by Cornelia Davis - 1:31 -
Good developer level discussion of running applications in Pivotal Cloud Foundry

Pivotal Podcast Episode #9 - "USING PIVOTAL CF FOR BLUE/GREEN DEPLOYMENT by Simon Elisha - 15 minutes
Discusses using Pivotal Cloud Foundry for Blue/Green deployments for continuous delivery

All Things Pivotal Podcast Episode #11: So What Happens When I Push An App Anyway? by Simon Elisha - 10 minutes

Build Packs: Delivering Heterogeneous Architectures by Casey West - 30 minutes
Great talk about how build packs work and why they are important to customers.

All Things Pivotal Podcast Episode #15: What is Pivotal Web Services? by Simon Elisha - 11 minutes
This episode takes a closer look at Pivotal Web Services

All Things Pivotal Podcast Episode #21–Platforms vs. Platform as a Service - 35 minutes
Michael Coté talks with Andrew Clay Shafer about “platforms” vs. “Platform as a Service.”

Enabling Continuous Delivery with Pivotal Cloud Foundry by  Matt Stine
Great Session An overview of the principles of Continuous Delivery, followed by a demo of building a CD pipeline with Pivotal CF, Spring, and CloudBees Jenkins Enterprise.

Pivotal CF Tech Overview by Cornelia Davis -  30 minutes
Reviews components in Pivotal CF and how they work together in depth

Cloud Foundry and Containers by George Lestaris and Julian Fiedman
Provides a great discussion of the role of containers and Docker in Cloud Foundry.

Diego Runtime Overview by Eric Malm - 30 minutes
Great in-depth discussion of the Diego architecture.  Provides an under-the-covers view of how cloud foundry does what it does.

Real Training: When you are ready to get serious and take a class, Pivotal offers three classes designed to get you certified to deliver projects with Pivotal Cloud Foundry.  They are:



And just for fun, here are my favorite Pivotal Cloud Foundry customer success stories:

 

Friday, July 31, 2015

Gemfire and Apache Geode


Most of you know that Pivotal donated over a million lines of code from our very successful Gemfire product to the Apache foundation to start the Apache Geode project. The main project web page is http://geode.incubator.apache.org/.  Since then the project has been gaining a lot of momentum.  This is important for those of you trying to become Pivotal experts because in addition to open-sourcing the code, we are also pushing a lot of material to help people learn about Geode / Gemfire out into the open and asking the community contribute to building that body of knowledge.

One of the best technical overviews on Geode / Gemfire I've ever seen is available at: http://bit.ly/POSH0068yt

There is also an introductory developers course with videos and labs available on Github at: https://github.com/project-geode/training

When you have questions about developing with Geode / Gemfire post a question on StackOverflow http://stackoverflow.com/questions/tagged/geode

Once you understand the basics of building a system with Geode / Gemfire and want to learn how to secure it there is a great article about integrating Geode with an LDAP server at: https://blogs.apache.org/geode/date/20150729

Open sourcing the core of Gemfire is just the beginning of Apache Geode and keeping up with it as time goes on will be important.   For that we've started a Geode community page at: http://geode.incubator.apache.org/community/

A few other great places to get the latest information about Geode are:

Real Training - When it's time to get to be certified to deliver Gemfire projects, the best thing is to attend both the Gemfire Developer class and the Gemfire Administration classes.  These classes focus on teaching you the proven techniques for building successful Gemfire applications such as using PDX serialization, and Partitioned regions for everyday things.  The classes also teach you about the  edge features of Gemfire that you should avoid in normal projects unless you know you need them.



Wednesday, October 1, 2014

The Journey Begins!

Many of my journeys begin on an airplanes flying to somewhere I've never been before.  This blog is about a journey I've been on since before Pivotal started as it's own entity.  It's about the journey to become an expert with Pivotal's technology.

I started this blog to share hints, tips, and tidbits of information to help others who are also on the journey to becoming an expert with Pivotal’s exciting technology.


Pivotal’s ambitious goal to provide a new generation of tools to help companies manage big data, get insights from that data, and put those insights into action with new applications faster than ever before is both an opportunity and a challenge.  It’s an opportunity because of what we can do for our clients, but it’s a challenge because there is so much new technology to master.   This new world brings together Data Science, tools from the Hadoop ecosystem, real-time in-memory data-grids, and agile development done using high level frameworks that help us deploy to cloud based environments where we can push our applications directly into the hands of our users with today's mobile devices.

If you don’t know where to start this journey, or if you are already well into your journey but you could use a little help finding your way to that next breakthrough, hopefully I can help.   I don’t consider myself an expert in everything - I told you I'm on this journey myself!   However, years of experience has given me a bit of a head start, plus I have exceptional access to resources and information, and I have luxury of being able to focus exclusively on Pivotal’s technology.  So maybe I'm a little further along on my journey that most people.  Hopefully I can share some insights I've picked up on my journey, so you won’t have to discover the path entirely on your own.