How to avoid circular dependencies between Cloudformation stacks

I am currently designing a cloud based solution based on AWS, Cloudformation and Chef, and using the concepts as described in an earlier post.

In short, I am a big fan of Cloudformation which is a perfect example of Infrastructure as Code, but developing and maintaining Cloudformation scripts and stack is a bit of a cumbersome process. In order to keep that manageable, we need to adopt good old design principles such as low cohesion/high coupling when deciding what to put in a particular template/stack.

Let’s assume two Cloudformation templates, one containing all database server related resources, while the other contains all web server stuff. Forget all other stuff, and assume each stack just have a server instance with a security group.

The Web server requires the database server to be present to do the provisioning of the application, while the database server security group requires the Web server security group to be created in order to provide access. Bang! Circular dependency, which is always a bad thing!

Of course, this can be solved, and the following options exists:

Forget about modularisation, put everything in one template. Of course this works, and in this simple example most likely the most pragmatic approach also. But in a real life setup, things are not as simple and you might have a multitude of service stacks having inter-dependencies. And putting these all together in one script will quickly become a head ache.

First create all security groups, then the server instances. Well, that will work as well, and is a relatively clean solution too. Few things I consider a drawback are:

• It introduces another template (and hence stack);
• It also introduces additional dependencies between these stacks, which must be catered for by means of input parameters;
• The definition of the SG, which is tightly related to the server is removed from the service stack.

The option I went for, was introducing a dedicated security group in one stack, and use it for identification purposes in another. This is shown below.

In the database stack I create an additional security group (WS-ID) which has an empty definition, and I only use it to allow the DB Security Group to provide access to it.

In the web server stack, I provide a reference to this security group (through an input param) and by this allows clean access to the database server. Basically you are using the WS-ID security group for identification purposes only.

Hence I have removed the circular dependency, kept the number of stacks the same and have limited the dependency between stacks to a minimum. In this example, there is only one dependency, but in reality there might be many more. 

These software design patterns prove to be useful even when designing infrastructure he!

Update: there is a significant drawback in this approach, as you cannot terminate and rebuild the DB Server stack without terminating the web server stack. To cope with that, you can create the WS-ID Security Group in a separate stack, and feeding the reference into both the DB Server Stack as well as the Web Server stack.

Why the headache of License Management could be a major push towards cloud adoption

I've been busy lately with various software license related assignments, not necessarily related to a cloud setup. 

And quickly I recognized all the headaches and frustration that comes with it, which are even more apparent when being active in cloud environments for the last decade or so.

Let's face it. Licenses are a major headache. Understanding the rules is ridiculously complex, and change all the time. And most of the time, these changes are not to lower your fees.

Strong software asset management discipline is a necessity, and while I will be the last one to claim that cloud will solve all these problems and make them redundant, it definitely will help in some cases.

For instance, one customer has an application (provided as a SaaS) with a very peaky behaviour. One week per month it needs massive power, the other three weeks it just cruises. Buying licenses for the peak capacity would be a very costly exercise, even more so given the SaaS characteristic of the service. You simply wants to scale with demand, both periodically as well as the growing customers, and traditional licensing models simply cope very badly in such scenarios. Cloud computing, in case licenses are included proves to be a major attraction to this types of customers.

Another customers needed to renew their enterprise agreement with another software vendor, and needed to make an inventory of what software is used, over time, and in what configurations. Should be simple enough, right? Not so, keeping track of everything, understanding all the rules and exceptions is close to impossible. Comparing that with ease in which you can launch enterprise applications in the cloud, and getting charged for using both hardware and software by the hour (or even minute), it is clear that there is something to be gained here.

Understand me correctly, I do not feel the cloud is a silver bullet that solves all such licensing issues, but the matter of fact is that it comes inherent with some features that makes life in this regard a lot easier. And in these two cases cloud computing was mentioned by the customer as an attractive setup, mainly because of the licensing restrictions.

I am convinced that this will prove to be a major battle ground, and by exploiting this cloud providers have a great opportunity to lure traditional enterprises to use their services.