Amazon's EC2 Virtual Private Cloud feature is very useful, it allows for much more control of the network architecture and provides additional features such as the possibility to add more IP addresses per instance.
Typically, a VPC consists of at least a public subnet, hosting publicly accessible servers and one or more private subnets hosting more sensitive servers such as a database server. The servers in the private subnet are obviously not accessible from the outside, but they still can communicate with external resources through a NAT instance.
Servers in the public subnet are only accessible when they are assigned an Elastic IP Address or through a load balancer (e.g. AWS ELB service).
So far so good. Something less obvious is that if a server in the public subnet does NOT have an elastic IP address assgned (e.g. it is connected to a load balancer) it also loses the capability of accessing external resources. This would be a show stopper, as for instance it is much more difficult to allow these servers to auto-configure themselves upon launch.
Well, wait, we have this NAT service, haven't we? Yes, we have but by default it is not possible for servers in the public subnet to use this NAT instance. Changing the VPC route tables is not going to work, as it will break the connectivity of the EIP-attached servers in the public subnet.
In order to still allow EIP-less public servers to use the NAT instance, the network configuration of these instances must be modified to point the default gateway to this NAT instance. This requires a static IP configuration of course, and to avoid running into all kind op IP conflicts it is possible to let AWS hand out the IP address using DHCP and then convert upon launch time (e.g. using a cloud-init script) to a static configuration with the desired default gateway.
