Thursday, July 24, 2014

AWS Facts - VPC(Virtual Private Cloud)

Amazon Virtual Private Cloud:-VPC

A virtual private cloud (VPC) is a virtual network that closely resembles a traditional network that you'd operate in your own data center, with the benefits of using the scalable infrastructure of Amazon Web Services (AWS).

Ø  Some of the key things in vpc:-

1. Subnet groups are different for different VPCs.


2. VPC is comprised of Subnets:
          1) Public Subnets
                 - Accessible from public
                 - Usually ELB, Bastion Hosts, NAT, VPN systems are launched under this subnet.
                 - Hosts can access internet by merely attaching EIP or public IP           
             
         2) Private Subnets
                 - Inaccessible from Public.
                 - Accessible from VPN, BastionHost, ELB
                 - Hosts can get access to internet only if configured through NAT instance.



3. Create new Internet Gateway for Public subnets (igw-xxxx)

4. You can categorize public and private subnets by creating two routing tables:

Target
Name
Description
Destination
igw-xxx
Route1
Public Subnet1
0.0.0.0/0
i-xxxxx
Route2
Private
0.0.0.0/0


5. Hosts on different subnets can access each other with Security Group allow rules.                              

6. Spin up NAT instance on public subnet against community provided AMIs (i-xxxxx).
            - Disable Source/Destination checks
            - Open ports from security group (Inbound- 80, 443 Outbound- All)


7. Type of firewalls:

• Security groups—Act as a firewall for associated Amazon EC2 instances, controlling both               inbound and outbound traffic at the instance level
           -        Termed as “first layer of defense”
           -        Supports allow rules only
           -        Associates to particular instance during or after launching.

   • Network access control lists (ACLs)—Act as a firewall for associated subnets, controlling both inbound and outbound traffic at the subnet level. It get applied VPC level
            -        Termed as “second layer of defense”
            -        Supports allow rules and deny rules
           -        Automatically applies to all instances in the subnets it's associated with (backup layer of defines, so you don't have to rely on someone specifying the security group)
    Note: To make blocking single layer, you can allow all ports in SG and then can manage the access from Network ACL resulting port management centralization


         


8. Subnet communication: Suppose we have VPC name "V1" and have two subnets down under as "S1" & "S2". Under S1 and S2, we have EC2 instances as I1 and I2 respectively:
o    I1 cannot access I2 ports unless the port is explicitly opened in SG and Network ACL.
         
9. Elastic IP v/s Public IP
o    EIP remains static during its lifetime, however public IP changes after every Shutdown-start(sometime remains same during restart).
o    EIP is chargeable, Public IP is free.
                                     
                                       
Practices:-

a)   Get your Amazon VPC combination right:  
·         You need to decide the right Amazon VPC & VPN setup combination based on your current and future requirements. It is tough to modify/re-design the Amazon VPC at later stage, so it is better to design it taking into consideration your NW and expansion needs for next ~2 years.

b)   Choose your CIDR Blocks:

·         While designing your Amazon VPC, the CIDR block should be chosen in consideration with the number of IP addresses needed and whether we are going to establish connectivity with our data centre.    
                     
·         Also when you design the Amazon VPC architecture to communicate with the on premise/data center ensure your CIDR range used in Amazon VPC does not overlaps or conflicts with the CIDR blocks in your On premise/Data center.
·         E.g., Your VPC CIDR block is 10.0.0.0/16 and if you have 10.0.25.0/24 subnet in a data center the communication from instances in VPC to data center will not happen since the subnet is the part of the VPC CIDR. In order to avoid these consequences it is good to have the IP ranges in different class. Example., Amazon VPC is in 10.0.0.0/16 and data center is in 172.16.0.0/24 series.
                  
c)   Securing Amazon VPC :

·         You can configure Intrusion Prevention or Intrusion Detection virtual appliances and secure the protocols and take preventive/corrective actions in your VPC
·         Configure VM encryption tools which encrypt your root and additional EBS volumes. The Key can be stored inside AWS (or) in your Data center outside Amazon Web Services depending on your compliance needs.

·         Configure Privileged Identity access management solutions on your Amazon VPC to monitor and audit the access of Administrators of your VPC.
·         Enable the cloud trail to audit in the VPC environments  ACL policy's. Enable cloud trail : http://harish11g.blogspot.in/2014/01/Integrating-AWS-CloudTrail-with-Splunk-for-managed-services-monitoring-audit-compliance.html
·         Apply anti virus for cleansing specific EC2 instances inside VPC. Trend micro has very good product for this.
·         Configure Site to Site VPN for securely transferring information between Amazon VPC in different regions or between Amazon VPC to your On premise Data center.
                            
·         Follow the Security Groups and NW ACL's best practices listed below

d)   Spread your NAT instance with Multiple Subnets:
·         What if you have hundreds of EC2 instances inside your Amazon VPC and they are making lots of heavy web service/HTTP calls concurrently. A single NAT instance with even largest EC2 size cannot handle that bandwidth sometimes and may become performance bottleneck. In Such scenarios, span your EC2 across multiple subnets and create NAT's for each subnet. This way you can spread your out going bandwidth and improve the performance in your VPC based deployments.

e) Plan your Tunnel between On-Premise DC to Amazon VPC:
Option 1: Secure IPSec tunnel to connect a corporate network with Amazon  VPC (http://aws.amazon.com/articles/8800869755706543)

Option 2 : Secure communication between sites using the AWS VPN CloudHub (http://docs.aws.amazon.com/AmazonVPC/latest/UserGuide/VPN_CloudHub.html)
          
Option 3: Use Direct connect between Amazon VPC and on premise when you have lots of data to be transferred with reduced latency (or) you have spread your mission critical workloads across cloud and on premise. Example: Oracle RAC in your DC and Web/App tier in your Amazon VPC. Contact us if you need help on setting up direct connect between Amazon VPC and DC.                    
                                     
                                     



Thursday, July 3, 2014

Chef Server Errors and their Solutions

While working on Chef, I came across multiple errors. After spending lot of time, I was able to find solutions. So, I came up with an idea of this post which is focused on errors related to chef and their solutions. 
Error 1 : Error while installing knife ec2 gem
ERROR:  Error installing knife-ec2:
ERROR: Failed to build gem native extension.

        /usr/bin/ruby extconf.rb
checking for libxml/parser.h… no
——-
libxml2 is missing.  please visit http://nokogiri.org/tutorials/installing_nokogiri.html for help with installing dependencies.
——-
*** extconf.rb failed ***
Solution :
yum install libxslt libxslt-devel
This command will install all necessary dependencies for knife ec2 gem.
——————————————————————————————————————————————
Error 2 : EDITOR environment variable error
ERROR: RuntimeError: Please set EDITOR environment variable
Solution :
This error can be resolved by issuing command  “export EDITOR=$(which vi)”.
But this environment variable will lost on system restart. For persistance, please add this above command to .bashrc or .bash_profile file.
Another alternative solution(suggested by Mohit - check first comment) for this error is by defining editor in knife.rb file.
Add knife[:editor] = ‘`which vim`’ in knife.rb file.
——————————————————————————————————————————————
Error 3 : Error while downloading cookbooks to your local laptop/PC from Chef Server (which is hosted on AWS EC2)
nameless@Nameless:~$ knife cookbook download apache
Downloading apache cookbook version 0.1.0
Downloading resources
Downloading providers
Downloading recipes
Check your knife configuration and network settings
Solution :
We face this error because the s3_url which is set to EC2 private dns address is used by erchef to talk to bookshelf and used to generate URLs that are given to clients that they use to access cookbook content.
To fix this, we need to add :
erchef[‘s3_url’] = https://<AWS-EIP>
to /etc/chef-server/chef-server.rb file. If the file is not present, please create that. Once done, please issue “chef-server-ctl reconfigure” command. [Thanks to Gionn and JB for highlighting this - see comments]
Another solution (Less Suggested) :
PATH : /var/opt/chef-server/erchef/etc/app.config
Modify s3_url from EC2 private dns address to EC2 EIP/Public DNS i.e., modify {s3_url, “https://ip-10-190-87-80.ec2.internal”} to {s3_url, “https://<AWS-EIP>”}.
After that, please perform a system restart. If you issue, chef-server-ctl reconfigure command, it will set s3_url value to original value.
——————————————————————————————————————————————
Error 4 : HTTPServerException : 403 “Forbidden”
FATAL: Net::HTTPServerException: template[/etc/httpd/conf/httpd.conf] (apache::default line 20) had an error: Net::HTTPServerException: 403 “Forbidden”
Problem: Objects from cookbooks (templates, cookbook_files) are lazily-loaded by default. If it's a very long run, deployment time runs out of time(15 min by default). 
Solution : This error is encountered when we have large chef recipes whose deployment time on clients is large than 15 minutes. In order to avoid this error, please increase “s3_url_ttl" value from 900 seconds to required time interval.To increase this value, please add
erchef[‘s3_url_ttl’] = 3600in /etc/chef-server/chef-server.rb file. Once done, please issue “chef-server-ctl reconfigure" command. (Thanks Gionn and JB for highlighting this - see comments).
Another Solution: Don't have access to chef server then: 
You can turn off lazy loading on the client by adding
                  no_lazy_load true
Another Solution (Less Suggested) :
We can find this value in app.config file under path  “/var/opt/chef-server/erchef/etc
After that, please perform a system restart. If you issue, chef-server-ctl reconfigure command, it will set s3_url value to original value.
——————————————————————————————————————————————
Error 5 : RabbitMQ does not start on Oracle or Amazon Linux
RabbitMQ doesn’t start when “chef-server-ctl reconfigure” command issued.
Solution : This error is encountered on Amazon Linux because amazon linux platform is not definited in runit chef recipe for chef server. 
To fix this issue, navigate to “/opt/chef-server/embedded/cookbooks/runit/recipes/default.rb” and add below mentioned line after ubuntu platform description :
when "amazon"
  include_recipe "runit::upstart"
After that, the default.rb will look like : 
case node["platform"]
when "ubuntu"
  include_recipe "runit::upstart"
when "amazon"
  include_recipe "runit::upstart"
when "redhat","centos","rhel","scientific"
  if node['platform_version'] =~ /^6/
    include_recipe "runit::upstart"
  else
    include_recipe "runit::upstart"
  end
Once done, please issue “chef-server-ctl reconfigure” command again.
——————————————————————————————————————————————
Error 6 : Error when uploading - NameError: uninitialized constant Chef::CookbookUploader::Queue
Solution : This error is mainly encountered on Amazon Linux while uploading cookbooks.
To fix this issue, navigate to “/usr/lib/ruby/gems/1.8/gems/chef-11.8.2/lib/chef/cookbook_uploader.rb” and add below mentioned line on top.
require 'threads'
——————————————————————————————————————————————