I see this board every where in Bangalore these days, and is one of the reason i love biking (aka cycling) instead of a car. There is also a bigger reason to worry about – auto’s going at the speed of rajanikanth, speeding call center cabs, heavy duty bus which occupies entire road …. all rushing thru city limits, making living here less joyful.

Am sure as a child we would have done loads of biking. Rekindle your inner child and start again. Its actually good fun cycling, first we dont have to worry about “no parking” boards, second cops wont stop you, third you burn unwated calories in your body which results in you enjoying every meal that you eat , fourth is i can still travel in city volvos to any where and do the last mile in bike (aka cycle), fifth is we are definetly more green (burn ur fat not fuel ), sixth you make tons of friend on you way, seventh is there is still a chance of seeing sky,cloud,stars,sun,moon,people, etc.

For sure biking (aka cycling)keep you fit, gives you some time to listen to music or some podcast and make your life bit more enjoyable.

(Veloscope a initiative started by couple of us to promote biking & photography in and around Bangalore, India. Visit our facebook page)

We are gonna use the command line from a shell. Our goal right now is to get ‘values’ for the key listed below. Line starting with ‘#’ are to be ignored and meant for explanation.

#JAVA_HOME -> where you java installation, the below path works for Mac. For linux and Windows it would be different
export JAVA_HOME=/System/Library/Frameworks/JavaVM.framework/Home/
# ec2-api-tools, this need to be downloaded from http://s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip and place it where ever you want.
export EC2_HOME=/Applications/ec2/ec2-api-tools 
# both the .pem has to be generated and download, and if you mis you will have to regenerate. I normally keep this is my dropbox.
export EC2_PRIVATE_KEY=~/.ec2/pk-xxxxx.pem
export EC2_CERT=~/.ec2/cert-xxxxx.pem  
# available from the http://aws.amazon.com/account/
export EC2_ACCNO=1111-2222-3333
export ACCESS_KEY=BABBIO4XTFPTE6UYD3BQ
export SECRET_KEY=jcbsdsadcusjkh867HJk+897bnHygTQoEHj
# for easy access
export PATH=$PATH:$EC2_HOME/bin

Login to your account and click security credentials:

Get your Access and Secret Keys: 

Generate your certificate and download pk***.pem and cert-***.pem

Get you ec2 account number:

Reference:
http://aws.amazon.com/documentation/

Detailed reference of commnad line
http://docs.amazonwebservices.com/AWSEC2/latest/CommandLineReference/

1. LAMP

• The traditional way of setting up a LAMP is installing the require packages from the OS repository which would involve setting up Apache, MySQL and PHP. (reference: https://help.ubuntu.com/community/ApacheMySQLPHP). Here the entire stack remains in one instance. This mean you will have to do the database tweaking and administration yourself apart from app development.

2. LA(M)P -> LA(RDS)P

Things to take care while setting up Apache/PHP
As said earlier EBS allows you to scale from GB -> TB and is re sizable. I recommend to create a EBS volume  (start with XGB based on what is currently used) and attach this instance. Upon attaching a EBS volume to a instance we can access it as a separate disk drive. Format with ext{3,4} and mount it. Use the newly mounted partition as your Apache Root folder.  This approach has multiple advantages as well.

• Separation of OS data vs application data
• In case of crash, you can boot another instance and connect the volume to new instance, yes data in EBS is persistent
• We can schedule more frequent snapshot for data volume.
• The snapshots are stored in S3 (a highly reliable data store ) and is replicated across multiple availability zone for higher redundancy, a huge ++++ for us.
• Creating a experimental setup with mock data is as simple as taking a snapshot of your vol and cloning it.

A reason to smile, a little bit of cost saving here
By default EBS volume snapshot are stored on to S3 and is replicated to multiple availability zone. But in some case we might have loads of data which are not critical but is generated at run time or during the session. Here we could choose to use Reduced Redundancy Storage (aka RRS). So go ahead create a EBS volume, set the attribute to make it RRS, attach to your instance and let all your transient data reside in this new low cost volume/partition.

how do we know if we should increase the instance size (RAM, CPU, IO, Network, etc..)
As we have started our implementation with basic instance type which matches our local hardware, we need to know if we are required to increase the CPU,RAM, etc… This brings in CloudWatch which allows us to monitor the instance very closly and take calculated decisions. Based on the monitoring details we can choose more memory or more cpu (more about instance type here).
Update from small to medium/large to give you users a better experience. Else downgrade  from large to medium/small to save some bucks for your company.

What next ?? customer is king, let’s give him the best experience.
Now that we have taken care of our basic need with respect to base OS, database and storage. Now let’s looks at some other aspect w.r.t reducing latency of the content that we serve users. Internet is something which is consumed globally and need to serve the content (be it images, video, etc.) to every user around the world becomes the next challenges. Amazon AWS provide CloudFront, a web service for global content delivery with focus on reducing the latency while serving any user any where in the world. Let your customer reside anywhere in the world, CloudFront is there to lead from front.

faster, faster & faster …. but f-a-t-a-l error and system crashed !!!
In the above section we saw that we have handled the Content Delivery part of it, we started monitoring and know if load is high or not. Being a start-up every user who is visiting us is a potential customer and he leaving with a bad impression is the last thing to happen. What is you server faulted and is down at the middle of night or while you were travelling ? Wont it be nice to have another instance serving your customer and make him happy ?

Don’t worry we have Elastic Load Balancer. The load balancing can kick in based on multiple parameter which are configurable. For example in our case we want to make sure if a instance is bad then route all the data to new instance. Elastic Load Balance uses Cloud Watch behind the scene to get the stats of instance and act upon it based on what we have configured.  

ok all well, now i have million of user at middle of night !!!
Cool, am happy to hear that more people are using your service, but sad to know that you need to awake to check when to allocate more resource. What if i give a option to do this dynamically ?? am sure your family will be really to have you home at mid night. Here we have Auto Scaling. Again choice is left to you to decide when to scale up and when to scale down based on parameters (like time of day, load, etc)

all is well, but how to achieve seamless working from my office and Amazon AWS
Since your office is gonna be connected to Amazon AWS network for managing the instance, copying critical data, testing a new piece of code, etc… you don’t want every employee to connect separately to EC2 machines. Also while transmitting critical data you don’t want a network hacker (or a competitor) stealing your critical data. To save you from all these head aches of seamless and safer connection, AWS has Virtual Private Cloud (aka VPC). This will make sure your IT infrastructure and Cloud is connected via an encrypted VPN connection. Once your VPC is enabled and configured your cloud is ready to serve you locally. Remember to tweak you security group to enable/disable network connections.

What have we done so far, a quick glance:
Our goal is to get a robust, reliable, secure, scalable, on demand computing resources for the next .com that you are building on top of LAMP stack. With ec2 running Linux is a click away. By choosing appropriate distribution (one which has Life Time Support like Ubuntu, CentOS,RHeL, etc.) you can choose to stay focused on you application development. By storing your ‘/’, ‘/data’ on EBS make sure you have safe backup available in multiple availability zone (powered by S3). By storing ‘/non_critical_data’ in EBS (with RRS) we make sure we save money (reducing redundancy). With monitoring (Cloud Watch) enabled you know what is you system doing and when w.r.t CPU, IO, Network, Memory, etc .. Once you have become popular around the globe, the Cloud Front (the content delivery network) make sure the latency is reduced to minimum and as a result customer experiencing best possible experience using your product. Just to make sure you sleep well and stay focused, the elastic load balancing system keeps a tab on faulty system and redirect traffic to new instance when required. Same is the case when you have more users hitting your system, the Auto Scaling does the magic behind the scene. Security group make sure that you are safe in the Internet by allowing only needed protocol/ports to public. Finally our VPC make sure that data is secure while your employees connect from office to all aws resource (ec2 instance, etc..) which make the machine in the Amazon AWS network transparent.

happy aws’ing

for any help am just a email away

In this blog post i will cover steps to launch a instance in less than 10 steps.

Step1:
Visit https://console.aws.amazon.com/ec2/home and login

Step 2:
Choose your region, in our case Asia Pacific

Step 3:
Decide which AMI to boot from. For those who are new to AMI, Visualize it as a OS template. For this example we will use an AMI listed in alestic.com

Step 4:
Create a Security Group (equivalent of firewall ). Let’s just enable SSH at the minimum and if we are planning to expose http then port 80 as well.

Step 5:
Create a key pair (The ssh connections to AMI is key based and hence this step). Store the key pair in your home folder (i created a ~/.ec2 folder to keep all ec2 stuff)

Step 6:
Launch Instance (choose ami, instance type, security group, key pair) and finally fire it.

Step 7:
ssh to the machine and see if its working.

In case if you are interested you can setup the command line API to connect to Amazon AWS (will cover this as a separate post)

happy aws’ing

I got mine wisdom tooth removed on a wednesday. The x-ray showed that i have a straight root and it should be relatively easy to pull it out but it turned out to be a disaster (had to cut in pieces before taking it out). Also the doc had to cut the tissues around for a safe removal. Had stiched it to make sure that i heal faster.

Day 0 night was crazy, i had to take injection for pain and antibiotic. Both on my left and right thigh. I managed to eat some thing, dont remember what but thanks to mom. Normally i sleep on a room in first floor (my parents in ground floor) and in the middle of night i had to come down for ice cubes (reduces swelling), my legs trembled and phat…i lay flat on ground. Immediately i got my conscience back and i could remember my enemies very well (took that as a sign that nothing serious happened).

Day 1 morning, i had developed severe neck pain and i assumed that it because of fall. Later i realized that it because of locals. Bleeding continued and was a vampire for entire day 1 (bleeding almost stopped end of day 1)

Day 2 could brush my teeth and felt a bit okay than previous days. Could even walk for few km (very slowly tho)

Day 3 am writing this blog

I continued to eat very well (grinded the food before eating) thanks to mom. Just after the medicine i dont go out or do things (feels giddy for almost an hour) and i gargle at least 3/4 times and occationally eat some ice cream if pain aggravates.

Overall it painful, but hoping that my tomorrow is gonna be better.

I know this is nothing to what other go thru in life when there is no medical support or any else to take care of them. When i read how Lance Armstrong recovered from cancer with mere 2% survival chance to winning tour de france this is jus nothing.

Experience is great teacher and feeling greatful to all master/patriots/freedom fighters/soliders who were/is ready to sacrifice there happiness today for a better tomorrow.

Oops too much gyan for today !

A screenshot of bash in adroid is attached. There are some basic apps that will make ur life simpler while using any droid phone. Like for example evernote for capturing any notes anywhere, a blogger client or google cli (helps u to post a blog from command line), google sync (comes as part of gapps package from google directly), sms back up+ (which backup all received sms and latest version it syncs call logs as well), this means all ur contact + sms + call logs are in safe hands (google god ). The other app which is must is titanium backup, the only app i paid some money (150/- indian rupee or $3) and really worth paying for. It has a cool feature to backup to dropbox (2gb free and if u r smart u can get upto 8gb free).

If you are planning to buy a phone and get it updated like the way i have done with openenta (or any other firmware) be ready with a standby phone and a new sim card, so that ur exisiting sim can be used in the spare phone and the temporary sim can be used in android phone till u set it up completely (i got a temporary uninor 6gb data plan and got everything tested before i started using my actual sim). Also be ready to go thru some hardship of reading, reading and reading some android forums post in case if you are stuck. The best part is android community in general r()cks.

The are few things i want to achieve:

• Use existing infrastructure like Google docs or something similar where people can collaborate easily and work together with a single standard tool chain.  (with zero or no approval cycle)
• There are loads of awesome documentation scattered around, need to figure out a way to scrape it and make it part of the this documentation.
• Should be easy to includes mindmaps and other visual charts (i suppose gdocs has some support for these already)
• A comprehensive documentation which can be source of reference for beginners and experts (build the required gdocs to support feature we want)
• Zero administrative maintenance/support (like for example the domain name, ftp, etc). Use as much as public domain tools (say like facebook for communication, drobbox public folder for sharing, gdocs for sharing/writing documents, etc).  

This is born out of a personal desire, i need to figure out way (irc, forums, posters, facebook page) to contact/interact people and evole the mission and goals for this project.

What to expect next:
I will perhaps put together a campaign which we can announce in multiple Linux forums and get some feedback. If you are reading this and going thru a similar thought feel free to drop in a email to me, perhaps u can be the first to volunteer. 

1. (An) Idea
2. Implement (the Idea, ofcourse using a programming language of ur choice )
3. Test/Validate/etc (exception if you are a awesome coder)
4. Package (or bundle) It (3 and 4 could be interchanged and depended on organization)
5. Take the packaged software to server or customer box and install it  (push it in package repository)
6. Provision the installed package (take the package from repo and untar it and apply system wide configuration changes)
7. back to step 2, with additional inputs to improve the software (add a beta tag, because this never stops, iterative development)
8. push it to update server
9. continue to use it or delete it

The 3 main activities we are focused here are Packaging or Bundling, Installation, Provisioning the already installed package, Upgrading existing installed software and finally deleting the package either from locally installed machine or from the software repository itself.

Some time the best (may be crude) approach is tar it (or zip it) and untar or unzip w.r.t particular directory (tar -C in Linux) and untar/unzip in destination host. This would be most primitive approach and could work clean in some scenarios. Other simpler approach is using a rsync from you pre-production environment to production environment. All the above mentioned steps has some dis advantages when it comes to large scale systems (more than 100+ servers), w.r.t loosing track what is installed, upgrading OS, selective update, etc…

A good packaging system will help us get the following:

1. Name of the package Installed
2. A brief description of what is suppose to do
3. A version number identifier (some companies obfuscate this )
4. Checksum, to avoid over the network tamppering 
5. List of other dependent packages 
6. After installation, the details of packages are normally kept in a local package database

What does a package management brings it a Operating System ?
Its help maintain things independently and co-evolve different components/libraries of Operating System, otherwise would have become a single monolithic design in nature.

Lifecycle of Package Management System:

In general, it will great to have the following principles as foundation principle for any package management system:

• Install & Deletion of package has to be first functionality
• Verification of a already implemented package has to be smooth, an verification of source of packages also will be good to implement
• Package builder is the king, me it simpler for him. If he is not happy we will not have packages generated
• Let the package builder refer to original source (otherwise can become a hell managing it)
• Make sure that the core package manager system works in multiple architecture, this means the package format can be safely built and used in multiple arch (no relearning of tools for different arch)

If you are one of the developer, who’s life revolves around amazon ec2 servers (or any cloud services) for production, development, testing then you have used provisioning systems already. When we issue, ec2-start-instance , the system working behind this command is a provisioning system. The ami-id is a unique identifier which is mapped to a specific template which has instruction to boot specific version of kernel, network, OS, etc…

In a physical machine after the BIOS POST it will start PXE and gets itself a IP address. Making use of various protocols like tftp it gets an Operating System image (Windows, Linux, Solaris, etc) and boots the kernel (which initializes the rest of the System).

In a virtual environment, the user (or web services api in amazon ec2) request the host operating system to provision the guest machine.

In theory provisioning systems are nothing but templates (or pre configured machine instructions) that a machine will execute while booting up (or during power on). In traditional system where we have a physical machine next to you and if you have a installation CD in hands things were easy, mount the CD and fire install and few minutes later you configuration screen pops up and you create users, set network, login, etc … Now with Cloud Computing (aka On demand) people have figured out way’s to automate things so that we dont have the repeat the same thing again and again while installing or configuration 1000+ of systems (imagine installing 1000+ servers using CD, its a nightmare, less productive and cannot scale up). From the age old days we had kickstart (find /root -name ‘ks.cfg’) from redhat which anaconda installed used to provision multiple systems with same configuration. 

With changes in way OS is deployed the need to have centrally managed easily configurable and which is extend able evolved over a period is the need of hour. (especially when we have On demand system based on XEN and other Virtualization technologies)

The beauty of Open Source/Free Software is that you think about it and by the time you have finished thinking someone else has already executed it

Secret (or sacred ) weapon for doing provision is Cobbler (i dont want to getting into a fight of which is better one, feel free to choose what works for you)

So has cobbler reinvented the wheel, according to me no because it does follow the famous unix principle, “get simpler tools work together” (eg: ls -l | grep name where ls, grep does it job and pipe connects it both). As a whole its a complete Provisioning Application with its api exposed as web services.

The bigger picture (than me writing the all the nuts and bolts)
 

One feature which i personally like is installation over SSH, especially useful for distributions like gentoo where we build things from scratch. Its also provides a neat built-in configuration management tool or can be used alon side with puppet. Some of the power feature Cobbler come with are
Power Features:

• Non program can get started easily using cheetah templates, http://www.cheetahtemplate.org/
• reusing common blocks using snippets, https://fedorahosted.org/cobbler/wiki/KickstartSnippets
• Yum Mirroring
• Advanced Networking Configuration — bonding, vlans
• DHCP Management
• DNS Management
• Power Management
• Configuration Management Hooks (puppet), u can choose built in conf management as well
• BIOS/Firmware upgrades (ex: Dell, HP), personally i have not used it
• Remotely erase machines (DBAN), http://www.dban.org/
• Extensive anaconda monitoring features, really helpful to see the remote logs, https://fedorahosted.org/cobbler/wiki/AnaMon
• Multi-server replication, for redundancy and high availability. Will be handy if you want to expand to other geography. 

Worth taking a look at Cobbler, try it out in your next project and drop in your feedback.

reference
http://mdehaan.fedorapeople.org/presentations/cobbler/index.html#%2812%29
https://fedorahosted.org/cobbler/
https://fedorahosted.org/cobbler/wiki/UserDocs

mailing list
https://fedorahosted.org/mailman/listinfo/cobbler


irc
#cobbler and #cobbler-devel in irc.freenode.net

To start with the entry formalities were so well orchestrated and so was the parking at the venue. The assembly area was sanitised by the army for Conty Club members. As the Conty’s rolled in, the other formalities were done and CCI was playing fantastic host. Getting members to know each other and their cars and looking inside out of each other cars was the norm of the hour.

Soon after the 10th car arrived, members were requested to proceed to have their refreshments. The refreshments itself was a brilliant arrangement and the army had set up the spread with such opulence, that it seemed more like a small wedding reception. Although there wasn’t a huge breakfast spread at you disposal, the stuff was substantial for a brunch. The military chai still lingers in my taste buds.

The photo sessions went on with members & families including children simply having a good time with 19 Conty’s in one place plus the sun playing hide and seek with a gentle drizzle to add to the aura of the event.

Everything was with clock work precision. A small talk by the hosts about the meet, the philosophy of the club in bringing people together for a purpose, the love of the car and members sharing their experiences with loads of laughter, fun and frolic.

The cars were finally flagged off by a senior army official and split in different directions to avoid disruptions to normal traffic from a big convoy and prevent inconvenience to other road users. Again a very thoughtful gesture by CCI.

The team regroup at Residency road parking with all the 19 cars for a round of chilled Red Bull and was greeted by the Scooter club making a cheerful pass, also by some of CCI’s friends from the Fiat Club. The cars were parked for public viewing and again an instant hit. The teams dispersed on time and the Sunday couldn’t have been spent in a better way.

I thoroughly enjoyed my time at the CCI meet and have volunteered to take pictures of the next meet also as their official photographer. The show they put up was simply awesome. This is by far the best car or auto meet by a group/club I have ever been to.

Some pics here

ContyClub_Meet