December 29, 2008
Here’s how to do serious scaling with PostgreSQL for pennies:
- Use pgpool for replication, load-balancing and failover
- Run the whole thing on Amazon Web Services like Soocial
Done. There were some examples in AWS’ docs suggesting an “average web database of 100gb” with 100 I/Os per second would cost around $36/month to run. I’m pretty sure I already pay $36/month for just the electricity in my server cabinet.
My only concern is durability… it seems like there are still some hoops to jump through to guarantee a node failure doesn’t take all of your data with it.
Anyone have experience running ColdFusion on AWS/EC2?
All Categories from Feedburner
Zack Steinkamp said:
on December 30, 2008 at 9:24 am
Brian, to ensure a node/instance failure doesn’t take your data, create and mount an Elastic Block Store (EBS) volume to house the data.
EBS volumes are highly reliable (running on big Hitachi hardware, e.g. http://www.hds.com/products/storage-systems/universal-storage-platform-v.html ), highly performant, and survive the reboot/shutdown/deletion of a machine instance. You can create a backup of your data in only a few seconds (literally) — just lock your tables, run the command to snapshot the EBS volume, then unlock tables. This is best done on a slave DB machine so as to not interrupt your service.
I’ve found the best AWS setup so far involves provisioning an instance from a base OS image (I use Rightscale’s CentOS 5 images), then configure your accounts, software, etc on that instance, then make a new machine image (AMI) from which you clone new machines from. It takes about 30 minutes to create your own AMI.
When making a new machine instance, you provision a new EBS volume using your latest data snapshot as a basis (as a DB slave, it should be set up already to connect and catch up to the master) and mount it to the new instance that is made using your custom AMI.
Provisioning a new DB slave is then only a couple of steps, and maybe 5 minutes total time.
I’m going to be “off” from Jan 5-19, so if you like I can stop by MSRHQ and give you a demo.
brian said:
on December 30, 2008 at 9:32 am
@Zack – you take time off??!
I will gladly take a demo – name your date and time and lunch is on me!
Zack Steinkamp said:
on December 30, 2008 at 9:38 am
Cool deal Brian. I will be in touch next week then.
Tony Brandner said:
on December 30, 2008 at 9:52 am
A friend points out this as a potentially larger-scale alternative to pgpool – http://www.slony.info
He says: “a bit more meatier implementation but pgpool seems pretty sweet for a smaller implementation”
Tony
charlie arehart said:
on December 30, 2008 at 1:20 pm
Hey Brian, you ask about those with experience doing CF and Amazon EC2, etc. I’ve had Adam Howitt on the CF Meetup a couple of times talking on the topic. You can find them listed either on the meetup recordings page (recordings.coldfusionmeetup.com) or via a search in my UGTV site list of recordings. Here’s a URL to go straight to the search results: http://www.carehart.org/ugtv/list.cfm?search=amazon
There’s one talk there also from an Amazon consultant, but it really wasn’t specific to CF. Hope that may help while you await other replies.