Amazon Web Services publishes our most up-to-the-minute information on service availability in the table below. Check back here any time to get current status information, or subscribe to an RSS feed to be notified of interruptions to each individual service. If you are experiencing a real-time, operational issue with one of our services that is not described below, please inform us by clicking on the "Report an Issue" link to submit a service issue report.
I was checking out the archive for this blog and realized that I wrote my first post exactly five years ago! In that time I've written posts to introduce each of our new services. Here's a quick recap of the highlights:
- We started out with the November 2004 introduction of the Amazon Simple Queue Service - When we rolled this out five years ago, I think the general reaction was "Huh? Why would Amazon do that?" I think it is now pretty obvious that a scalable messaging model is an essential component of a scalable system architecture.
- The next step was Amazon S3 in the spring of 2006. That service has grown and grown (and grown) and now holds a remarkable 64 billion objects. Our early adopters quickly realized that Amazon S3 was an efficient and reliable way to store any amount of data.
- In the summer of 2006 I took a break from my summer vacation to announce Amazon EC2 and with it the concept of computing in "the cloud." As we've added features such as the Elastic Block Store, Elastic IP Addresses, Elastic Load Balancing, Amazon CloudWatch, and additional instance types, Amazon EC2 has become more flexible and more powerful, step by step.
- Amazon SimpleDB was announced at the end of 2007, giving developers a fast and highly scalable way to store and retrieve indexed data.
- We also introduced Amazon DevPay at the end of 2007, consistent with out plan to allow developers to build and to monetize their own web services. At the beginning of 2009 we also introduced the S3 Requester Pays model to make it easier to monetize content stored in Amazon S3.
- With increasing adoptions came increasing requests for more support options, so we introduced AWS Premium Support option in the spring of 2008, along with the Service Health Dashboard.
- We rolled out Amazon CloudFront in the fall of 2008, making it easier and more efficient to distribute content stored in Amazon S3.
- 2009 started off right, with the introduction of the AWS Management Console, making it even easier to manage the use of AWS resources.
- In the spring of 2009, Apache Hadoop became an official part of AWS with the introduction of Elastic MapReduce.If you read my 2006 post introducing EC2, you'll see that I thought that EC2 was a perfect host environment for Hadoop way back then!
- Come the summer of 2009, we introduced the Amazon Virtual Private Cloud, giving our customers the ability to connect their existing networks up to an isolated set of AWS resources across a VPN connection.
- This past fall we continued to push forward, giving developers access to the Amazon Relational Database Service, making it easy for them to set up and run a relational database in the cloud.
- Over the years we've also reduced our prices for bandwidth, Amazon EC2, and SQS requests. We also lowered prices and introduced a free tier of usage for Amazon SimpleDB.
We continue to listen to our customers and to innovate on their behalf. We've got plenty of work ahead of us, so stay tuned to this blog and keeps those cards and letters (and comments) coming!
-- Jeff;
Hadoop provides two filesystems that use S3.
S3 Native FileSystem (URI scheme: s3n) A native filesystem for reading and writing regular files on S3. The advantage of this filesystem is that you can access files on S3 that were written with other tools. Conversely, other tools can access files written using Hadoop. The disadvantage is the 5GB limit on file size imposed by S3. For this reason it is not suitable as a replacement for HDFS (which has support for very large files). S3 Block FileSystem (URI scheme: s3) A block-based filesystem backed by S3. Files are stored as blocks, just like they are in HDFS. This permits efficient implementation of renames. This filesystem requires you to dedicate a bucket for the filesystem - you should not use an existing bucket containing files, or write other files to the same bucket. The files stored by this filesystem can be larger than 5GB, but they are not interoperable with other S3 tools.
AWS Hadoop Filesystem(HDFS)
- ネイティブ・ファイルシステム( s3n:// )
- ブロック・ファイルシステム ( s3:// )
Didn't see this documented anywhere. Install ruby-aws with 'gem install ruby-aws'. If you are using soap4r and ruby 1.9 you may need to follow these instructions if they haven't updated soap4r yet.
Tomorrow I'm invited to the Amazon Webservices Customer Advisory board in London. And I'll attend with wico from the #vz team at the http://aws.amazon.com/enterprise_london/.
At my former company - Lovely Systems - we were using Google App Engine and Amazon Webservices all the time.
Moore’s Law has prevailed in the online services space - the cost of online infrastructure is now at a point close enough to nothing to cost nothing. Thanks to Google, only pay for what you use, on demand and accessed anywhere because it’s stored in ‘the cloud’ for $0.10 a gigabyte of bandwidth, or $0.0001 per email.
Check out Google App Engine, a platform that let’s developers host and run applications. Up to a generous quota for free, and then by the gigbyte.
http://code.google.com/appengine/docs/whatisgoogleappengine.html
One of the cool ‘useful’ apps using this ‘zero cost’ platform is behavioural targeting and segmentation app BTBuckets.
A behavioural targeting and segmentation tool running off the Google App Engine, along with Amazon Web Services AWS platform, it’s delivered free for up to 5 million impressions a month. The interface is 2.0 with easy to use, obvious functions that so often get missed in more expensive options. The app segments and triggers different kinds of events for different users. It’s useful, and equivalent to traditional software options worth $10,000 a month or more. But, there’s a catch, this app also aggregates all of the user information in its system to provide demographic and behavioural data into ad-serving systems. Subsidised by the new marketers quest for more granular targeting of their audience. Be prepared to have to change your privacy policy.
Having run a couple of startup companies in the cloud on Amazon Web Services, the issue of sending email has cropped up a number of times.
The problems
Amazon doesn't allow you to set reverse DNS lookup entries for your mail server and a large number of mail servers will not accept mail from a server that doesn't have this. This is a spam prevention feature to try and cut down on mail being relayed through insecure mail servers.
The other blocking issue you will come across is blacklisting. Because of the nature of cloud systems it is likely that someone else has had your IP address in the past. If the person wasn't entirely honest then they will probably have been blacklisted at some point. Here is an example error message you will receive if you try and mail from a blacklisted host:
553 Mail from xx.xxx.xxx.xxx not allowed - 5.7.1 [BL21] Connections not accepted from IP addresses on Spamhaus PBL; see http://postmaster.yahoo.com/550-bl21.html [550]
or
550 DY-001 Mail rejected by Windows Live Hotmail for policy reasons. We generally do not accept email from dynamic IP's as they are not typically used to deliver unauthenticated SMTP e-mail to an Internet mail server...
The solutions
So, what can you do about it? The simple answer is do not send mail from the cloud, instead relay your mail through a server hosted somewhere else.
The first option is to send email through GMail. This works fine for small volumes of email but as your web application becomes more succesful you will quickly start hitting the volume limits imposed by Google and will start seeing entries like this in your mail log:
550 5.4.5 Daily sending quota exceeded.
The best long-term solution is to run your own mail server hosted outside of the cloud. I recommend slicehost as a provider, you can have a mail server up and running from as little as $20/month and this will allow you to send masses of email. This might sound a bit ominous if you haven't set one up before but they have a great tutorial on setting up a mail server.
Dodging the spam folder
Once you have your mail server up and running you should set an SPF record for your domain. SPF records are TXT DNS records that you create to announce that your new mail server is allowed to send mail for your domain. Lets say you have just finished setting up your mail host as mail.example.com then you need to create a TXT entry for example.com with the following contents (including the quotes!):
"v=spf1 include:mail.example.com -all"
You should now be sending mail happily from the cloud and with a bit of luck your recipients will receive the mail in their inboxes.
Dynamo: Amazon’s Highly Available Key-value Store
Giuseppe DeCandia, Deniz Hastorun, Madan Jampani, Gunavardhan Kakulapati, Avinash Lakshman, Alex Pilchin, Swaminathan Sivasubramanian, Peter Vosshall and Werner Vogels
Amazon.com
今日は, SimpleDBを使ってみることにした. SimpleDBは, 要はクラウドの中にスプレッドシートを持って参照や更新ができるようなイメージ. 公式ドキュメントのココのページの図が特徴を良くあらわしている.
(dev)hdknr@domU-12-31-39-00-D9-A1:~/.ve/dev/src$ hg clone
https://hdknr@bitbucket.org/david/django-storages/
destination directory: django-storages
requesting all changes
adding changesets
adding manifests
adding file changes
added 43 changesets with 105 changes to 51 files
updating working directory
32 files updated, 0 files merged, 0 files removed, 0 files unresolved
(dev)hdknr@domU-12-31-39-00-D9-A1:~/.ve/dev/src$ cd django-storages/
(dev)hdknr@domU-12-31-39-00-D9-A1:~/.ve/dev/src/django-storages$ ls -al
total 68
drwxr-xr-x 6 hdknr users 4096 2009-10-11 05:20 .
drwxr-xr-x 3 hdknr users 4096 2009-10-11 05:20 ..
-rw-r--r-- 1 hdknr users 562 2009-10-11 05:20 AUTHORS
drwxr-xr-x 2 hdknr users 4096 2009-10-11 05:20 backends
drwxr-xr-x 2 hdknr users 4096 2009-10-11 05:20 docs
drwxr-xr-x 4 hdknr users 4096 2009-10-11 05:20 examples
drwxr-xr-x 3 hdknr users 4096 2009-10-11 05:20 .hg
-rw-r--r-- 1 hdknr users 103 2009-10-11 05:20 .hgignore
-rw-r--r-- 1 hdknr users 1539 2009-10-11 05:20 LICENSE
-rw-r--r-- 1 hdknr users 315 2009-10-11 05:20 README
-rw-r--r-- 1 hdknr users 21229 2009-10-11 05:20 S3.py
-rw-r--r-- 1 hdknr users 988 2009-10-11 05:20 setup.py
