Clever Advertising Campaigns of McDonald’s
sunilshenoy
's favorite posterous posts
Here's some stuff sunilshenoy has liked. To find more cool stuff, check out Explore »
We should always have a reason to wake up!
We should always have a reason to wake up, and we shouldn't miss this reason.
I think we should follow the signs, me yes sure I'm following the signs:)
Remember when you were a kid and you thought you could do anything? You still can.
via my old friend Sherman's posterous: the juice is worth the squeeze
I'm trying to tackle some pretty tough stuff tonight... Sunday night, of all nights, usually ends up being my most productive. It's definitely a long hard slog, but this definitely brightened my evening.
The blood, sweat, and tears are worth it. You're right, Sherman! The juice is worth the squeeze.
I want a house like this!
This incredible eco-house was designed by the 74 year old green architect Micky Muenning.
Located within the rolling mountains of Cooper Point, California, the
2,745 sq. feet house has 3 bedrooms, a series of solar panels that
provide all of the enrgy needed to upkeep the house, and loads of
light…
Rails framework upgrades with git-bisect
When Rails 2.2 hit the shelves, the features were not compelling enough for us to move Poll Everywhere from 2.1 to 2.2. Now that 2.3 is out, we've got a number of reasons to upgrade and as expected, there was a lot of breakage. To make matters work, we have some code that does some really hacky things to the Rails framework. We tried running regression tests but that only helped us fix about 80 of 100 failing tests (out of 600 total tests). For the last 20 some failing tests, we needed to dive deep into the Rails framework and find out what specifically changed to help us find what might be breaking in our code. Fortunately my new best friend, git-bisect, was there to save the day. Here's how:
First, clone Rails from github somewhere on your local machine. We're going to symlink this into your project vendor/rails directory.
my_project$ ln -s /local/github/rails_clone/ ./vendor/rails
You're going to want to checkout the version of rails you're testing against from the rails git repo. Fortunately the rails team does a great job tagging their releases so its pretty straight forward
rails $ git checkout v2.1.2
Do a sanity check by running your test suite to make sure everything is passing. Now the fun starts; checkout the version of rails you plan on upgrading to:
rails $ git checkout v2.3.2
Run your test suite again and see what's broken. In the case of Poll Everywhere, we had 100 tests failing and were able to fix 80 before we had to dive deep. If you're luckier than we are, you might be able to fix everything and verify it through automated tests. If not, get ready for a bunch of server restarts.
Now we want to start git-bisect to see what change in Rails broke your application. Finding out what commit in Rails broke your app makes it a hell of a lot easier to hunt down the offending code in your app. Lets start git-bisect:
rails $ git bisect start
rails $ git bisect good v2.1.2
rails $ git bisect bad v2.3.2
Bisecting: 1104 revisions left to test after this
[550fbcceddceabdb4fe000ee52142e8461b28d54] Fix test warnings
Git will checkout a commit that you need to mark as good or bad. The determining factor for good or bad could be the tests you wrote (do they all pass/fail?) or does your application crash when you are running it. If all is well, you'll mark the commit as "good" and you'll get the next commit to test:
rails $ git bisect good
Bisecting: 552 revisions left to test after this
[3b35366d5df8c8d8a7b216c42dd96b0cfa38fee4] Use more generic test env flag
If the next test run is bad:
rails $ git bisect bad
Bisecting: 275 revisions left to test after this
[d8a555e1376ed509c9f81c42229c5e923153eeb3] Mocha 0.9.0 compatibility for test setup/teardown callbacks
And so on until you find the offending commit:
Bisecting: 1 revisions left to test after this
[f7f113610e7cdca8ef03e206f2cbeb8400cdfefa] Add a rake task to apply a template to an existing application.
rails $ git bisect good
Bisecting: 0 revisions left to test after this
[e4eadf3910d04fcdcab3e084d249f3a881a3ca35] Fix message when running TemplateRunner#git. [#1526 state:resolved]
rails $ git bisect good
ebec9d43e262d28d742ff10acd828bad6cbb28ed is first bad commit
commit ebec9d43e262d28d742ff10acd828bad6cbb28ed
Author: Joshua Peek <josh@joshpeek.com>
Date: Sun Dec 7 16:37:48 2008 -0600
Make integration test runner more Rack friendly and clean out old CGI cruft
:040000 040000 66590983a84e62e949b326db9b8d7da717a15a79 d333e0e0580849752bf4d192b8061a64c73e49fc M actionpack
Now you can take a look at the patch, see what code is changed, and hopefully follow it to a point in your code that is blowing up.
When you're done fixing everything, remove the vendored version of rails and call it a day.
my_project $ rm -rf ./vendor/rails
Want to buy Twitter? Better find $3 billion.
Valuing Twitter is a purely theoretical exercise. There are no revenues and we know little about their cost structure. That said, there is lots of speculation on Twitter's suitors and the potential price they should pay. From my own back of the envelope view of the potential of Twitter, I think all the speculation wildly underestimates Twitter's value.
Here is how I think about their value. First off, Twitter, as I have argued, is a search business. Regardless of how they monetize it in the future, the biggest value lies in the ability to siphon off search volume from the current leaders. So, the key questions to answer in coming to a value are: how much share can they get and what is that share worth? Let's start with the second question first. Here are the current search market share stats:
So, what is a point of search share worth? There really are only two semi-pure play search companies:
Google: Market cap ($115bn) divided by share (81%) = 1.5bn per point of share
Yahoo: Market cap ($18bn) divided by share (10%) - 1.8bn per point of share
Is this a fair way to get search share value? Both these companies have other businesses that certainly contribute to value. Of course, they also have businesses that detract (lots of high cost projects), but there is no question, search drives the vast majority of these market caps. Yahoo has more diversified revenue source thus creating a premium. Let's take the Google value and discount it since they get a dominant position premium. So, let's cut it by 20%. Therefore, for these purposes, a point of search share is worth $1.2bn.
Twitter is not even on the share map. Why? Well, they are not a search engine. But, people are increasingly using Twitter to get information on products, reviews on events, news updates, etc. And, this does not include anything creative they could do by aggregating links, retweets, influencer appeal, etc. to build real search results.
In the last week, I did about 1 twitter search for every 3 on google (this does not include my constant search streams I have in Tweetdeck). Twitter has about 25% of my search volume. Now, I am not your typical Internet user. Let's say only 5% of people are like me. That is 1.25% share.
That creates a current value of $1.5bn (the discounted $1.2bn per share point x 1.25% share). Now, you can poke all kinds of holes in this logic. Even if you cut it in a third, you are looking at 1bn of real value TODAY. Google and Yahoo are increasingly valued as mature companies - not on potential. Their growth is slowing massively, so their market caps are based on mostly current results. Does Twitter have upside on their current position? You better believe it.
The potential of Twitter is enormous. It is just hitting the mainstream. The search model is not even developed. The data created has option value above search. And, the costs of running twitter have to be low. No need to index the internet. No need for millions of servers.
3x current value seems very reasonable to me given the current momentum. So, $3bn to buy Twitter. Not only is it worth this much, but the founders don't need to sell. They have made money and they know the downsides being bought. It will take an incredible offer to even create consideration. $3bn could be low.
If you think this is crazy, consider Ballmer was willing to pay $20bn plus for Yahoo - a business with many assets but decreasing traction and tons of complexity. Is the hottest Internet property, with huge search game changing potential not worth 1/7th of that? I think it is a no brainer.
Building and Scaling a Startup on Rails: 12 Things We Learned the Hard Way
There are a bunch of basic functional elements to building out a popular Rails app that I've never really seen explained in one place, but we had to learn the hard way while building Posterous. Here's a rundown of what we've learned, in the hopes that some Google linkjuice may bring an intrepid wanderer to this forlorn part of the woods and help you out.
Static Storage
S3 is awesome. Yes, you can host everything off S3. Is it a little more expensive? Probably. But if you're engineer constrained (and if you're a startup, you absolutely are) -- set it and forget it. If you absolutely must have local storage across app servers, then MogileFS, GFS, or HDFS or even NFS (yuck) are possibilities. For alternatives to S3, Mosso is supposed to be good too.
Images, files, whatever. Just drop it there. People say a lot of stuff about the Cloud, but it's real and a game changer for anyone doing user generated content.
HTTP Cache Control
The HTTP protocol lets you tell browsers what static content they can cache. You set this in apache. Rails automatically will put timestamps in the IMG / javascript / CSS tags, assuming you're using the helpers. The Firefox plugin YSlow coupled with Firebug are your friends here. The improvement is significant and well worth your time, especially if you add gzip'ing. 100KB initial page load can be brought down to 5K (just the HTML file) on subsequent clicks around your site.
Search
You're not going to run full text search out of your DB. It's totally not worth it to roll anything custom here. The smart money is on Sphinx with the ThinkingSphinx plugin is probably your best bet. If you have more than one app server, you'll want to use this. Alternatively, Solr with Acts as Solr can be used if you're a Java geek / have Lucene/Solr experience previously.
Storage engine matters, and you should probably use InnoDB
MyISAM is marginally faster for reads, but InnoDB will make you more crash resistant and will not lock tables on writes. Read about the difference, because when your servers are on fire, you will realize MySQL feels like a pretty thin layer of goop on top of your storage engine. MyISAM is actually the default on MySQL, which makes sense for most crappy phpBB installations -- but probably not good enough for you. The default can hurt you.
Oh yeah, and if you can start with some replication in place, do it. You'll want at least one slave for backups anyway.
Fix your DB bottlenecks with query_reviewer and New Relic
This basically saves your ass completely. Everyone complains that Rails is slow. Rails is not slow, just like Java Swing is not slow. Rails makes it easy to shoot yourself in the face. If you do follow-the-textbook-example bumbling around with Rails ActiveRecord objects, you will end up with pages that drive 100 queries and take several seconds to return.
Above is a screenshot from query_reviewer. It tells you every single query being run, and alerts you to things that use temporary tables, file sorts and/or just damn slow queries.
In a nutshell, you need indexes to avoid full table scans. The traditional way is to run EXPLAIN manually on queries coming out of your dev log. Query_reviewer lets you see it all right there in the left corner of your web browser. It's brilliant. You also need to eager load associations that will use in your views by passing :include to your ActiveRecord find method call, so that you can batch up SQL queries instead of destroying your DB server with 100 queries per dynamic page.
New Relic is new for us, but it helps us see what is really happening on our production site. If your site is on fire, it's a freaking beautiful gift from the heavens above. You'll see exactly what controllers are slow, which servers in your cluster, how load is on all your machines, and which queries are slow.
Memcache later
If you memcache first, you will never feel the pain and never learn how bad your database indexes and Rails queries are. What happens when scale gets so big that your memcache setup is dying? Oh, right, you're even more screwed than you would have been if you got your DB right in the first place. Also, if this is your first time doing scaling Rails / a db-driven site, there's only one way to learn how, and putting it off til later probably isn't the way. Memcache is like a bandaid for a bullet hole -- you're gonna die.
You're only as fast as your slowest query.
If you're using nginx or Apache as a load balancer in front of a pack of mongrels (or thins or whatever else is cool/new/hip), then each of those mongrels acts like a queue. The upshot is that if you EVER have a request that takes a long time to finish, you're in a world of hurt. So say you have 4 mongrels, and Request A comes in to port 8000 and it takes 10 seconds. The load balancer is naive and keeps passing requests to Port 8000 even though that port is busy. (Note: This might help, but we don't use it)
Then what happens? Sad town happens. 1 in 4 requests after Request A will go to port 8000, and all of those requests will wait in line as that mongrel chugs away at the slow request. Effective wait time on 1/4th of your requests in that 10 second period may be as long as 10 seconds, even if normally it should only take 50msec!
Enter the wonderful mongrel proctitle. Now, you can see exactly what is blocking your mongrels. I keep this on a watch in a terminal at all times. It's what I look at immediately if our constant uptime tests tell us something's wrong. Super useful.
The answer is: a) run some mongrels dedicated to slow running jobs (meh) or b) run Phusion Passenger, or c) run slow stuff offline... which leads us to...
Offline Job Queues
So you gotta send some emails. Or maybe denormalize your DB. Or resize photos, or transcode video or audio. But how do you do it in the 200msec that you need to return a web request? You don't. You use Workling or Delayed Job or nanite. It'll happen outside of your mongrels and everyone will be happier.
I don't know why people don't talk about this more, because if you run a site that basically does anything, you need something like this. It *should* be a part of Rails, but isn't. It isn't a part of Rails in the same way that SwingWorker in Java wasn't a part of Java Swing core like forever, even though it absolutely had to be.
If you don't monitor it, it will probably go down, and you will never know.
Test your site uptime, not just ping but actual real user requests that hit the DB. Sure, you could use pingdom if you're lazy, but it seriously takes like 10 lines of ruby code to write an automated daemon that runs, does a user action and checks that your site is not hosed. open-uri is your friend. You don't know if you're up if you're not checking. Do not tolerate downtime.
Also, use god for mongrel and process monitoring. Mongrels die or go crazy. You gotta keep them in their place. (What's funny is that god leaks memory over time with Ruby 1.8.6 *sigh*). Munin, monit, and nagios are also great to have.
Keep an eye on your resources -- IO ok? Disk space? It's the worst thing every to have a site crash because you forgot to clean the logs or you ran out of disk space. Make cronjobs for cleaning all logs and temp directories, so that you can set it and forget it. Because you will forget, until you are reminded in the worst way.
Read the source, and cut back the whining
You will learn more reading the source and debugging / fixing bugs in plugins and sometimes Rails itself than a) complaining on a mailing list or b) whining about shit on your twitter. It's Ruby open source code -- if it's broken, there's a reason. There's a bug, or you're doing it wrong. Fix it yourself, drop it into a github fork, and submit back.
Beware old plugins
They don't work well. And they sit around on Google sucking up time and effort. Acts as paranoid is one. They look legit, with famous names who created them. Don't fall for it. Insist on using code that has been updated recently. Rails changes pretty fast, and plugins that don't get updated will waste your time, cause random bugs, and basically make your life crap.
Github is new on the scene and has totally revolutionized Rails. When in doubt, search Github. If it's not on Github, it's probably dead/not-maintained. Be wary.
Beware old anything
Actually, if this blog post is older than even 6 months or 1 year -- you might want to go elsewhere. Rails moves fast. What's hot and "must have" in Rails now may be totally a piece of crap / barely functioning garbage later. Same with any blog posts. Be super wary of the Rails wiki. There be dragons -- I mean, really stuff that references Rails 1.2.6 or earlier!
And that's a wrap.
There's tons more stuff, but this is a pretty decent list of stuff to watch out for. If you have any suggestions for other things I missed, or questions, please do leave a comment below!
If you liked this article, please try posterous.com and/or follow me on twitter at @posterous and @garrytan!
A comprehensive primer about CSRF, Session attacks, Redirection, XSS attacks and MySQL security for Rails
Totally awesome. Must-read for any production Rails dev. Heiko Webers is totally
providing a huge service by compiling a comprehensive list of items to
check for.
Scott Adams says early popularity lets you achieve and create quality
If you are planning to create some business or other form of entertainment, you will need quality at some point to succeed. But what is more important than quality in the beginning is some intangible element that makes your project inherently interesting before anyone has even sampled it. That initial audience will give you the luxury of time to create quality.
via dilbert.com
Be popular, then be excellent. Words to live by.
