Why Adam Smith Would Be Impressed by AWS

In “The Moment I Bought into AWS” I mentioned how the father of modern economics, Adam Smith, would have been impressed by AWS.  Why is that so?

This is best illustrated by one of my most favorite EconTalk episodes ever, “Roberts on Smith, Ricardo, and Trade“.  I enjoyed that episode so much that I’ve probably listened to it 3 or 4 times.  The point I’m referring to gets made about 20 mins in, although you have to listen through the first 20 mins for the proper buildup.

Smith’s point is that if the extent of the market is large enough, only then does it make sense to invest in capital (technology) that will make it cheaper to produce output.  If the extent of the market is not large enough, investing in that capital will actually make the output more expensive.

Or in a more concrete example: If you wanted to make a sandwiches for yourself at home every day for lunch, it wouldn’t make sense to buy a deli meat slicer (the technology) for just yourself. The slicer is expensive and turning it on and cleaning it would be too much hassle for a single sandwich.  However if you were making 500 sandwiches per day as a sub shop owner, then having the slicer would allow you to produce sandwiches more quickly and more efficiently (less labor needed per sandwich).

This relates back to the the Amazon video in the examples of innovative datacenter optimizations they’ve created for themselves.  Since Amazon is operating at such a massive scale, they’re able to invest resources into optimizing power supplies, BIOSes, UPS firmwares, and custom storage systems in ways that a smaller player would be foolish to spend time investing in improving for their own benefit which is of a much smaller scale.

Advertisements

The Moment I Bought into AWS

I bought into AWS after watching this video that Alan G had sent me.  It’s an amazing example of how due to economies of scale, Amazon is able to invest in efficiencies in storage, servers, UPS’s, etc that it would never make sense for a smaller organization to invest in.  Then, because of these efficiencies that they develop, Amazon is able to pass better products and cost savings onto their customers.

Adam Smith would be proud.

Rewriting an Email’s Sender Address w/Postfix

At Virtual Instruments we rely upon emails from cron to get output from periodic tasks within our infrastructure.  These emails will come from the user account and the machine from whence they ran.  For example, I can expect emails from “jenkins@vi-jenkins.vi.local” or “root@vi-cron1.vi.local”.

Our IT staff recently made a security change as to who could send emails to company distribution lists, blocking emails that do not come from “<xyz>@virtualinstruments.com” email addresses.  This broke our aforementioned cron email notification setup.

Apparently new versions of cron support a “MAILFROM” environment variable, but unfortunately Ubuntu 12.04 LTS does not support this.

Not to fear!  Our production VMs are all running the Postfix mail daemon, managed by Puppet.  With a little bit of Postfix sender address re-writing magic, we can make all of the emails sent by cron appear to be sent by someone else.

The file “/etc/postfix/generic” is a table of patterns to match against the sender of each email.  If there is a match in the left hand column of this table, postfix will replace the sender with the corresponding address in the right hand side of the row. In our case, I wanted all emails being sent from the current host to go out with a different sender. You could easily change the pattern to only match for specific users but since our machines are all service machines I didn’t see the point in limiting this to specific users.

I made the following changes to our Postfix configuration with Puppet:

/etc/postfix/main.cf:
    smtp_generic_maps = hash:/etc/postfix/generic

/etc/postfix/generic:
    @<%= @hostname %>.virtualinstruments.com     <my_sender_account>@virtualinstruments.com

Then you just need Puppet to exec the following “postmap” command and restart the Postfix service like so:

    file {'/etc/postfix/generic':
      owner   => 'root',
      group   => 'root',
      mode    => '0644',
      content => template('postfix/generic.erb'),
      require => Package['postfix'],
      notify  => [
        Exec['update postfix postmap generic.db'],
      ],
    }

    exec {'update postfix postmap generic.db':
      command     => '/usr/sbin/postmap /etc/postfix/generic',
      user        => 'root',
      refreshonly => true,
      notify      => Service['postfix'],
    }

Voila! Emails from cron no longer come from “root@mymachine.vi.local”, but instead as whatever sender account you defined in “/etc/postfix/generic”.