Quick Tip: Moving the Insertion Point in the Mac OS X Terminal

It’s been a while, but I got a quick tip to share that I bet a lot of Mac power users don’t know about and will be interested in. If you forget to sudo a command, what do you do? Press the up key, hold the left arrow for 10 to 20 seconds while you look around for something to do with your other hand so you don’t feel like you’re wasting as much time, type “sudo”, and hit enter? Try this: Hold down the alt key and the mouse cursor will turn into a pair of crosshairs. Then click a character and the insertion point will move there instantly!

Does BitTorrent Equal Piracy?

Apparently, it doesn’t:

http://www.doesbittorrentequalpiracy.com

The newly launched site lists takes users through many of the legal uses of BitTorrent. Here are the more interesting ones in my opinion (in other words, the tech-related ones):

  • BitTorrent is how Facebook scales code deployment.
  • BitTorrent is what is powering a faster, congestion-aware Internet.
  • BitTorrent is how Twitter scales code deployment.
  • BitTorrent is how the Internet Archive preserves and distributes over a petabyte of the world’s cultural artifacts.
  • BitTorrent is what will make it possible for creators to broadcast anything live, without bandwidth restrictions.
  • BitTorrent is what makes grabbing open-source software a 6-minute job, not a 60 minute one.
  • BitTorrent is what Wikipedia uses to keep video bandwidth costs down.
  • BitTorrent is what Etsy uses for search replication.
  • BitTorrent is what Blizzard uses to distribute content and patches for games like Diablo III, StarCraft II and World of Warcraft.

Multi-Keyed Lookup Table Using C++11 Templates

So yesterday I decided to exercise my template chops.  What I wanted to create was a lookup table that would allow you to insert and retrieve values based on any of several keys.  It’s easiest to explain with some sample usage:

struct Thing {
    Thing(int id, int group, std::string name) : id(id), group(group), name(name) {}

    int id;
    int group;
    std::string name;

    void print() const {
        printf("id - %d, group - %d, name - \"%s\"\n", id, group, name.c_str());
    }
};

int main(int argc, char* argv[]) {
    LookupTable<Thing, LookupKey<'__id', int>, LookupKey<'_grp', int>, LookupKey<'name', std::string>> table;

    Thing a(1, 2, "a");
    Thing b(2, 5, "b");
    Thing c(3, 3, "c");
    Thing d(4, 5, "d");
    Thing e(5, 8, "e");

    table.insert(a, a.id, a.group, a.name);
    table.insert(b, b.id, b.group, b.name);
    table.insert(c, c.id, c.group, c.name);
    table.insert(d, d.id, d.group, d.name);
    table.insert(e, e.id, e.group, e.name);

    printf("searching for id == 3\n");
    for (auto thing : table.lookup<'__id'>(3)) {
        thing.print();
    }

    printf("searching for group == 5\n");
    for (auto thing : table.lookup<'_grp'>(5)) {
        thing.print();
    }

    printf("searching for name == \"b\"\n");
    for (auto thing : table.lookup<'name'>("b")) {
        thing.print();
    }

    return 0;
}

Kinda cool, right? One import design goal was to make this as fool-proof as possible. Everything is checked at compile time so if you slip up and mistype a key or use a wrong type as an argument, you’ll get a static assertion. For example, trying to call table.lookup<'namw'>("b") in the above example would give you a “key not found” compile message.

Here’s the source:

https://s3.amazonaws.com/daxnitro/misc/LookupTable.h

There’s a decent chance that this code is unlike anything you’ve ever seen or written before. It demonstrates several new C++11 features and many older, yet rarely seen ones. The key parts are the ways it utilizes variadic templates and static assertions, while leveraging SFINAE and partial ordering. If you’re into C++ and any of these buzz terms are unfamiliar to you, I recommend reading up on them. Templates can be magical (and really disorienting).

It should be noted that the above code isn’t meant to be used with objects that aren’t trivially copyable and might be more useful with the addition of a mechanism to remove items, but the primary purpose here is to demonstrate concepts.

Inventing on Principle / Development Environments


Bret Victor – Inventing on Principle from CUSEC on Vimeo.

A while ago, a friend sent me a link to this and it was pretty inspiring to me. Bret Victor‘s guiding principle is one any programmer can relate to, and this video will make for an hour well spent. It might even change the way you think about development altogether.

While I’ve been unable to find or create tools for myself to achieve the levels of immediate connectivity demonstrated, the video at least contributed to my decision to create a programmer’s text editor: CodeX.

The hope was always that eventually this could turn into a full-fledged IDE, and by using my own IDE, I could have complete control over my programming environment, give it the feel I want, and implement any features that I find useful, including these. Unfortunately there are some challenges. The main one is that in order to see things like live previews, on-the-fly function simulation, or even intelligent auto-complete and in-line error checking for files that are part of larger projects, the IDE needs to have full knowledge of what it takes to properly build the code you’re working on. It needs to know where to find headers, what macros to define, where to get symbols, etc. and to do this with a language like C++, it needs a firm concept of projects. Of course, I could implement support for the common types of projects: make projects, Xcode projects, Visual Studio projects, Eclipse projects, etc. but trying to support another IDE’s project format is the type of thing nightmares are made out of. I’d much rather create a new project format for the IDE to use. But I’m not quite convinced that’s the best way to go either.

Lately I’ve decided to take my development one step further than simply using my own editor. I recently finished creating my own compiler for a high-level C-inspired language, and I’m at the point where I need to figure out how multi-file projects are going to fit together. In an ideal programming language, would we even need project files? I’ll surely touch on this more later.

Migrating to AWS – The Final Step: Transporting the Database

Okay, so when I made the first migration post today, I expected to do this whole thing over the course of a few days.  Turns out the sluggishness of the site bothered me too much to let that happen.  I just finished copying the database over to AWS and with that, my site is serving up pages a thousand times faster (slight exaggeration).

I originally said I would put the database on an EBS volume attached to an EC2 instance.  Instead, I actually decided I would try out Amazon’s RDS service since I’ve never actually used it before.

Getting up and running on that was extremely straight-forward:  I just created a database instance in the same zone, added my own IP as well as the EC2 instance to the security group, then moved everything over.

Easy as pie.

Step 3 complete.  The site feels great now.  I even added the page generation time to the footer so we can quantify its speed.

Just to summarize the differences:

  • Before step 1 – Page generation times varied wildly.  Were occasionally in the 30-60 second range.
  • After step 1 – No change.
  • After step 2 – Average of about 2 seconds.
  • After step 3 – Average of about 0.18 seconds.

I’m happy with this, but maybe I’ll try to further optimize it later for kicks.

Migrating to AWS – Step 2: The EC2 Instance

Time to set up the site’s EC2 instance.  To make this incredibly easy, I’ll be using Amazon’s Elastic Beanstalk.

First of all, the MySQL database needs to be accessible externally.  If I were too concerned about security to leave the database in this state, I could make it available over localhost only later after moving the database.  Making it accessible externally is primarily just to make the transition seamless.

Since the database is accessible externally, and the site’s configuration uses the external address, I’m able to create an almost fully functional mirror of the site by just copying the files to my local machine.  The only thing that doesn’t work is the links, since they point to the site’s online domain.

Screen Shot 2012-12-14 at 4.11.26 PM

My local mirror.

Now that I have a local mirror, this can act as a sort of staging area for the site.  I can make changes locally, test them out, then push them to the world if they look good.

From here, I’m essentially going to run through the Develop, Test, and Deploy guide for Elastic Beanstalk:

dnmbp:Desktop dax$ cd /Volumes/Macintosh\ HD/Users/dax/My\ Sites/daxnitro 
dnmbp:daxnitro dax$ export PATH=$PATH:/Users/dax/Desktop/AWS-ElasticBeanstalk-CLI-2.1/eb/macosx/python2.7 
dnmbp:daxnitro dax$ git init .
Initialized empty Git repository in /Volumes/Macintosh HD/Users/dax/My Sites/daxnitro/.git/
dnmbp:daxnitro dax$ git add .
dnmbp:daxnitro dax$ git commit
[master (root-commit) 2c96e54] initial commit
 1067 files changed, 268464 insertions(+)
 ...
dnmbp:daxnitro dax$ eb init
To get your AWS Access Key ID and Secret Access Key, 
  visit "https://aws-portal.amazon.com/gp/aws/securityCredentials".
Enter your AWS Access Key ID: XXXXXXXXXXXXXXX
Enter your AWS Secret Access Key: XXXXXXXXXXXXXXX
Select an AWS Elastic Beanstalk service region (current value is "US East (Virginia)").
Available service regions are:
1) US East (Virginia)
2) US West (Oregon)
3) US West (North California)
4) EU West (Ireland)
5) Asia Pacific (Singapore)
6) Asia Pacific (Tokyo)
Select:  (1 to 6): 1
Enter an AWS Elastic Beanstalk application name: Daxnitro
Enter an AWS Elastic Beanstalk environment name:  Daxnitro
Select a solution stack.
Available solution stacks are:
1) 32bit Amazon Linux running PHP 5.3
2) 64bit Amazon Linux running PHP 5.3
3) 64bit Windows Server 2008 R2 running IIS 7.5
4) 64bit Windows Server 2012 running IIS 8
5) 32bit Amazon Linux running Tomcat 7
6) 64bit Amazon Linux running Tomcat 7
7) 32bit Amazon Linux running Tomcat 6
8) 64bit Amazon Linux running Tomcat 6
9) 32bit Amazon Linux running Python
10) 64bit Amazon Linux running Python
11) 32bit Amazon Linux running Ruby 1.8.7
12) 64bit Amazon Linux running Ruby 1.8.7
13) 32bit Amazon Linux running Ruby 1.9.3
14) 64bit Amazon Linux running Ruby 1.9.3
Select:  (1 to 14): 2

Except rather than use “eb start” to start up the environment, I’m going to do it via the web console since it gives me a few additional options (or maybe my eb version is out of date):

Elastic Beanstalk Environment

Now all it takes is a “git aws.push” to push the site to the EC2 instance!

Step 2 complete.  I can go ahead and modify the DNS entries to point an elastic IP address bound to that instance.  Once that happens there should be a significant increase in the site’s speed (though it still won’t be as fast as I want until we move the database).

Note: If you’re pretty sure you’ll never need the auto-scaling and load balancing functionality of Elastic Beanstalk, you can save yourself a potential $18 / month by just creating a single EC2 instance and uploading the web site manually. However, the auto-scaling / load balancing does come free for the first year under the free usage tier.

Migrating to AWS – Step 1: Moving Uploads to S3

So I’ve begun the process of moving the site to AWS.  First, a few words about AWS.  AWS is Amazon’s highly renowned “Infrastructure as a Service” product (IaaS for short).  It allows you to rent out computing power, storage space, databases, load balancers, the works, mostly operating within this amazing new mysterious thing known as “the cloud”.

There are other IaaS providers out there.  For example, there’s Google’s Cloud Compute Engine.  But at the moment, I consider AWS to be far superior (and last time I checked, cheaper).

If the features and pricing aren’t enough incentive to go with AWS, this should at least be enough incentive to give them a try:  AWS has a Free Usage Tier that allows you to use their services for a year, at no cost.  You’re a bit limited in what you can use under the free usage tier, but it’s more than enough to run a low-to-moderate traffic website.

Now then, the end result of this migration will be the website running on an EC2 instance, with the MySQL database stored on an attached EBS volume (avoiding RDS for the time being simply to minimize costs when the free usage tier expires), and the uploads (images and other attachments) placed in an S3 bucket.

The first step, as indicated by the title of this post: Moving uploads to S3.  The reason we do this rather than store them within the WordPress filesystem is to avoid saving any sort of state within the site’s filesystem itself, because when EC2 instances are terminated, any changes to the filesystem disappear forever.  This is also a great start to making your site horizontally scalable and able to adapt to large workloads should it rapidly gain traffic.  The idea is that if needed we could spin up multiple EC2 instances and use a load balancer to distribute the work amongst them with only minor adjustments.

The method of moving your mutable assets to S3 varies depending on the site, but luckily, for WordPress sites, there are some existing plugins that make it easy.  First and foremost, there’s WP2Cloud.  This is probably the best solution out there for many, and I may end up using it in the future, but right now I’m only interested in moving uploads.  I’m also very skeptical of the claims by the author that the required ClouSE MySQL plugin performs as well or better than using a local MySQL server on an EBS volume.

The plugin I’m currently using: Amazon S3 for WordPress with CloudFront.  So far it does what it says.  When I upload new media, it gets sent to my S3 bucket.  I had to transfer the previously uploaded media manually, but in my case that was only a single image.

Step 1 complete.  All uploaded media is now stored at “http://daxnitro-wp.s3.amazonaws.com/wp-content/uploads/……”.

Well, one thing’s for sure: GoDaddy isn’t cutting it.

The site’s currently hosted by GoDaddy (on a cheap hosting package I’ve had for years).  I have no qualms with them as a domain registrar, but when it comes to web hosting . . . just say “no”.  They get nothing but bad marks from me in both features and performance.

GoDaddy hosting

Even trivial PHP scripts here occasionally take 30-60 seconds to execute. Before the site hits prime time, I’ll definitely be moving it to another host, likely AWS, which I’ve had nothing but great experience with in the past and highly recommend.