Employee Equity and Startups – A Primer

Anyone who takes a job with a startup should be well educated with respect to matters related to employee equity, both to make sure they understand the risks they are taking on and to ensure that they do not wind up in a tricky situation with respect to taxes. In an ideal world startup companies would clearly and correctly explain all of these issues to each of their employees/prospective employees.

However some of them don’t or can’t, and unfortunately some will even deliberately feed bullshit information on the subject to their employees in order to avoid having to actually make good upon their promises of equity. So for the benefit of anyone who has unanswered questions on the subject of employee equity I am providing this article.

Be aware that I am not a lawyer, and do not claim to be. However I have worked with several startup companies and seen the employment contracts and equity agreements used by several others (startups and established firms alike), and invested a significant amount of my time in researching this subject. I think that makes me qualified to speak on it. I’m also not a liar, a bully, or a manipulator, and that ought to count for some measure of credibility as well. If that statement seems out of place to you, feel free to ignore it. Those who should know it to be true and relevant, will. And naturally you are free and encouraged to do your own research and to check the references I’ve used.

Anyhow, all equity agreements vary somewhat as far as the specific wording used. Some features, however, are essentially universal. These are the allocation, a vesting schedule, and a cliff. Many agreements will also include some verbiage around the subject of dilution, what happens if the company is purchased or goes public, and similar topics.

There will be more on all of those shortly, but first let’s discuss a couple of common misconceptions that are related to this subject.

Common Misconceptions
There are a number of myths about employee equity and compensation that you may find perpetuated at the less ethical startup companies. A couple of these are debunked here.

Equity is the “prize at the end of the race”. False. The value that the equity takes on over time is the prize at the end of the race. The equity itself is your incentive to run the race as hard as possible. Because that’s the only way your equity will ever have value. And it only works if you are allocated the equity before you start running. The fact of the matter is that you want to be allocated your equity when it is still worthless. If you wait for the company to become successful you’ll be stuck in one of two undesirable situations. Either 1) you will get hit with a massive tax bill when you finally do receive your equity that will probably force you to immediately sell off a significant chunk of it (which can be very difficult unless the company has gone public), or 2) you will be forced to take your equity in the form of options instead of as shares. Note that option #2 means that your equity will have zero value when you finally do get it (that’s the only legal way around the tax issue), unless your employer back-dates the grant date for you. And do note that back-dating options may be illegal. If your employer thinks you should run hard before they allocate you any equity, you should run all right; run away from your employer as fast as you can!

Market rate is lower at a startup. False. There’s only one market, and only one market rate. Regional fluctuations exist, naturally, but even those are diminishing as distributed teams, telecommuting, and similar practices become more widespread. In any event, a startup does not get a discount on the prevailing market rate simply for being a startup. Wages are virtually always lower at startups, because employees are allocated large chunks of equity to make up for the difference between market rate and what they are actually being paid. If a startup does not give this equity to its employees, then it has no excuse for offering any employee a below-market-rate salary. A correct reiteration of the original assertion would be “your cash wage is lower at a startup because you have been issued equity to account for the difference between your cash wage and your market rate”. If your employer says it any other way, that should be a red flag that something isn’t right. Doubly so if they say it in any other way in defense of not giving employees equity or of not paying employees the full market rate.

If you are unsure about the fair market rate for the job you perform, I encourage you to do some research on Glassdoor.com. For instance, here is the going rate for a technical architect in Australia. Here are some others:

Glassdoor is full of useful information. But it’s time to move on with the rest of the article.

Allocation
With equity agreements, the allocation is simply the amount of equity that a person is being granted. This may be specified as a total number of shares (at an established/well-organized startup) or as a percentage of the company (at a fledgling/less organized startup which has not set up a pool of shares yet). Either approach is fine, although when shares are used you should be sure to ask your employer what the total size of the share pool is, since obviously receiving 10,000 shares out of a pool of 100,000 is quite different than receiving 10,000 shares out of a pool of 100,000,000.

The bottom line with any allocation is that you should be able to work out what portion of the business, as a percentage of the company, that you are being granted. This is useful for answering questions like ‘if the company is sold tomorrow for $1 billion, how much of that would I get?’ ($10 million per each percent allocated). There is no “standard” allocation, however your allocation should be appropriate to 1) your amount of experience/expertise, 2) the importance of your role to the company, 3) your market rate and the amount of salary sacrifice you take on, and 4) how early (or late) you joined the startup. Depending upon these factors, allocations may be anywhere from 1% to 20%, with exceptions made in exceptional cases.

When joining a startup company early in its life and/or at a significant salary sacrifice you should be reticent about accepting any offer that is below 1% of the company. Never sell yourself short; if you’re going to be working for a fraction of your normal/market rate you deserve to have full confidence that you will win big if/when the company is successful. Otherwise you are just wasting your time.

Vesting / Buyback
The allocation specifies how much of the company you get. However, you will virtually never receive your full allocation up front (and if you are offered such a deal and aren’t a founder/co-founder, it’s a sure sign that the company you’re working with is very inexperienced in these matters; you should do the right thing, and let them know that they really ought to be using a vesting schedule). The reason for this is simple. If the entire allocation is ‘paid’ immediately, an employee can join a company, resign the next day, and walk with their entire allocation. That’s very bad for business, for obvious reasons.

To solve this problem, you will typically receive incremental portions of your allocation over time. Should you leave the company (due to resignation or any other reason), provisions in your equity agreement will only permit you to retain those incremental portions that you received up to your final date of employment. That portion is referred to as the vested shares (or vested equity, if working off a percentage).

In essence, vested shares are the shares that the employee now owns, while unvested shares are shares which have been allocated to the employee but not actually handed over to them yet (the employee will get them eventually if they stay with the company, but doesn’t own them yet). Unvested shares gradually convert to vested shares based upon the vesting schedule. A vesting schedule will specify a timeframe over which vesting occurs, and at what increments unvested shares will convert to vested shares.

A typical vesting schedule will specify a timeframe of 4 years (this is basically industry-standard), with the allocation vesting in equal increments across that span of time. Increments are usually handled on a monthly or quarterly basis, although half-yearly and yearly increments are sometimes used. So for instance, if an employee is given 4% of the company and has a 4 year vesting schedule with quarterly increments, they will receive 1% of the company per year they remain employed, at a rate of 0.25% per quarter. Usually a vesting schedule will not prorate partial increments (i.e. if your last day of employment falls in between two increments, you will not usually receive vested shares for the partially completed increment).

Buyback is a feature unique to Australia (or at least, not seen in the U.S.; it’s possible other countries use similar provisions). In the U.S. a tax loophole exists (known as an s83(b) election) that allows the recipient(s) of equity to avoid owing tax on it incrementally as it vests. An equivalent loophole does not exist under Australian tax law, and so some companies will use a buyback provision to work around this fact for their employees.

Buyback works almost like an inverse of the standard vesting schedule. Instead of the allocation being held by the employer and then incrementally transferred to the employee as it vests, under a buyback arrangement the employee will receive the entire allocation up front and the employer will retain a right to buy back a portion of the allocation from the employee for some nominal (close to zero) value. This buyback right diminishes incrementally, such that at the end of the vesting term it has been reduced to zero and the employee now has full control over their shares. Separate provisions will generally prevent the employee from freely trading any shares that are still subject to buyback, typically by the employer retaining right of first refusal over any such shares, or other prohibitions on trading.

Recent changes to the tax code in Australia may have removed some or all of the taxation benefits associated with buyback provisions, so they may be less common now than in the past.

Cliff
The simplest possible vesting schedule will convert unvested shares to vested shares in equal increments across the entire vesting timeframe. Few businesses will use that approach, however, again due to the (entirely legitimate) desire to prevent people who stay with the company only a short while from walking away with chunks of equity. To mitigate that possibility, most startups will include a cliff in their equity agreements.

A cliff is essentially a modification of the vesting schedule to delay the vesting of some portion of shares until some timeframe has elapsed. For instance, a cliff may specify that no shares shall vest until an employee has been with the company for a total of six months, at which point any shares that would have normally vested during that timeframe (as specified in the vesting schedule) will vest all at once.

So if our employee from the previous example (who was granted a 4% allocation vested quarterly over 4 years) is also subject to a six month cliff and resigns after five months, they will not be able to take any equity with them. Similarly, if they instead resign after 7 months they will have satisfied the cliff, and be entitled to take their first two quarters’ worth of equity (0.5% of the company) with them.

Typical cliff durations range from six months to 2 years. Some startups may not use a cliff at all, and you should be wary of accepting any cliff that is more than a year or so in duration. Also note that a cliff may have tax implications (particularly when the cliff timeframe is longer than one year in duration), as it causes you to receive a larger block of equity all at once. When an equity agreement involves options rather than shares (i.e. when joining an established company instead of a startup), a cliff is not typically used as the allocation is generally much smaller and its real/immediate value is significantly less than when dealing directly with shares.

Dilution
Shares in a company don’t appear out of thin air. As new allocations are granted, the equity being allocated to the new employee needs to come from somewhere. This necessarily means taking some portion of equity away from at least one existing shareholder (or out of a preallocated pool, if one exists). When this happens, it is referred to as dilution.

Dilution is a relatively straightforward subject in and of itself, but failure to understand it can lead to disaster. For instance, let’s say that Alice and Bob are co-founders at a startup, and that Alice holds 80% of the equity and Bob holds the remaining 20%. If a new employee is allocated a 10% stake in the company, the following scenarios are possible; 1) all shares dilute equally, meaning that Alice’s holding will be reduced to 72% and Bob’s will reduce to 18%, 2) only Alice’s shares will dilute, meaning that she is left with 70% and Bob retains his entire 20%, or 3) only Bob’s shares will dilute, meaning that he is left with only 10% while Alice keeps her entire 80%. In all cases the new employee ends up with 10%, at least until the next employee comes along. Note that a 4th scenario is also possible, in which the company maintains a pool of unallocated shares to cover new hires coming on board; this effectively prevents dilution, at least until the pool is exhausted.

It should be apparent from those example scenarios that being the only person with dilutable shares is a bad idea. As The Social Network so entertainingly underscores, if you are the only person with dilutable shares you will quickly see your equity evaporate. Note that in scenario #3 Bob lost a full 50% of his original allocation due to a single new employee coming onboard. It would not take many more hires to reduce Bob’s shareholding to essentially nothing.

When dealing with an equity agreement that is specified in terms of shares (i.e. with an established/well-organized startup), the terms of the agreement should state whether the shares allocated are Common (which they will be 99% of the time), Preferred, or any other special type. Any special dilution rules that apply should be mentioned, although it is always wise to ask about how many different classes of shares the company has issued, and about whether or not any class of shares has preferential treatment with respect to dilution. In the vast majority of cases, all shares will simply dilute equally.

Conversely, if your equity agreement is specified in terms of a percentage of the company (i.e. with a new or disorganized startup) you should ensure that your percentage is specified as being non-dilutable until such time as proper shares are allocated. This provides incentive for the business to allocate actual shares swiftly, and protects you from any number of shady behind-the-scenes things that might be done when dealing with a dilutable percentage.

Generally speaking, it is fair and commonplace for everyone’s shares to dilute equally once shares have been allocated. Prior to that point, it is fair and commonplace for only the founders’ holdings to dilute up until the moment when actual shares are allocated. If your equity agreement is significantly different than this, think twice. And if it tries to make you the only person with dilutable equity, run.

Options vs. Shares
Depending upon where the company is at the time you join it, your equity agreement may deal either in options or shares (if your equity agreement deals in terms of a percentage of the company, expect your portion of the company to be delivered to you as shares). Shares are quite simple, and the norm when dealing with an early-stage startup. Each share has a value that is determined by the company’s net-worth/market cap, and the value of an allocation is simply the share value multiplied by the number of shares. Simple.

The problem with shares is that they have a real value at the time they are allocated, and this means that share allocations are taxable as income (though note that when a vesting schedule is used your shares will be taxable as they vest and not when they are allocated). This can cause some very significant issues. For instance one share of Google is currently worth a little better than $1,100. If Google were to give 500 shares to an employee, that employee would owe taxes on some $550,000+ in income, probably forcing them to sell 30-40% of their shares just to cover the tax bill.

The way around this tax issue is to use options. An option is not a share; instead it is the right to purchase a share at a set price at some time in the future. The price associated with an option is generally (unless the company has back-dated the option) the market price on the date the option is issued. This means that when options are granted they have zero real value, which solves the tax problem nicely and is why established companies use options in their equity agreements almost exclusively. It also means that unless back-dating is involved, options are completely worthless when you receive them.

To return to our previous example, Google could issue an employee options on 500 shares, granting them the right to purchase those 500 shares now or in the future for $1,100/share. The options have no immediate value (the employee can get 500 shares of Google, but only by paying $550,000 for them; the same as buying them on the open market). If the value of Google increases to, say $2,200/share, at that point the employee can buy 500 shares for $1,100/share, and then immediately turn around and sell them for $2,200/share, netting a $550,000 profit (which is taxable as capital gains). So options only attain value if the company’s share price continues to rise. If it remains the same or goes down, the options remain completely worthless.

If you’re joining a newly founded startup, you do not want to receive options unless the strike-price is fixed on or against your hire date (meaning that it is <= $0.01/share, since the startup should be close to worthless when you join). You should be offered shares in the company. If you are not then that's a good indication that you should seek other opportunities elsewhere. Words are Trivial
Now that’s all a fair bit to take in. And this is a good moment to underscore the importance of always getting a written equity agreement from your employer. Verbal agreements are notoriously unreliable and subject to differences in interpretation and perspective. Particularly when dealing with subjects like vesting and dilution that are extremely nuanced and virtually impossible to adequately capture in a verbal agreement.

If the startup company you are working with will not provide a written equity agreement, or fails to provide one in a timely fashion, that’s a strong indication that you should seek another employer. There is little reason for a startup company to not provide written equity agreements, unless the managers of the startup company simply do not intend to actually give anybody any equity (some people just don’t like to share; but thankfully their actions make them easy to spot). And you really don’t want to end up working at startups like that.

If your employer cannot tell you (in exact terms) your allocation, your vesting schedule, your current vesting status, or exactly what will happen in the event of an acquisition, resignation, or termination, they haven’t given you a real equity agreement!

Example Equity Agreements
For reference, I have included a some example equity agreements here. All are real agreements that have been used in practice, although any identifying information has been obliterated. Note that although some are far more verbose than others, all agreements cover essentially the same subjects.

The equity agreement that you get from your employer should be reasonably similar to these agreements. You should be able to easily pick out the allocation and the vesting schedule. If you cannot, be wary. Or if you have only a verbal equity agreement from your employer, be triply wary.

Thought Exercise
Now let’s consider a hypothetical example company, Transnational Application Production Services. At this company, employees are routinely offered salaries that are significantly below prevailing market rates. The typical “discount” is in the range of 50-75% off of market rate, which means that some employees are even asked to accept offers that are below the legal minimum wage. In exchange for this salary sacrifice, employees are offered equity in the company as part of their compensation. However, this offer of equity is only ever stated verbally, and although several employees request written equity agreements both before and after signing their employment contracts, no written equity agreements are ever provided by Transnational Application Production Services. Nor does the company provide any statements relating to vesting schedules, allocation size, or other issues typically covered in an equity agreement.

Can you guess what happens if/when an employee leaves the company? Do you think they will get to take any portion of the equity that was promised to them when they leave? If you answered ‘absolutely not‘, you are probably correct. In a normal startup company, every employee would have a document the specifies their cliff terms (if any), and what happens should they leave the company before or after the cliff period has elapsed. An Transnational Application Production Services employee has none of those things; and since their equity agreement relies upon a verbal contract, it would be very difficult for them to ever prove otherwise. A verbal contract is about as good as having no contract at all.

However, let’s say that we’re wrong, and that Transnational Application Production Services agrees to make good on its verbal promises of equity. Can you work out how much equity the employee will receive? If you answered ‘absolutely not‘ again, you are most definitely correct. Because the verbal agreement from Transnational Application Production Services never specified an allocation or a vesting schedule, there’s absolutely no way of knowing how much equity a given employee should or will receive, or how much equity they possess at any given point in time. Without a written agreement, everything is left up to the whim of Transnational Application Production Services.

Contrast that with a standard startup company, in which each employee’s contract specifies an allocation and vesting terms. This makes the task of determining how much equity a departing employee gets to take with them trivial, since it’s all there in black and white. By not having a written equity agreement, employees at Transnational Application Production Services are exposed to a significantly larger amount of risk than their counterparts at virtually any other startup. One hopes that the hypothetical management at Transnational Application Production Services would at least take the time to explain this extra risk to employees as part of their hiring talks. Though you may use your own imagination to decide whether or not that happens in our hypothetical world.

But surely there’s an upside‘, you say. And that’s a reasonable thought. Taking on increased risk is supposed to result in increased rewards, after all. But unfortunately no, that’s not how it works in this case. Because no allocation has been specified, each employee’s rewards have been left entirely up to the whim of Transnational Application Production Services and its management. There’s no minimum amount of equity that an employee at Transnational Application Production Services can count on getting. There’s no maximum amount either, but that’s not an added benefit because the same case exists at every other startup. Having an allocation specified does not preclude management from allocating additional equity to people it feels deserve it, it simply provides a minimum amount that every employee can be confident that they will get.

So employees at Transnational Application Production Services are taking on more risk than their counterparts at other startup companies. While at the same time they have less certainty about how much of a reward they will receive. There is no upside; the employee loses on both counts.

Things are quite a mess over at Transnational Application Production Services. Our hypothetical company is most definitely not a shining example of how real companies should be run. If Transnational Application Production Services reminds you of your own employer, you’d be wise to start seriously seeking out other options.

Of course, it’s entirely possible that the management at Transnational Application Production Services is simply uninformed about how to best handle equity allocations, as it is a startup company after all. It’s important to always give people the benefit of the doubt.

However, imagine that at our hypothetical company multiple employees have raised the issue of getting written equity agreements in place. And that some of them have even gone so far as to discuss the very issues covered above with their employer. And that our hypothetical company has made repeated promises with respect to getting together a written agreement but not delivered upon a single one. And that at Transnational Application Production Services the only person who has been given an actual allocation is a manager who takes home a six figure income while his colleagues work on below-minimum-wage salaries, while simultaneously and routinely asserting that everyone is being paid market rate and that there’s no money available for pay increases. And that those employees on below-minimum-wage salaries have repeatedly brought up the issue of their low rate of pay. And that the only employees to successfully petition for a legal wage were the ones who pointed out that their wage was in fact illegal. And that all of this has gone on for years with no corrections being made.

A certain pattern of inequity begins to emerge at our hypothetical company. Will this pattern of inequity exist at every startup company that doesn’t provide written equity agreements? No, probably not. But the lack of one should be a red flag that you may be working at Transnational Application Production Services. And that is not a company that you want to work for.

Ignorance is understandable, if deplorable (and also 100% correctable). Deliberately taking advantage of people, on the other hand, is just plain despicable. Companies which operate like our hypothetical Transnational Application Production Services are taking advantage of their employees and wasting everyone’s time. The company is getting work at a significantly discounted rate, and not providing anything whatsoever in return. ‘Thievery’ and ‘exploitation’ are words that come to mind.

Our hypothetical startup is about as bad as they come. We should all be grateful that such companies rarely exist in the real world.

This example further underscores the importance of always getting a written equity agreement that specifies your allocation, vesting schedule, and cliff whenever dealing with a real-world company.

Random Quote
No matter how you spend your life, your wit will defend you more often than a sword. Keep it sharp!

Disclaimer
All characters, companies and events in this blog–even those based on real people–are entirely fictional. All celebrity voices are impersonated…poorly. The preceding blog post contains useful information and due to its content should be read by most people. But not the elderly, people with heart conditions, or CEO’s of disreputable startup companies.

References

Posted in banter, editorial | Tagged , , | Leave a comment

NSObject+SimpleJson

Certainly the subject of JSON processing on iOS is fairly old-hat at this point. After all, Apple has provided native support for JSON serialization and parsing since the release of iOS 5. So why bring it up now?

Personally I’ve never been entirely happy with the implementation provided by Apple, and in particular their decision to build their API around NSData instead of NSString. Granted there’s probably a reason why Apple chose to go that route, and also a reason why they didn’t bother to provide convenience methods that allow people to work with whichever type they prefer. But as far as I’m concerned, JSON is a textual/non-binary data format, so therefore the natural iOS data structure to use for outputting and inputting JSON data is an NSString. Using NSData instead is counter-intuitive, and clunky as a result.

So because of that clunkiness, I’ve stuck with the tried and true SBJson library, even though the author of that library himself recommends just using Apple’s implementation. This preference is largely based around the very helpful (and also very deprecated) ‘NSObject+SBJson‘ category that is included in the library. This category imbues objects with a couple of very convenient methods:

- (NSString*)JSONRepresentation;
- (id)JSONValue;

These methods can be used to turn an appropriate object (i.e. an NSArray or an NSDictionary) into a JSON string, and to turn a JSON string back into a proper object, respectively. No parameters, no options, no boxing to or unboxing from NSData; just a simple API that is basically just ‘[thing turnIntoJson];‘ and ‘[json turnBackIntoThing];‘. It doesn’t get much more convenient than that.

And that was all well and good, until I wanted to build a framework which internally used SBJson, and then use that framework within a project that also needs to make its own use of SBJson. Linker hell was the result. And long story short, I decided that it would be faster to build a shim around Apple’s JSON API to preserve the convenience I’m after than it would be to appease the linker gods.

So in case anyone else has been avoiding Apple’s JSON implementation because it’s too clunky to work with (or too painful to migrate away from SBJson), here’s my 5-minute shim:

NSObject+SimpleJson.h

#import <Foundation/Foundation.h>

@interface NSObject (SimpleJson)

- (NSString *)JSONRepresentation;  //turns an object into a JSON string
- (id)JSONValue;                           //turns a JSON string back into an object

@end

NSObject+SimpleJson.m

#import "NSObject+SimpleJson.h"

@implementation NSObject (SimpleJson)

- (NSString*)JSONRepresentation {
    //write ourself to a JSON string; only works if we're a type that 'NSJSONSerialization' supports
    NSError* error = nil;
    NSData* tempData = [NSJSONSerialization dataWithJSONObject:self options:kNilOptions error:&error];
    if (error) {
        return nil;
    }
    
    return [[NSString alloc] initWithData:tempData encoding:NSUTF8StringEncoding];
}


- (id) JSONValue {
    //converts from a string back into a proper object; only works if we're an NSString or NSData instance
    if (! [self isKindOfClass:[NSString class]] && ! [self isKindOfClass:[NSData class]]) {
        return nil;
    }
    
    NSData* jsonData = nil;
    if ([self isKindOfClass:[NSData class]]) {
        jsonData = (NSData*)self;
    }
    else {
        //we must be an NSString
        jsonData = [((NSString*)self) dataUsingEncoding:NSUTF8StringEncoding];
    }
    
    return [NSJSONSerialization JSONObjectWithData:jsonData options:kNilOptions error:nil];
}

@end

If you have code that uses SBJson’s ‘NSObject+SBJson‘ category, the above category should be drop-in compatible. And just as importantly (in my case) does not invoke linker hell when used as part of an iOS framework.

Note that the above code assumes you are using ARC. If not, you’ll want to add an ‘autorelease‘ call in there.

Posted in coding, objective-c | Tagged , , | Leave a comment

The Essence


You can feel it in the space around you
It’s like there’s something in the way that you can’t find

And you can’t hide

You’re only thinking it a day at a time
Somehow you’ve got to make a change
Make it right

So I will try to let you walk away

Stop holding it down
Stop thinking it over
Decide that you’ll live your life awake

Why don’t you live your life today?

I know you feel it in the space around you
I’m coming round to make a change for you this time

And you can’t hide

You’re only thinking this a day at a time
You’ve only ever walked away
So make it right

I still try and let you walk away

Stop holding it down
Stop thinking it over
Decide that you’ll live your life awake

Why don’t you live your life today?

Got this whole thing underground
I can feel you in every sound
I see your hand in every little thing
I see your mind in every little thing

Not every little thing will go your way

If I could offer you one piece of advice, it would be this: Keep your promises; every single one of them.

Or prepare for armageddon. The choice is yours.

Posted in Uncategorized | Leave a comment

On Notice

Until you crash, until you burn
While you lie, until you learn
Until you see, until you believe

I will fight you.

Until you fall,
Until the end of everything at all

I will fight you.

Until you give, until you have used
Until you are lost, until you lose
Until you see

I will fight you.

Until you have lived a thousand times
Until you’ve seen the other side

I will fight you.

Until your truth becomes a lie
Until you change, until you deny
Until you believe

I will fight you.

This is my chance
I take it now because I can
I want it now

I will fight you. And you will lose.

Posted in Uncategorized | Leave a comment

[JavaScript] Finding Equidistant Lat/Long Coordinates

Here’s a quick one that was inspired by a StackOverflow question and built using the references posted here. It allows you to compute the position of an arbitrary number of points (expressed as latitude/longitude pairs) equidistant from a given centerpoint (using a given radius/distance, of course) on the surface of the Earth. The crux of the code goes like so:

window.pointsAround = function(centerLat, centerLng, radius, numInterpolations) {
    var result = [];

    //do annoying trig maths to work out the delta in latitude between our start and end points
    var targetD = radius;          //km
    var fractionalDistance = (targetD / radiusOfEarth) / 2;
    var sqrtA = Math.sin(fractionalDistance);
    var a = sqrtA * sqrtA;
    var sinHalfDLat = Math.sqrt(a);
    var dLat = Math.asin(sinHalfDLat) * 2;
    var dLatDegrees = degrees(dLat);
    
    var minLat = centerLat - dLatDegrees;              //furthest valid latitude above the origin
    var maxLat = centerLat + dLatDegrees;            //furthest valid latitude below the origin
    
    var centerLatRadians = radians(centerLat); 
    
    //topmost and bottommost points in the circle
    result.push({lat: minLat, lng: centerLng});
    result.push({lat: maxLat, lng: centerLng});
    
    //step from minLat to maxLat, interpolating coordinates that lie upon the circle
    numInterpolations = numInterpolations ? numInterpolations : 360;
    var step = (maxLat - minLat) / (numInterpolations / 2.0);
    for (var count = 0; count < (numInterpolations / 2) - 1; count++) {
        minLat += step;
        var minLatRadians = radians(minLat);
        dLat = radians(centerLat - minLat);
        
        //more annoying trig to work out the delta in longitude for our interpolated coordinate
        var dLon = 2 * Math.asin(Math.sqrt((a - (Math.sin(dLat/2) * Math.sin(dLat/2))) / (Math.cos(minLatRadians) * Math.cos(centerLatRadians))));
        var dLonDegrees = degrees(dLon);
        
        var newLng = centerLng + dLonDegrees;
        var deltaLng = newLng - centerLng;
        result.push({lat: minLat, lng: newLng});
        result.push({lat: minLat, lng: centerLng - deltaLng});
    }
    
    return result;
};

This code works out the northernmost and southernmost points that are the specified distance away from the given centerpoint. It then steps between the minimum and maximum latitude of these two points, interpolating the appropriate longitudinal value(s) to satisfy the equidistant constraint. The total number of coordinates returned can be controlled using the optional ‘numInterpolations‘ parameter. When not specified a total of 360 points will be returned by default, because that is a nice, round number.

If you use something like the following code to plot some points on a map:

window.map = new GMap2(document.getElementById("mapDiv"));
    var centerPoint = new GLatLng(37.4419, -122.1419);
    map.setCenter(centerPoint, 8);
    map.addControl(new GLargeMapControl3D());
    map.addControl(new GMapTypeControl());
    
    var marker = new GMarker(centerPoint);
    var markerText = "Center @ 37.4419, -122.1419";
    GEvent.addListener(marker, "click", markerClickHandler(marker, markerText));
    map.addOverlay(marker);
    
    var points = pointsAround(37.4419, -122.1419, 50);
    for (var index = 0; index < points.length; index++) {
        var point = points[index];
        var latlng = new GLatLng(point.lat, point.lng);
        marker = new GMarker(latlng);
        markerText = "Marker @ " + point.lat + ", " + point.lng;
        GEvent.addListener(marker, "click", markerClickHandler(marker, markerText));
        map.addOverlay(marker);
    }

…you will get a nice circle of points around the hard-coded centerpoint. You can see a working example of this here: http://jsfiddle.net/HmchC/27/.

Posted in coding, javascript | Tagged , , , , | 1 Comment

[iOS] Jira Mobile Connect

Not long ago Atlassian released version 1.0 (now up to 1.0.7) of their Jira Mobile Connect plugin. This is a plugin for Jira (obviously) that aims to simplify testing, error-reporting, and feedback collection/management for iOS applications. Assuming that you are doing iOS software development and have a Jira server instance running (which you really should if you are doing any nontrivial amount of development work of any variety) then using this plugin in your apps is really a no-brainer. Jira Mobile Connect includes a number of very cool features, such as the ability for users to attach annotated screenshots/images to their feedback reports, to record audio to attach with their feedback, and even to chat back and forth with the developer(s) working on their issue/ticket . And of course it does basic crash logging and reporting, as well.

Previously if you wanted a free/open-source crash reporting framework for iOS your options were basically limited to QuincyKit, which is a serviceable but basic solution. Sadly, the backing architecture used by the QuincyKit server is not well designed and scales very poorly with the number of crash reports in the system. Once you have around 5,000 you’ll notice the server slowing down significantly, and go much beyond 10,000 that and the system grinds to an unusable halt. With a day or so of database and code refactoring, these scalability issues can be resolved, creating a system performant enough to track millions of error logs or more. But that’s a subject for another time. The point for today is that despite its basic level of functionality and flawed server architecture, there are a few key areas in which QuincyKit blows the default implementation of Jira Mobile Connect out of the water:

  1. Automatic grouping of crash reports.
  2. Automatic symbolication of error logs.

If you have any experience whatsoever with supporting multiple high-volume iOS applications you will instantly realize that these are features that you want. They might even be features that you want more than annotated screenshots, audio feedback, user chat, or seamless integration with Jira and all the awesomeness that Jira brings. In short, Jira Mobile Connect’s lack of support for these two key features may cause serious developers to pass it over in favor of other solutions.

Without grouping every single crash will be creating a new ticket in Jira that you need to track and resolve. Multiple instances of the same crash will have to be manually flagged as duplicates within Jira. And without symbolication trying to actually map back from an error log to the line of code that caused it is an exercise in futility, or at best, tedium.

In any case, rather than abandon the excellent potential shown by Jira Mobile Connect I decided that instead I would attempt to patch it up and add the missing features myself. It’s all open-source code, after-all, and if the tangled mess of PHP that is QuincyKit server can provide these features then they can’t be that difficult to implement. Unfortunately I had to change too many files in too many different places to show the code here, but if you want the short version of it I was able to implement both grouping and symbolication, and you’re welcome to view the complete diffs on bitbucket:

https://bitbucket.org/aroth_iapps/jiraconnect-ios-iapps/compare/..atlassian/jiraconnect-ios (client/native iOS code)
https://bitbucket.org/aroth_iapps/jiraconnect-jiraplugin-iapps/compare/..atlassian/jiraconnect-jiraplugin (server/Java code)

One interesting side-effect of adding groups was that it became possible for multiple client UID’s to be associated with a single Jira ticket. This had an important implication for feedback notifications/chat in that the reference implementation allowed only a single UID to be associated with each Jira ticket. Since the UID is used for determining what notifications/updates to send to the native client, this restricted update notifications to a single user per ticket. Not too useful if you have a common crash that thousands of users have experienced. The implementation above extends the data model to allow multiple UID’s to be stored against a single Jira ticket, allowing each UID to be updated when new feedback is posted by the developer on a ticket. In essence, implementing grouping also required the implementation of group feedback/chat.

There is one caveat with my server implementation, in that it assumes the existence of the ‘symbolicatecrash‘ utility on the system’s runtime PATH. This means that it will only work if your Jira server is hosted on a Mac, with the proper XCode developer tools installed (and with your application’s .app and .dSYM files copied to the local filesystem). This is of course a requirement regardless if you want automatic symbolication to work on any sort of system; somewhere there needs to be a Mac with ‘symbolicatecrash‘ available. In any case, it is a fairly simple matter to either turn this off or otherwise make it more intelligent, if your Jira server is incapable of running ‘symbolicatecrash‘.

Also note that the native iOS code has been restructured to build a universal iOS framework as opposed to an architecture-specific static library. This is done using Karl Stenerud’s excellent XCode4 Project Template. You will need to install this template in order to actually build the modified code. Or you can just refactor it back to build a static library again, but why would you want to do that?

When using the iOS framework, be aware that you will need to set the ‘-all_load‘ linker flag and also include all the images and nibs in the framework’s ‘/Resources‘ folder as part of your build. You will probably also want to include the ‘JMCLocalizable.strings‘ file in the same folder as well, to provide proper text and labels on Jira Mobile Connect’s UI elements.

Regardless, if you do a lot of iOS development and are already using Jira (like you should be) then I encourage you to check this out. This is the Jira Mobile Connect plugin, now with automatic grouping and error log symbolication.

Posted in banter, coding, software | Tagged , , , | Leave a comment

[JavaScript] Creating an Animated Hurricane Tracker

Here’s a fun little diversion I came up with when the news was all abuzz with information about Hurricane Irene. Now I use Wikipedia almost exclusively for keeping track of this sort of thing as I find that it consistently provides the most up to date data in the cleanest and most easy to digest format, but I also noticed a large volume of comments on other news sites complaining about the accuracy of hurricane forecasting. And this got me to wondering, just how accurate are the historical forecasts when overlaid with the actual storm track?

Well Wikipedia doesn’t provide this information directly, but interestingly enough Wikipedia does in fact keep all the historical data necessary to answer this question. Thanks to Chrome’s awesome JavaScript console (and the fact that Wikipedia uses jQuery on all of its pages) it is a very simple matter to extract this historical information in a format that is easy to work with later. The following code will do the trick:

var result = []; 
jQuery.each($(".filehistory a[href$=5day.gif]"), function(){
    result.push(this.href)
}); 
result;

This simply constructs an array containing the URL’s of every historical forecast/storm track graphic (ordered by date from newest to oldest) and outputs it to the console. We can then copy/paste the console output to wherever we like, such as jsFiddle for instance. Combine this with some basic HTML structure:

<div class="title">
    Hurricane Irene Animated Forecast Map
</div>
<img id="display" src="http://fcf.teambeachbody.com/fcf/images/spinner5.gif" />
<div id="status">
    (<span id="numLoaded"></span> of <span id="total"></span> frames loaded)
</div>

…and some simple JavaScript to handle the actual rendering:

document.getElementById("numLoaded").innerHTML = "0";
document.getElementById("total").innerHTML = data.length;

//preload all the images so that they are cached for the animation
window.imagesLoaded = 0;
for (var index = data.length - 1; index >= 0; index--) {
    var node = document.createElement("img");
    node.style.display = "none";
    document.body.appendChild(node);
    $(node).bind('load', function() {  
        window.imagesLoaded++;  
    });
    $(node).attr('src', data[index]);
    //node.src = data[index];
}

//run the animation
window.currentImage = data.length - 1;
window.updateAnimation = function() {
    if (window.imagesLoaded < data.length) {
        //keep waiting
        document.getElementById("numLoaded").innerHTML = window.imagesLoaded;
        return;
    }
    $("#status").hide();
    document.getElementById("display").src = data[currentImage];
    currentImage--;
    if (currentImage < 0) {
        currentImage = data.length - 1;
    }
};

setInterval(updateAnimation, 200);

…and you get an animated map of the entire forecast history, including the actual storm track. The result looks something like this:

Not too bad for a few dozen lines of code and about 30 minutes worth of effort. You can also access a full-scale, editable version of this animated storm history here.

So what does this say about the accuracy of the hurricane forecasting for Irene? Personally I think it shows that the forecasts are pretty damn accurate. In fact, if you focus just on the solid white portion of the predicted storm track it appears that the storm never deviates from its forecast path. It’s only when looking at the dashed white portion of the forecast that any significant deviation can be noted. And the dashed white portion indicates forecasts that are more than 72 hours in the future.

Of course we should always be working to improve our ability to predict and forecast these things, but I have to think that being able to accurately predict a storm’s movement for up to 72 hours into the future is adequate for most purposes.

Posted in coding, javascript | Tagged , , | 1 Comment

Natural Scrolling Isn’t

Mac users and other developers such as myself who have no choice but to use a Mac for certain projects will already know what I’m talking about. For everyone else, however, the newest version of OS X includes a most bizarre feature, called “natural scrolling”. Despite its innocuous-sounding name, this feature (which ships enabled by default) breaks the interface to the OS by inverting the direction content scrolls when using the mouse-wheel.

Apple’s explanation (and seemingly, justification) for this atrocity of a feature is that “content should move in the same directions as your finger”. Now that’s all well and good, except for one thing; there is no “finger” when using a mouse. Touch- and mouse-driven are distinct input paradigms.

Apple is correct in that in a touch-driven system content should indeed track in the same direction as the user’s finger moves. If a hypothetical touch-based interface were ever released which scrolled content in the opposite direction of the user’s finger, then people would say that it was broken. And rightly so. Moving content in the same direction as the touch is the intuitive operating mode of a touch interface.

Similarly, moving content in the opposite direction of the scroll (or more accurately, moving the scrollbar in the direction of the scroll) is the intuitive operating mode for a mouse-driven interface. And it follows that Apple’s mouse-driven interface in Lion is just as broken as that hypothetical touch-driven interface would be. By Apple’s logic scrollbars themselves should also be inverted, tracking from the bottom of the scrollbar to the top as the content scrolls from top to bottom. Confused yet? Apple’s blunder here is that they have conflated mouse-driven input with touch-driven input, without bothering to translate properly between the two.

As an interesting aside, a comparable analog to touch-style scrolling does exist in the mouse-driven paradigm; the drag operation. You’ve probably seen it in things like Adobe PDF documents, and it can also be observed on any scrollbar. In the drag operation you choose an anchor-point, and then that anchor point moves in the same direction that you move, and it all intuitively makes sense. The problem with scrolling using a scroll-wheel is that it has no anchor point from the user’s point of view (they have done nothing to nominate one, and the UI does nothing to indicate that one has been chosen). In the mouse-driven paradigm, scrolling is a distinct operation from dragging, and by conflating the two Apple has broken their interface. At least until they start incorporating a touch-screen into every computer they sell.

If Apple’s position is to be taken seriously, then we must accept that the current cursor position must always represent where the user’s “finger” is. There’s one major problem with that, however. With a mouse and cursor, the cursor never leaves the screen. And if the user’s finger is in constant contact with the screen, then why should there be a separate scroll operation at all? It would logically follow (since mouse-driven == touch-driven and cursor == finger in Apple’s world) that since a touch-point exists, and since it is moving, the content (all of it, everywhere) should simply scroll whenever I move the mouse (all the time, everywhere). But of course that is nonsense, and so is Apple’s “natural scrolling” mouse behavior. The mouse cursor is not a finger, it does not represent a finger, and its location is not where your finger is touching. Mouse-driven and touch-driven are different modes of operation, with different intuitive behaviors.

Thankfully, you can turn off “natural” scrolling and rid your OS of its nonsense. I’ll be happy to turn it back on when my mouse is replaced with a touch-screen display, but until that day comes you can forget about it. Apple, please stop trying to convince me that moving my mouse cursor around the screen is supposed to be the same thing as dragging my finger all over a touch screen. It isn’t, and I’m not stupid enough to be convinced that it is.

Posted in banter, operating systems | Tagged , , | 3 Comments

[JavaScript] Naming Your Function ‘$’ is not Clever

The atrocity that is the dollar function dates back at least 2005 and the introduction of the Prototype Framework. Back in those days, ‘$()‘ was simply a shorthand way of saying ‘document.getElementById()‘ without having to type all 24 characters. And don’t get me wrong, aliasing ‘document.getElementById’ to something shorter is not a bad idea, in and of itself. But why should ‘$()‘ be the alias? Such a name gives no clue that the function is looking up a DOM element by its id. Something like ‘findById()‘ (or even ‘$findById()‘ if there are concerns about naming collissions) would have been infinitely more clear, and still a good deal shorter than writing ‘document.getElementById()‘.

But sadly, Prototype decided to use ‘$()‘ as the name of their alias. And even worse, other frameworks have followed in this pattern, adding their own dollar functions that have semantics that can be completely different from the original ‘$()‘ introduced by Prototype. Consider jQuery, which introduces its own version of ‘$()‘ which can be used to find (and wrap and clone) elements in the DOM by a number of different criteria, but which sadly fails miserably when given a string containing a literal element id. This creates a lovely situation where now the developer cannot even guess at what ‘$()‘ is actually doing without first checking to see which framework is being used by the code. It also means that any code that was written using Prototype’s ‘$()‘ will be broken by including jQuery (and vice-versa).

“But that’s what ‘jQuery.noConflict()‘ is for”, you say. But you miss the point. It shouldn’t be necessary for ‘jQuery.noConflict()‘ to exist in the first place. If the jQuery developers hadn’t decided that it would be cute to define their own version of ‘$()‘, then jQuery would never conflict with anything except possibly an older version of jQuery. The thing that really baffles me is how so many developers (obviously talented developers, at that, considering the work that goes into something like Prototype or jQuery) can all decide that it is a good idea to try to lay claim to a name that exists in the global scope. Granted JavaScript does not have any concept of a proper namespace, but if jQuery limited itself to only setting properties on an object named “jQuery” and Prototype did the same on an object named “Prototype” and so on, the current JavaScript landscape would look ever so much more sane.

And of course, this says nothing of the ‘$$()‘ and ‘$F()‘ functions that were also introduced by Prototype (and the ‘$E()’ and ‘$ES()’ functions inexplicibly introduced by MooTools) and whose names are so semantically worthless that a developer has no hope of inferring their function without consulting the reference documentation. Which again can only be accomplished after checking the entire codebase to determine which framework it is that is defining these functions with its woefully awful naming convention.

If the developers of Prototype, jQuery, Mootools, and the like wanted to turn JavaScript into an unreadable and unmaintainable mess of a language like Perl, then I applaud them for their efforts. But if not then I call them all out on their atrocious naming convention and design decisions. There is no excuse for this nonsense. If you’re building a JavaScript library or framework, then pick a reasonable “namespace” to use for it. This namespace should be the only thing that you bind in the global scope. Everything else that is part of your library should be bound as a property inside of this namespace. And there should be no automatic creation of shorthand aliases in the global scope.

I don’t think this is too much to ask. And of course, any developer that wants to use a global shorthand alias can always do:

window.$J = jQuery;
window.$P = Prototype.$;  //or:  window.$P = document.getElementById;

It takes all of one line of code to set up an alias. But when you start doing this automatically for everyone that uses your framework you create nothing but problems. Especially when you jump on the bandwagon and choose the same crappy shorthand alias names that other frameworks are already using.

There is absolutely no reason why multiple JavaScript frameworks should all be competing over the same terrible function names. This isn’t a contest to see who can break the most legacy code, or who can come up with the most obtuse API. They do have such contests if you are interested in such a thing, but this is not one of them. The only thing accomplished by parading the dollar sign around as an example of a well-chosen function name is an increased brittleness in the code that glues together the majority of our modern websites. It muddies semantics, and creates cases where completely unrelated libraries are not easily compatible with each other, and where simply importing a new library can cause existing code to break. Even worse, some novice developers get the wrong idea from all those dollar signs flying around think that they need to (or should) prefix all of their function names with “$”. So let’s put this bad practice to bed before it’s too late.

So please, can well all stop writing frameworks that give people the impression that calling a function “$”, “$$”, “$ES”, or “$anything” is a good idea? Is it so difficult to choose a reasonable namespace, minimize the number of things that you declare in the global scope, and allow the developer to define a global shorthand alias for this-or-that utility function if they decide they want to? I think not.

Posted in coding, javascript | Tagged , | Leave a comment

Resurrecting sun.misc.Unsafe

Here’s one that only the hardcore Java hackers will enjoy. Perhaps you are already familiar with sun.misc.Unsafe. This heavily protected internal class provides access to a number of low-level memory operations and system functions that are generally hidden away from user-code executing in the JRE. If you’ve never heard of it before (or even if you have) then I encourage you to read this article to get an idea about the sort of things that can be done using ‘Unsafe‘. Note that we are talking about very low-level operations here. Operations where if you make a mistake, you won’t just get a simple ‘Exception‘ complaining that you did something bad. Instead you’re liable to bring the entire JVM to a crashing halt if you do something wrong (it’s called ‘Unsafe‘ for a reason, after all), so you should proceed only if you’re sure you know what you are doing.

Also note that the method described in the other article for getting your hands on an instance of ‘Unsafe‘ no longer works. Using the latest JDK, the following code will not work (you’ll get compiler errors for even trying to import sun.misc.Unsafe):

private static Unsafe getUnsafeInstance() throws SecurityException, NoSuchFieldException, IllegalArgumentException,
    IllegalAccessException {
    Field theUnsafeInstance = Unsafe.class.getDeclaredField("theUnsafe");
    theUnsafeInstance.setAccessible(true);
    return (Unsafe) theUnsafeInstance.get(Unsafe.class);
 }

Here is a variant that still works:

	public static Object getUnsafe() {
		try {
			Class unsafeClass = Class.forName("sun.misc.Unsafe");
			Field unsafeField = unsafeClass.getDeclaredField("theUnsafe");
			unsafeField.setAccessible(true);
			Object unsafeObj = unsafeField.get(unsafeClass);
			
			return unsafeObj;
		}
		catch (Exception e) {
			return null;//UNSAFE_ERROR_CODE;
		}
	}

Note that as importing sun.misc.Unsafe now causes a compiler error, it is only possible to return an Object reference to the ‘Unsafe‘ instance. This means that any method that you want to call on it must be done through reflection, which is tedious at best. Luckily, I’ve taken care of this tedious bit for you, and created a wrapper that provides the same public API as sun.misc.Unsafe and uses reflection to delegate method calls to the real ‘Unsafe‘ instance. The source code for this utility is quite long and not very interesting, so I’m not going to post it here. Instead you can use this direct link to download a copy.

Here is a simple example of how to use this class to perform some basic tasks:

		System.out.println("Testing sun.misc.Unsafe...");
		double[] averages = {-777.0, -777.0, -777.0};
		UnsafeUtil util = new UnsafeUtil();
		System.out.println("addressSize=" + util.addressSize() + ", pageSize=" + util.pageSize());
		
		long memPointer = util.allocateMemory(1024);
		long memPointer2 = util.allocateMemory(1024);
		System.out.println("1K memory blocks at addresses:  " + memPointer + ", " + memPointer2);
		
		//valid copy
		util.copyMemory(memPointer, memPointer2, 1024);
		
		//invalid copy
		//util.copyMemory(memPointer, memPointer2, 1024000);
		
		int result = util.getLoadAverage(averages, 1);
		System.out.println("getLoadAverage:  Result=" + result + ", averages=" + averages[0] + ", " + averages[1] + ", " + averages[2]);

Do note that if you uncomment the “invalid copy” line then running this code will very likely crash your JVM. As noted above, you should use this utility with caution, particularly if you are running it in an environment shared by other Java code.

And why do all this when Sun (now Oracle) clearly does not want people mucking around with this class? Because ultimately power belongs in the hands of developers. All developers, not just the select few chosen to work on privileged system code.

Posted in coding, java | Tagged , | 2 Comments