Apr 03 2008

Your Software Is Only as Good as the Testing You Do

Published by Justin under Project Management

In the process of designing and managing the development of several web-based applications, your correspondent has come to learn a critical lesson in software development (that many others in the software industry likely already know):

Your software is only as good as the testing you perform.

Your correspondent worked on the team that recently released an online chat tool called University WebChat. Powered by jQuery, University WebChat enables college and university admissions recruiters to easily host web chats for talking with prospective students.

The Bug

The application was working great during beta testing, except when more than 20 chatters joined a room. Once we hit north of 20 chatters, odd things started to happen: chatters got booted from the room, some couldn’t join the room at all, others just got an endless connection waiting when trying to load the room URL.

But none of these issues happened consistently, only consistently intermittently. We couldn’t regularly reproduce the issue, so it was impossible to fix. We focused on other bugs.

A few weeks went by and we still had no luck in finding the cause of the issue. Then we decided to really test out the application by hosting a high-profile chat where we expected more than 75 chatters.

What a difference a little pressure makes. We were forced to take a different approach to debugging the issue. So we ran a more complex test on the bug, and dug a bit into how Apache works (it’s a LAMP application). Eventually we found the source of the bug, fixed the issue and felt confident that the application would support 80 chatters in one room.

Test, Debug, Refine and Repeat

If we hadn’t been forced to test the application harder then the true source of the bug would never have been identified, the software wouldn’t scale as well as we needed it too and (worst of all) customers would have had problems with our software.

Your software won’t get better on its own - you have to force it to become better with great developers, useful features and most of all hard testing. Over and over again. From different angles and vectors. From different computers, networks, browsers, operating systems, locations, monitors and processors.

This isn’t a new revelation, granted. But an important one.

Missing features axiom #7: Your software is only as good as the testing you perform.

One response so far

Feb 24 2008

Are You An Efficient Programmer?

Published by Justin under Software

Most programmers are really bad at programming. Grab 10 random students from a computer science masters program and your correspondent will happily wager a case of Brooklyn Brewery Chocolate Stout that at least nine of those programmers will fail a basic programming task.

Guns Don’t Kill People, Bad Code Kills People

Why should anyone care if most programmers are bad? Well, what if the same were true of surgeons? Or commercial airline pilots? Sure, human life doesn’t depend directly on the daily work of a programmer. But the software they create sure does.

Think about all of the bad code in the control systems of nuclear power plants or in the computers under the hood of your car as you speed down the highway at 115 mph.

Are You Experienced?

Experience has little to do with programmer productivity or even in determining if you are a good programmer. But good programmers do have one trait in common: good programmers work very efficiently.

Why? Well, it mostly boils down to two things:

  1. They are really smart;
  2. They are really good at figuring things out.

When a smart person who is good at figuring things out asks him or herself how can I do this faster? productivity gains are realized.

Zen and the Art of Being a Productive Programmer

Whenever your correspondent teaches a web technology class — be it a course on project management or XML-based web services — there is always a class spent talking about how to work efficiently.

Beyond tips for basic productivity gains, here are some specific techniques for web programmers:

  1. Use ALT-TAB to move between your code and your browser;
  2. Use keyboard shortcuts;
  3. Use a good IDE;
  4. Isolate and toggle when debugging;
  5. Use small successful iterations; and,
  6. Use a good query editor.

ALT-TAB is straight forward enough. Don’t waste time moving your hand from the mouse to the keyboard. Sure, it only saves a second. But you are making this motion hundreds — perhaps thousands — of times per day. It adds up.

The same goes for keyboard shortcuts. Learn how to outdent. Learn how to highlight an entire line of code. Learn how to highlight all of the code from your cursor to the end of a file. Learn how to save the file. Again, it adds up.

My IDE Can Beat Up Your IDE!

It’s quite simple to know if you are using an IDE that will make you more efficient:

  1. You like to program in it;
  2. It’s not VI;
  3. It tells you the parameters (and data types!) for built-in methods;
  4. It tells you the parameters (and data types!) for the custom methods in your application; and,
  5. It colors your code.

Isolate and Toggle

The concept of isolate and toggle is a helpful way to speed up bug fixing (don’t worry, you will still spend more time fixing bugs than actual programming).

When you have a bug in your code, it’s caused by one or more outliers. Your job as a debugger is to eliminate each outlier until you find the root cause. You can only do this by eliminating each possiblity one at time. If you don’t, you might miss something.

Identify all the possible causes of the bug, isolate each one individually, and test (read: toggle your code) to eliminate each possible cause. One at time: isolate, toggle and test. Eventually, you will find the cause of the bug and you will know how to fix the issue.

Sure, sometimes it takes a little longer to fix a bug this way. But most of the time it takes a lot longer if you don’t.

One Step at a Time

Understanding and using the technique of small successful iterations is probably the single easiest way to be a more productive programmer. The idea is simple: when you code, code a little bit at a time then save (use a keyboard shortcut!), switch to your browser (use a keyboard shortcut!) and refresh (use a keyboard shortcut!).

Not even Carmack writes perfect code every time. You will make stupid mistakes frequently. So if you code just a few lines and test out those lines right away, you will know immediately where you need to look for the bug.

Query Editors

Using a good query editor is also a big help to productivity. MySQL has nothing compared the productivity gains from using Microsoft SQL Server’s outstanding SQL Management Studio. It is simply the single best platform for SQL development on the market, and a big competitive advantage for the Microsoft SQL platform.

The reason having a great query editor to use during development is a big productivity boost is that the alternative — writing the SQL in your application code — introduces a layer of abstraction complication between the database and your query. And this layer of complication isn’t designed to make debugging your SQL query any easier.

You can’t comment chunks out, run two similar queries at the same time and compare the results or see the results in the same window as the actual query. Worse still, it typically takes three commands (save, toggle and refresh) to see the results of a change to a query in your application code while you can get the same result in one command (execute) in a query editor tool.

Get Your Own Money Bin

The secret to becoming more productive at anything is to remember the lesson we learn from Scrooge McDuck: work smarter, not harder.

No responses yet

Jan 30 2008

Where’s the CC?

FogBugz — an outstanding bug tracking tool — has a really neat feature called remind. Essentially, a user can select one or more open cases and have the system send a reminder email to a user.

FogBugz includes links to the tickets you select and formats a draft email body accordingly. Most times you can select a dozen cases and fire off a remind email in under four seconds with just two clicks. Very cool.

This is a great feature for any project manager, as part of the job is frequently kicking the developer about due dates. Your correspondent frequently uses this tool when reviewing a list of past due tickets among his development team.

Here is the remind compose screen:

Remind compose email screen in FogBugz

However, this handy tool does have a really simple, but needed, missing feature: a CC field.

Perhaps the CC feature was overlooked because most people don’t really know what the purpose of a CC field does in an email. Can’t you just use the TO field?

Well, no, not really.

A CC on an email is one of two things:

  1. A way to keep a team member informed; or,
  2. A handy way to add weight to an email.

While a BCC is a soft notification (in that you let the boss know something but not the recipient that the boss knows the something), a CC is a hard notification in that it adds authoritative weight to any email sent. A BCC is used to let a valued employee save face while still keeping the boss in the loop. A CC is used to reinforce the email with the superior authority of the person CC’ed.

When used correctly, a CC can say Hey! This is important. As you can see, I’m letting the boss know I told you this.

Since the remind feature in FogBugz is really just a quick way to send a you dropped the ball email, including a CC feature can only add to the effectiveness of any emails sent with it.

No responses yet

Jan 25 2008

What Hospitality Teaches Us About the Software Business

Seth Godin has a great post today where he talks about how the last impression is far more important than the first impression:

“I recently had some waterproofing done in the basement. The first date was great. The company was professional and had every single element down, from their AdWords to the web site to the way they interacted on the phone and in person.

I think that stuff is pretty important, but I’m way more interested in the last interaction than I am in the first (and if you care about word of mouth, you should be too).

After they finished the job, they left my basement a mess.

Forever, my only memory of the job is going to be the mess.”

Essentially Seth is arguing that his memory isn’t set by the first interaction, but how the closest memory he has — the last interaction — matters most.

This isn’t a new idea. In fact, Seth is touching on a lesson so important to any business that New York restaurateur Danny Meyer was able to build a fine dining empire on top of his complete understanding of this lesson.

Setting the Table, by Danny MeyerWhat lesson? In this outstanding business-auto-biography of sorts — Setting the Table — Danny talks about the importance of writing your own last chapter.

In a chapter aptly titled “The Road to Success is Paved with Mistakes Well Handled”, Danny tells the story of greeting a Senator one evening. The Senator mentioned to Danny that the night before, at another one of Danny’s restaurants, his salad was served with a live beetle in it.

A free meal or a bottle of wine might be nice way to say sorry, but it won’t change this Senator from telling everyone he knows about the beetle incident. In his own words, Danny instructs the Chef to write a great last chapter:

“Whether or not Senator Kerrey or his guest order a salad during his lunch, I want you to deliver a beautiful salad and garnish it with a small piece of paper. On that piece of paper I want you to write the word RINGO, and when you deliver it, you can tell them, ‘Danny wanted to make sure you knew that Gramercy Tavern wasn’t the only one of his restaurants that’s willing to garnish your salad with a beatle.’”

The lesson here is clear: Everyone screws up, every company makes mistakes. And when a company screws up, a customer has a story worth telling to someone, somewhere, over a beer, about the horror that happened.

But if you take control of that mistake, correct and end it better than expected, you’ve turned a negative story that would have scared patrons away into a positive story that might actual help the bottom line.

Mistakes happen but only you decide how the mistake will end in the customers eyes.

No responses yet

Jan 24 2008

Don’t Waste Time: Write an Agenda

Published by Justin under Project Management

Office SpaceToo many meetings are a wasted effort mostly because discussions end up being circular or branching. Either way, the discussion never helps the meeting reach any goals, and thus everyone’s time is wasted.

Why bother running a good meeting? Because the old adage is true: time is money. Or maybe in the software world the adage is better written as time is bugs.

Try this little exercise to see just how expensive meetings are:

  • At the next meeting you attend make a mental note of how many people are involved
  • If you work at a consulting firm, calculate the hourly rate of each person and calculate the cost
  • If you work at a traditional software development firm, assign two bugs per programmer and one testing document, one module technical specification and three emails per project manager or architect at the meeting

How much did your meeting cost? Was it worth what was achieved?
A 2-hour meeting with three .NET developers, a database developer, a project manager and a database architect just cost the firm 8 bug fixes, two testing documents, two technical specifications and six emails to clients.

If the team bills at $200/hour, that meeting just cost $2400. Ouch.

Meetings are expensive and most people hate them. But they shouldn’t. They hate them probably because they aren’t run well. But it isn’t hard to run a great meting, it just takes a little work… and an agenda.

Why do I need an agenda?

The whole point of a meeting is simple: to make a decision that involves more than one person. This decision might be a feature list, a schedule, an upgrade plan or a technical outline to solve a problem. Whatever you might need from the meeting, it’s still a decision.

Where does the agenda come in?

Well, in order for a meeting to come to a decision, you need to have a clear goal. Why? Because a goal makes it clear to all involved what needs to be determined by the end of the meeting, makes it easy to evaluate the success of the meeting and most importantly leads to a decision.

So where does the agenda do?

It makes the goal clear (by stating it succinctly in the agenda) and it sets a framework for writing the discussion topics so that they help attain the ultimate goal of the meeting: the decision.

The agenda clothing rule

There is no set format for an agenda and no hard and fast template that you can apply to every kind of meeting. An agenda can be a simple three item list sent to the team in an email, or a full and formal two-page agenda as a PDF attachment in an email sent to a client.

The trick to selecting an agenda format is agenda clothing rule: the format of the agenda should match the clothes of the meeting attendees.

If you are meeting with paying clients and everyone is wearing pressed pants and ties, you need a nicely formatted, formal agenda. If you are meeting with the development team wearing flip-flops and wrinkled t-shirts with trite, trendy statements, a simple email agenda will probably do just fine.

The short agenda – for the informal meeting — is usually written as part of a meeting reminder email and contains a one line goal for the meeting and a short list of 2 - 5 discussion items. Short, sweet, targeted and informal.

The long agenda — for the more formal meeting — is usually a full page PDF that contains a few parts:

  1. Document title;
  2. Meeting location;
  3. Meeting date and time;
  4. Meeting goal(s); and,
  5. Topics/discussion items.

If the long agenda meeting is meant for more than an hour you probably want to include times for each discussion topic. This helps you keep topic discussions targeted and keep the meeting moving.

If you had listed 30 minutes for the current topic and time is clearly up, it becomes really easy to say “I want to respect everyone’s time, so I think we really need to move to the next topic” without offending anyone.

A long agenda should probably end with a “Next steps” topic to allow the person running the meeting to wrap-up and outline what happens now.

Topics, topics, topics

The core of any agenda are the discussion topics you outline. These should be easy to write if you have identified a goal for the meeting.

Here are a few points to keep in mind:

  1. Items should be very short, less than 7 words usually (the thought process that goes into watering down a complex issue to just a few words tends to make clear the core issue that should be discussed);
  2. Be as specific as possible in each topic (the more vague the topic the more vague and unhelpful the discussion will be); and,
  3. Ensure that each topic helps achieve whatever goal you have outlined for the meeting.

Putting so much thought into an agenda might seem like overkill. But remember, a meeting is a lot like what you eat: what you get out of a meeting can only be as good as what you put in.

No responses yet

Jan 22 2008

They Just Don’t Get It

Published by Justin under Software Industry

ScrabulousScrabulous is an online version clear rip-off of Scrabble, the board game jointly owned by Hasbro and Mattel. It’s only available at Facebook.com and is built on the Facebook application framework.

Unsurprisingly, Hasbro and Mattel aren’t pleased and are trying to shut down the application. As TechCrunch reports today, Scrabulous must either shut down tonight or sell itself to Hasbro at a price that is sure to reflect the mood in the negotiation room (with Hasbro firmly pressing a gun against Scrabulous’ head).

This is probably the worst move Hasbro could make to protect its copyright and to serve its customers. The success of this game on Facebook isn’t due to the game of Scrabble, but due to the designers who built the application and made it popular.

Forgetting the fact that Scrabulous’ popularity will help retail sales of the boxed version of the game Scrabble, Hasbro is missing an even bigger opportunity: working with the creators to bring more Hasbro and gaming magic to Facebook.

They are missing out on a few sales now, and loads of sales in a few years.

Young consumers — who happen to use Facebook in large numbers — will be in the position to buy the kind of consumer products that Hasbro happens to sell in a short amount of time. What a better opportunity does Hasbro have to get them hooked on a game they will later bring to the family room when these young users grow up and marry and have little babies (as most young people tend to do, in droves).

So what is the right move?

Hasbro has huge leverage against the creators of Scrabulous in the form of threatened litigation. If they use this leverage to steal Scrabulous from the developers, they aren’t really going to want to work for Hasbro and create other great online games.

Scrabble isn’t a valuable asset to Hasbro- people like the guy who created Scrabble are. Lawsuit filed, lawsuit won, talent lost.

Instead, Hasbro ought to use their leverage to “invest” a 50% stake, let them keep Scrabulous online, and start work on bring other new exciting Hasbro initiatives to the Facebook world (with the help of the Hasbro marketing machine).

This is the only option Hasbro can commit to if they truly wish to increase shareholder value (as all publicly traded companies are obligated to do).

Unsurprisingly, when revolutions happen entrenched organizations just don’t get it.

No responses yet

Jan 16 2008

What a 10-Minute Rice Bag Teaches Us About Usability

Published by Justin under Consumer products

The Flying Spaghetti Monster, courtesy venganza.orgJust as evolution is the great unifying theory of humanity (in that evolution is the core catalyst for life, behavior, war, sex, innovation, etc), so to is usability the great unifying theory of consumer product design.

Since all consumer products have to be used by your customers, understanding how to make your product more useful is an absolute must.

Still, it’s rather troubling how many usability mistakes we see in consumer products.

Still, why bother usability testing consumer goods? The 10-minute Success rice bag is a great example of why it’s not only helpful, but absolutely necessary.

Take a look at this bag your correspondent recently cooked:

A bag of Success 10-minute rice, opened

Look closely at how the bag tears: the built in tear-point — positioned by Success — causes the bag to split the cooking instructions in two pieces, splitting the text.

A simple test of sending this rice to a few real homes to be cooked and served to real people would have quickly shown this usability flaw, and saved Success from this embarrassing mistake.

A few tips on consumer home products usability testing:

  • It’s vital to test in a real setting, e.g. in the home of a person cooking
  • Don’t interact or help the user in any way, just watch (or better yet, video tape it so you don’t even need to be there interrupting the kitchen setting)
  • Interview the tester after the cooking, but don’t ask leading questions
  • Run about 3 sets of tests, with each test set including about 5 homes

No responses yet

Jan 15 2008

One Easy Step to Lose Your Customers

Published by Justin under Axioms

Far Side - push, don’t pullThe need for usability testing in consumer products cannot be overstated. But even if it doesn’t have an electrical plug, it still probably needs usability testing.

In fact, here is a really easy (and free!) test for all of the product managers out there to determine if your paycheck product needs a usability test:

Is your product used by humans?

If you answered yes, then you need to be running usability tests. If you answered no, then you aren’t selling a product, so you aren’t really a company.

Is it that easy to tell?

Let’s look at a great example of a consumer product usability mistake that could have been easily caught with some user testing. Take an item from a recent Crate and Barrel catalog, the bamboo tea holder:

Bamboo Tea Holder, from crateandbarrel.com

Looks perfectly functional, except it isn’t.

When you put a common brand of tea in this holder, it doesn’t close:

Tea holder closed, full of tea

Crate and Barrel must spend millions of dollars on marketing and product development, and yet a simple usability test costing about $100 wasn’t performed. Instead, Crate and Barrel chose to sell an inferior product to consumers who might start to question the quality of the entire product line.

If you sell products to consumers then you need your customers. Spending $100 is lot a cheaper than losing even one customer.

Missing features axiom #6: Test it or forget it.

No responses yet

Dec 22 2007

Why Ad Blindness Is an Unforgivable Usability Sin

Published by Justin under Abuseability, Web Design

There are lots of easy ways to make subtle usability errors in web sites. But there are only a few unforgivable usability sins. And ad blindness is one of them.

Take a look at the new design launched last week for the USC Credit Union:

USC Credit Union Ad Blindness

Can you spot the unforgivable usability sin? Need a hint? Try to find the link to the online banking site.

It’s in the top right of the screen, and looks like this:

USC Credit Union Ad Blindness logo

Why do most users miss this link? The problem is ad blindness.

Ad blindness is when visual cues alter the perception of a GUI element and cause users to interpret that element as an ad, not a valid link. Since users ignore ads in all contexts (except search results), the link is often ignored.

Ironically, it was likely the desire of the design team behind the redesign to make this link as easy to find and clear as possible that led to making the link hard to find.

In the case of credit union, there are two major visual mistakes that lead to this particular ad blindness problem:

  1. The placement of the link in the top right corner of the screen (where ad banners go); and,
  2. The boxed graphic style applied to the link (styling it like a banner ad).

Punish the sin, not the sinner

Here are some guidelines to help prevent ad blindness from affecting your designs:

  1. Place critical, deep links in the focal point of the site’s main content body or in the navigation menu;
  2. Don’t make critical links images, make them text;
  3. Perform a few paper prototypes on early design mock ups to identify potential ad blindness issues; and,
  4. Style the highlighted link similar or identical to the standard link style used on the site.

This topic offers up missing features axiom number 6: don’t over-design your critical links.

No responses yet

Dec 19 2007

Everything I Learned, I Learned from Scrooge McDuck

Published by Justin under Axioms, Software

Scrooge McDuckYour correspondent learned his most important business life lesson from Scrooge McDuck at a very young age.

In a scene with a young Scrooge just starting out, our hero struggles to shine enough shoes to make a living. No matter how hard he tries, he just can’t seem to shine enough shoes. Then he gets some advice from a customer that changes his life:

Work smarter, not harder.

Inspired by this advice, Scrooge proceeds to build a machine that allows him to shine several shoes at the same time: he exerts the same effort (it’s bike powered) in the same amount of time, but he reaps 4-5x the revenue per session.

He was well on his way to filling that amazing Money Bin.

This is an important lesson for software design, specifically in the realm of user experience. You can spend hours as a developer creating a really cool, complex feature that you think just rocks. But if the feature doesn’t help the user be more productive, then the user isn’t going to care.

You have just wasted days of developer time that you could have used to fix bugs or do something helpful like build an AJAX-powered foot massager for college linebackers.

What they need

As a project manager at a consulting firm, your correspondent is often torn when developing software for clients between doing what the client asks and what we think the client needs. If we help them develop a better work flow, make it easy to work on that work flow and actually make the user more productive, then the client can actually realize cost savings from the money spent on the software.

You don’t make the user more productive by just working hard, you have to work smart and design smart software the understands the needs of the user, and serves those needs.

You can program the easy, or the beautiful. But if you don’t build what end-users actually need, you just won’t create create great software.

Missing features axiom #5 (with thanks to Scrooge): Work smarter, not harder.

No responses yet

Next »