A Lesson from Wall Street

by Luke 5. August 2010 10:37

I recently had the chance to spend some time talking with the founder of a startup in the financial industry. He was looking for better ways to express the decisions he wanted to automate for a hedge fund management tool. And it struck me that the finance folks must know something that the engineers haven't caught on to yet.

Look at the Rules Fest speaker lineup and see all the finance folks. What do they know the we engineers don't? The cynic might say that they know the rules to skimming profit off a transaction. That seems to be the general feeling about high-frequency trading algorithms these days. But from my experience it's more than high-frequency trading - it's about automating all the little decisions.

The simple fact is that there is money to be made and in the financial industry it's easy to keep score. But when you look deeper, it's clear that beyond simple opportunism, the financial industry has learned how to automate the simple tasks. A large trade could be made any number of different ways. But rather than have people explore the myriad of options, computers do the heavy lifting. Doing it that way makes them more money. It's that simple.

That's a lesson we engineers need to absorb. The simple fact is that we need to get more efficient and we need to keep score if we're going to stay competitive. We've gotten good as a profession at using computers to automate calculations, and the next logical step is to use them to automate the simple decisions. That will make you a more efficient engineer, and efficiency is a great way to get more business.

Nobody Gets Paid to do Addition

by Luke 3. June 2010 03:02

There was a time when "computers" meant a small army of secretaries doing tedious math problems in the service of war or business. Today, nobody gets paid to do addition. Or multiplication. We have machines to do that.

For that matter, we don't use lookup tables for trigonometry or logarithm calculations. Machines do those, too. We don't manually calculate the standard deviation for a data set. We click a button in a spreadsheet to do that. We don't even have to type in the data since most of the time a machine collected it. The machine calculates integrals. It solves partial differential equations numerically. For almost any type of math an engineer could want to do, we're running out of problems that a computer can't solve.

Simple problem solving is the latest class of work to be automated. My GPS can route me around traffic jams. Netflix and Amazon can suggest movies and books you might like. Google made a fortune deciding what adds to show with your search. All of that problem solving was more than number crunching.

Today we use computers to manage supply chains, balance stock portfolios, and decrypt terrorist communications. We are so far beyond automating simple addition that it is amazing no one seems amazed. And this is a trend that is still accelerating. Be ready because the engineers that can harness the power of decisions in software will be the only ones able to manage and build on the complexity that's growing around us.

Making Computers Make Decisions

by Luke 14. December 2009 12:11

The most valuable work that people do is make judgments. We decide what we want. We decide how to make it. We decide when we're happy with the result.

The problem we have today is complexity. The things that we want (from fancy cell phones to space satellites) require so many decisions that it takes huge teams of people to make sure they all get made. But many of those decisions aren't very interesting and, while necessary, don't add much value.

Any time I see a situation like that, my first thought is to make a computer make the decision. There are a number of approaches you can take. Decision tables and decision trees are straightforward tools that most engineers with some software skills can implement. Moving up the difficulty scale is implementing a rule-based system for your project. Beyond that, you'll be moving into the hard AI arena.

There are a lot of decisions that you can automate using the more basic tools and you should do some research and give those a try. Mostly I want to encourage you to think in terms of making decisions rather than crunching numbers. There are plenty of number-crunching tools but few decision-making tools. If you exhaust your options and are looking for other approaches, let me know and I can point you to some of our work or point you in the direction of some other experts that may be able to help you.

Mindviews Labs Blog

Sharing about the future of automated engineering.