JoelBlog

This is a surprisingly unabashed admission from Science magazine (i.e. the world’s flagship scientific publication):

Science’s publication workflow relies on Microsoft Word97.  To translate LaTeX files into Word97, we use an intermediate MS-DOS routine that converts the TeX source into HTML.

Just think about that the next time you hack something in to meet a deadline. Hacks layer upon existing hacks until you have some kind of living monstrosity.

Of course, I’m often guilty of these kinds of crimes. On the optimistic side, their system sounds quirky and interesting (from a Rube Goldberg point of view), but not something I’d like to work on in particular.

If you’ve ever needed to throw up a quick real-time plot from C++, it can be a hassle. Data visualization is a great way to sanity check your code, especially in scientific computing where you might have some long-running iterative code (for me it’s genetic algorithms).

A quick and dirty way to do this is to pipe out to a gnuplot process. It can update in real-time, and although there are some issues, it generally works pretty well. I just hacked up a lightweight C++ header that does this (note: only tested with linux), which I called plotpipe. Hopefully it’s useful to others as well (requires a gnuplot install to run).

Right now it just handles 1D and 2D datasets. Here’s an example of it plotting an animated moving window of a quadratic function:

#include <stdio.h>
#include <stdlib.h>
#include <vector>
#include "graph.h"
using namespace std;

int main(int argc,char **argv) {
 plot p; 
 for(int a=0;a<100;a++) { 
  vector<float> x,y; 
  for(int k=a;k<a+200;k++) {   
   x.push_back(k);   
   y.push_back(k*k); 
  } 
  p.plot_data(x,y); 
 }  
 return 0;
}

An insightful post made its way on hackernews today, reminding us of the ever-present danger of echo chambers. The idea is that if we lose vigilance we may simply subscribe to media available on the internet that reinforce our existing beliefs. I think a more graphic term is helpful here: Intellectual Masturbation.

Strangely, the internet facilitates both concrete and abstract masturbation. Everyone knows that porn websites account for a significant fraction of web traffic (although perhaps not as much as you’d think!), but you can think of all the strange conspiracy websites as intellectual porn for those that subscribe to those theories. And all the websites we frequent have a particular intellectual bent that’s perhaps not so overt — whether it is hackernews with its centricity on startups or reddit’s obsession with cats and Ron Paul.

The problem is that we get complacent and comfortable with the content sources we consume, which can lead to intellectual stagnation and worse, to dogmatic thinking, a “us” vs “them” mentality — which is exemplified to me (leaning slightly liberal myself) by fox news. And perhaps those on the right would argue for MSNBC fulfilling the same role. There’s probably some truth in both of these accusations.

Anyways, there’s no easy solution, except to try and force yourself to be open-minded and to think critically — even when it seems as if a new factoid either entirely aligns or conflicts with your worldview.

In closing, challenge yourself to think — and think for yourself.

Musicians, designers, game programmers, tenured professors.

There are some jobs that seem great, and the demand vastly outweighs the supply. There’s a deluge of people applying for a small number of jobs. In those sorts of professions, a system of hoops naturally forms. It’s no longer enough just to work hard and be really good — you’ve got to jump through the hoops.

For musicians, it’s playing dingy clubs night after night trying to build some kind of following, hoping for an eventual record deal. For game programmers it’s paying your dues as a code monkey working hellish hours in the run-up to release, hoping to become a game designer so you can realize your own vision. And for wanna-be professors it’s living in the academic limbo known as a “post-doc,” hoping one day to have a research group of your own (with your own army of post-docs to do your bidding!).

It’s a filtering mechanism. It weeds out those people who truly aren’t passionate — otherwise why would they jump through those painful hoops? You don’t become a game programmer for the pay. If you’re smart you don’t become a musician because you want to be rich (only the most elite musicians are).

It’s clear that you need some sort of filtering. There are just so many musicians out there — some are terrible, many are incredibly talented. But pointless hoops aren’t the best answer. Perhaps before technological solutions playing dingy clubs was the best filtering mechanism for musicians. But in the internet age we’re starting to discover that becoming well-known musically is a different ballgame. You can become a you-tube sensation instead.

And indie game developers are finding a way to exploit the internet as well to avoid those hoops. But for other occupations it’s less clear. Is there some sort of disruptive hoop-killer for academia?

To check, you can visit this handy web-app: doesmyvotematter.com

Your vote should matter in this upcoming 2012 presidential election, right? We live in a democracy, after all. Well, it turns out that for most people in the US, your vote won’t impact the national election one bit.

The problem is the electoral college system we have in the US. Entire states are won or lost in the electoral college. This means that if you are a democrat in a republican state, your vote has no effect. It’s extinguished by the republican majority and never makes it out of the state (and the same argument applies for republicans in overwhelmingly democrat states).

So most states are locked up for one party or the other. In this current election there are only a couple of states that could go either way — the swing states: Colorado, Florida, Iowa, New Hampshire, Nevada, Ohio, Pennsylvania, Virginia and Wisconsin. If you look at the numbers here, these 9 states only make up 33% of the population of the US. In other words, the election basically rests on how only a third of the US votes.

While you can say it more gently, I’ll be blunt: It’s dumb. We have better and more fair information available to us (the popular vote), or if you want a smaller change we can there at least divide electoral college votes more fairly instead of a winner-takes-all system (two states do this already).

The electoral college is dumb. And our current “majority vote” system itself is dumb (you only vote for one candidate, not a list of your preferences among candidates). There are much better voting systems, that at least make it so third party candidates can’t change the entire outcome of an election (although no voting system is perfect).

Anyways, the voting situation here is like using a mainframe computer (from the 1700′s) because it worked okay back then — when you could have a modern laptop that is a thousand times better. Everyone who studies voting knows it’s broken.

I guess the reason that it doesn’t change is that politicians don’t want to give up a game they know how to play (even if it’s busted) for one that’s more fair but less predictable. Or maybe there’s just nothing politically to be gained from it.

There’s an alternative universe where no one ever writes legal contracts. The negotiating parties gather in a room and wait for text messages from the future. One might be: “What should we do in the event of shares of company being granted to an ex-spouse in the case of divorce?” Then the parties argue and negotiate, and eventually there’s a decision about ex-spouses getting shares. “Their shares shouldn’t include voting rights within the company because the ex-spouse will likely lack expertise in the company’s specialty”. And when the situation arises in the future with emotions hot and impartial interests, it’s handled as it was planned objectively in the past.

Motivation

I’ve recently been part of forming a company, and part of the frustrating bureaucracy is drafting legal documents. For example, you have to write an “operating agreement” that codifies exactly how certain situations are handled. What should the company do if one member isn’t pulling their weight? What should the company do if there is a death of one of the members?

It’s a time consuming and boring process. And the standard for great legal documents are those that are airtight: They foresee every possible contingency. What should the company do in the event of alien invasion? What if gravity reverses itself in Kansas City for two hours?

What’s frustrating to me is that when we write some legal document, we all end up arguing about all different sorts of contingencies (that likely won’t happen), so that we can put it into the operating agreement. It’s a waste of time that distracts us from building our product (which is stuck in an everlasting ‘one-month to launch’ limbo). On the other hand, at some point one of these contingencies may actually come into play — and it’s then that you would wish you’d spent more time on your legal document.

The main problem is that at the time of the contingency the people involved aren’t objective anymore — the contingency might mean that one person would benefit directly from something at the cost of the others. And when huge sums of money are involved any bet on rational and benevolent action are unwise. For example, if the president of the company dies, the other members might all think they would be best successor (especially if it entails a large raise). The document protects everyone from everyone else.

So the reason you draft a legal document ahead of time is that everyone is rational and reasonable and acting in the companies’ and each others’ interests at that time. You’re trying to freeze and bottle up the essence of what the company should do ahead of time.

Law By Time Travel

So imagine in an alternative universe, you didn’t ever make a contract. Instead, what you did was gather all the negotiating parties together in a room and wait for text messages from the future. And imagine that they poured in. “What should we do in the event of shares of company being granted to an ex-spouse in the case of divorce?” might be the first one. Then the parties argue and negotiate, and eventually there is a sentence in the contract about ex-spouses getting shares (you probably don’t want them to have any voting control of the company because they lack expertise in the company’s specialty).

The idea is that sometime in the future this issue arose, and so it became important to deal with it. However, rather than attempting to foresee it and write it into the document, it is dealt with by sending a text message to the past, at which point it is debated by the parties and decided then. The benefit is that only the issues that actually happen are dealt with and take time, and that unforeseeable issues don’t end up having crazy unintended consequences.

Importantly, the essence of the legal agreement is what those parties believe at the time of signature, and is corrupted in the future by extenuating circumstances. At the time of signature, there’s more objectivity. A particular circumstance that benefits one party ruins rational thinking and impartiality.

Law By Acting

Unfortunately, we can’t send text messages to the past. However, the key insight is that you want the essence of what you meant at the time to decide legal matters. We can get to this in other ways. For example, if a negotiating party wrote down a few pages on their motivations and important interests in the contract, an actor could later use these pages to negotiate on their behalf in the future.

In other words, you can try to freeze the essence of what you care about in a negotiation, and then have actors work from that essence in the future to hash out an unforeseen issue in a reasonably fair way. The actors are hired to work from the pages that the parties wrote at signing time and are provided the most general scenario to debate (without any impartial information, i.e., whose spouse now has shares?) so it remains an approximation of the debate that would have happened at the time of signature.

The benefit of this is that you don’t have to deal with all sorts of weird contingencies (and you can’t anticipate all of them, even then) before they actually begin to matter.

Moral of the Story

I think there’s something wrong with law and how it’s handled. Laws shouldn’t require years of study to write — or more importantly, to understand. I’m not sure yet of the best solution, but I think computers can play an important part. If there were some way to write laws in a way computers could understand (which is a level of logical consistency and clarity that lawyers aspire to), they could point out the logical implications that might not be obvious to humans. Or perhaps there are ways around actually writing many legal documents (such as law by acting).

Bureaucracy is the art of making the possible impossible.
-Javier Pascual  Salcedo

You will never understand bureaucracies until you understand that for the bureaucrat, procedures are everything and outcomes are nothing.
-Thomas Sowell

If an idea can survive a bureaucratic review and be implemented, then it wasn’t worth doing.
-Mollison’s Bureaucracy Hypothesis

Pointless bureaucracy is infuriating for us all. It seems to spring up naturally in all large institutions. I think of it like some sort of ratcheting process — it’s easy to create but hard to get rid of. As a result, bureaucracy accumulates slowly like cruft. How many times have you heard the following phrases at a big company or university:

“Yes, I know it’s silly, too. But to get your request approved you’ve got to do it.”

“I’d like to help you — I understand that this is an exceptional situation — but I honestly don’t have the authority to bend the rules here.”

“I’m sorry, that’s the way it’s always been.”

I recently defended my Ph.D. dissertation at a large university. Successfully defending consisted of two main parts: 1) Thoroughly preparing for the defense itself, and 2) Completing the most boring real-life “point-and-click” adventure game ever, one that I call, in the spirit of the old Sierra Online games: Dissertation Quest IV: The Hunt For Signatures.

My advisor was out of the country a few weeks before I was to defend. This resulted in a gradual intensification of what I call my “nerd anger” over the procedures for getting forms approved. Now, nerd anger is that feeling we all have when we realize there is a much simpler way of doing things — but the convention requires us to do it in a roundabout inefficient way. It is what drives us to program automation for monotonous tasks.

Now let me describe the roundabout procedure for getting a form approved:

1] Find the form online and fill it out
2] Print it out
3] Sign it
4] Scan it in
5] Email it to my professor

My advisor, in a foreign country with spotty internet access, is then expected to:

4] Print out the form from the email
5] Sign it
6] Scan it in
7] Email it back to me.

Now, my responsibility is simple:

8] Keep going to the office of whomever needs the form until they are actually there
9] Give them the form

So completing this for various forms was just a massive pain.  And my anger grew. At first I didn’t understand why it was making me so angry. But then I just thought about how I would code the solution.

First of all, any physical form is completely and utterly unnecessary. This also implies: requiring physical signatures is completely and utterly unnecessary. Both of these are just stone-age activities. I cannot state enough how ridiculous it is: What physical form will not just need to be re-typed into a computer system of some kind eventually by a secretary? You should just have a simple web form instead!

Also, with regards to signatures, computer-signing of electronic forms is not only vastly more efficient, simple to implement, and independent of physical locality, but also, it is a much better solution because it is far more secure. A squiggly signature can be faked (and whoever verifies them anyways?) while the firepower required to bust an encrypted signature is significant.

Anyways what this all boils down to is that the bureaucratic cruft of forms and signatures (while it will go away some day in the future), is here to stay for the near term. But the simple system that replaces it will look something like this:

1] Find the form online and fill it out

This is where my personal responsibility ends!

The form is then forwarded automatically to my advisor, who:

2] Approves it via electronic signature.

And then the form is forwarded automatically to whoever needs the information, and automatically entered into a database.

This kind of system is probably in place in many institutions already, but I think there is room for a start-up to completely disrupt this space with a simple-as-pie works-out-of-the-box product that could make them a pile of cash while saving huge institutions a pile of cash at the same time. Win-win! And I have won’t to be angry about hunting for signatures anymore.

Ubuntu, Chrome, and I’ll be damned if I can’t find a way to pop this sucker into 16-color mode.