In the context of exploratory programming

Ah, the joys of exploratory programming! Taking a piece of small and trivial code, swiftly hacking several layers of functionality, refactoring occasionally when more flexible abstractions are recognized, continuing on until Experience is screaming at you to start documenting your work in some sort of design specification. At this point, the need is to stave off madness; a clear and manageable set of assumptions has now become a crowd of sophisticated design decisions dancing Tangos inside your head.

So exploratory programming is just wonderful! Code slinging wearing merely your underwear and a cowboy hat. If you're a lady, I dunno, something similar maybe? Okay, probably not. Seriously, if you're in unfamiliar technical waters, it's a big help to start with example code, adding your own custom code as your solution progresses. It's possible to learn and create from reading only some library reference docs, assuming that they are complete, but it's usually a long and miserable experience.

Sometimes it's a little bit of Catch-22, where you don't understand foreign example code because it uses an unfamiliar library. But after turning to the reference docs, you often discover a chain of dependencies with incomplete explanations that are difficult to make sense of without good matching examples, which are usually absent. So, now the situation digresses to hunting around the web with your trusty Google chops and hoping that something useful eventually falls out of the sky. A good answer usually does, or maybe try pleading for a morsel on the newsgroups. Indeed, these are time consuming hobbies!

To sum up my previous rambling, a clear and adequate context is the foundation of lasting programming progress. This context can be found in the discovery of good examples, in the organization of one's problem solving plans, in reflection on what has already been tried, and in the result of many other useful habits.

The context of past experiences is perhaps the most important guide of all. This is simply known as 'understanding based on experience'. For example, I had decided to continue investing time and effort in learning the Haskell programming language and many of the powerful usage patterns made available by it's design, in part because I had used many other programming languages in the past, and was able to contrast their capabilities with those of Haskell's. If I was just starting out as a programmer and Haskell was the first programming language presented to me, I would probably have had a much harder time justifying the effort in continuing to learn a esoteric technology that can easily appear to make a more difficult experience out of programming than necessary. I suspect that most beginning programmers are initially unable to see past trivial programming examples when trying to get a feel for the most appropriate programming language to use. I've wondered how many novice programmers have decided Object Oriented Programming was "the bee's knees" after only seeing it used in the classic Shapes example. (Square, Circle, Triangle, etc. classes inheriting from class Shape) If a person has always lived just two blocks from where they work and had never encountered a bicycle before, how could they justify the seeming difficulty in riding a bicycle to work and back, if it is so easy to walk there?

Exploratory programming is experience building, which is why it plays such an important part. Without this context, you would have only the wise and obscure words of others to direct your steps; ultimately, not as effective. Doing is learning. Pretty obvious statement, I know.

Contrast this with the philosophy of first deciding upon a formal design specification before any implementation is written. The thinking here is that without mapping out a design, you risk implementing dead ends. In reality, unless you understand your target domain to a high degree and it's already clear in your mind what data structures your solution needs, it's usually more fruitful to begin doing something constructive with code first, then evaluating what's feasible as you progress.

It's a valuable skill to be able to alternate your focus between experimental implementation and formal design specification. Which brings me to a subtle but critical context needed for continued progress. The context of current achievement; deciding how to move forward in the context of what you have already achieved. For example, when coding up a complex real-world solution there is always many unexpected subtleties to encounter and work through. Complications are encountered that couldn't have been anticipated by preparatory design. Because of this, good insight into which direction to proceed next is often found by studying the micro-decisions already made in the code and recognizing the awkward parts that need to be refactored. True, there is always the risk of wasting lots of time flailing around with implementation changes. Here it is really important to encourage code tractability by implementing using either a language like Haskell, with static typing but automatic type-inference, or a dynamically typed language, because when you need to change directions and significantly modify your code, you don't want most of your time and effort being soaked up with the hassle and frustrations involved with explicit typing considerations.

In conclusion, it's often better to begin a project by writing code, testing the results, and exploring the solution space, rather than feeling stuck insisting on developing an uncertain design. A realistic design may not be revealed to you except after a little exploration. There is an art in knowing at what point to pause and reflect on the code for the purpose of fleshing out more of the formal design. A programmer shouldn't be afraid to make early implementation mistakes as long as the implementation is scrutinized regularly. Save your greater concerns for the much more serious mistake of solving the wrong problem for lack of considering what your problem space really is, something I will write about in a future posting.