Development Strategy

From Elemental

 
 Keep on the lookout for novel ideas that others have used successfully.  
 Your idea has to be original only in it's adaption to the problem you're working on.
   -Thomas Edison (1847-1931)
 

Contents 1 Introduction 2 Research, or Thesis 3 Development, or Antithesis 4 Production, or Synthesis

Introduction

... project management for researchers.

Research and development is like drawing a map of a dark room inside of that same dark room. Like the room, all technology has an inherent structure to it. This is the Archetypal shape of algorithm. The complication with research (or bleeding-edge) development is that the technology's shape is unknown in the beginning. To uncover the shape in a reasonable amount of time, a clear development philosophy is needed. A method by which we can set about the task of mapping the (potentially giant) dark room...

If the complexity of the dark room is unknown, then it can be difficult to estimate a proper development time-line. More than development, research requires extremely flexible task and time management systems to handle unforeseen development requirements. Using a milestone tracking system allows each developer the flexibility to rotate priorities, build their own schedule, and swap assignments with other coders. Inherently, this provides merit-based development system which produces better quality code while encouraging teams dynamics and impromptu co-development amoungst group mates. Developers also tend to "police their own", which makes the project manager's job of weeding out non-hackers easier.

We combine the process of Hegelian Synthesis with Agile development methods to address complications with developing projects with unknown scopes. Our method allows researchers to avoid getting stuck in the details, and to foresee and navigate around complicated development problems by combining best practices from bottom-up, top-down, continuous-redesign, back-tracking, and organic-growth strategies.

Research, or Thesis

Using a bottom-up approach we add functionality to the codebase based on brainstorming sessions where we form milestones. While new ideas are fresh in the mind, we develop a POC (Proof Of Concept) model to explore it's potential. After letting the new tech settle in the mind, we redevelop it and transplant it to the best location in the main codebase.

 Invent <---> Integrate 
          |
          L -> Backtrack

Bottom-Up We use a bottom-up approach for research. We take base-level technologies and grown them organically into a top-level architecture. This helps us find the architecture's most natural structure. Invent/Integrate works as a spiral model with production returns slowing over time. Development cycles work outward from the base architecture. Programmers rapidly invent new functionality and apply it to the core spindle in prototype fashion. This is very efficient for brainstorming, however, coders must be rotated often to avoid divergence and stalling in development. Backtracking is time consuming, and produces an endless amount of miscellaneous code cleanup. Our measure of success is how far we can get before backtracking, thus coding endurance and discipline is key to our strategy.

Back-Tracking Backtracking specifically refers to starting with the last code you developed and working backwards in your development timeline. Using the rapid-development approach, our codebase often reaches a point where forward progress is no longer feasible. When progress slows to a point where the forward rate has lost it's return, we Backtrack out of our development cycle.

Incremental Backtracking over time allows the base architecture to morph it's inherent structure. As programmers integrate their functionality back into the core, the core itself grows organically in on itself. The core architecture grows in layers. As development progresses, functionality crystalizes under the pressure of development.

Most of the time, we find that cleaning up our code base gets us enough cycles to continue development. Each time this point is hit, we stop forward progress and work in tandem to properly engineer our original work. Over time, several clean cycles produce a system architecture, and solidify through a process like the simulated annealing of code. Standard procedures include general cleaning, optimizing algorithms, and Abstraction.

After the code has been cleaned up enough to provide room for forward development again, we go back to Inventing and Integrating

Transplanting At certain times the project must be transplanted out of the current architecture and into a new one. The aspects of the original codebase that are to be kept should be cleaned and modularized. We pull these components into the new project space, and attach controllers and camera first to get base functionality online. Then we build in the new architecture and integrate components into the main procedural code-branch.

Development, or Antithesis

We use top-down development along with our backtracking procedure to produce the final application.

Top-Down Perspective - Footprinting Footprinting is the art of plotting our/your final application to include all viable technologies. As the final application scope begins to emerge, we reverse our procedure, from bottom-up to top-down. This usually takes breaking away from the code for a bit to clear your perspective and look at the code fresh. We begin by examining top-level descriptions of the application such as is seen in UML class and sequence diagrams. The main components and procedural lines of the system's inherent structure should become increasingly obvious as the UML structure is organized and reworked, producing a center-line for the application's architecture.

Forward-Tracking Perspective Once the footprint has been established, we work the codebase against the grain of our original bottom-up approach. Starting from the main entry point of the system, we work outward checking for compatibility with the footprint. Conflicts are solved with interest to maximizing the functionality of the footprint, or final requirements.

During this development time, integration, abstraction, and cleaning of the code base will strengthen the inherent framework. Multiple iterations of cleaning are required, and should each start at the strongest centerlines of the code and work outwards as far as progress is productive.

In forward tracking, again we find ourselves continually bogged down in "currently unsolvable problems". When our forward progress has slowed, we track back into cleaner code and proceed forward with another component. Our unsolvable milestone will likely unravel itself as other components mature and define their interfaces.

Remember, code has an inherent structure.. like a jigsaw puzzle, building around an unknown piece may reveal it.

Transplant Cycle At certain times the project must be transplanted out of the current architecture and into a new one. The aspects of the original codebase that are to be kept should be cleaned and modularized. We pull these components into the new project space, and attach controllers and camera first to get base functionality online. Then we build in the new architecture and integrate components into the main procedural code-branch.

Production, or Synthesis

Switching between bottom-up to top-down allows us to develop concepts rapidly. We can code with less precision initially the knowledge that the concepts will be revisited and industrialized. Our approach has saved a substantial number of work hours.

The speed up comes from _not_ getting lost in the details. In our experience, we find that programmers have a tendency to get bogged down in the implementation details that are often removed or recoded later anyway, due to broader structural changes. By leaving the research implementation in a vaguly "hacked-in" state, the codebase is more flexible to change as the research continues.

Industrialization of the code occurs during the development stage, and once the full application scope has been hashed out. Our cross-grain approach, or switching to top-down passes, provide different perspectives on the codebase which helps to ensure that code is clean around the edges.

Finally, we maximize the talents of both the researcher and the developer by using seperate passes. Researchers are given the freedom of creative coding, without having to worry obsesively about coding standards and best practices. Developers, on the other hand, are able to take a well defined footprint of the project and industrialize without getting bogged down with complicated research areas that may stall development.