muttley hates software: software with surprising depths of superficiality

• From: Simon Wistow
  Date: 10:59 on 28 Aug 2003
  Subject: software with surprising depths of superficiality
  
  I'm actually talking about a particular bit of code that I've inherited 
  but I've seen this phenonomem elsewhere so I'll paraphrase.
  
  I'm not sure if there's a neat buzzword for this particular species - 
  neither lasagne nor spaghetti code seems to work. Perhaps 'spirelli' 
  code.
  
  Anyway, lurching back to the point. I've inherited this code. The task 
  seems simple enough - indeed I have a suspicion that if I was to 
  reimplement it from scratch it would take a week or two. Instead, on and 
  off, I've been poking at it for nigh on six months now. 
  
  Now to be honest this is a week here and a week there but still. Six 
  months. You could rewrite Emacs in Perl in that space of time. Or 
  Mailman :)
  
  Why is this code so bad? Well, its lineage for a start - I can 
  practically see its evolutionary tree.
  
  Currently it is a rag tag collection of shell and Perl scripts. Some of 
  the Perl scripts obviously used to be shell scripts to the extent that 
  they actually SHELL OUT TO OTHER PERL SCRIPTS. Not for forking reasons, 
  oh no. Just because, well, the original author had partaken of the 
  crack pipe and not drunk deeply from the cup of refactoring. I have to 
  concede that they did move some of their 'require'-d code into modules 
  but the design of those, plus the fact that they lurk in various lower 
  case namespaces or, and I'm not sure if this is better or worse, in the 
  Test:: namespace, means that this is not enough to let them off the 
  hook.
  
  
  Even as I write this I'm struggling to explain why this has caused me so 
  many problems. On paper, and I know this because I had to write it down 
  yesterday, it all seems perfectly easy. But, obviously there is, 
  otherwise I wouldn't be writing this.
  
  My two, no wait, three major issues with this stuff are these :
  
  1. The code is *incredibly* brittle 
  This to the point that if I leave it for a week and then run it again I
  can practically guarantee that it will not work, often failing silently,
  and I will spend a full day poking around trying to diagnose the
  problem. Often there will seem to be no reason for this.
  
  2. It's amazingly hard to follow. 
  There are several reasons for this - my particular favourite being the
  lack of indenting, the terse variable names, the overuse of hash
  references (which are normally fine but become incredibly difficult to
  follow when nested 8 deep and passed between required files to functions
  with 10 arguments with no consitent naming scheme or ordering), the fact
  that there are many "actions at a distance" not least the use of
  Environment Variables as some sort of perverse global variable system,
  like the author went "well, I'm told globals are bad so if I put then in
  ENV then nobody will know". Truly, the list goes on and on.  I haven't 
  even mentioned the lack of comments and the long, undocumented regexes 
  with unprintable characters in. On and the fact that various libraries 
  are in various people's home directories. 
  
  Testing is also a problem because a trial run takes 4 hours and requires
  a full days worth of data meaning that a fix checked in on Monday won't
  be checkable until after lunch Wednesday.
  
  3. I'm not allowed to refactor. 
  Apparently the main reason I'm not allowed to is that then it won't be
  'trust worthy'. I've mentioned Unit tests but these have been roundly
  ignored. Also, the code has been internationalised from the original
  which is also being developed (don't ask why there are mutliple
  branches). So I can't refactor unless I want my life to be a manual
  patch applying hell. Hell, I can't even use Perl Tidy (I tried -
  waaaaaay too many spurious deltas)
  
  
  Actually, there's a fourth.
  
  4. It makes me feel shit
  I know I'm not a bad programmer[0] (stop sniggering Richard). I've 
  written far more compilcated programs than this. Yet it has single 
  handedly been the most disasterous project I've ever worked on. I seem 
  to find it impossible to give, let alone keep, a deadline. It saps my 
  confidence and I have huge problems forcing myself to work on it which 
  in turn makes me feel worse. Et cetera, ad nauseaum, ad infinitum.
  
  Right, that's enough moping.
  
  
  

• From: =?ISO-8859-1?Q?Ask_Bj=F8rn_Hansen?=
  Date: 11:05 on 28 Aug 2003
  Subject: Re: software with surprising depths of superficiality
  
  
  On Thursday, Aug 28, 2003, at 02:59 America/Los_Angeles, Simon Wistow 
  wrote:
  
  > Right, that's enough moping.
  
  And now to look for New Place of Work!
  
  Sounds like you need it very badly. :-)  At least get an offer or two, 
  then you can tell your local PHB that you either get support and time 
  to refactor The Monster - or you are out.
  
  
    - ask
  
  
  

• From: Piers Cawley
  Date: 13:29 on 28 Aug 2003
  Subject: Re: software with surprising depths of superficiality
  
  Ask Bj=F8rn Hansen <ask@xxxxxxxxxx.xxx> writes:
  
  > On Thursday, Aug 28, 2003, at 02:59 America/Los_Angeles, Simon Wistow
  > wrote:
  >
  >> Right, that's enough moping.
  >
  > And now to look for New Place of Work!
  >
  > Sounds like you need it very badly. :-)  At least get an offer or two,
  > then you can tell your local PHB that you either get support and time
  > to refactor The Monster - or you are out.
  
  Ah, the old "If you can't change your organization, change your
  organization." technique.
  
  

• From: Ann Barcomb
  Date: 11:34 on 28 Aug 2003
  Subject: Re: software with surprising depths of superficiality
  
  We've probably all had to maintain horrible software, so even if I
  haven't met your particular burden, you have my sympathy.
  
  I think I'll avoid listing which jobs and places of employment were
  responsible for these disasters, but I have at one time or another,
  had to maintain all of the following:
  
  * poor directory structure (modules, logs and cgi in the same directory)
  * variables shared between scripts and modules without ever being explicitly
    passed
  * the same sort of data (such as translations) stored in multiple
    locations--sometimes in a file, sometimes in a database
  * credit card numbers stored unencrypted in databases
  * code littered with exceptions for projects or features which have since
    expired or faded away
  * unused code which hasn't been deleted
  * unused database fields
  * modules which export all 200 functions by default
  * hardcoded deployment information scattered across dozens of files
  * no revision control in sight
  * no tests
  * no documentation
  * multiple versions of the same application which differ just enough to
    make maintaining each a seperate nightmare
  * no development environment; everything is done on the live server
  * multiple HTML pages embeded in a single script
  * thousands of lines of code without a subroutine
  * subroutines which neither accept arguments nor return values, but set
    global variables
  * dry run code which shares nothing whatsoever with the real run code
  * code which avoided nested data structures by having hash values which
    were in fact the names of the other hashes
    (NOT: $hash{foo} = \%bar, BUT: $hash{foo} = 'bar')
  * subroutines which can take dozens of optional arguments, with each specifying
    a slightly different result in the routine (without using named
    arguments)
  * OO modules that did nothing which required OO, but instead exported a
    bunch of 'methods'
  * passwords in plain text in cookies, databases and URLs
  * really odd home-grown templating/cgi syntax
  * error messages with creative values like 'ERROR 2'
  * single letter variable names (when variables > alphabet, use 2-character
    names)
  
  I think I have to stop.  I'm getting depressed.
  
  

• From: Piers Cawley
  Date: 13:32 on 28 Aug 2003
  Subject: Re: software with surprising depths of superficiality
  
  Simon Wistow <simon@xxxxxxxxxx.xxx> writes:
  > 3. I'm not allowed to refactor. 
  > Apparently the main reason I'm not allowed to is that then it won't be
  > 'trust worthy'. I've mentioned Unit tests but these have been roundly
  > ignored. Also, the code has been internationalised from the original
  > which is also being developed (don't ask why there are mutliple
  > branches). So I can't refactor unless I want my life to be a manual
  > patch applying hell. Hell, I can't even use Perl Tidy (I tried -
  > waaaaaay too many spurious deltas)
  
  Have you been in touch with the other team? Directly, with the
  developers. Maybe they feel just as bad about it as you do, but
  daren't touch anything because there are other developers like
  you. Maybe they're not, but you can't know unless you communicate. 
  
  At best you'll all be able to present a united front to management
  when you say it needs refactoring. At worst, you'll know they're a
  bunch of hacks and you can start pushing (hard) to be reassigned or
  ship out.
  
  
  

• From: David Cantrell
  Date: 21:52 on 28 Aug 2003
  Subject: Re: software with surprising depths of superficiality
  
  Simon Wistow wrote:
  
  > Currently it is a rag tag collection of shell and Perl scripts. Some of 
  > the Perl scripts obviously used to be shell scripts to the extent that 
  > they actually SHELL OUT TO OTHER PERL SCRIPTS.
  
  Gosh, that sounds familiar. [looks around, hopes $colleague isn't on 
  this list] One of the many things on my to-do list at work is replacing 
  such an abortion.  Well, actually it's to re-jig it so that it does a 
  whole bunch more, but I'm damned if I'm hacking on that mess, so a 
  re-write it is.  And the re-write will not involve any of that XML crap.
  
  But in the mean time, I have far more interesting things to do, thank 
  $deity.
  
  > 2. It's amazingly hard to follow. 
  > There are several reasons for this?- my particular favourite being the
  > lack of indenting, the terse variable names, the overuse of hash
  > references (which are normally fine but become incredibly difficult to
  > follow when nested 8 deep 
  
  Are deeply nested data structures that naughty?  With sensible keys I 
  find them easy to use, even if you do things like ...
  
  $foop->{boing}->{$foop->{wibbles}->{$wibble}}
  
  Provided the keys are sensible - which does not preclude being terse - I 
  see little to be gained from creating temporary variables to hold the 
  individual bits of that just to make that one little bit of data access 
  shorter.  It certainly won't make the code easier to read IMO.
  
  
  

• From: Gavin Estey
  Date: 22:24 on 28 Aug 2003
  Subject: Re: software with surprising depths of superficiality
  
  On Thursday, August 28, 2003 4:52 PM, David Cantrell
  <david@xxxxxxxx.xxx.xx> wrote:
  > Are deeply nested data structures that naughty?  With sensible keys I
  > find them easy to use, even if you do things like ...
  >
  > $foop->{boing}->{$foop->{wibbles}->{$wibble}}
  >
  
  I had a colleague who insisted on doing this:
  
  $foop->{boing}->{$foop->{wibbles}->{$wibble}}{look} = 1;
  $foop->{boing}->{$foop->{wibbles}->{$wibble}}{at} = 1;
  $foop->{boing}->{$foop->{wibbles}->{$wibble}}{all} = 1;
  $foop->{boing}->{$foop->{wibbles}->{$wibble}}{this} = 1;
  $foop->{boing}->{$foop->{wibbles}->{$wibble}}{copy} = 1;
  $foop->{boing}->{$foop->{wibbles}->{$wibble}}{and} = 1;
  $foop->{boing}->{$foop->{wibbles}->{$wibble}}{pasting} = 1;
  
  To make it even harder to read he'd use lines well over 100 characters wide.
  
  Gavin.
  
  
  

Generated at 10:27 on 16 Apr 2008 by mariachi