Skip to main content

That Time I Wrote a Calculation Engine That No One Knew How To Use

♾️ He talks in maths, he buzzes like a fridge...

I work on software, and I've been doing so professionally for over a decade now. Shortly after starting at my current job, I was pulled into a brand new project working on quoting tool for one of our sales divisions. These quotes had a lot of complexity and cascading effects involving margins and markups, sometimes deriving final amounts from those, sometimes going the other way, and it all had to add up correctly.

This is, just so we're clear, a lot more complex than it sounds. Computers are machines built to do math, but they are very opinionated about the math they do. You see, for humans a number is a number, but to a program running on the JVM, a number is a byte, a short, an int, a long, a float, or a double (and sometimes also a boolean) and these different types don't inherently play nicely together. If you take $1 and add $.75 to it, the computer may just tell you it can't do that unless you convert the 1 to a 1.0 first. However, any time you introduce a decimal in computer math, you necessarily introduce a loss of precision as the value is cast as a floating point decimal. Loss of precision in business accounting is not desirable, ergo, the base unit of money is not the dollar, but the penny. But things also need to be human-readable, so you're constantly switching back and forth. $1.00 gets turned into 100 pennies then manipulated to become 175 pennies and then turned back, and then anything that relies on that amount converts it again. There are libraries to make this simpler, but they don't play nicely with your basic math operators, so you have to do everything in method calls which ends up feeling like reverse polish notation, but in a text document where you can't actually see the numbers.

There's all sorts of silly crap like this that is not hard, per se, but it gets to be a lot to keep track of. And if you're not very deliberate with how you wrangle it, you end up with something that's difficult to read and impossible to debug. One of the approaches we chose for this project was to break calculations down into smaller chunks that could be individually unit-tested and maintained. The challenge, then, was one of orchestration. How do you make sure things fire in the right order? How do you avoid circular dependencies?

So I had an idea, and the rest of the team was happy to let me run with it since I'm one of those people who doesn't find math intimidating at all. I wanted to create a dynamic orchestration engine that would take calculators, determine their dependencies, and order them appropriately. First I made sure that every single calculator in the stack had only one output. Then I created a Spy of the data model (a Spy is a type of Mock object used in testing that keeps track of how it's interacted with). I ran the entire calculation stack against it and used reflection to determine which fields they read from (inputs) and which ones they wrote to (outputs). From there I was able to build a dependency tree which I could use to order the calculation stack. If you wanted to calculate something for a new field, all you had to do was write a calculator with any number of inputs and only one output and register it with the orchestrator and it would get picked up and integrated automatically. And for good measure, I had a unit test that would fail if it ever detected a circular dependency. I built and it I tested it and it became the backbone of the entire calculation matrix. And speaking of the matrix...

If I may paraphrase The Architect: The first calculation engine I designed was quite naturally perfect. It was a work of art, flawless, sublime. A triumph equaled only by its monumental failure.

You see, it's not that it didn't work. It worked spectacularly. The problem was that I had written a framework, and it was opaque. People wanting to edit or add a calculator had to grok the entire framework first. They didn't inherently understand why a calculator could only have one output. Or the recursive dependency test would fail and they would have no idea why. Suddenly I was a gatekeeper, the only person who could interact with the damned thing. So a few months later, I ended up re-writing it entirely.

There's a lesson that any programmer worth their salt eventually learns, and this was the incident that drove it home for me. You spend more time reading code than writing it, and you spend more time working on other people's code instead of your own. It doesn't matter how brilliant this thing you've constructed is if the person who comes after you doesn't know what to do with it. Far better to keep things obvious and simple, and if they can't be simple, they should at least be intuitive.

]{p

Comments

Popular posts from this blog

On Getting Laser Eyes

Last week I got Lasik. I was looking forward to not having to deal with glasses getting smudged by my kids or slipping off my face. I figured that not needing them would be pretty convenient. However, the words I heard over and over from other people who'd already done it were: "life-changing." That seemed to be overstating a bit. Convenient, yes, but life-changing? I didn't get it. I get it now. I've had some kind of vision correction, either glasses or contacts, for the last thirty-odd years, which is nearly as far back as I can remember. And what I hadn't realized was the extent to which this had become part of my identity. It's not that I thought glasses were cool because I wore them--although I did and they are. It's that the ability to see was, for me, artificial and temporary. And my vision was pretty bad, so my natural state was one of... not so much "blindness" as "isolation." There was a layer of vagueness that sat bet

100 Album: "Game Of Thrones Season 3 Soundtrack" by Ramin Djawadi

Kurt is going through his favorite records. Read the  explainer  or view  the master list . Artist:  Ramin Djawadi Title:   Game Of Thrones, Season 3 Soundtrack Released:  2013 Genre:  DAH duh, duh-duh-DAH duh, duh-duh-DAH duh He's not as big a name as Hans Zimmer or John Williams or the various Newmans out there, but Ramin Djawadi is easily the most interesting composer working in television right now (with due respect to Bear McCreary). Soundtracks, especially television soundtracks because they're produced so quickly, have a tendency to serve more as a wall of atmosphere than anything else. But Djawadi's work here and on Westworld  has generated some amazing musical themes. There's a strong undercurrent of leitmotif informing the way the music flows together and the themes those motifs are built around are damned  catchy--which you know if you got the joke in the genre description above. While all of the soundtracks for GoT  are very listenable, this is m

100 Albums: "Fashion Nugget" by Cake

Kurt is going through his favorite records. Read the  explainer  or view  the master list . Artist:  Cake Title:   Fashion Nugget Released:  1996 Genre:  lo-fi indie alt-rock There was a summer when I was in college that I spent every spare minute playing Super Bomber Man  on the SNES and listening to Cake's Fashion Nugget  (and one other album that I will get to shortly). Cake broke in the late era of grunge with The Distance , a--ahem--driving song about a man racing to get back to his love, or something like that. The metaphor was unclear, but the song was catchy as hell. They followed it up with a cover of I Will Survive  that was much more indicative of Cake's sound: lo-fi vintage guitar, a lead trumpet, John McCrea's deadpan just-off-rhythm singing and sarcastic lyrics, and Victor Damiani's frenetic bass-playing. Fashion Nugget  was independently produced under the ethos of "if you can't make it sound clean, make it sound dirty in an interesti