My Coding Approach

I was thinking about my typical approach to coding. When writing a new feature, I tend to implement in the direction of where the data flows, starting from the user interface then to the backend and back to the frontend and wherever else that goes. I will build incrementally, using debugging tools or console printouts to ensure that each step is working correctly.

As an example, here’s how I did a recent web-based function:

  1. The user needs to see a new button on the screen, so I add the HTML for that button and just have the click handler be an alert confirming the button has been clicked.
  2. The button needs to open a modal dialog. I write the HTML for the modal dialog, then add the code to the button to launch the modal.
  3. On load, the modal dialog needs to fetch some data from the server. I write the Javascript to make the ajax call on launch of the modal.
  4. On the backend, I map the URL route used by the above ajax call to a controller method. The controller method simply outputs a debug message to the console. I refresh the page on the front end, click the button to launch the modal and verify on my server’s console that the controller method has been called.
  5. I modify the controller method to call a method on the relevant model. I write the stub method on the model with a dummy return value, which the controller method then outputs to the console. I verify the entire flow again.
  6. I write the retrieval logic in the controller method. I verify the entire flow again, using server console output to verify the correct data is being returned to the controller.
  7. I have the controller write the returned data to the response as JSON. I go back to the frontend and modify the success handler of the ajax call to log the response to the Javascript console. At this point, I also update the error handler to do whatever is appropriate. I verify the entire flow again, this time confirming that the correct data is displayed in the browser console.
  8. The user needs to see the data from the response, so I write the Javascript and HTML to render the response inside the modal dialog. I verify the entire flow again, this time confirming that the correct data is shown in the modal dialog.

I’m not sure if this is the best approach, but it works for me. It feels incremental, systematic, methodical, and easy to see where things go wrong (which they inevitably do). As kind of an upside, I also strangely feel compelled to not stop until I have working complete flow. Of course the downside is that if I have to stop at any point, I now have a nonworking UI component, but at least that’s a clear indicator of where I need to pick up next time.

I realize that a lot of modern software engineering advocates recommend some kind of test-driven approach. I imagine with such an approach start with the smallest iota of functionality, write a test for it, then write the actual code, then work outwards from there, building more complexity on top. So for the example above, maybe I should have started with the retrieval logic on the backend then work my way outwards from there.

I do that sometimes but I find that as a full-stack engineer my natural tendency is work my way across a new feature in this manner – from user interface to backend and back. Combined with the difficulty of writing code to test UI/HTML/JavaScript, this means I rarely get into the mindset of test-driven development. It’s not something I ever got used to. Maybe someday a switch will click and I will get it and my mind will be blown.

My approach does have some benefits though, in that I’m defining the interfaces first before diving into the guts of the logic. In that sense it’s kind of “test-driven” except all my testing is manual.

I do think having such a systematic, incremental method of implementation is a good skill for the junior programmer to learn though. My experience is that many younger programmers (especially those fresh out of college) tend to write huge chunks of code/logic/functionality then get surprised when the very first thing breaks down.

Recently when sitting down with a junior to help them walk through some problem I often show them how I debug problems this way. I trace the entire flow from start to end, adding debug messages as I go, to help identify at which step the process goes awry. I also try to describe my proposed approach for complex new functions this way. I’ll prototype a small part of it from frontend to backend, writing debug messages all the way, and it conveniently leaves the junior with a template for writing the rest of the functionality. (Although I am as of yet unable to attest to the efficacy of this approach)

I’ve been writing code for a long time, and while I wasn’t self-aware enough back when I started out to document how I was writing software, I like to think I’ve improved a lot since then. And for sure there’s still a lot to learn. Maybe some years from now I’ll look back on this blog post and chuckle about how naive I was and write a new blog post detailing an even better approach.

Software Development and Government

Random thoughts while walking at night: The structure of government can be a bit analogous to the structure of a software development project.

The Constitution is like the requirements for a project. It’s kind of high-level and (I believe) shouldn’t be too detailed. Supposedly the requirements are written by the client. For a country like the Philippines the client is “we the sovereign Filipino people”.

Slight tangent: I used to know this guy who was one of those rabid “we need to amend the constitution” types and he asked me to review a “mathematical model to track the budget as a function of tax collection and monetary policy” that he wanted to include in a proposed new constitution. I told him I didn’t believe such detailed rules shouldn’t be in the constitution – that would unnecessarily tie us down to a specific model that we may or may not regret later on. I ended up declining to help him with his strange hobby.

The Legislative branch are the analysts. They make the detailed specs/laws to define how we’re going to satisfy the requirements/constitution. In a real project, the client usually signs off on the specs. In government, we supposedly sign off when we vote for the legislators.

The Executive branch are the developers. They’re responsible for implementing the specs/laws in order to fulfill the requirements/constitution. They can decide the technical approach to use, as long as they satisfy the specs and the requirements.

The Judiciary is the QA, they review the work of the developers/executive and make sure they’re following the specs/laws and adhering to the requirements/constitution. Sometimes they even have to review the specs/laws themselves to make sure they aren’t contradicting the requirements/constitution.

Sometimes a project is so large and you need to form smaller subteams to specialize in different areas or modules. That’s called Federalism. Or maybe local government.

Sometimes one person handles specs, development and QA. That’s called dictatorship.

Sometimes people want to rewrite or amend the requirements/constitution. That’s called agile development. Or maybe scope creep.

Sometimes the bureaucracy needs to be reorganized to be more efficient. That’s called refactoring.

The big difference between government and software projects is that in software projects, team members are selected based on meritocracy. In theory at least, you choose the best people for the job. In government, who gets the job is determined by who is best at bamboozling the largest number of people to vote them in. Maybe someday we’ll get a meritocracy in government too, but until then I’m probably sticking to software development.

Web Frameworks – Open Source or Roll Your Own?

A while back I wrote about my experience coding and maintaining an in-house web framework at a previous job. It was a full-stack web framework. We had libraries for front-end Javascript up to server-side database connections. And the entire stack was tightly coupled. But while the framework was serviceable, it was almost always behind modern trends in web development. I always felt like we were playing catch-up. And as a developer I wanted to widen my horizons and try out more things. So more than once I had discussions with higher management about using open source web frameworks in some projects.
Unfortunately, our in-house web framework already had a long history and most of the devs in our company were used to it. The company had tried using a different Java-based framework stack before. It was back in the days when things like Struts, Spring, Hibernate, etc were beginning to ramp up. It kind of ended in disaster – that project ended up taking a lot of effort, had a lot of technical problems, and so on. I believe this gave the company leadership the impression that investing in other frameworks are not worth the risk and effort. It’s a form of Not Invented Here syndrome.
I admit there are some advantages to having your own in-house web framework. After all, all the popular web frameworks today started out in-house in some company and later they decided to release as open source. And many companies do fine using in-house frameworks. Using an in-house framework means full control over the behavior. And you can tailor the functionality and coding style to your internal processes. In fact it can be a good value add to your company if your in-house framework did something better or unique compared to open source ones.
But there are also significant advantages to using open source frameworks. Maybe I should have used some of these during discussions when I was there:
  • With an open source framework, we wouldn’t have to maintain the core functionality of the framework ourselves. We wouldn’t have to maintain the full stack ourselves. We would only need to maintain any customisations and extensions we write for our own needs. For our in-house web framework, every so often we’d spend some effort to come out with a new version with incremental improvements. With an open source framework, we could instead redirect that effort to higher-value work.
  • There’s more learning material available online, and they don’t have to be maintained in-house.
  • There’s a wider base of experience to draw from. For our in-house framework if you encountered a problem requiring deep framework knowledge, there were only a handful of high-level experts in the company. For an open-source web framework, that expertise is widely available on the internet through sites such as Stack Overflow.
  • Having knowledge/experience in established frameworks means your company can get more contracts. You can take on projects that use those frameworks. You don’t have to spend proposal space trying to convince clients that your in-house framework is great. With open source frameworks you can reuse marketing copy by someone else!
  • Working on well-known, established frameworks is better for your developers career-wise. It gives them more knowledge that could be transferable to other jobs. While this isn’t a benefit to the company per se, it will make the company more attractive to developers. It even allows the flexibility of hiring developers experience with that framework.

There are some disadvantages too of course:

  • As mentioned above, there is time and effort involved in learning a new open source framework. This effort is mostly for the experienced developers – for new hires they will need to learn something regardless
  • Having an in-house team developing your own framework means you have a core set of developers experienced in the full stack. Reliance on open source frameworks means most of your developers won’t be familiar with the low level details.

In the end, there’s no guarantee that using an open source framework will be painless or be better than developing one in-house. So I can understand the decision to stick with what you know. But for me as a developer, I feel that it’s more rewarding to be exposed to different frameworks.

In fact I wrote this post because recently someone asked me what my “go-to web framework” was and I said I didn’t have one. I’d rather be flexible enough to learn and work with any existing framework. In our industry where change happens quickly and can catch you by surprise, I think that flexibility is a much more valuable asset to have.

MarkLogic NoSQL

I recently attended a few training sessions for MarkLogic held at an office in a nearby business center. Now, I’ll forgive you for not knowing what MarkLogic is, as even I hadn’t heard of it before six months ago. MarkLogic is (apparently) the leading Enterprise NoSQL provider.
NoSQL is big and sexy right now because of the supposed advantages in handling big data, and large web companies like Google and Facebook use a lot of NoSQL in the backend. Most of the popular/well-known NoSQL solutions are open-source/free ones: MongoDB, Cassandra, CouchDB, and so on. But these aren’t actually very popular on the enterprise side, hence “Enterprise NoSQL” isn’t a very common phrase.
One of the reasons NoSQL isn’t currently very popular for enterprise projects is that the popular open-source solutions such as MongoDB don’t guarantee ACID transactions. In fact, MongoDB has the concept of “eventual consistency” for their distributed servers, which implies that they don’t have real-time consistency.
MarkLogic does guarantee ACID transactions, along with government-grade security. Both of these are things that enterprise clients love. So that’s the market they’re in. Their highest-profile project to-date was the backend datastore for (also known as Obamacare/ACA). That project involved consolidating data with different structures from multiple sources and needed to scale up to a ridiculously high capacity, so it seemed tailor-fit for a big data level NoSQL solution.
During the second-to-last day of the training, Jason Hunter, CTO of Marklogic for Asia-Pacific, dropped in to answer some questions. He talked a bit about how MarkLogic started and how they got him on board, and a bit about their competition (some disparagement towards MongoDB and Oracle) and about
He also had a really good sales pitch about why NoSQL is just a better approach compared to RDBMS. (Although it would have been more fun if he had given this talk with like an Oracle salesperson there to debate with him.)
One of his points was that RDBMS restrictions such as limits on column size are outdated. They were sensible in the old days when disk space was at a premium, but these days you don’t need to limit how many characters you store in a last name field. MarkLogic stores all records as documents with no file size limit (AFAIK) to avoid such an issue. From experience developing web forms, there’s the occasional client or system user who doesn’t understand why we need to have character limits on fields. We also had this system where they wanted to be able to type unlimited-length rich text content (we stored it as HTML in the backend) in memoboxes. These were fields where they had to write stuff like notes and assessments and most of their data was pages of text. I feel like that sort of thing would have been a great use for NoSQL.
Another point he raised was that RDBMS systems were very bad at allowing the user to search like three columns at random. You needed to know in advance which three columns to index. For document stores like MarkLogic, typically the entire document is indexed so that problem is avoided. Of course, that means that functions like “sorting by last name” need a bit more setup. You need to build a range index in MarkLogic to sort by a specific field. So it’s kind of a trade-off either way.
Now to be fair, Oracle does support indexed full-text search over any number of columns via Oracle Text. But it’s not the default behavior and definitely not straightforward. I used to work with Oracle Text a lot in some of my older projects, and the amount of time it took to index any nontrivial amount of data often gave us a headache.
I should do a test sometime to determine how well MarkLogic’s indexing performs. The story goes that MarkLogic started out as a document search application before they changed gears to become a database. They even approached VC’s with the intent of competing with Google. They’ve had a lot of time to get good at this, so I have high expectations.
The MarkLogic server itself is an interesting piece of engineering. It’s basically a document store, a search engine and an application server all rolled into one package. Upon installation you get some administrative web applications for configuration purposes. The admin interface seems robust and thorough. Contrast that with Oracle where you often find yourself needing to tinker around with configuration files and such
You can run web applications on the MarkLogic server itself. The supported languages are XQuery and server-side JavaScript. Odd choices I know. I suspect due to historical reasons they started out with XQuery, but the SJS side has the same capabilities (or so we’re told). If you’re not a fan of either option, you can also expose the server’s functionality via a REST interface. They also provide existing Java and Node.JS APIs on top of that REST interface. All of this means you can deploy any kind of webapp in front of MarkLogic server.
The world is moving towards bigger data stores, so it’s not unreasonable to think that NoSQL is on the way up and will be big players in the future. So I think the training was worth it (even if I did have to stay in Ortigas for a while). It’s early still. MarkLogic might still turn out to be as painful to work with as Oracle was. But at the very least it’s interesting to try a different approach to enterprise data storage. Looking forward to see what kind of applications we can build with this tech.

Google Code Jam and Competitive Programming

Back in 2004, I signed up for the Google Code Jam for the first time. Unfortunately I didn’t make it past the qualifying round.

I was a bit luckier in 2008 and 2010, making it to round 2 both times. In fact in 2008 as I recall I was one of only two participants from the Philippines who made it to round 2, which allowed me to jokingly brag about being the #2 programmer in the country.

I registered again a few times after that, and tried again last year (2016), where I only made it to round 1.

I’m no stranger to programming competitions, although the GCJ was the only one I’ve ever really participated in. Back in college, one of my friends usually joined such competitions, and I had a good time going over some of the problems with him.

Programming competitions don’t usually reflect the kind of programming done in most real-world jobs (although it might be interesting to use competitive programming problems in whiteboard interviews and see what happens). They focus on pushing the boundaries of the programmers talents in problem solving. You also need to be fairly familiar with algorithmic approaches to searching, sorting, and so on. Having a decent background in math also helps a lot.

The Google Code Jam in particular focuses on scaling up your code to handle very large amounts of data – well past the integer limit sometimes. Each question usually has a small and a large input. For the large inputs, you only have a short amount of time (I think it was 5 minutes?) to submit the output. Presumably you wrote and tested your code before starting the timer, so the short time is really only for your code’s execution. You can afford to be inefficient or maybe even brute force your way through the small input, but if you want to solve the large inputs on time you have to be able to optimize your approach.

When I first signed up, I dreamed of winning the entire thing (and of course being able to brag about being the best programmer in the world). But realistically, a lot of the people joining these competitions are “pros” more or less, so the chance of a “casual” like me winning are not that high. They are “pros” not in the sense that they do it for a living, but more of they do a lot of these competitions and regularly practice for them. Websites like TopCoder provide such competitions regularly for those who are interested. Also, I feel like I am very old and rusty, having worked too long in web development stuff where my focus is usually on a different kind of problem.

Still, I think it’s a good idea for programmers of all skill levels to give it a try, just to see where you are at. Who knows, you might even make it all the way! The next Google Code Jam starts soon – the qualification round is on Apr 7. I signed up, although I’m unsure if I will have the time this year even if I advance, due to scheduling problems. But a man can still dream, right? Who knows, maybe I’ll see some of you there too in the final round!

Fake Devs and Whiteboard Interviews

Recently, a developer needed to undergo a tech interview at US immigration:

This may surprise some people I’ve worked with, but I didn’t have formal computer science training in school. I’m not actually a computer science major. Yet I’ve worked as a software developer for more than a decade now. Literally zero times have I needed to write a sorting function or balance a BST.

I have a rudimentary understanding of some sorting algorithms (mostly just bubble sort and selection sort), and I have some idea of how to balance a BST. And if I needed to implement any of those I could probably whip something up after a few tries. (Something buggy and/or inefficient probably.) If I had to interview at a company that used one of these whiteboard interviews, there’s a good chance I wouldn’t pass.

For maybe 90% (or even as high as 99%) of modern-day programming work, it simply isn’t needed to know these things by heart. Those skills are more necessary if you’re doing low level things like writing an OS or a graphics driver or such. For everyday web development or Java or Rails or whatever, it’s an indulgence. And even for those instances that you need them, well, the internet is right there.

I’m not a fan of the idea that not knowing data structures and algorithms means you aren’t a real software developer. I have a couple of developer friends who jokingly refer to themselves as “fake devs”. It’s supposed to mean someone who doesn’t have any formal computer science background and has at best a passing familiarity with data structures, algorithms, and other computer science concepts. As one of them stated “I wouldn’t know how to make a map without using a map.” (I thought about writing a post about that, but it’s surprisingly easy – just look it up.). One of them even asked me to give them a quick crash course in data structures and algorithms because he was applying in a large and well-known software company.

For me, this term is silly. If you’re doing software development work, then you’re a software developer.

I’m not saying knowledge of these things isn’t useful. If you want to do more complicated work it is certainly useful to have these tools in your toolbox. Knowing them also makes you more flexible and expands your thinking. And certainly one should be the type of developer who is willing to seek out that knowledge if it turns out you need to use it.

But the problems with the whiteboard interviewing process isn’t that they require knowledge of these things; it’s that they require it under pressure, off the top of your head, without access to the internet. This doesn’t reflect how programming work happens in the real world, where you often have a team, an IDE, and even the entire internet to help you.

Unfortunately, it’s often far too costly for companies to evaluate you under real-world conditions, so many resort to this common interviewing process. After all, it is difficult to evaluate the performance of even developers you have been working with for months, what more someone who just whipped up a short function on a whiteboard in twenty minutes? We tried some variations on it at our last company, but for me it is a problem yet unsolved. Maybe someday…


Running Python 2.x and 3.x on Windows

I’ve been hesitant to try Python 3.x because it’s not backward compatible with Python 2.x which I’ve been using for scripting since forever. But recently I found out that since Python 3.3, they’ve included a launcher in the Windows version that supports having both versions installed.

You can use the launcher to specify the Python version to use at the command line (it defaults to whichever version was installed first):

Even better, the launcher will recognize a header in your .py files that can specify which version of python to use:

#!/usr/bin/env python3

If the launcher sees this header, it will automatically launch the appropriate python version. Handy!

Weekend Project: Twitter Trivia Bot

I had been meaning to try writing a Twitter bot for a while now. I figured a trivia bot would be pretty easy to implement, so I spent some time a couple of weekends to rig one together.

It’s (mostly) working now, the bot is active as triviastorm on Twitter, with a supporting webapp deployed on The bot tweets out a trivia question once every hour. It will then award points to the first five people who gave the correct answer. The bot will only recognize answers given as a direct reply to the tweet with the question, and only those submitted within the one hour period.

Some technical details:

My scripting language of choice for the past few years has been Python 2.7. I’m using Tweepy to interact with the Twitter API, PyMySQL to connect to the database, and Flask to run the webapp. I haven’t used Flask in some time, but it’s still very straightforward. I actually had a harder time configuring the webapp with mod_wsgi on my host.

The main problem with a trivia system is that you need a large and high-quality set of questions. Right now the bot is using a small trivia set –around a thousand questions I got from a variety of sources. If I want to leave this bot running for a while, I’m going to need a much larger trivia set. However, reviewing and collating the questions is a nontrivial task. Hopefully I can add new questions every so often.

Feel free to follow the bot and help test it out. I’d be grateful!

Integrating Open Source Libraries

There are a few things that one should consider when using and integrating an open source library into your application:

  1. What are the licensing terms for the library? There are some liberal licenses that mostly let you do anything you want. The MIT license is an example of a very permissive license. Other licenses may provide a number of restrictions. Can you integrate with closed-source software? Can you distribute binaries without the source? Do you need to put some kind of attribution somewhere in your software? Another thing to look our for are the so-called viral licenses. Viral licenses specify that if you integrate their code into your system, then the terms of that license apply to your own software as well. These can be very dangerous from the standpoint of a company developing a commercial product. The most well-known example of such copyleft licenses is the GPL. Integrating GPL code into your system will often mean your software needs to be open source as well. While more open source is good, it may not be in the best interest of your company, so tread lightly.
  2. Which version of the library will you use? Many open source libraries will release updates on a regular basis. There may be some compatibility issues depending on the version you use. For example the popular open source CMS WordPress releases regular updates. If you are integrating with WordPress, you need to decide which version you are supporting. You will also need a plan for upgrading to future versions and ensuring compatibility. (See the next item.)
  3. How will you handle bugs and updates in the version you’ve chosen? It’s happened that our system was having problems due to a bug in an open source Javascript rich text control library. After investigating, we found that a fix was available but was only applied in a future version. Upgrading to that future version would require that we test the entire system for compatibility problems. That was out of the question. I ended up having to apply a fix to our local version only. This meant that we now had a different version from the “official” released version. This may lead to further compatibility problems in the future. You should also be aware of licensing issues here too. The license may require that you submit any modifications to the community as open source contributions.
  4. Is there an active help and support system for the software? The demands of enterprise software development often need timely support. There was a time we had to use a certain open source document management system. We later discovered that it was very hard to get help on any issues encountered, either from official channels or the community. This led to many difficulties and issues in that project. You might also want to check popular programming forums such as StackOverflow. Check how many questions about that library usually remain unanswered. If the number is low, that’s a good indicator that you might have trouble looking for help later on.
  5. What dependencies does the library have? You might want to make sure the dependencies don’t have any problems with the previous items above.

The great thing about modern software development is that we have a large number of open source libraries available to use and draw inspiration from, but that doesn’t mean you can just willy-nilly pick up just any library. The above questions provide a good starting point for evaluating open source libraries for your use.

Working with Client-Server Programs

Back when I was starting out as a software developer, webapps weren’t really a thing. Not as much as they are now anyway. My company provided training to new hires, but I didn’t get any web development training at the time, even though they already had a few web development projects in play at the time.

Instead my initial training involved mostly development of so-called client-server software. This was software that was installed and run on the client machine but they would connect to a remote database server. Up until the early 2000s most enterprisey-type systems used these kinds of program.

I was trained in using mostly two tools:  Borland Delphi and Oracle Forms and Reports. These were the sort of tools that would have been billed as reducing the need for specialized programmers or developers. They featured drag and drop user interfaces to design forms or report layouts. The Oracle tools featured robust database integration that let you drag and drop to associate form fields to database tables and fields. Supposedly even people with minimal training would be able to do the work of application programmers.

In practice of course, the only people patient enough to work with these tools in-depth were the programmers themselves. And client requirements always eclipsed the capabilities of these tools, such that expert programmers were still needed to push the tools to their limits and beyond.

Tools like Oracle Forms and Reports had a lot of problems with modern software development practices. For one thing, the “source” files weren’t actually text files. They were mostly binary format files with some PL/SQL interspersed, and could only be opened in the proprietary IDE that only Oracle provided.

Binary formats meant that while they could reap some of the benefits of source control (namely version history and such), actually figuring out the differences between revisions involved opening up both versions in the IDE and comparing each item/script/setting in the two files. Doing things like global find and replace were also a pain in the ass, especially in systems that had hundreds of forms. It was so bad I remember spending some time trying to reverse engineer the binary format so that I could attempt to make some of this work easier. No dice.

Another issue was that Oracle’s IDEs were notoriously buggy. Their Reports IDE in particular often had me incensed. If you dragged the wrong thing to the other wrong thing, you would crash the IDE entirely. In fact, we had a list of things to avoid doing that would crash the entire IDE entirely. The built-in text editors were so bad, I often had a entirely separate text editor open in a separate window so I could copy-paste PL/SQL code there for any complicated edits. (Back then we were for some reason fond of Crimson Editor although it wasn’t as full-featured as UltraEdit or as reliable with updates as Notepad++) This was actually also sometimes a problem as some versions of the Reports IDE also had a crash on paste.

Most of my first two years of work involved maintenance of Oracle Forms-based systems (with some Delphi work thrown in occasionally). I didn’t get an introduction to web development until late into the second year of my career. I was so impressed with web development I made a mock-up of one of the really complicated screens from one of our Oracle Forms projects and it seemed so pretty. I secretly hoped we could convince our clients to port them over to a web application. (1) No dice again; and (2) I probably would have regretted such a thing.

I know I would have regretted it because some number of years later, we got a project to migrate an existing client-server system to a web application. The original application was written in Powerbuilder. I figured it was a fairly straightforward reverse-engineer and implement as a webapp, but nooooo. One of the higher-ups decided that to save on costs, we should look into attempting to automate the porting process somehow. We were to write translation software that would take the Powerbuilder source and convert it into the appropriate web application screens.

This was ridiculous for a good number of reasons, but as a tech lead, I had to look into it and had to prove it was in fact not viable (Spoiler alert: it was not). The first concern was that Powerbuilder also had a binary format. This was solved when I found that someone had already written a tool to export Powerbuilder binaries into some kind of text format. It was a bit crude, but it was a way forward.

The second concern is that client-server form behaviors do not map well to web applications. My favorite example of this is editable grids. These are some kind of excel-like grids that were commonplace in client-side systems. And when users upgrade their legacy systems to web apps, they inevitably expect that their editable grids should work the exact same way on the web as they did on the client-side forms.

Back then Javascript-backed editable grids were still in their infancy, and each one we tried had their own sets of problems and limitations which would lead to some number of client complaints. Some clients even wanted editable grids that could accept copy-pastes from excel! Yeah, we had to develop Google Sheets! I personally had to roll my own editable grid framework from scratch at least once too. I ranted so often against client’s editable grid requirements that a few developers often quoted something I said in one of our internal chat rooms:

    <jaeger> you say “editable grid” i hear “problems”

Anyway yeah, porting client-side behavior to a web framework would not have been straightforward. At least not to the way our internal web development framework liked to do things. In order to have any kind of reasonable mapping, I would have needed to build some kind of Javascript rich-client framework (or see which ones of the emerging ones we could use). That also meant rewriting a lot of the backend code to support a bunch of new operations. It was a lot more work than the expected output of “a series of steps to easily or automatically migrate PowerBuilder source code to web applications.”

When I raised that the idea was not viable, I was asked to give a more detailed explanation. This involved taking one of the more complex Powerbuilder source files, putting it into an Excel, and line-by-line explaining how and why that line could or could not be translated into a web application. As an alternative option to an automated porting program they wanted a guide or process so simple that even junior developers with minimal experience could follow the guide and port the programs quickly. I think they eventually went with this approach. (Which was closer to our normal reverse-engineer-and-reimplement methodology.)

These days web applications are the norm and client-server programs are a thing of the past. They were relics of a different age. The browser is our universal client now, on the desktop at least. On mobile, “apps” are effectively client-server programs, though they have a different set of advantages and limitations. Maybe in the future mobile apps will converge as well, into a unified browser client – although probably mobile browsers need to be more feature-rich before this convergence can take place. My historical disdain for client-server programs carries over to mobile apps though – I don’t like to use too many of them, although that is a story for another blog post.