Boxes

We put people into boxes because it is convenient. It’s easier for our mental model of the world to say to yourself things like “This guy works with computers, maybe he can tell me how to fix my printer.” or “This person is from [school] and they are very arrogant.” or “You’re from [country]? You guys do [that country’s thing] right?” or  “This person is a supporter of [politician] so he must support all the things that politician does, even the things I hate.” Or in the modern world, there are even worse stereotypes.

Boxes and stereotypes are convenient because it means we don’t need to relearn our reactions and opinions when presented with a new person. We simply choose a convenient box to put them in and then we convince ourselves we are treating them appropriately.

But the real world is not so discrete. The real world is a lot more granular, a lot more nuanced. While it is inconvenient for your mental model, people will usually not fit exactly into any of your preconceived boxes. Or they may fit in multiple boxes. Or maybe none at all.

Every time you put a person into a box, you discredit her by some amount. In your mind, you shave off some of who he is a person so that he better fits your box. And for many instances, maybe that’s okay. Maybe you don’t have any significant interaction with that person, and the simpler mental model is sufficient. You don’t need to understand the character nuances of every grocery store cashier for example.

But beyond that, in order to pursue any kind of meaningful interaction with another person, that is insufficient. We must be willing to set aside whatever box you have put a person into and be willing to understand all those little bits we set aside. Those nuances characterize and define the individual, not whatever box we have put him into.

Weekend Project: Twitter Trivia Bot

I had been meaning to try writing a Twitter bot for a while now. I figured a trivia bot would be pretty easy to implement, so I spent some time a couple of weekends to rig one together.

It’s (mostly) working now, the bot is active as triviastorm on Twitter, with a supporting webapp deployed on http://trivia.roytang.net/. The bot tweets out a trivia question once every hour. It will then award points to the first five people who gave the correct answer. The bot will only recognize answers given as a direct reply to the tweet with the question, and only those submitted within the one hour period.

Some technical details:

My scripting language of choice for the past few years has been Python 2.7. I’m using Tweepy to interact with the Twitter API, PyMySQL to connect to the database, and Flask to run the webapp. I haven’t used Flask in some time, but it’s still very straightforward. I actually had a harder time configuring the webapp with mod_wsgi on my host.

The main problem with a trivia system is that you need a large and high-quality set of questions. Right now the bot is using a small trivia set –around a thousand questions I got from a variety of sources. If I want to leave this bot running for a while, I’m going to need a much larger trivia set. However, reviewing and collating the questions is a nontrivial task. Hopefully I can add new questions every so often.

Feel free to follow the bot and help test it out. I’d be grateful!

Write Smarter, Not Harder

There’s this well-known idea that it takes ten thousand hours of practice to become an expert in something. But of course, it has to be ten thousand meaningful hours of practice. Meaningful here means that you are actually learning something from your practice. If you are repeating the same hour ten thousand times, that’s not worth very much.

Instead, we should be actively learning while we practice. This means identifying our weak points and learning how we can improve. It requires that we have a feedback loop that tells us whether we are doing well or not. It also means taking on bigger and more challenging tasks. Only then can we walk the path to expertise.

As an example, I’ve been posting on this blog on a regular basis since 2016. Aside from that, I’ve also been trying to write fiction in my spare time. So I figure I’m getting a lot of writing practice and I’m doing well, right?

I recently took an online exam about how well I would do in editing work. Now, I’m pretty confident in my grammar and spelling, so it was no surprise that I got perfect scores in those categories. Yet I got poor scores in diction and sentence parallelism and style and such things. My overall score was still above average, but the test highlighted some weaknesses that I can work on.

To help with those, I’ve started going through the Chicago Manual of Style. It’s a bit boring, especially near the start, but I’m learning a lot. I also found out about this webapp Hemingway which scores your writing for readability issues. I’ve found it pretty useful – already I can see that my blog posts use way too many adverbs and have a lot of hard-to-read sentences.  My 2016 Nanowrimo manuscript got a higher score on this app compared to my posts though. (But I’m not proud of that work for nontechnical reasons haha). I’m also looking into a few other tools that can give me some feedback on how good my writing is.

The best feedback of course, would be from those of you who for some reason deign to read my posts. I already corrected my tendency to not end paragraphs with punctuation since someone called me out on it. So I would appreciate any further feedback about how I write and where I can improve.

Review: Duelyst

After Hearthstone, I tried out a couple of other digital CCGs: Spellweaver and Eternal, but neither one hooked me. The one I enjoyed the most and did pick up to play regularly was Duelyst. So this review is written from the perspective of someone who has played both Magic the Gathering (MTG) and Hearthstone (HS).

Hearthstone, Spellweaver and Eternal played like digital MTG with some advantages, as I outlined in the HS post linked above. Duelyst keeps many of the same elements and advantages, but adds an extra dimension. Besides being a card game, Duelyst is also a board game.

Board game

In a game of Duelyst your General (the analogue to Hearthstone’s hero) and his minions are played into a 9×5 board. Minions can only be summoned on empty spaces adjacent to one of your units. During your turn, each unit can either: attack an adjacent enemy; or move up to two spaces, then attack an adjacent enemy.

Having a board and unit movement greatly enhances the “positioning matters” mechanic as compared to Hearthstone. In HS, it only mattered whether minions were adjacent, to the left or to the right. In Duelyst, you can position minions to prevent enemies from reaching your General, or to restrict enemies from moving completely.

That means a number of spells and abilities care about positioning too. Each of the available Generals has a Bloodborn spell (BBS), the equivalent of Hearthstone’s Hero Power. One of the Generals has a BBS which allows her to deal damage to all enemy units in the same column as the opposing General. This means you have to be careful where you place your minions whenever her BBS is active! There are spells that affect a small area (2×2 or 3×3 and so on). There are spells that care about adjacency (“Destroy target minion that is not nearby any general”). And so on.

Factions and Generals

Hearthstone has different heroes, and each hero has a different card pool available. By contrast, Duelyst has factions and Generals. Each faction has its own card pool, and of course there is a global or “neutral” card pool as well. Each faction also has two Generals, each of which has a different BBS. The factions and Generals encourage a lot of different playstyles.

Some factions like the Songhai are more focused on spell-based damage and backstabs. The Lyonar focus on cheap, efficient creatures. Vetruvian has obelisks that can generate temporary minions. Abyssian focuses on swarm tactics and shadow creep. The Vanar rely on placement and trickery. My favorite General, the Magmar Vaath, likes to go toe-to-toe with the enemy General and punch him in the face!

Keywords

As expected, many of Duelyst keywords care about positioning, and thus have no meaningful equivalent in either MTG or HS. Namely:

  • Airdrop – minions with Airdrop can be summoned anywhere on the board
  • Backstab – this unit deals extra damage when attacking from behind (yes, even facing matters!) and doesn’t receive a counterattack
  • Blast – attack hits all enemies in the same row or column
  • Flying – may move anywhere on the battlefield
  • Frenzy – normal attacks hit ALL adjacent enemies
  • Infiltrate – gains bonus effect if its on the enemy’s starting side of the board
  • Provoke – somewhat like HS’s Taunt. Adjacent enemies cannot move and must attack a Provoke unit if there is one nearby
  • Ranged – can attack from anywhere on the board
  • Shadow Creep – this is a modifier that can be added to board tiles. An enemy standing on Shadow Creep takes 1 damage at the end of the Shadow Creep owner’s turn
  • Zeal – gains bonus effect as long as it’s next to the General

Duelyst also has a number of keywords that are analogous to abilities in MTG and HS. Rush is the same as MTG’s Haste and HS’s ChargeOpening Gambit and Dying Wish are the same as HS’s Battlecry and Deathrattle. (I worry that at some point new cardgames will run out of names for “enters the field” and “leaves the field”.) And so on – no need to cover everything here.

Skill Level, Competitive Play and F2P

The added dimension of positioning means that Duelyst is strictly more skill-based than HS. This is obvious if you note that there are a lot more choices for where to place minions. Hearthstone is also more reliant on RNG than Duelyst is, although I can’t say that will hold for the future. I enjoy some level of RNG as it creates variance, but the randomness means a higher chance of losing due to bad luck.

I find competitive play on the ranked ladders for both Duelyst and Hearthstone to be about the same. I’m not saying either meta is healthy, but as I see it there are more viable decks in Duelyst ladder as compared to HS.

Since I’ve already spent a bunch of money in MTG, I don’t like spending money on digital card games. So for both HS and Duelyst, I am strictly free-to-play (F2P). Both games offer similar monetization models (pay money for booster packs/cosmetics) but I was able to get further up the ladder in competitive Duelyst.

In my first full month (December), I was already able to get to Duelyst’s Gold Division. This should be similar to around rank 10 in HS. After a lot (and I mean a lot) of grinding from Diamond rank, I managed to hit the S-Rank Division last month. This is the equivalent of HS’s Legend rank. The highest rank I ever got in HS was around rank 6 or 7.

If I think Duelyst is more skill-based than HS, why did I find it easier to get to S-Rank than HS’s Legend rank? Two reasons come to mind:

  1. I found Duelyst games more fun. This means I was playing Duelyst on a daily basis since it started. At the start, I played regularly because I wanted to get Steam Achievement completion. But I found myself enjoying the game and continuing to play even after that. There are also a lot more variance in decks being played on the Duelyst ladder. By comparison, HS ladder seemed to be almost 50% the one best deck (used to be Midrange Shaman, nowadays Pirate Aggro…) while the other 50% are trying to beat it.
  2. It was easier to get the powerful cards and build the tier one decks in Duelyst. The pricing of packs is about the same (100 gold), but on average I could earn around 100-150 gold per day from Duelyst’s quests, compared to 40-60 for HS. The drop rates for Legendaries (highest rarity in both games) is higher in Duelyst as well. Unlike HS, Legendaries in Duelyst are not restricted to one per deck, you can play as many as three. This meant you need more of them, which annoyed me at first. But I think that’s counterbalanced by the higher drop rate. For comparison, I’ve been playing HS for more than a year and have only gotten two premium (golden) Legendaries. Three months into Duelyst and I’ve opened the same amount of premium (prismatic) Legendaries. (Well, this comparison may be skewed since I open more Duelyst packs due to the increased gold rate.) Another factor was the Bloodborn expansion, which you could complete by playing enough to grind 3900 gold. That expansion came out in December, and I completed the set early in January. This set gave me access to many useful cards that were important in the meta. This allowed me to build a tier one deck and reach S-Rank in January. Even after that deck was nerfed, I was also easily able to shift to another tier one deck from another faction. In HS, I still haven’t found a new competitive deck after Midrange Shaman.

The Future

At first I told myself I would stop playing Duelyst after I completed the Steam Achievements. Then I said I would stop playing after I got to S-Rank. But I’m still playing it now. By comparison, I now launch HS once or twice a week, and I don’t clear the quests.

Duelyst is still a young game – the second expansion just came out last December. So in the future it may yet be plagued with the problems Hearthstone currently has. The developers have shown themselves to be responsive in nerfing problematic cards and shaking up the meta though. Whether they are able to keep this up remains to be seen. In the meantime, I’m willing to keep playing as long as the game is still enjoyable.

 

Integrating Open Source Libraries

There are a few things that one should consider when using and integrating an open source library into your application:

  1. What are the licensing terms for the library? There are some liberal licenses that mostly let you do anything you want. The MIT license is an example of a very permissive license. Other licenses may provide a number of restrictions. Can you integrate with closed-source software? Can you distribute binaries without the source? Do you need to put some kind of attribution somewhere in your software? Another thing to look our for are the so-called viral licenses. Viral licenses specify that if you integrate their code into your system, then the terms of that license apply to your own software as well. These can be very dangerous from the standpoint of a company developing a commercial product. The most well-known example of such copyleft licenses is the GPL. Integrating GPL code into your system will often mean your software needs to be open source as well. While more open source is good, it may not be in the best interest of your company, so tread lightly.
  2. Which version of the library will you use? Many open source libraries will release updates on a regular basis. There may be some compatibility issues depending on the version you use. For example the popular open source CMS WordPress releases regular updates. If you are integrating with WordPress, you need to decide which version you are supporting. You will also need a plan for upgrading to future versions and ensuring compatibility. (See the next item.)
  3. How will you handle bugs and updates in the version you’ve chosen? It’s happened that our system was having problems due to a bug in an open source Javascript rich text control library. After investigating, we found that a fix was available but was only applied in a future version. Upgrading to that future version would require that we test the entire system for compatibility problems. That was out of the question. I ended up having to apply a fix to our local version only. This meant that we now had a different version from the “official” released version. This may lead to further compatibility problems in the future. You should also be aware of licensing issues here too. The license may require that you submit any modifications to the community as open source contributions.
  4. Is there an active help and support system for the software? The demands of enterprise software development often need timely support. There was a time we had to use a certain open source document management system. We later discovered that it was very hard to get help on any issues encountered, either from official channels or the community. This led to many difficulties and issues in that project. You might also want to check popular programming forums such as StackOverflow. Check how many questions about that library usually remain unanswered. If the number is low, that’s a good indicator that you might have trouble looking for help later on.
  5. What dependencies does the library have? You might want to make sure the dependencies don’t have any problems with the previous items above.

The great thing about modern software development is that we have a large number of open source libraries available to use and draw inspiration from, but that doesn’t mean you can just willy-nilly pick up just any library. The above questions provide a good starting point for evaluating open source libraries for your use.

War

As of today, our country (The #Blessed Republic of the Philippines) is already at war with:

  1. Drugs
  2. Illegal gambling
  3. Communist rebels

Some other things we might consider declaring war on (in no particular order):

  1. Poverty
  2. Ignorance
  3. Misinformation (sorry, “Alternative facts”)
  4. Abusive government officials
  5. Traffic
  6. Rights abuses
  7. Pollution
  8. High power rates
  9. Political dynasties
  10. Poor quality of local cinema offerings
  11. Politicians putting their names everywhere
  12. Internet trolls and bullies
  13. Lack of critical thinking
  14. Redundancy
  15. Overtime without overtime pay
  16. Government officials blatantly lying or pulling statistics out of thin air
  17. Slow and expensive internet
  18. The MRT breaking down
  19. Cruelty to animals
  20. Poor quality of local anti-piracy ads
  21. Jejespeak
  22. SMS spam
  23. Typhoons
  24. Taxis that don’t give exact change
  25. War
  26. Irony
  27. Spoilers
  28. Pineapples on pizza
  29. Poor grammar and/or spelling
  30. Hashtags
  31. Hypocrisy
  32. Multi-level marketing
  33. Working at “Edi sa puso mo”
  34. Redundancy
  35. Low effort blog posts that start out serious but end up trying a bit too hard to be funny
  36. People who don’t understand sarcasm
  37. People who stand in malls and shove fliers in your face
  38. Commenting on posts without reading the actual article
  39. Lists that end abruptly at weird numbers so you’re not sure if there’s more or what

Working with Client-Server Programs

Back when I was starting out as a software developer, webapps weren’t really a thing. Not as much as they are now anyway. My company provided training to new hires, but I didn’t get any web development training at the time, even though they already had a few web development projects in play at the time.

Instead my initial training involved mostly development of so-called client-server software. This was software that was installed and run on the client machine but they would connect to a remote database server. Up until the early 2000s most enterprisey-type systems used these kinds of program.

I was trained in using mostly two tools:  Borland Delphi and Oracle Forms and Reports. These were the sort of tools that would have been billed as reducing the need for specialized programmers or developers. They featured drag and drop user interfaces to design forms or report layouts. The Oracle tools featured robust database integration that let you drag and drop to associate form fields to database tables and fields. Supposedly even people with minimal training would be able to do the work of application programmers.

In practice of course, the only people patient enough to work with these tools in-depth were the programmers themselves. And client requirements always eclipsed the capabilities of these tools, such that expert programmers were still needed to push the tools to their limits and beyond.

Tools like Oracle Forms and Reports had a lot of problems with modern software development practices. For one thing, the “source” files weren’t actually text files. They were mostly binary format files with some PL/SQL interspersed, and could only be opened in the proprietary IDE that only Oracle provided.

Binary formats meant that while they could reap some of the benefits of source control (namely version history and such), actually figuring out the differences between revisions involved opening up both versions in the IDE and comparing each item/script/setting in the two files. Doing things like global find and replace were also a pain in the ass, especially in systems that had hundreds of forms. It was so bad I remember spending some time trying to reverse engineer the binary format so that I could attempt to make some of this work easier. No dice.

Another issue was that Oracle’s IDEs were notoriously buggy. Their Reports IDE in particular often had me incensed. If you dragged the wrong thing to the other wrong thing, you would crash the IDE entirely. In fact, we had a list of things to avoid doing that would crash the entire IDE entirely. The built-in text editors were so bad, I often had a entirely separate text editor open in a separate window so I could copy-paste PL/SQL code there for any complicated edits. (Back then we were for some reason fond of Crimson Editor although it wasn’t as full-featured as UltraEdit or as reliable with updates as Notepad++) This was actually also sometimes a problem as some versions of the Reports IDE also had a crash on paste.

Most of my first two years of work involved maintenance of Oracle Forms-based systems (with some Delphi work thrown in occasionally). I didn’t get an introduction to web development until late into the second year of my career. I was so impressed with web development I made a mock-up of one of the really complicated screens from one of our Oracle Forms projects and it seemed so pretty. I secretly hoped we could convince our clients to port them over to a web application. (1) No dice again; and (2) I probably would have regretted such a thing.

I know I would have regretted it because some number of years later, we got a project to migrate an existing client-server system to a web application. The original application was written in Powerbuilder. I figured it was a fairly straightforward reverse-engineer and implement as a webapp, but nooooo. One of the higher-ups decided that to save on costs, we should look into attempting to automate the porting process somehow. We were to write translation software that would take the Powerbuilder source and convert it into the appropriate web application screens.

This was ridiculous for a good number of reasons, but as a tech lead, I had to look into it and had to prove it was in fact not viable (Spoiler alert: it was not). The first concern was that Powerbuilder also had a binary format. This was solved when I found that someone had already written a tool to export Powerbuilder binaries into some kind of text format. It was a bit crude, but it was a way forward.

The second concern is that client-server form behaviors do not map well to web applications. My favorite example of this is editable grids. These are some kind of excel-like grids that were commonplace in client-side systems. And when users upgrade their legacy systems to web apps, they inevitably expect that their editable grids should work the exact same way on the web as they did on the client-side forms.

Back then Javascript-backed editable grids were still in their infancy, and each one we tried had their own sets of problems and limitations which would lead to some number of client complaints. Some clients even wanted editable grids that could accept copy-pastes from excel! Yeah, we had to develop Google Sheets! I personally had to roll my own editable grid framework from scratch at least once too. I ranted so often against client’s editable grid requirements that a few developers often quoted something I said in one of our internal chat rooms:

    <jaeger> you say “editable grid” i hear “problems”

Anyway yeah, porting client-side behavior to a web framework would not have been straightforward. At least not to the way our internal web development framework liked to do things. In order to have any kind of reasonable mapping, I would have needed to build some kind of Javascript rich-client framework (or see which ones of the emerging ones we could use). That also meant rewriting a lot of the backend code to support a bunch of new operations. It was a lot more work than the expected output of “a series of steps to easily or automatically migrate PowerBuilder source code to web applications.”

When I raised that the idea was not viable, I was asked to give a more detailed explanation. This involved taking one of the more complex Powerbuilder source files, putting it into an Excel, and line-by-line explaining how and why that line could or could not be translated into a web application. As an alternative option to an automated porting program they wanted a guide or process so simple that even junior developers with minimal experience could follow the guide and port the programs quickly. I think they eventually went with this approach. (Which was closer to our normal reverse-engineer-and-reimplement methodology.)

These days web applications are the norm and client-server programs are a thing of the past. They were relics of a different age. The browser is our universal client now, on the desktop at least. On mobile, “apps” are effectively client-server programs, though they have a different set of advantages and limitations. Maybe in the future mobile apps will converge as well, into a unified browser client – although probably mobile browsers need to be more feature-rich before this convergence can take place. My historical disdain for client-server programs carries over to mobile apps though – I don’t like to use too many of them, although that is a story for another blog post.

Gaming Walkthroughs

Old gamer rants follow.

Gaming has changed a lot over the years.

For one thing, there’s the internet now. If you got stuck in a game, you just head on over to GameFAQs or some other site and someone on the message boards will tell you how to get unstuck. Or you can even watch Youtube videos on how to do it! (Side note: I dislike having to watch Youtube videos to figure stuff out. I read really quickly so I prefer some descriptive text.) (Side note #2: I have amazingly written about the youtube thing back in 2006)

Back in the olden days of gaming, there was no internet. All you had was word of mouth and lots of free time for trial and error. There was this guy we knew where whenever we came over to his house he would have some new Super Mario Bros. 3 tricks for us, we had no idea how he figured them out.

Sometimes you’d be lucky and you’d have a friend who had an issue of Nintendo Power or something and there’s a whole lot of secrets in just one issue and you wanted to show off to your friends so you tell them all these tricks. That’s how word of mouth happens.

Even fighting games like Street Fighter II, when they first came out in the arcades, the machines didn’t have movelists or anything like they do now. We had to guess. People figured out how to do fireballs by word of mouth. I remember the first time I played it in the arcade, I was jiggling the joystick randomly trying to figure out how to dragon punch. I had to read an issue of GamePRO to find out how many different throws Zangief actually had. Then GamePRO did an April Fools article where they claimed that there was a secret character Shen Long in the game (based on Ryu’s poorly-translated win quote “You must defeat Shen Long to stand a chance”). The article had screenshots and everything. What a cruel trick to play on a playerbase where secrets spread by word of mouth.

I finished Day of the Tentacle Remastered  on PS4 a while back. It was great. DoTT was one of the first PC games I ever played. I was in 3rd year high school when I played the first game Maniac Mansion at a friend’s house. Back then we didn’t have walkthroughs or anything so we could literally be stuck for days on one puzzle or whatever. We had to try all combinations of inventory items with verbs and environment items. The Remaster even had a command to highlight all interactible items in the environment, that would have been a big help back in the day! I don’t know how long it took me to finish DoTT back then, but I loved it so much I installed it in a secret directory on one of the computer labs in high school (we were the first batch to actually have a computer lab!). The Remaster I finished it in around 3-4 hours. I didn’t need a walkthrough to finish the game (I still remembered a lot), but I did for getting all the achievements (some of them were very obscure).

I’m actually surprised with how well the point and click adventure game genre is doing so well lately (what with Telltale leading the charge and all), as you really have to be disciplined with trying to avoid walkthroughs, otherwise what’s the point? Well, you could enjoy the story I guess, but it’s a point of pride to finish without a walkthrough, kind of. I’m still stuck somewhere on episode one of Telltale’s Back to the Future. I think I had to open a door or something. If only achievements could detect whether you used the internet to cheat.

When GameFAQs first came out, I was very active on those messageboards. In fact, I probably contributed to a few walkthroughs myself. The great thing about these sites is that they build communities of people actively trying to find secrets and explore every nook and cranny of a single game. I don’t have the time nor patience to do that sort of thing anymore, but I’m glad that kind of culture exists. It’s that culture that enables people to push games to the extreme – speedrunners, extreme challenges, etc. can be very entertaining to follow.

It’s actually a bit ironic that walkthroughs are more commonplace now, given that games are certainly a lot less difficult than they used to be. “Nintendo-hard” games are few and far between and appeal to niche audiences only. In the older Dragon Quest or Final Fantasy games you often had to walk around and talk to every available NPC until you find the guy who will move the story forward. These days that guy has an exclamation point over his head and a marker on the minimap and you can fast travel to him too.

To be entirely fair, when we were kids we didn’t really need walkthroughs because we just had so much damn time. I mean, we had enough time to try to hit every single brick in that stage of Super Mario Bros 3 that was basically pyramids full of bricks, just to see if any of them held secret coins. (Later we found a glitched cartridge that showed us where hidden blocks were, that sped things up a lot.)

I used to have a strong sense of pride when playing video games, I almost never used walkthroughs when playing through RPGs for example. But things change when you’re an adult playing computer games. You have real world stuff to do and not enough time, so you just say screw it when a game makes it too difficult for you. Either you give up on the game or you youtube that stuff.

There’s some sort of mystery lost when you’re playing a game and you know that anytime you get stuck you can just google as a fallback. Well, that’s the tradeoff I guess. In exchange though, we get a lot more online gaming content these days, not just walkthroughs: we get let’s plays and livestreams and speedruns and all that kind of stuff. Sometimes you don’t even have to be playing a game to enjoy the content, so that’s kind of cool too.

And you can still finish a game without a walkthrough…you just need the discipline to stay away from the internet.

 

Handling unexpected errors in web applications

So after so many months of development you deployed your webapp to production and it’s up and running and everything is fine and you celebrate and your work is done right?

Not really.

Two days later you get an urgent support call in the middle of the night. (Your clients are halfway across the world.) They’re asking why the website is inaccessible. You check via your browser and sure enough there’s an error 500. You have to ssh into the server. Half-asleep, you have to quickly comb through the log files to figure out if there’s something you can do about it right now or you have to sadly tell the client that the team has to handle it in the morning.

Nobody expects software errors. And nobody can predict how they will happen. (If we could do that, there wouldn’t be any errors!) But in general you should stay ahead of future headaches during development time by having robust error handling and reporting principles in place for your development team to follow.

There are two classes of errors:

  • Validation errors – checks on user inputs. These errors are expected, and the most important thing here is to present clear error messages that tell the user what was wrong and how they can correct it.
  • Everything else is an unexpected error, which we can breakdown further according to how the development team would react:
    • “That shouldn’t happen!” – the broadest category, usually caused by human errors in coding/configuration or hardware problems. Examples include: database failure, typos in configuration files, different data formats used by different developers, unexpected system crashes and so on.
    • “Hmm, oh yeah, we didn’t think of that.” – i.e., design flaws or unhandled cases. These are errors caused by program flow or usage that was not anticipated in the program’s design but are considered valid use.
    • “That can’t possibly happen!”, also popularly known as “We have no clue how that happened.” Most unexpected errors start out under this category then after some detective work (some developers use the term “debugging”), it transfers to one of the above categories.

Unexpected errors may also be either recoverable (the user can either attempt to repeat the operation to continue or use other functions in the system) or catastrophic (the entire system or subsystem is completely unusable.)

Ideally, what happens when an unexpected error occurs? There’s a number of things that probably need to be considered:

  1. The user needs to be aware that it’s not an error on his part and that the problem needs to be reported to support or the system administrator.
  2. The user needs to be presented with enough information to give to the support team to allow them to determine the cause and fix the error.
  3. The program needs to capture what information it can to help the support team find and fix the error.

For client-side errors (typically JavaScript problems, but can also be problems in HTML/CSS rendering), all of the above are difficult, since the mechanisms by which you present #1 and #2 and by which you can do #3 may be compromised by the error. Typically for any client-side errors, you are almost entirely reliant on the user’s description of the error and the steps that led to it – not the most reliable source of information, but you make do with what you have. If you are unable to reproduce the error during debugging, you are in for a whole lot of speculation and trial and error.

For server-side errors, it’s a bit easier than client-side errors. The first two items are typically handled by having a catch-all error page in the application that gives an error code and message for unexpected errors.

For the last item, it generally means logging, either to a file or a database-backed audit trail (the file is more reliable). Generally we log the error/exception encountered, including the stack trace, and some log messages on each request/page access. A majority of the time, having the stack trace is sufficient. The access logs allow us to back-trace previous steps which might have contributed to the problem. Any special conditions encountered in the code should also be logged – this helps to identify errors that can only happen under some combination of special conditions.

You don’t want to log too much information (what to log might need to be a separate post all on its own), but just enough to help the support team determine the cause of the problem.

Sometimes, the error is specific to the data being used by the users, or happens only in the environment they are using. And the support team investigating the problem will often not have access to that data or that environment, so your logging and the error reports have to be good enough to allow them to deduce the conditions that caused the error.

Error handling for deployed systems is already hard – so make sure not to make it harder on the future support team by having standard error handling and logging mechanisms.

12 Signs You Should Think About Leaving Your Job

Signs you think about leaving your current job:

  1. The company culture has changed in ways you don’t like or recognize
  2. You are no longer proud of the work you do
  3. You are always feeling tired, even when you just got to work
  4. You more easily notice your coworkers’ screwups and are more easily annoyed at them
  5. You feel unappreciated for the work you do
  6. You feel like the company doesn’t listen to your inputs
  7. Many of your close friends in the company are unhappy and want to leave or are already leaving or have already left
  8. You no longer trust your boss to make good decisions about the company’s future
  9. You are always looking for more free time to pursue other pursuits
  10. It’s Monday and you’re already looking forward to Friday
  11. You hate your commute
  12. You start reading articles about signs you should think about leaving your job

A lot of people feel trapped in their current job. They feel that they will be unable to find better or at least similar work and will rather stay than risk it. It’s a false sense of “job security” vs fear. It might be true that you have no good options. But if you’re not looking at all, it’s 100% surely true.

Even if you don’t find better options, the act of looking for them helps you through some sort of catharsis. It confirms if you really have no choice but to stay. It helps you understand your place in the job market and what you can do to improve your position and maybe open up more options later on.

Bottom line: if you’re unhappy in any way (as indicated by any of the signs above), you should at least be looking. Otherwise you are doing yourself a disservice.