The Promise of Social Media

When blogging and social media started to explode in popularity, it held a great promise: it would give a voice to the masses. Where previously there were gatekeepers in traditional media channels who controlled whose opinions could be published or broadcast, the internet meant that anyone with an internet connection could publish and voice out their thoughts and people would be all smart and there would be lots of intelligent discussion and it would be great.

And that did kind of happen – there was now a wealth of information and opinions and discussions online – but people being people there was a lot of silly, frivolous stuff too. Memes, jokes, 4chan, and all that. And that was all fine and great.

But there was some dark stuff too. Spammers and scammers and whatnot trying to take advantage of people. And more recently with all the political activity it seemed like it had come to a head. Lies, damn lies and propaganda. It had always been happening before, but now the scales were larger and the stakes were higher.

Instead of discussing opposing points of view, people joined social media groups  supporting one side or another, discussions giving way to echo chambers and crowds castigating people who didn’t share their views. Online trolls were paid to harass opposing personalities and reinforce their benefactor’s propaganda. Outright false information was spread using social media.

And it was our fault too. We seek out those opinions that feed our biases and don’t bother questioning their veracity. Many times we don’t even bother reading past the clickbait headlines and make judgments without all the information because who has time for all that context right? Many acted like fanatics where their side could do no wrong and the other side were demons who could do no right.

A friend of mine, disgusted with the outright lies and propaganda of the opposing camps, has started censoring comments he disagrees with on his posts. I felt that stifling discussion was counterproductive, but he insisted that in this new era, a wide variety of approaches was necessary, with both moderates maintaining balanced views and encouraging discussion; and fanatics pushing aggressively to counter the opposing propaganda. Perhaps it was true, but it lessened my hopes that we would be able to raise the standard of online discussion to a higher level.

When I started writing this post the original title was “The Unfulfilled Promise of Social Media”, but I realized the adjective there wasn’t appropriate. If anything, the promise of democratized discussion has been completely fulfilled – I had just forgotten that democracy did not necessarily mean people would be smarter or more enlightened. Just like in real-life we are vulnerable to charlatans and demagogues and lies and propaganda.

All the more reason we need to be vigilant with what we read, share, comment and post online. We need to give adequate consideration to opposing points of view, to weigh the facts against speculation. We need to learn and apply critical thinking, and to teach other people to do so. We should strive to elevate discussions to higher levels (although it can often be tiring to do so!). It is an ongoing struggle for me to improve and be critical of my own thinking, but I believe it to be a worthy cause.

Learning to SQL

Since I wasn’t a graduate of computer science, there were many concepts of software development I really only got exposed to when I started working. One of those was the concept of a relational database, and hence SQL. The company I worked at gave all new hires a training regimen that started with about a week of SQL. Despite not knowing anything about it beforehand, I took to it like a mouse takes to cheese. At the end of the week, they give you an exam to see how well you understood the material, and my batchmate who I took the course with likes to tell people that halfway through the allocated time for the exam, I was already taking a nap.

The fact that in my first project I had to port a lot of reports meant I was able to apply what I learned and learn even more easily. I would find as time went on that these bespoke projects typically included hundreds of different reports. The users would always want to see so-and-so data over time or compared with other sets of data, or summary reports of data over weeks, months, years, and so on. And when these systems are being developed, the reports are typically among the last to be finalized, which means that the database structure is almost always optimized towards the needs of the functions needed to maintain the data, and not towards reporting on that data. This typically led to reports that had really long and complicated SQL statements that would look intimidating to newbies. I didn’t mind, I treated each large SQL as a puzzle to be solved and understood.

About a year and a half later, I got tapped to be a trainer in the company for the first time, and I would be training people on SQL. Training other people gave me a better appreciation of how hard SQL was for other people. When I was in college, I understood that certain subjects could be considered as “filters” that many people will not pass and should take as a sign that the course of study was not for them. An example was the Introduction to Computer Science class that most engineering students have to take. Programming is hard and needs a certain mindset, so many people would fail or do poorly in that course, and if you were a Computer Science major it was a clear sign as to whether you should continue that course or not. SQL had that same property – I found out that it was a pretty good indicator of whether certain developers would do well in the company or not.

As I mentioned, the company’s SQL training course was a week long. You could typically tell which trainees would be having trouble by the third day – that was the day that joins would start to be discussed. Joins where were the men got separated from the boys as it were. SQL joins could not be treated the same way as other programming problems, they were not best solved in an iterative or procedural matter, but rather should be approached by an understanding of set theory. You had to figure out the particular set of results you wanted to retrieve and what set operations needed to be done on the data in order to get those results. Most of the trainees having trouble with SQL would find it difficult to wrap their heads around this concept.

Our SQL training was decidedly Oracle-specific, instead of adhering strictly to the ANSI syntax. We preferred teaching Oracle-specific syntax: DECODE instead of CASE, joining by saying “from TABLE1, TABLE2” instead of the more modern “from TABLE1 join TABLE2 on …”. I am not even sure if at that time Oracle already supported the more modern/standardized syntax.

Another thing about the SQL course was that we had to teach SQL performance tuning. It was a subject matter that was hard to teach because providing examples was so difficult. SQL that has a performance problem is typically long, complicated, and involves a number of tables that would require a certain level of domain knowledge. So we would end up providing simple examples and a list of tips for writing SQL (avoid sorts, full scans, distincts, etc) and we had to hope that the trainees would remember these tips when they start writing their own SQL for their projects. Even after a decade later, I would find myself glaring at developers who never learned to avoid using UNION instead of UNION ALL where possible.

For projects with a lot of reports, SQL performance was a big deal, especially when reports were expected to be generated interactively by the user online (as opposed to in the background by batch)

SQL performance tuning involved a relatively straightforward procedure: review the execution plan for the SQL (can be extracted from the database), and try to minimize the number of full table scans and unnecessary sorts. Oracle execution plans typically will even give you a cost for each operation, making it easy to identify which operations are more important to optimize. You also needed some idea of the volume of data on each table. It was more important to minimize full scans and sorts for larger data sets.

To remove full table scans, you typically had to introduce new indexes. You had to identify which WHERE conditions could benefit the most from an index. Some WHERE conditions could not use an index, and you would need to figure out if there was a way to rewrite the conditions that would allow an index to be used. Minimizing sorts means removing unnecessary set operations that require sorting (typically UNION, INTERSECT, MINUS, DISTINCT, that kind of thing). Sometimes the order in which tables in the SQL were evaluated would also matter, so you had to rewrite the SQL such that the faster operations would be done first in order to reduce the data volume before the more expensive operations come in.

Sometimes all of these tips and tricks wouldn’t work and you would spend hours on end trying to figure out how to get an SQL performing better. I once spent more than a day in my first project trying to figure out how to tune a report that was taking more than one day to execute. The usual tricks weren’t working, but I eventually figured out that it would be faster to separate the data into two subsets and perform different filtering conditions on each set, then combine using a UNION ALL. (In Oracle a UNION ALL performed no sorting, so it was relatively safe). It was the first real SQL tuning I was proud of, my team leader was amazed when the succeeding runs of the report took a matter of seconds.

Evolution of Society

I found this draft blog post that I had written back in 2010. I’m not sure why I never published it, though it does end on a dire note and seems a bit incomplete. Perhaps I had some further thoughts percolating in my brain back then that never came forward. I am also not sure if I wrote this before reading about The Great Filter. I figured I’d just publish it now without further comment:

I was thinking about all the stupid things humans do, and the fact that there are still so many stupid humans about, and because of the stupid things human do like not focusing on education, we are creating even more stupid humans. It seemed to me like some sort of failure of evolution that we have not yet got it through our thick skulls how important stuff like education and critical thinking are.

But at some level, we’re experiencing a different form of evolution, one that is applied to our society as a whole.

Consider that in some other part of the universe, maybe at a nearby star or somewhere so far we have no way to imagine the distance, there probably exists an earthlike-planet. Maybe it has differences in terrain or weather, but for the most part it has seas and oceans, continents and archipelagos and glacial poles and all that jazz that makes earth interesting. And maybe on this other earth, life has evolved similar to the way it has here. Maybe some species are different, maybe some survived there that did not survive here, but for the most part there are plants and animals and insects and fungi and of course, there are humans.

Of course these humans won’t be exactly like us. Maybe they were spawned in different weather conditions, making the dominant skin tones different. Maybe they evolved slightly better or slightly worse eyesight or sense of smell. Maybe their superstitions were slightly different. Though in most ways this other human society on this other earth is similar with our own, all the slight differences are going to add up and the fate of their society may be vastly different with our own.

Maybe because of their different superstitions, they are unable to survive massive plagues and disease in their middle ages. Maybe because of their differences they are unable to resist the temptation to devolve into thermonuclear war. Maybe they lack focus on technological advances and are unable to bridge the gap to the stars before their ever-growing population outgrows the resources their earth provides for them.

If the nature of humans on this other earth is close to what we have here – selfish, bickering, superstitious, xenophobic then the odds of these humans able to evolve past their world and into the stars is pretty low.

Certifications

I never really put much stock in certifications. I felt that they were no guarantee of knowledge or expertise, and that many people who did have knowledge or expertise wouldn’t necessarily have a certification to say so. Add to that it often seem overpriced to even apply for the certifications, so I didn’t have a high opinion of them. That being said, I have had the opportunity to take professional certification exams twice in my life (both luckily paid for by my employer at that time).

The first was the JITSE – the Japanese IT Standards Examinations. This was back in 2004 when our company was planning to undertake some projects based in Japan. Luckily, the exam did not require us to learn Japanese. The exam mostly covered programming and computer science concepts. The scope was largely academic in nature, not really related to the actual practice of software development. At first I thought it might be a problem for me since I had no formal computer science schooling, but it turns out I need not have worried. We were able to find ample review materials / sample exams that gave us a good coverage of the scope of the exams. We even had like five thick textbooks full of review material to cover, which typically I didn’t end up going through (seemed like a bother), I felt like the sample exams were more than enough. And to add even further to my disdain of certification exams, a significant percentage of the questions during the actual certification were already in the sample exams we reviewed. So yes, easily passed.

Of course I should probably note ever since high school I have considered myself to be a fiend when taking multiple-choice exams. I am supremely confident when taking any sort of multiple choice exam, no matter the actual level of knowledge I possess. During the JITSE for example, I finished it very early. If I didn’t have any kind of shame, I would have submitted the AM exam a whole hour-and-a-half early just so I could get some sleep before the PM part. (Some people later told me they noticed me dozing off). Multiple choice exams are really easy to go through quickly, especially if there’s minimal problems solving involved. There’s no reason to linger at each question: you either know the answer or you don’t, and if you don’t you quickly decide on a candidate that looks the most likely. (If there’s some sort of problem solving or math involved then you go into that weird fuzzy logic where you try to figure out which choice is mathematically closest to the answer you got. Or yes, you could just redo your work, but don’t be afraid to assume that the exam itself can be wrong if you come up with the same answer three times)

The second time I took a professional certification exam was when I was asked to take a Java 6 certification exam for reasons I now cannot remember. This one I was more confident in – by this point I had already been using Java for the better part of a decade and was pretty sure I knew it inside and out. That being said, you always find out about obscure language quirks when preparing for these exams. For example, I never had to use the keyword volatile so I really only learned what it was for while reviewing for the certification exam.

Being the exam-taking fiend I was, I took and easily passed the Java 6 certification exam, which of course led to jokes along the lines of “Hey, we don’t need a compiler anymore, Roy is now a valid JVM”. I never did get to use that certification for anything though. I took the exam and passed it (finally earning my Java certification) in Oct 2012, after 8 years of being a Java programmer.

Over the years as I’ve dealt with a wider range of people across society, my disdain for certifications has dimmed a bit. The most important bit I have learned is that it’s very hard to evaluate software developers in terms of skill (a topic itself worthy of an entire series of blog posts), so certifications allow people in hiring capabilities (or any other capability that requires an assessment of a developer’s skill) to cheat a little bit. They save a little bit of effort because a certification basically says “this guy knows his stuff well enough to pass a certification exam”, which can be good enough depending on your specific purposes. Sure, it’s possible to pass a certification exam and have no real idea how to apply the topics covered in practice, but at least the certification gives them a bit more confidence in you, even if unjustified. It’s a bit like school diplomas and driver’s licenses: it doesn’t really guarantee that you know what you’re doing, but at least you were good enough to fool other people who evaluated you, so even if you turn out to be not so good, at least the evaluator is not alone in making that mistake.

And if you’re the sort of developer who’s all about the money, having certain certifications can be very good for your bank account. I’m pretty sure getting an in-demand certification like Oracle DBA or Cisco Networking stuff can get you a significant bump in pay grade. This also means that it can cost a significant amount of money to even apply for taking those certification exams – it might be best to try to find a way to fool – er, I mean persuade – your current employer into paying for it.

Dealing with Failure

"Success is not final, failure is not fatal: it is the courage to continue that counts" - Winston Churchill

During my the first semester of my second year in University, I failed four out of five classes I was taking due to slacking off a lot. Those same classes were only available during the first semester of each year, which meant I could not retake them during the second semester or over the summer to catch up. So effectively, I had fallen one year behind everyone I knew.

When I got the news, it was devastating. Up until that time I had always been a pretty good student and I felt like I had always made my parents proud and happy. I did not know how I would tell them about it, I felt that I had failed them utterly. I knew I only had myself to blame as I had let most of these classes go by without paying much attention to them. Sometimes when you’re used to doing well academically all the time you might get lax and feel complacent and then you’re surprised to find out there’s a lot you haven’t picked up but it’s too late to catch up.

My parents were very understanding. My friends did not make fun of me or anything like that. I decided to take the failure for what it was – a failure – and simply learn from my mistakes and move on. I had to toughen up. I accepted that I would not be graduating on time, and that it was my own fault. I spent a semester under academic probation, but from that point onwards I did not fail any other classes, and I successfully graduated some years later, got a good job and so on.

Failure is a setback yes, but it’s a setback you can learn from and that you can recover from. Failure is not fatal (most of the time – obviously try not to fail at skydiving). As long as you can find a way to move forward there is no reason you would not be able to surpass it. In time you will look back on your failures as struggles that you learned from and made you stronger and more resilient

This post was originally written as a Quora response

Web Security: File Upload Vulnerabilities

A friend of mine had an informal consultation with me the other day (read: asked me questions over FB messenger) about what their IT staff was telling them about a file upload vulnerability that had been recently exploited in one of their applications. Obviously it was difficult for me to judge given that I didn’t know all the details, but for me it was most likely a vulnerability introduced in the application code itself.

If you’re not familiar with file upload vulnerability, the simplest type of attack goes something like this: the user (or attacker in this case) uploads a file using a file upload function on the system (a common functionality), and the system allows the user to execute that file as if it were code. This can happen if the application saves the uploaded file directly into the web application path, such that it can be executed directly from the client. This is a common vulnerability listed on the OWASP page.

For example, your application provides a function to upload an image to use as the user’s profile picture, then the applications stores the uploaded image in the file system directly in the application path such that it is served to the user as something like /images/uploadedfile.png. However, if the user uploads instead a file that can be executed directly by the web server, such as a .PHP file for PHP webservers or a .JSP file for Java containers, the user can now execute any code that was in his uploaded file simply by invoking /images/uploadedfile.php! And this access would be absolute, everything your system has access to will be fair game: file system, network, database, etc., and provides the attacker with more information for succeeding attacks.

The key to protecting yourself from file upload vulnerabilities is one of the basic tenets of web application security: Never trust user submitted data. This means that you should validate uploaded files – maybe you only allow image files for example – and your validation should be thorough. You shouldn’t validate only be checking the file extension or the content type in the HTTP header, as these can be faked. Use a server-side API call to verify the file type.

In addition, do not store or serve the uploaded files in a way that they will be executed by the web application. For Java webapps I’ve worked on, often we will store uploaded files in the Oracle database, and when they need to be served they are dumped directly into the response OutputStream. If you must store the uploads in the file system, do it in a location that is not accessible from the web server directly. For J2EE containers that can be inside /WEB-INF/ or outside the application path itself. For PHP, it should be outside the HTTP doc folder of the Apache server.

For the PHP example above, you should also avoid serving the image files using a PHP include. If the URL to access the image is something like uploadedimage?image_id=1 and your code maps that image_id to a file on the file system and serves it using a PHP include, it will still be executed by the system as code. Instead the file should be dumped directly into the response. One way to do it is as follows:

<?php
header('Content-Type: [whatever your content type is]');
header("Content-Disposition: inline; filename=\"$file_name\"");
echo $file;
?>

This is only the most basic attack. File upload functions can be exploited in a number of other ways. For example, the attacker could upload a number of large files to attempt to take down your server due to insufficient disk space (prevention: restrict file upload size, prevent user uploading too many files). So developers need to be carefully in how they allow file uploads in their system and be aware of possible vulnerabilities and exploits, the OWASP page linked above lists some more prevention measures.

Continuous Learning

During my adult life, I’ve tried to learn or at least expose myself to one new skill or programming language every year. For example, over the past year or so I’ve been studying, dabbling, or trying out the following: game development using Unity, technical analysis of stocks, Spanish, driving a car, and even some simple cooking! I’ve also been regularly practicing to improve my skills in writing and sketching. I probably even forgot a few things I’ve tried to learn. I also have a backlog of programming languages and frameworks I’ve been wanting to learn and/or try out.

There can be a tendency for some people, especially those tired of formal education, to fall into the trap of thinking that once they become adults they no longer need to learn new things or improve their skills, but an attitude towards continuous learning and improvement pushes us towards more fulfilling lives.

There are even certain professions such as software development or medicine where you pretty much have to keep up with the latest trends or new technologies in order to maintain your skill set. But even outside of such professions, a lifetime of learning can be enriching.

That’s literally too. Learning and exposing yourself to new things is a process of growth and discovery that can lead you to better career options and be more financially rewarding. Scott Adams (creator of Dilbert) gave some advice that you can make yourself more valuable by combining two or more skills. The more skills you learn, the more likely you are to find a combination of skills that gives you the edge towards financial success.

At work, I’ve always been happy to take on tasks that required me to learn new frameworks or programming languages. Without an attitude towards continuous learning, your skills will stagnate. You’ll be working on the same sort of tasks again and again, leading to boredom and unhappiness.

Apart from the possible financial rewards, continuous learning has a few other soft benefits. It keeps you mentally sharp and healthy even as you grow older. It can help you meet new people. It can give you new conversation topics with your friends. It builds your confidence and leads to trying even more new things. It helps you look at things and situations with fresh eyes or different angles. And it means you’re almost never bored!

If you have kids, it sets an example for them that learning is a good thing and definitely you should be trying to instill the same example in them. Encourage their curiosity and facilitate their learning whenever they show interest in something, especially outside of school, to reinforce the idea that learning isn’t something that can only happen in the classroom.

It can be exhausting sometimes to be trying to keep up with everything and learn new things all the time, but you owe it to yourself (and perhaps even to the world!) to continuously try to be a better person.

Performance Optimization for WordPress Blogs

Some time ago, one of my many intrepid followers pointed out that this blog tested poorly on web page performance according to this Speed Testing Tool. Now, I’m of the opinion that for a personal blog such as this, web performance isn’t really a mission-critical sort of thing, but as a software developer who has often had to work hard to optimize the web applications we delivered to our clients, it kind of became a matter of pride :p

Unacceptable!
Unacceptable!

I also ran the site through Google PageSpeed Insights. The results were mostly the same, but the Google site gave me results faster, so this was the one I referenced for subsequent testing. The most important problems to address were:

  • Optimizing image sizes
  • Removing render-blocking JS and CSS in above-the fold content
  • Leverage browser caching (through the use of HTTP headers)
  • Minify CSS and Javascript

All of these are familiar to me since they are very common performance optimizations done in web applications. However, this blog is running on WordPress, which means I should try to look for existing WordPress-based solutions instead of hacking the code myself. (I used to run a custom-built Django-based blog system here, but I got tired of maintaining it myself so I went for tried-and-true WordPress). Here are the optimizations I ended up doing:

  1. Upgraded to the default WordPress Twenty-Sixteen Theme. Not strictly necessary for optimization, but I was previously using the Twenty-Twelve Theme, and the problem with using older themes is that they may not be using newer web development techniques to improve performance. Also, Twenty-Sixteen looked better haha. And yes, I do use the default WordPress themes. Too lazy (as of yet) to look for a nice custom theme, and I don’t have the design chops myself to build it, so it will do for now.
  2. For the image size problem, I found that the biggest offender was the Sketching Daily post where I broke down how I do my daily sketches. The problem was that I was uploading full-size images of the drawings and they were being inlined into the post – which meant the browser was loading the full-size image even though it was shown in a smaller size on the post itself. Instead, I manually changed the settings of each image in the post to use one of the smaller-size images generated by WordPress on upload. wp-image-settingsChoosing “Large” here meant the Large one would be inlined, while I could still set the image to link to the full-size one on click. The full size image is a whole fifteen times larger than the “large” image! It’s something I need to remember in the future if I ever upload huge images again. I also found a link that described how to set the default image size (so that it doesn’t default to full size all the time), but I don’t think it’s necessary for now.
  3. To further help with image optimization, I installed the WP Smush plugin, which automatically optimizes (“smushes”) uploaded images. I found that WP Smush reduces the uploaded image sizes by around 10-15% on average, it’s not too bad.
  4. Next was render-blocking JavaScript. If you’re not familiar, any <script include=”…” /> tag in your HTML blocks the rendering of any content below the tag itself until the content of the JS has finished executing. The typical solutions are either to (a) move the script tags to below the content (“below the fold”) or (b) add an “async” or “defer” attribute to the script tag. It turns out that there was already a plugin that did (b), Async Javascript. So I gave it a go, but for some reason it wasn’t working. After a bit of debugging, I found that there was an “aj_enabled” option that wasn’t being set. I couldn’t figure out any settings page to update it, so I went ahead and edited the plugin directly to set the value: wp-async-jsIt’s generally a bad idea to hack the local copy of your WordPress plugins, since you have to check the impact every time the plugin has a new version, but I decided to live with it for now. After the update, sure enough the JS no longer blocked page rendering.
  5. Lastly, I installed the W3 Total Cache plugin. I disabled the WP Super Cache plugin I was using before since they would probably have some sort of conflict. In addition to page caching, this plugin addresses a couple of other issues: it supports HTML/JS/CSS minification (reduces file sizes and HTTP requests) and it allows adding the expiry headers to static content (to leverage browser caching). If you’ve done any work on website performance, the settings for the plugin are straightforward to understand, but I also referenced the guide at this link. The plugin looks okay, but there’s an unfortunate side effect: the minified JS created by the plugin does not respect Async Javascript, so it blocks above the fold content from rendering. It’s a problem I’ll have to try to resolve in the future.

After the above changes, I retest the site:

Much improvement!
Much improvement!

Much improvement indeed! There’s still a few issues to iron out, but for the most part I’m happy with the results (and my pride has been satisfied :p). I may revisit this issue at a future date to see if there’s anything else I can improve on.

 

Ice Candy

I read an article recently about how we should encourage entrepreneurial spirit in kids from a very young age. It made think of a time when we were kids and we tried running a business

It was a summer from years ago. Perhaps 1988 or 1989, or maybe a year or two earlier, I can’t be sure. I was young, my brother was younger by a few years, my female cousin older by a few years. Back then our families live under the same roof, my grandfather’s house

We needed something to keep us occupied during the summer. We probably already had a Nintendo around this time, but my brother and I were only allowed one hour of video games a day, so we needed something else to fill the time. I think it was my mother who suggested we run a small business over the summer, sell some food things to people

Our house had a sari-sari store. Sari-sari stores are like small family-owned convenience stores in the Philippines, often run out of the house directly, with the storefront little more than holes in the wall that make up the house’s facade. The one in our house had one small window and one large window, each covered by rusty metal bars half a foot apart. It sold soft drinks (cokes and sprites and whatnot), beer to the local tambays, small bags of chips (Chippy, Clover Chips, Tortillos, etc), small chocolates and packs of nuts and such. As kids we spent a lot of free time at the store, especially when the parents were away. After my grandfather died, the store was taken over by a beloved family friend who had been living with us forever. She often indulged us kids our whimsies, which often meant our parents came home to find out they now owed a bunch more pesos to the store. That summer, we had decided to sell ice candy to the people in the neighborhood through the store

You could be forgiven for not knowing what ice candy was, at least if you didn’t live in the Philippines. It’s a treat made out of frozen flavored water, usually in a long plastic bag that gives it an elongated shape. In the US, they might be called “freeze pops”

My parents gave us some “seed money” (and I suppose my uncle gave my cousin some as well). I think my brother and I would have a separate stock from our cousin, but we would both be selling in the same store. We went to the nearby grocery store and bought the materials we’d need: some powdered flavoring and packs of small plastic tubes. When we got home, our mother showed us how to prepare the ice candy. We would mix the flavoring (I think we used orange flavoring. I liked orange back then) in some water, then pour the flavored water into the plastic tubes and tie a knot in the end to seal them. Then we would leave the tubes in the freezer overnight, and we would sell them the next day

I remember them being a hit, especially with the neighborhood kids. We sold them for I think a peso each, or maybe a peso and fifty centavos. (Things were simple and cheap back then.) I think my cousin used slightly larger plastic tubes and sold her ice candies for a bit more. In theory, we would then reinvest the day’s income into more flavoring and plastic tubes for the next day, but I think it turned out that our parents just gave us more money for the next day’s materials. So perhaps it was less of capitalism and more of “here’s a fun way to earn allowance during the summer”

I specifically remember my younger brother enjoying the business and dreaming of making it big as a businessman. His imagined future business was still ice candy. He imagined having an ice candy empire well-known throughout the land, with rich and famous personalities such as the president coming to his stores to buy ice candy. So maybe the exercise did manage to impart a little bit of entrepreneurial spirit to us after all

I remember at the end of that summer, we were happy with the amount of money we had earned, and we talked about doing it again the next summer. But we never did

Working with Testers

I had my first taste with working with software testers during my first project where I was involved with porting an old system to a new version of the software. My first task involved porting reports which were to be generated by the users then printed out. The task wasn’t too difficult: basically you took the source code of the report (it was some weird binary format recognized only by the particular reporting tool – that was how it was done back in the day) and open it using the newer version of the tool, and the tool did some sort of migration magic to adapt it to the new format, then you just save it back again. You compile and run the report using the new format and verify that it’s getting the correct output (needs to be the same output as the old format), and that should be it. Most of the time, at least. Sometimes there were some things that the new format couldn’t handle, or maybe there were some minor compile errors after migration, but such problems were easily fixed, so it was pretty straightforward

Or so I thought. All the reports I was porting were being tested by one of my team members, who was also a newbie like me. Coincidentally, both of the testers I had on the project were also ECE graduates same as me; I was the unusual one who took the developer role. They had their own training focused on tester concepts, so initially I had no idea what to expect. When you’re fresh out of school you don’t have any idea yet how much iteration and rework goes into software development, so there’s this attitude of, “Hey it’s working. Looks good!” without being too thorough

I soon found out that the testers were trained to be very thorough, so I would get a bunch of bug reports. One of my testers would print out the reports, both the new format and the old format, and hold both reports up against the light, overlaid on each other. And they had to match, pixel-perfect! If the new report generated in a slightly different manner, that was a bug! (Of course, this made no sense to me, since logically the users wouldn’t care or notice if some column in the report was 1 pixel over to the right compared to the old format, but those were the instructions so whatever). And then they would use like really long nonsense strings in the data, and in some cases it would make the text wrapping in the report work differently, so I had to fix those too. Then one of the testers would print out reports with a lot of data, generating like 30 to 40 pages or more (we wasted a lot of paper in those days), and the pagination had to match exactly between the old format and the new format. And if it didn’t, I had to spend some time poring over the printout to find the exact point at which the pagination changed and what caused it. Eventually I learned a set of tricks and knew what to look out for when porting the reports

When you’re starting out as a software developer and you’re first introduced to the idea of QA or software testers, there’s a slight tendency to be offended or defensive about bug reports and to be a bit annoyed at the testers. After all, these are people whose job is specifically to find your errors! It’s a problem especially if you have a sense of pride about your work, but once you let go of that pride and recognize that you are not your code, you get to realize that you and the testers are working as a team. The testers are the shield that make sure that stupid stuff you missed or overlooked doesn’t get sent to the users. It’s like how in a buddy cop movie you need to have a partner to watch your back or to cover the other exit so that the bad guy doesn’t get away

After working with the same testers for some time you get used to their individual quirks and figure out which bugs they are more likely to find, so you start focusing on those areas more. You also learn some tricks to sometimes troll around with them especially when you get annoyed with too many bug reports

One of my favorite things early on in the same project was when I would send one of programs over for testing. In addition to the reports, often we would also be doing forms running using the same tool. In those days, we didn’t yet have Hudson or Maven or Ant any sort of automated build system. We would commit our source to version control and generate the binary file manually and copy the binary file over into a shared network drive. And there was no versioning on the network drive, just one folder where all the binaries were. Then we can tell the tester, hey you can test so-and-so program now

The tester would copy the binaries from the network drive to their own machines for testing, and after a while he would come back with some bug reports. The bug reports would be logged in an Excel file (we didn’t have any bug tracking software then either), and I would do a quick skim, and if the changes were easy enough (minor UI stuff), I’d do some quick edits, rebuild the binary, and copy it to the network drive. Then I would message the tester, pretending I’d just seen the bug reports, “Hey, these bug reports don’t seem right. I think you forgot to get the latest files from the drive.” And sure enough, they get the latest files and check again and the bugs would already be fixed. I managed to do this a couple of times before my tester caught on

With the advent of dedicated test servers and modern build scripts, of course this was something I couldn’t regularly do anymore – although once or twice I did have the opportunity to do a live edit on a test server (to verify my fix works) and troll the tester a bit by telling her it worked fine!

After a few months of working with testers, you quickly get used to working with them and get into a good rhythm. You’re kind of partners in crime, except instead of crime it’s quality in software programs. Even with modern developer unit-testing and automated testing practices, having a second set of eyes looking at your program and attacking it from different angles helps improve the quality of the system. A good tester is the shield to the developer’s sword, they make sure the developer keeps his guard up and doesn’t lose to his own mistakes