On software testing

No software is an island, tells a thesis. It means that practically the code we have running in computers, smartphones, and appliances rely on a lot of other code.

Sometimes implications of a small bug are big: Ariane 5 unmanned space rocket crashed due to a SW bug.

Even though there’s been a lot of improvements in technology, the fact remains that essentially computers are blind: the computer follows instructions given by a programmer.

Program code takes input from people. And sometimes people intentionally try to cheat the machine into doing things it shouldn’t. This is called many names:

  • hacking
  • cracking
  • intrusion
  • data theft
  • breaking code

Software quality is about making sure the software is as bug-free as possible, and also if something happens, the process would mitigate the possibility of exploits. In English: gracefully failing systems are better than those that leave a gap wide open, if they fail. It’s not all there is to making secure systems, but software security plays a big part. If the software is insecure, it helps an attacker a lot: they don’t need to, for example, travel to anywhere (instead attack the parts of a server via the weakest point, remotely). They don’t need to convince anyone over the phone, to make dumb things.

Often we might think a small program stays “alive” for maybe a few months. Believe me – this comes from a veteran developer: you might well see your little program live as a crucial part of a business, 10 years from now.

You never know for sure. That’s why high quality and uncompromising work ethics is crucial. You never really know how long the software will live on.

It only takes one bit in the wrong place, a capital letter where a small letter was supposed to be, or mis-interpretation of a value such as ‘undefined’ vs. “None” to make bugs.

What is shared code?

“Other code” from the viewpoint of a software developer is called:

  • software library
  • package
  • a module
  • shared code
  • extension (sometimes)
  • public code

The idea of sharing and reusing pieces of code is that thus we can elevate the level of thinking in software.

Example: searching for text inside text

A common problem to solve is checking does a piece of text occur in another piece, the source text. This is called string matching.

In trivial cases, string matching is very easy to perform. If you want to just get down to Yes / No answer, here’s the simple formula:

  • take first letter of the target string (which you want to find in source)
  • set stillNeeded to be length of target (ie: ‘cat’ => stillNeeded is 3)
  • go through source string, one letter at a time: read the next available, store in variable C
  • if target[0] == C, then decrement stillNeeded by 1
  • if stillNeeded == 0 then answer is Yes
  • if target[0] is not C, then reset stillNeeded to original length, and advance the loop to next character in source
  • if we are at last character in source, answer is No (not found)

Special cases in string matching include:

  • whether character upcase / lowercase makes a difference?
  • how to handle 16-bit Unicode characters in string matching, where there’s larger character set than ASCII’s 8-bit
  • depending on what the programming language is, certain special characters might cause extra coding needs -for example, in C language the strings themselves are “terminated” using value zero, which thus cannot be part of the strings to be compared

There’s a lot of nuances and it requires attention to get string matching algorithms right. If we had to do it every time we developed a piece of software, it would be outright insane.

Tradeoffs in using ‘other code’?

When we take a library to use in our own project, there’s a few implications that happen.

  • check license compatibility with our project
  • documentation helps using the code, a lot!
  • you need to rely on the source of the other code’s supply being stable (repository, servers)
  • have a back-up plan B: what if the library vanishes, or would not be available for use? Is it critical? Can you choose another one?
  • there should be enough people working on maintaining the other code so that you can rely it will be bug fixed, for example

I’m at times participating in the maintenance of a tool that has to do with software testing. It’s a test result reporter. The code produces XML-format chunks of data, and gives that over to a another software, a dashboarding tool. The quality dashboard relies on this “pipe”. Some of the ideas arose when I started to dig deeper into how actually our modern software is tested for high quality.

How code operates in a computer?

Some important distinctions to make first. Modern computers are made in a van Neumann architecture. That means the memory banks (RAM) can hold both code and data. It’s a technical detail, but had importance when the shift to modern van Neumann’s happened.

The main brain of your computer, CPU, runs code. Code is done by writing a source program (a listing of text), which is then compiled by another software tool. The source code is turned into some form of executable byte code. The form (syntax) depends on your computer’s make and model.

Developers have to hold a mental image of the task at hand. In code, things can be divided into hierarchies: functions, and modules.

Independence of a function

Functions are designed so that they do an understandable, self-standing piece of service: functions return a value or a structure of many values to the caller.

Related thing is functional programming. There, the whole idea of doing code is about avoiding “state”. Functional programming may become more important as we’re looking for new ways to better handle the complexity of code. Functional programming could be described as removing a lot of the fragile dependencies of individual pieces of code.

Photo by Wes Hicks on Unsplash

Functions should be written so that their actions can be carried out regardless of the situation “outside the function”.

Divide-and-conquer

Let’s take an example that clarifies the problem-solving process. The brain’s capacity to do programming is based on a few things. Mathematics is believed to be crucial to programmers, but possibly much more than abstract math, we need good nerves and a unrelenting attitude to tackle “boring problems”. Not all code is super fascinating to do.

All puzzles in developer’s daily grind can be solved, at least theoreticaly, in a very mechanical fashion. It’s called divide-and-conquer method. But this method is not easy, even though it is one that yields a solution. Using divide-and-conquer reminds a bit of the onion:

Photo by K8 on Unsplash

The craft of software development is about learning theory, applying it, and socially learning how other people solve similar problems. The craft is about learning “templates” (patterns of programming).

It’s just that sometimes we try to skip corners, and bugs happen.

What is TDD?

Test -driven development. Idea:

  • write first your test code
  • then write the actual payload code
  • when your tests turn “green”, ie pass, your code is ready
  • benefit: you won’t over-engineer the code, since you have a clear goal to fulfill

Let’s go to Software Testing

If you know, that during development, you will be making mistakes (bugs), then there’s a powerful idea: test your code!

  • test is a piece of instructions, that tells a testing software to automatically test your project’s code
  • tests call parts of the code, and see if predictable (expected) answer come back
  • correct answer makes test pass
  • a wrong answer makes test fail
  • the idea is bit obscure at first.. think like this: I can build a machine to push the levers and buttons of another machine
  • a test is very simple code, that is for certain correct
  • this test code is run automatically on (against) your real code
  • you will get test results indicating how many % of the test suite “passed”
  • for future, now you have also guarded your own code, against unintentional changes: you’ll see immediately if you (or someone else in the team) tampers with your code in a way the breaks it

What are drawbacks in software testing?

  • you have to write test code (takes time)
  • tests have to follow your code base – if you make new features, ie. knowingly decide to change existing code, the test suite must also be altered
  • writing effective and proper test code is a skill in itself
  • it’s easy to write huge amounts of tests that don’t do anything useful (missing the target)
  • psychologically developers are prone to making new endeavours, instead of delving in mundane testing

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: