The Idea of Software Testing

Unlike most other forms of engineering, software development is incredibly accessible to the general public. In fact, the widespread infiltration of software comes from, in part, software development’s high level of accessibility. Namely, all one really needs for software development is a computing device and some basic understanding of computer programming. This accessibility even allows young children to be capable of developing basic software!

The Lack of Software Testing Competency

Unfortunately, software development’s low bar to entry often causes people independently pursue the field without learning proper software etiquette. This in turn often results in informal and/or insufficient training on critical aspects of software development. For example, unless one explicitly learn about software security, developers rarely consider software security at any point in software development. Thus, hobby / informal developers often release low quality “functional”^​*​ software that malicious parities can easily exploit. While this is not necessarily a problem for systems strictly used personally by the developer, it quickly becomes problematic as more people use the system.

A common example of this phenomenon might occur when a developer’s personal project rapidly gains popularity. As users subject the system to new environments (e.g., Windows 8 instead of Windows 10) and use cases (e.g., using the application in unprecedented fashions), the project may begin to fail, behave unexpectedly, or even allow dangerous activities to occur (e.g. safety violations or hidden security exploits). As long as users actively use the project, these issues must ultimately be patched out, otherwise exploits involving the project will occur.

The Three Necessities for Adequate Software

Fortunately, there exist several software processes and techniques which help facilitate high quality software possessing key characteristics. Any serious software developer must learn these processes and actively employ them during software development, especially as systems grow complexity and size.

These characteristics are as follows. First, (1) systems should perform as expected from the developers and users. Second, (2) systems must maintain some minimal quality. Finally, (3) developers must be capable of efficiently working on the systems. These respectively correspond to (1) software functionality, (2) software quality, and (3) several areas including software management, understandability, and maintenance. Missing (1) can render the system useless to all users while missing (2) will make users desire other better systems. Finally, missing (3) makes developing systems a truly awful experience.

Implementing quality (e.g. loading a webpage within 100 seconds) generally involves high-level software design (the field of software architecture). The issue of efficient software development, likewise, involves properly managing people and establishing processes to make all aspects of development easier. The primary topics relating to this smooth development includes requirements engineering, project management, and software maintenance. A software system loses all purpose, however, if it fails to achieve its intended functionality. Thus, a system’s intended functionality is the most important single factor about a system. The system’s actual functionality however often differs from its actual functionality over the course of development. The verification of actual functionality vs intended functionality lies primarily in the area of software testing.

Maybe you have yet to hear about software testing. If you are any kind of developer, you have hopefully heard of software testing. If you are unfamiliar with it, here’s a quick rundown on the general idea of software testing!

What is Software Testing?

In short, software testing is about … testing your software! Simple, right? At the end of the day, software testing is all about ensuring software (1) functions as intended and (2) possesses satisfactory quality (e.g. loading a webpage in 10ms instead of 10 minutes).

Testing Functionality

The most prominent way to achieve the former involves developing supplementary code which explicitly tests system functionality under real-world scenarios. This supplementary code is often called system tests. The general idea is that modifying a system in any fashion should be followed by execution of system tests. If all system tests pass, then the system conforms to the functionality tested by the system tests. This may seem an odd way to state it, but essentially anything not explicitly tested has unverified functionality. Such unverified functionality can cause issues for future development. Developers can mitigate this risk by employing strong tests (which thoroughly test system functionality) rather than weak tests (which minimally test system functionality), but this falls into the subject of properly designing system tests, an article for the future.

Testing Quality

Software quality is often measurable through system tests, but implementable via fine tuning a system’s architecture. Again, a rather large field itself, most software quality results from successful software architecture. Rather than designing low-level items such as classes, methods, properties, and so forth, developers can group these low-level items in some fashion and then thoroughly define how these grouped elements may interact with each other. A common example of such architecture is the Model-View-Controller (MVC) architecture pattern used as the basis for displaying webpages. Though related, ensuring software quality vastly differs from ensuring software functionality. A future blog post shall cover this topic.

Why Test Software ?

Analogy: Writing a Book

So with a rough idea of software testing, why is it important? Well, think of developing software systems analogous to writing books. Outside the most trivial of books, you will need to proofread every page. This proofreading process at least involves checking grammar and spelling, but can also optionally involve checking other important aspects, such as checking literary devices. Even in a best-case scenario where you write entire the book in one setting, you still need to follow the entire proofreading process. This means you need to check for grammar, spelling, plot consistency, formatting, and other aspects specifically important to your book. But what happens if you decide to change a key plot point? Well now you must adjust all references to the old plot point and redo the checking proofreading process.

As these books gets longer, you must invest increasingly more effort and time to modify outdated plot points, check grammar, check consistency, check spelling, and other proofreading items. Sure it’s lot of time + effort, but it is to ensure that the final deliverable has (1) satisfactory content and (2) sufficient quality! You may take 100 passes on the book, but upon publication, the book will receive several magnitudes more passes from all readers. Thus, you (should) heavily review and revise the book initial drafts to ensure the final revision remains satisfactory. This applies to any form of content; books, movies, art, and so forth!

Sample Software Development Models

Back to the Context of Software

Every one of these points also applies to developing software systems! Making anything nontrivial? You need to explicitly check to ensure sure the functions as needed. Do you add, modify, or delete functionality? Thoroughly test all new functionality and ensure existing functionality remains unchanged! Software systems are generally quite a bit more complex than books, so it highly probably that some intended behavior faulty. Even more likely is some unintended behavior manifesting, furthering showing the need for software testing on all systems. After all, if you decide to develop a software system, putting in extra effort to ensure it achieves intended functionality makes common sense. It prevents users from being discouraged due to bugs, poor performance, terrible understandability, and so forth. Put more concisely, properly testing software makes observing unintended system behavior easier as system development progresses.

What Does Software Testing Look Like in Practice?

public class SimpleFunctions {
    public Integer simpleAddition(int x, int y) {
        return x - y; // This should be "return x + y;"
    }
}

@Test
public void SimpleAdditionTest1() {
	// Create an instance of SimpleFunctions
	SimpleFunctions simpleFunction = new SimpleFunctions();

	// Represents some value generated from source code
	int calculatedValue = simpleFunction.simpleAddition(0, 0);

	// Represents some measurable output
	int expectedValue = 0; (0 + 0 = 0)

	// assertEquals(a, b); Fails if a != b
	assertEquals(calculatedValue, expectedValue); // Passes
}

@Test
public void SimpleAdditionTest2() {
	// Create an instance of SimpleFunctions
	SimpleFunctions simpleFunction = new SimpleFunctions();

	// Represents some value generated from source code
	int calculatedValue = simpleFunction.simpleAddition(1, 1);

	// Represents some measurable output
	int expectedValue = 2; (1 + 1 = 2)

	// assertEquals(a, b); Fails if a != b
	assertEquals(calculatedValue, expectedValue); // Fails, 2 != 0
}

Software Testing in Practice: Unit Testing

The concept behind unit tests is simple. Ideally, developers can decompose systems into methods / functions. Each of these methods should perform one specific utility. Developers can test this utility this for correctness. So, rather than designing multiple tests for the entire system, you can test individual methods for correctness. Any errors found in individual methods are likely to propagate throughout the system. One can thus prevent a significant portion of future testing issues by testing methods in isolation.

There exist numerous testing frameworks supporting unit tests and JUnit is one of the most prominent for the Java programming language. Figure 1 shows two example unit tests from the JUnit framework and the source code for these unit tests. At the end of the day, all these tests compare some generated data (e.g., data output from a system function) against some expected data (e.g., some truth value such as 1+1=2). Methods generating output different from these truth values are not operating according to specifications and thus need manual review for correctness.

Upon executing system tests, your IDE of choice should output detailed test results (e.g, exactly what tests failed, why did they fail). Figure 2 shows a sample of such output generated from the NetBeans IDE. Here, it is evident that one test passes while one test fails. From these results, it becomes much easier to find erroneous code.

Unfortunately, some classes of errors are undetectable via unit tests, so additional techniques are needed to compensate for unit test weaknesses, but that will be covered in other technical posts.

Summary

In short, this technical post provides the following key points:

• Software systems are crucial for modern society and they need to be developed with minimal errors.
• Software testing is a software engineering technique that primarily aims to (1) test software systems for intended functionality and (2) detect unintended behavior.
• Testing is a necessity for any nontrivial software system; done properly, it mitigates the likelihood of erroneous system behavior existing during system deployment / release.
• Numerous software testing techniques exist, but developers often employ some form of unit testing (e.g., JUnit for Java).
• Output from proper software testing will make finding erroneous behavior trivial and correcting faulty behavior easier.

Now this is just an extremely high-level overview of software testing. There exist quite few nuances involved in achieving proper system testing, so keep reading these technical posts and share this article if you feel other developers or businesses will benefit!