Software Testing

Ambiguous Software Testing Terminology for 2025

October 18, 20246 min read
Software Testing image
Image by testbytes from Pixabay

Understanding Ambiguous Software Testing Terminology

After reading this article, I hope you will gain a better understanding of the often-confusing terminology surrounding software testing. I've made a concerted effort to research the subject, but if you notice any areas for improvement or inaccuracies, please feel free to leave feedback.

Functional Tests, Unit Tests, Acceptance Tests… What?

There’s considerable confusion around the terminology used in software testing. These terms are often interpreted differently depending on the context. For instance, many associate integration testing with the validation of entire infrastructures, whereas others view it as testing how two services or modules interact.

Let’s attempt to clarify these ambiguities by categorizing some key testing methods.

White-Box vs Black-Box Testing

White-box testing refers to tests designed at the source code level. It encompasses techniques like control flow testing, data flow testing, branch testing, path testing, statement coverage, and decision coverage, including modified condition/decision coverage (MC/DC). In essence, white-box testing evaluates the internal structure and logic of an application. This category predominantly includes unit and integration tests, typically authored by developers.

On the other hand, black-box testing does not require any knowledge of the underlying code. The software is tested without insight into its internal structure or implementation details. Black-box tests can be executed by various stakeholders and include both functional and non-functional testing, which we will explore further.

black box testing

image source

Functional Testing

Functional testing validates whether a system's functionalities meet the specified business requirements. This involves black-box testing, where the focus is solely on the output generated in response to specific inputs, without concern for the internal workings of the system. Key forms of functional testing include:

  • Smoke tests
  • Sanity tests
  • Regression tests
  • Usability tests

For example, a functional test might verify whether data is correctly saved to a database or ensure that a particular business scenario behaves as expected.

Smoke Testing

Smoke testing is a quick, surface-level check to ensure that the core functionalities of the system are intact, allowing more in-depth testing to proceed. It is often used to weed out severely broken builds.

Sanity Testing

Similar to smoke testing, sanity testing checks the fundamental parts of the software in a broad, non-comprehensive manner. Its goal is to verify that minor updates to the software haven't introduced critical bugs.

Acceptance Testing

Acceptance tests are high-level evaluations that ensure the system meets the client's requirements. These tests are used to validate whether the product is ready for delivery. The distinction between acceptance and functional tests can vary: some consider them synonymous, while others see them as distinct. The key is to adopt a consistent terminology.

Snapshot Testing

Snapshot testing compares the current state of a system's output to a reference snapshot. This technique is particularly useful for detecting unintended changes, such as verifying that an HTML page remains unchanged between different versions of an application, even when unrelated components are updated.

Non-Functional Testing

Non-functional testing assesses aspects of the system beyond specific functionalities, such as performance, scalability, security, or general behavior. For example, it might evaluate how many users can log in simultaneously. Types of non-functional testing include:

  • Availability tests
  • Load tests
  • Maintainability tests
  • Performance tests
  • Security tests

Performance Testing

Performance testing ensures that the system performs efficiently under expected workloads. It evaluates factors like speed, scalability, and stability. For instance, load testing assesses how the system behaves under real-world conditions when multiple users access it simultaneously.

Security Testing

Security testing identifies potential vulnerabilities that could result in information loss or exploitation. The goal is to uncover and mitigate risks. There are several forms of security testing, such as:

  • Vulnerability scanning
  • Security scanning
  • Penetration testing
  • Risk assessment
  • Security auditing
  • Ethical hacking
  • Posture assessment

An example might involve ensuring that sensitive data, like passwords, is properly encrypted, or that financial websites prevent unauthorized access via the browser's back button.

Functional vs Non-Functional Testing

Functional testing is typically performed before non-functional testing. It is driven by clear business requirements, making it easier to define. In contrast, non-functional testing focuses on broader customer expectations, which are often harder to quantify. Functional tests evaluate the system’s behavior, while non-functional tests assess aspects unrelated to behavior, such as performance and security.

Manual Testing

Manual testing is the most fundamental form of testing, executed by human testers without the use of automation tools. Testers manually execute test cases, verify functionalities, and report bugs. Any type of testing can be performed manually, including:

  • Black-box testing
  • White-box testing
  • Unit testing
  • System testing
  • Integration testing
  • Acceptance testing

An example of manual testing is exploratory testing, where testers proactively try to break the system by simulating unusual or destructive behaviors.

Unit, Integration, and E2E Tests as the Test Pyramid

I hope this article has helped clarify some of the terminology used in software testing. If you're interested in learning more about unit, integration, and end-to-end testing and how they fit into the well-known test pyramid, check out my follow-up article here: Why the Test Pyramid is a Misleading Guide to Testing.

Sources:

  • Mike Cohn, Succeeding with Agile: Software Development Using Scrum, Addison-Wesley Professional, 2009
  • Testing of Microservices, André Schaffer, https://labs.spotify.com/2018/01/11/testing-of-microservices/
  • What is functional testing with example?, Neha Sharma, https://www.quora.com/What-is-functional-testing-with-example/answer/Neha-Sharma-3421
  • 3 typy testów, które musisz zrozumieć, by robić efektywne TDD, Wojciech Zawistowski, https://it.esky.pl/programowanie/3-typy-testow-ktore-musisz-zrozumiec-by-robic-efektywne-tdd/
  • White-box testing, Wikipedia, https://en.wikipedia.org/wiki/White-box_testing
  • Black Box Testing, Software testing fundamentals, http://softwaretestingfundamentals.com/black-box-testing/
  • How deep are your unit tests?, Kent Beck, https://stackoverflow.com/questions/153234/how-deep-are-your-unit-tests/153565#153565
  • Błędy popełniane przy automatyzacji testów, Radek Smilgin, http://testerzy.pl/baza-wiedzy/bledy-popelniane-przy-automatyzacji-testow
  • TestPyramid, Martin Fowler, https://martinfowler.com/bliki/TestPyramid.html
  • Test Pyramid as a Risk Filter, Amit Kulkarni, https://amtoya.com/blogs/test-pyramid-as-a-risk-filter/
  • Piramida automatyzacji testów, Code Coverage, http://pisz-kod.pl/2018/06/07/piramida-automatyzacji-testow-code-coverage/
  • The testing pyramid, Automation Panda, https://automationpanda.com/2018/08/01/the-testing-pyramid/
  • The Evolution of the Testing Pyramid, James Willett, https://james-willett.com/2016/09/the-evolution-of-the-testing-pyramid/
  • Functional Testing Vs Non-Functional Testing: What’s the Difference?, https://www.guru99.com/functional-testing-vs-non-functional-testing.html
  • What is functional testing with example?, https://www.quora.com/What-is-functional-testing-with-example
  • Acceptance Testing, Software testing fundamentals, http://softwaretestingfundamentals.com/acceptance-testing/
  • Sanity Testing Vs Smoke Testing: Introduction & Differences, https://www.guru99.com/smoke-sanity-testing.html
  • Functional Testing, https://www.techopedia.com/definition/19509/functional-testing
  • Snapshot testing, https://jestjs.io/docs/en/snapshot-testing
  • Write tests. Not too many. Mostly integration., Kent C. Dodds, https://blog.kentcdodds.com/write-tests-not-too-many-mostly-integration-5e8c7fff591c
  • Performance Testing Tutorial: What is, Types, Metrics & Example, https://www.guru99.com/performance-testing.html
  • Load Testing Tutorial: What is? How to? (with Examples), https://www.guru99.com/load-testing-tutorial.html
  • Manual Testing Tutorial for Beginners: Concepts, Types, Tool, https://www.guru99.com/manual-testing.html
  • Software Testing