What is API Testing? (The Beginner’s Guide)

Introduction

In modern web architecture, Application Programming Interfaces (APIs) have become the integration glue that allows functionality to be delivered to users. In this article, we explain the concept of APIs and why testers need to understand and test them. We also explore examples of API testing, how defects and failures can occur, and the tools available for testing APIs.

What is an API?

An API is an Application Programming Interface which allows various components to communicate, share data, and achieve other functionality. In this article, we assume the context is that of web-based APIs which allow connectivity to web services and microservices. This distinction is important because operating system APIs are also used in desktop applications to interface with operating system components. In Figure 1, we see an example of how an API is used to perform a user query on a website by connecting to a web service which queries a large database.

Figure 1 – Example API for Database Query

What is API Testing?

API testing can have multiple objectives, such as testing:

• Functionality of the API
• API performance
• API security
• API reliability

Since this is a basic guide to API testing, we will focus on testing API functionality. API is best considered “grey-box testing”, as it combines elements of black-box testing (which is purely an external view, such as testing through the user interface) and white-box testing (which is purely an internal view). To test an API, you must know things like the endpoints (URLs, for example) and security keys, along with the data to be sent and expected in return.

Types of APIs

APIs can be classified in a variety of ways depending on their source and function.

• Internal or private APIs – These are not shared with the public or any other organization. Due to the nature of these APIs, they often require secure keys to keep information private.
• Partner APIs – These are APIs used to communicate with other entities such as business partners. An example is in retail applications that share information such as pricing and inventory availability between many stores.
• Composite APIs – These APIs can access multiple endpoints in one call. They are used to eliminate the need for multiple APIs.
• Open APIs – These APIs are available to the public, often in an open-source approach. Open APIs may require some level of security, but others may have no security requirement at all. It is important to know the type of APIs you are working with so that the appropriate level of security can be obtained.

Benefits of API Testing

While black-box functional testing would invoke APIs and test them from a user perspective, this is often inadequate to fully test APIs. Some key benefits of API testing include the following.

To Test Error Handling

One of the primary reasons to test at the API level is to focus on testing just the interface, without any UI (user interface) influence. The UI may have data input edits, for example, that help ensure the data passed in an API is correct and valid.

However, you should test for invalid data inputs at the API level to see how invalid data is handled – not only by the API, but also by the sending and receiving components. As an example of this, consider the situation where a website uses an API to access a currency conversion service. (Figure 2)

Figure 2 – API for Currency Conversion

When a customer is making a purchase in a particular currency, the website detects if a different currency value is needed. If so, the purchase amount is sent to a currency conversion service via API, along with the desired currency code. In return, the service sends back the converted amount which the website uses to complete the transaction.

User interface edits are in place to prevent invalid data input such as alpha characters, very large amounts of money, and so forth. However, how do we know if the currency conversion service is returning correct values? Is there any validation on the website for values that are obviously in error? Also, in currency conversion, small amounts add up over time. What is the expected level of error tolerance, and what happens when such errors occur?

APIs will return certain completion codes, such as 200 for good completion, 404 when an endpoint is not found, and so forth.

API testing lets you simulate error codes you might not otherwise be able to see in a black-box test. The importance of error code testing is that you can see how your application is handling the completion code, whether it be an error code or a successful code.

To Test When There is No User Interface

The lack of a user interface, or a user interface data input field, to access for a test is sometimes known as “headless testing”. In this situation, the API may be the primary way to test data input provided to an interfacing service or application.

To Test When the User Interface Changes Frequently

One of the most common issues in test automation is that of UI changes which may cause the automated test to fail. The failed test is not due to failed functionality, but due to things such as invalid locators.

One strategy to help minimize the impact of UI changes is to test at the API level. However, API testing should not totally replace UI testing, manual or automated. UI testing verifies and validates the user perspective of functionality, which API testing does not achieve.

To Test Large Volumes of Data Quickly

Using API test tools, it is possible to provide a large volume of data, or to receive a large volume of data quickly through an API. Trying to test the same volume of data through the user interface could be very time consuming and subject to other errors that could occur during the testing which are not related to testing the API alone.

One key test objective in API testing is to validate that the service or application receiving and/or sending the data can handle the expected volumes in the required time frames.

As an example, consider a travel website application that processes millions of requests per day to reserve rental cars, hotel rooms and so forth. While a test consisting of such a large volume of data might be impractical or unachievable, testing a substantial subset of data through the API can reveal performance issues as well as integration issues.

Understanding API Protocols

Therefore, it is up to any person or organization to understand and follow the required protocol and formatting. This is found in the API documentation which is published by the owner of the API. Be aware, however, that it is possible for the API to change with no associated change to the documentation. Or, perhaps the documentation change may come well after the API is changed.

The most common API protocols

• REST – Representational State Transfer

This is a lighter weight protocol that uses the basic architecture of the web, such as HTTP/HTTPS. In REST, the API calls use methods such as GET, POST, and PUT to complete the request. GET is used to retrieve data, POST is used to send new data to the receiving application or database, and PUT is used to update existing data.

• SOAP – Simple Object Access Protocol

This is the earliest protocol that was used in Service-Oriented Architectures. SOAP uses an envelope structure of header and body with XML as the self-describing markup language for the data being exchanged.

API Testing Tools

The most efficient and effective way to test APIs is by using a tool designed for API testing. Examples of these tools include Postman and SoapUI.

API test tools allow you to create the body of the call using various formats, such as JSON and XML. Also, the tools facilitate handling of security keys and tokens, which will be required in most cases where private APIs are used. Public APIs may be available without the use of a key, but the risk is that messaging and the payload itself may be non-secure.

As each API call is performed and completed, the tools allow you to see the return code(s) and any values returned in formats such as JSON and XML.

Keep in mind that real-world API testing often consists of many APIs, each of which may require multiple tests. To create, maintain, run, and evaluate the API test set requires a tool which can handle the task (Figure 3). To try this API yourself, the API documentation is here.

Figure 3 – Representative Example of an API Test in Postman

In the figure above for the current weather conditions, note the values passed to the API and the return code of 200, indicating a successful response. Also returned is the response time in milliseconds and the size of the payload. One other thing to note is that the body in this example is graphical output. The more common output is data, such as in XML (eXtensible Markup Language) or JSON (JavaScript Object Notation) format that is consumed by the application that invoked the API (Figure 4).

Figure 4 – API Response in XML Format

With a simple change in the GET request to “output=JSON”, the results are returned in JSON format (Figure 5).

Figure 5 – API Response in JSON format

Because JSON uses the same syntax as JavaScript, a JavaScript program can easily convert JSON data into native JavaScript objects. This makes JSON a popular choice whenever compatibility with JavaScript is needed.

Dealing with Responses

API testing, like API usage itself, is based on a request with a corresponding response. In the response, a return code is received, along with any data.

Normal return codes:

• 100 – Informational – The request was received.
• 200 – Success – The request was received, and the response was completed successfully.

Error codes can include:

• 300 level - Redirection – The request is pending, client action needed.
• 400 level - Client error – The request is invalid. This includes the common 404 error – Resource not found.
• 500 - Server error – Such as an internal server failure occurred during a request.

When examining the response, error codes are obviously indicators that something is wrong. Perhaps the endpoint was specified incorrectly (as in the case of a 404 return code), or there is a server configuration problem (as in a 500 return code).

One should not simply accept a return code of 200 as a “passed” test. While the response may be successful, the data returned may not be correct due to functional defects on the other side of the interface. This is where much of the functional testing of API occurs.

API Test Automation

It is possible to automate API tests. In fact, automation should be the goal of API testing. Like API testing in general, an API test tool is required for this.

Postman and SoapUI both provide the ability to run sets of API tests. The thing to consider is the robustness of reporting the tests, plus the ability to have access to all your tests in one test management tool.

API Test Management

When considering test management, a key consideration is how to organize tests in a way that can be centralized and measured. There is great value in having all your tests in a single test management tool.

While API testing is a type of testing that can be managed in an API test tool, it is also possible and even beneficial to house API tests in a test management tool such as PractiTest, along with other tests (Figure 6).

Figure 6 – API Test as Seen in the PractiTest Test Library

In the above figure, we see various tests along with run status. The API test is #24 for Account Info. If issues are observed with an API test, or any other test, those issues can be filed and linked in PractiTest. In addition, API tests can be traced to specific requirements to establish levels of test coverage.

Another reason to manage API tests with other tests is the ability to report test completion as part of a larger test set as opposed to having multiple sources for test reporting (Figure 7).

Figure 7 – API Test Status Reporting in PractiTest

Challenges of API Testing

Unexpected changes

It is not uncommon for APIs to change with little or no notice to the users of the API. A case in point is the Internet of Things (IoT), which relies heavily on APIs to function correctly. You might say a voice command to your digital assistant, such as, “Turn on the lights”, but due to an unexpected API change outside of the control of the IoT developer, nothing happens.

This is obviously an undesirable failure, but one that is not uncommon. This places the developers and testers of the IoT device firmware to scramble to deal with the API changes as quickly as possible.

The ripple effect of API changes

Not only are API changes sometimes unexpected, API changes can have impacts that are far-reaching in the applications in which they are used. Many times, it is very difficult to test the impact of API changes unless a true end-to-end test is performed.

If a variety of APIs and associated test tools are involved, this highlights the need to have a centralized repository of API tests, such as PractiTest.

Uncertain ownership of the API

Even though we might think we know who is the owner of the API, our assumptions may be incorrect. This becomes a huge issue when there are defects in how the API functions and the owner needs to fix the defect.

Keeping track of the APIs to be tested

Keeping an inventory of API tests seems pretty straightforward, right? However, consider how challenging it can be to keep up with all functional tests, all test automation scripts, all test data, and the code behind it all. The same challenges extend to APIs. Keeping track of the APIs to be tested, along with the tests, their associated requirements, and the test outcomes are all key reasons to use a robust test management tool such as PractiTest.

Dealing with the diversity in APIs

This diversity can be seen in protocols, formatting, endpoints, and levels of security (including encryption). Having a centralized repository of API tests can help manage the diversity in testing APIs.

The need for continual testing

Depending on the specific purpose of the API, it may be necessary for some testing to be performed in a live environment to detect incorrectness and performance issues that may be introduced as a result of unexpected changes to the API.

Care must be taken, however, to ensure test data is not seen in production. Performance testing must also be carefully planned to avoid impacting normal production operations.

Conclusion

An API is an Application Programming Interface that allows various components to communicate, share data, and achieve other functionality. APIs are the primary way that modern web applications communicate and share data.

Because APIs have such a key role in achieving integration and interoperability between many types of applications, the correctness, performance, security, and reliability of APIs must be tested.

API testing is one of many types of testing. While it may be possible to store API tests in a separate toolset, there is much value in storing and managing API tests along with other tests in a centralized test management tool such as PractiTest.