Hubway Connect API Performance Test Results

When developing an application, knowing how well your Application Programming Interfaces (APIs) are performing is vital. That’s why benchmarking your APIs against other ones known for good performance is crucial.

Hubway Connect is no exception. To benchmark an API created by Hubway Connect, we compared its performance to that of a raw API created by OutSystems, known for having a good response time.

The steps we took, along with the results, are outlined below.

Why benchmarking?

By benchmarking APIs, developers and businesses can gain insights into the performance of their solution under different conditions, such as varying traffic loads, network latencies and user behaviour.

This knowledge can help developers identify potential bottlenecks that make business users underperform, slowing down business operations and impairing customer-facing processes. It also helps pinpoint areas for improvement in their solutions, leading to a better user experience.

Setting the goal

It’s essential to define the goals and indicators to be measured. They must be comparable to another solution or platform that delivers a good result around the metrics adopted.

For this test, our goal included measuring the response time (or how long it takes for the server to respond to a request) of a Hubway Connect API and comparing the results to the average response time of an OutSystems API.

The data, server and client tool location, overall usage of the servers and the time the tests were run were relatively similar, reducing the risk of noise affecting the results. Also, the same test cases were run multiple times with both APIs to counter any temporary variance.

Security aspects

Authentication and authorisation are also essential considerations in API benchmarking as they ensure the testing is accurate and reflects real-world usage scenarios.

Authentication refers to the process of verifying the identity of a user or system attempting to access an API. It typically involves some form of credentials, such as a username and password or API key. This makes sure only valid users can access the API.

Authorisation refers to the process of determining what actions a user or system is allowed to perform once they have been authenticated. This involves defining roles and permissions and ensuring only authorised users can perform certain actions.

As this test was not focused on validating authentication or authorisation mechanisms, but on response times, we used basic authentication for both solutions, leveraging user accounts provided out-of-the-box by OutSystems.

Tools nd settings

Postman was the selected tool for this API benchmarking test. The OutSystems server used was a production environment in the OutSystems cloud. The server is based in Sydney, and its machine specifications are as detailed below:

• Processor Intel(R) Core(TM) i5-8365U CPU @ 1.60GHz 1.90 GHz
• Memory 16.0 GB (15.8 GB usable)
• Operating system Windows 10 Pro

The target

The data model used for this API benchmarking test is illustrated in the following figure:

Person500, Person20k, Person500k and Person all have the same attributes, just a different number of records – 500, 20k, 500k and 1 million, respectively. These entities were used for the test ramp-up strategy. Each Person is related to a specific Department and a specific City. We created a Hubway Connect project to expose these 6 entities for benchmarking.

Test Scenarios

To get accurate results, it’s essential to use realistic scenarios when benchmarking APIs. We ran 3 main scenarios, each with a ramp-up strategy, i.e., increasing output records per request.

Request scripts were implemented as Postman collections and run without using or storing any cookies from Portugal. Each request was run 10 times (the chart below shows the average) at approximately 10 pm Sydney time, with low network traffic and server usage.

Scenario 1 – Simple Select

The first and simplest scenario requested a simple list with 4 entities combining the number of records in the output, 500, 20k, 500k and 1 million records. This is probably the most common scenario when using Hubway Connect for data analytics and reporting purposes.

As illustrated in the chart below, as the entity size increases, the time it takes for the server to respond to the Hubway Connect API request is less than that for an equivalent OutSystems API request.

Scenario 2 – Filter

In this scenario, a filter is applied to one of the entity’s attributes using a like operator. We have filtered Person by the GivenName attribute. In OData terms, we have used the supported string function named contains and the $filter query option.

We observed a similar pattern as in the previous scenario where, as the entity size increases, the time it takes for the server to respond to Hubway’s requests is lower than to a standard OutSystems equivalent request.

Scenario 3 – Expand

Last but not least, we have an expanded scenario. For the OutSystems API, we have joined each of the Person entities with both City and Department entities. For Hubway’s API, we used the $expand query option, which joins the 3 entities and returns a hierarchical result set.

As illustrated in the chart below, as the entity size increases, the time it takes for the server to respond to Hubway’s requests is lower than to a standard OutSystems equivalent request.

Conclusion

OData APIs are a viable solution when it comes to democratising corporate data. It boosts business intelligence and makes reporting on top of real-time data easier, supporting data-driven decisions at an enterprise scale.

Combining best practices when handling a high volume of data, such as incremental synchronisation and pagination, is of fundamental importance. Integration with 3rd party solutions is also a valid use case that can be accelerated with the flexibility and performance of Restful APIs underneath the semantic layer provided by the OData protocol.

On top of that, Hubway Connect implements an optimisation mechanism that can improve the overall performance of its APIs while reducing the impact that heavy data operations could cause in runtime environments.