Intelligent Process Automation
Revolutionizing IT: A Leap in Workload Migration
Modernized migration platform boosts process visibility, accuracy, and customer satisfaction in IT services.
Professional Services
Real-time communication with OutSystems using Server-Sent Events for webhook callbacks, avoiding resource-heavy alternatives and streamlining server-to-client updates.
TL;DR: Server-Sent Events allow for real-time and unidirectional communication from the server-side to the client, this makes it an ideal tool to consume webhook callbacks from external services with OutSystems.
Webhooks
By definition, webhooks are HTTP callbacks that can be created and configured so that events on one site can invoke behavior or send data to another.
An example of this is if an application A requests a new process from a remote service B, and, once the process is ready, the service B replies back to a webhook on application A. When this happens, application A will immediately receive the process data from B, and can update the client UI with the new data.
And yes, most endpoint APIs use the more common HTTP request/response protocol, but in this article we will be focusing on the use of webhooks.
Roughly speaking, the flow of data on an OutSystems application, consuming a traditional Rest API, goes like this:
But if the external service needs to reply using a webhook, an API needs to be exposed in our infrastructure in order to receive the callback, as such:
The Problem
Here’s the catch when using the webhooks approach with OutSystems: the endpoint instance that receives the callback has no way of directly and immediately sending the received data to the client UI!
A workaround for this limitation could be to store the data received from the callback in the OutSystems infrastructure database (step 8 on the image above), and have the client poll the database for updates (step 6). This should be avoided, as not only is it bad practice, but it also defeats the whole purpose of using webhooks.
Another possible solution would be to use WebSockets, connecting the client browser to the back-end server.
WebSockets allow for real-time bidirectional communication between the browser and the server, and as such, the callback endpoint API on the infrastructure could use this protocol to send real-time events to the client.
The problem with this approach is that, with OutSystems, you would need an external service provider, such as Pusher or Firebase.
Server-Sent Events to the Rescue!
Server-Sent Events (SSE) is a long-lived HTTP connection of the type event-stream, and it enables unidirectional communication from the server to the client.
Once the client establishes an SSE connection with the server, the connection stays open, and the server can send messages or updates to the client at any time without the client needing to initiate a new request. At the end of the data transmission the server can, if desired, end the connection.
A simple example of this could be to render a progress bar on the UI, with the value of the percentage of completion sent from time to time from the server to the client.
This makes SSE a good candidate to send to the client the data that is received server-side on our webhooks callback endpoint. Also, it has a great selling point in that there is an OutSystems Forge component that allows the implementation of a SSE hub in your infrastructure, without the need for any external service providers.
The data flow, using webhooks and a SSE hub to send updates to the client, will look something like this:
Not Everything is Sunshine and Roses
Still, SSE has its own limitations and shortcomings, and there should always be a technical analysis to decide if it’s the best fit for a given project.
For example, SSE lacks the ability for bidirectional communication, and it also has a limit on the number of simultaneous connections (100 per hub, as described in the Forge component documentation). If bidirectional communication is a requirement of the project or if you need hundreds of permanent live connections, then maybe WebSockets are best fit.
Considering resource consumption, even if SSE can be lighter than WebSockets, they still are a continuous connection that consume server resources. There’s no such thing as a free lunch.
Security-wise, SSE uses a Rest API endpoint that must be opened in the infrastructure and configured when implementing the component. This is a potential vulnerability point, and care should be taken in order to properly authenticate and validate all incoming requests (as with all exposed APIs).
Wrap Up
OutSystems 11 does not have any built-in mechanism that allows real-time server-to-client updates (this might change with OutSystems ODC), and as such, webhook callbacks consumed with OutSystems can be problematic to handle if real-time behavior is desired.
As a workaround to this issue, developers can sometimes use polling, but depending on the use case, it can become very heavy and resource-wise inefficient.
WebSockets need external services, and as such, they are an extra layer of expense, complexity, and possible failure points. Also, for the simple case of handling webhook callbacks, WebSockets can be an overkill.
The use of SSE in OutSystems solves various problems, including:
- Allows sending real-time updates from the server to the client, not just for webhooks, but also for a lot of other situations where the need to send real-time updates from the server to the client arises (BPT processes come to mind).
- Does not need any additional external services, as communications happen all within the OutSystems infrastructure and the client.
- It can be implemented with a readily available and well documented Forge component.