Integration Concepts 3 - Reliable transactions

Using Transaction ID's in REST APIs and resend loops

Flowfinity contains built-in features to ensure REST API data operations are saved reliably.

Every REST API request is validated to contain fully formed and valid JavaScript Object Notation (JSON) values which start and end with matching curly brackets {}. If Flowfinity detects incorrect JSON syntax the request will be aborted and respond with an error.

Every operation request should include a unique 'ExternalTransactionId' value. The external transaction property can be any string text that uniquely identifies the record. This property is used to detect duplicate operation calls which can occur when HTTP connectivity is interrupted between an external system and Flowfinity.

When Flowfinity receives a request with an 'ExternalTransactionId' that already was processed successfully it simply responds with the status code '200' without processing the request body, avoiding the creation of a duplicate record.

To take advantage of this functionality the external system should also implement a loop of resending requests to Flowfinity Actions until it receives status code '200.'

'ExternalTransactionID' values can also be used with REST requests to a Universal Endpoint URL (which is designed to accept operations for multiple records that should be processed as one transaction). In this case all operations included in the request should be stamped with the same transaction ID on the top level of the request and the success code will only be returned when all the operations are processed and applied successfully.

Implementing resend loops

There is an increasing number of visual integration tools available in the market. For this example, we outline how to implement resend loops in two of the most commonly adopted platforms, Power Automate (Microsoft Flow) and Zapier.

Power Automate (Formerly Microsoft Flow)

A common way to retransmit failed submissions in Power Automate is to set the 'Do until' function to run until the HTTP status code equals 200, indicating a successful return. Alternatively, you may achieve similar results without using the 'Do until' flow block by configuring the HTTP element retransmission setting to attempt retransmissions before returning from the HTTP block.

If you plan to deploy this approach be aware that there is a 30 day time limit on how long an individual flow can run. If using 'Do until' it will execute only up to 5000 times and the maximum number of retries for an HTTP element is 90. As soon as any of these limits are met, the respective element block ('Do until' or HTTP) will exit even if there is no successful response received and the flow will end.

If you need to ensure that data keeps retransmitting for longer than the limits imposed by Power Automate, and no records are discarded regardless of time passed or number of retries, you can design a flow that keeps records due to be synchronized in a separate, persistent queue or database table.

Flows can be configured to run on a schedule, querying the queue, then submitting or updating the records with results read from Flowfinity. If you chose this option you will still need to use 'ExternalTransactionId' logic to avoid duplicates, however this method avoids the time limit on flows and execution limits on the 'Do until' function.

Zapier

To retransmit failed record submissions Zapier implements a special AutoReplay feature that automatically replays failed Zap steps, up to a maximum of five times.

If the record still cannot be submitted after 5 attempts a notification email will be sent. The failed Zap will be saved in Task History and is available for manual submission if necessary.

You can also configure a special Zap with a Zap Manager trigger, which executes when a Zap fails with an error, this can execute custom notifications or trigger additional actions as required.