Insights · 5 min read
Connecting two systems is easy on the happy path. The real work is in retries, idempotency, rate limits, and data mapping. Here's where the hours go.
By GGP Editorial
People talk about API integration as if it is plumbing: connect system A to system B and you are done. The happy path is easy. The work is in everything that is not the happy path, and that is most of the project.
We have wired payment gateways, market data feeds for a multi-market trading system, and the APIs behind mini-programs, and the lesson repeats every time. Budget for the edge cases, the retries, and the data mapping, because those are where the hours actually go.
The happy path is the demo. A request goes out, a response comes back, everyone nods. It works in the sandbox. Then the real world shows up. The third-party API is down for maintenance. It returns a 429 because you hit a rate limit. It sends a field as a string on Tuesday and a number on Wednesday. Each of those is a small piece of code, and together they are most of the project.
A realistic integration has to handle authentication and token refresh, retries with backoff, idempotency so a retry does not double-charge a customer, rate limits, timeouts, and enough logging to debug a failure three weeks later. None of that is hard on its own. It is just a lot of it, and it is invisible until it is missing.
Webhooks add another layer. When the third party calls you back with an event, that event can arrive out of order, twice, or hours late. You need a queue and a way to dedupe, or you will process the same payment confirmation twice and spend a day unwinding it.
The three that cause the most real-world pain are retries, idempotency, and rate limits, and they interact. If a payment request times out, you do not know whether it went through. Retry it blindly and you might charge the customer twice. So every write we send carries an idempotency key, and the receiving side dedupes on it. That one decision has saved clients from explaining double charges.
Rate limits are the other one. A third party will cut you off if you call too fast, and a naive retry loop makes it worse by hammering the endpoint. We add exponential backoff and a small queue, so bursts get smoothed out instead of rejected.
Here is a quick look at the layers most integrations need and what goes wrong without them:
| Layer | Why it exists | What breaks without it |
|---|---|---|
| Auth and token refresh | Keeps credentials current | Calls fail silently after the token expires |
| Idempotency keys | Stops duplicate writes on retry | Double charges and duplicate records |
| Retry with backoff | Survives blips and maintenance windows | One outage becomes a pile of support tickets |
| Rate limiting | Stays inside the provider's limits | The provider blocks you mid-transaction |
The last big cost is mapping data between two systems that were never designed to talk to each other. A field called "amount" on your side might be "total_cents" on theirs. Your customer IDs might not match their account IDs. Time zones, currency formats, and status codes all need translation.
This is not a technical detail. It is where your business rules live. When we integrated market data for a trading client, the feeds from three exchanges each had their own clock, their own symbol naming, and their own way of marking a trade as settled. Making those agree was a bigger effort than the connection itself.
The practical advice is to write the mapping rules down before you write the code, with examples of real records rather than just field names. A mapping that looks right on paper falls over the first time it meets a value nobody anticipated.
Integration work is steady, unglamorous, and valuable, and it fits how we are set up. The team is built on Java, Spring Boot, Node, and Go, which covers most of the APIs you will ever touch. We run the work across time zones with overlapping hours and a shared group, so a payment issue in your morning gets looked at while our team is still awake. We also put alerting on every integration from day one, so a silent token expiry becomes a notification in the group instead of a customer email.
If you are planning to connect a payment provider, a market data feed, or the APIs behind a mini-program, tell us what you are connecting and what it has to survive. The answer you get will be more useful than a quote for the happy path.
Tell us what you are building and where you are today. We typically reply within 24 hours.
