A reliable custom API integration connects systems through explicit ownership, validated data contracts, retry-safe operations, and reconciliation. Decide which system owns each field before choosing webhooks, polling, or middleware.
They are about getting existing systems to exchange data reliably.
A typical workflow might involve:
Website form -> API -> validation -> database -> email -> CRM
Each tool may already work well independently. The engineering problem is the connection between them.
Start with the source of truth
Before writing integration code, decide which system owns each piece of data.
For example:
- customer identity may belong to the CRM;
- submitted form data may belong to the application database;
- email status may belong to the email provider;
- analytics events may belong to the analytics platform.
If two systems can independently overwrite the same field, synchronization bugs become inevitable.
A simple ownership table can prevent a surprising amount of complexity.
Prefer explicit data contracts
Third-party systems rarely use exactly the same schema.
One service may send:
{"email":"person@example.com","company_name":"Example"}while another expects:
{"contactEmail":"person@example.com","organization":"Example"}The integration layer should normalize these differences explicitly.
Useful steps include:
- validating required fields;
- normalizing dates and time zones;
- standardizing identifiers;
- mapping enums;
- cleaning empty values;
- validating URLs and email addresses.
This keeps vendor-specific details from leaking through the whole application.
Decide between polling and webhooks
If the external system supports webhooks, they are often preferable for event-driven workflows.
A webhook can notify the integration immediately when:
- a form is submitted;
- a payment completes;
- a CRM record changes;
- a report is ready;
- an email bounces.
Polling is still useful when:
- webhooks are unavailable;
- historical reconciliation is needed;
- updates can be delayed;
- the external API is not event-oriented.
Many robust systems use both: webhooks for speed and periodic reconciliation for correctness.
Every integration needs idempotency
External systems retry requests.
Users double-click buttons.
Network calls time out even though the remote server successfully processed the first request.
Without idempotency, this can create:
- duplicate contacts;
- duplicate emails;
- duplicate payments;
- duplicate database rows.
Use a stable request ID, event ID, or business key to recognize repeated operations.
This is one of the differences between a demo integration and a production integration.
Use an atomic uniqueness constraint rather than a separate check followed by a write. Scope an event identity by provider and account, and record completion in the same transaction as database-only effects. External actions need their own stable idempotency keys or reconciliation. See the durable webhook inbox example.
Reconcile even when webhooks work
Webhooks reduce latency, but missed events, expired permissions, or provider outages can leave systems out of sync. A periodic reconciliation should compare authoritative state and repair differences without overwriting newer destination changes.
Track update versions, deletions, and the last successful synchronization boundary. The incremental API sync guide shows how to commit a page and checkpoint together so an interrupted run can resume safely.
| Failure | Appropriate next step |
|---|---|
| Duplicate event | Recognize the existing scoped event identity |
| Rate limit | Apply the provider’s documented wait and retry policy |
| Invalid payload | Quarantine with sanitized evidence; fix the contract |
| Expired credentials | Restore authorization before replaying pending work |
| Timeout after a write | Reconcile or reuse a supported idempotency key |
Provider-specific behavior matters. For an authoritative example of duplicate and unordered event handling, see Stripe webhook documentation.
Retries should be selective
Retrying every error is dangerous.
A timeout or temporary 503 may deserve another attempt.
A malformed payload or rejected credential usually does not.
A retry policy should distinguish:
- transient errors;
- rate limits;
- validation errors;
- authentication errors;
- permanent business-rule failures.
Exponential backoff and a maximum retry count prevent one failing integration from generating an endless loop.
Store enough information to debug failures
When a synchronization fails, “API error” is not enough.
Useful logs include:
- integration name;
- request ID;
- timestamp;
- operation;
- response status;
- sanitized error body;
- retry count;
- final state.
Sensitive values should not be written to logs.
The goal is to make support possible without exposing secrets.
Keep secrets on the server
API keys, private tokens, and admin secrets should never be embedded in browser code.
Public frontend applications should call a controlled backend endpoint when privileged credentials are required.
That backend can then:
- authenticate the request;
- validate input;
- call the external service;
- enforce rate limits;
- hide credentials;
- return only the required output.
This pattern is especially important when a static frontend is connected to paid APIs.
Design for provider changes
A third-party integration is a dependency.
Providers can change:
- API versions;
- authentication flows;
- rate limits;
- field names;
- pricing;
- webhook formats.
Keeping each provider behind a small adapter makes replacement easier.
The internal application should depend on a stable interface rather than on provider-specific details everywhere.
A useful mental model
Treat integrations as pipelines rather than glue code.
Receive -> Validate -> Transform -> Execute -> Confirm -> Record
This structure makes the workflow testable and observable.
It also scales from a simple form-to-email connection to more complex CRM, reporting, billing, and data synchronization projects.
For a lightweight version of this pattern, see Static Forms with Google Sheets and Apps Script.
Need several business tools to work as one workflow? Describe the systems and desired outcome here.