ikieEngineeringGet started
All posts

TasksSyncPorts and adaptersWebhooks

Seven other apps' tasks, one engine

Tasks from Notion, Slack, Todoist, Linear, Google Tasks, Microsoft To Do and GitHub land in the ikie inbox. The only thing that differs per source is the field mapping, so each adapter is 160 to 350 lines. Three fields merge three ways, and webhook bodies are treated as signals, never trusted.

한국어

Tasks don't live in one app. Work goes in Linear, personal things in Todoist, shared docs in Notion, and what someone hands you arrives as a Slack message. ikie's inbox aims to be where all of them gather. Open tasks assigned to you in other apps come in, and you decide when to do them in ikie.

On October 16 we connected seven sources in a day. With the first (Notion) we built the port and the engine; the other six only added adapters.

  1. 04:16Notion: port, three-way merge, engine and tables together (#557)
  2. 13:42Notion webhooks, changes in seconds (#583)
  3. 17:06Slack: add a task with a message shortcut (#592)
  4. 19:45Todoist: one list per project, webhooks (#598)
  5. 22:52Linear: issues assigned to me, webhooks (#600)
  6. 23:02Google Tasks, Microsoft To Do and GitHub issues (#603)

Shaped like calendar sync

ikie already merged external calendars field by field, three ways. Task sources got the same shape. The difference is the number of fields: calendars have many field groups, tasks have just three: title, done and due.

  1. Decisionpackages/core/src/task-sync.ts

    Merges the three fields three ways and decides what to write to ikie and what to send to the source. Knows no database or network.

  2. PortTaskSourcePort

    Source name, lists, add a field, open tasks, reread one, send changed fields. Those six are all of it.

  3. Enginelib/data/task-sync.ts

    Reads through the port and writes only what the decision says. Locks a source so two places never sync it at once.

  4. Adapters ×7lib/data/<source>

    Turn each source's API into the port's shape. Field mapping sits in core, one pure file per source.

The engine doesn't know which source it's talking to. It calls one port, and pure calculation decides what to write.

All that differs per source is "which of its fields are title, done and due". So the adapters are small.

See the numbers
SourceAdapter (mapping + API)
Notion345 lines
Todoist254 lines
Linear226 lines
Slack196 lines
GitHub185 lines
Google Tasks164 lines
Microsoft To Do160 lines
Lines per source in its core mapping file plus its lib/data adapter folder, tests excluded. The engine (155 lines) and the decision (104 lines) are shared by all seven.

Notion is the largest because its fields aren't fixed. Every Notion database has different properties, so someone has to pick which one means done and which means due. Sources with fixed fields, like Todoist, Linear and Google Tasks, have nothing to pick.

Three fields, and the source wins a tie

Every sync compares three values per field: the last synced value (the base), ikie's current value and the source's current value. Whichever differs from the base is the side that changed.

FieldBase (last synced)ikieSource (Linear)Merged
TitleFix checkout screenFix checkout screen copyFix checkout screenFix checkout screen copyOnly ikie changed: send to source
DoneOpenOpenDoneDoneOnly source changed: write to ikie
Due10/20 15:0010/20 15:0010/2010/20 15:00Same day, date only: unchanged
Three sides per field. If both changed, the source wins. The source owns the task; ikie adds when to do it.

The due row was the tricky part. Linear and Google Tasks store due as a date only. Set "the 20th at 3 pm" in ikie and send it, and the source returns just "the 20th". Compared naively, it looks like the source changed the due date, and ikie's time gets wiped. So when the base has a time and the source returns the same day as a date only, the source counts as unchanged (#600).

Microsoft To Do also stores a date, which is read and written in the person's time zone. The same 3 pm can fall on different days in Seoul and London.

What differs per source is the mapping

SourceListsFields syncedNotices changes by
NotionPer databaseTitle, done, due (you pick the fields)Webhook
TodoistPer projectTitle, done, dueWebhook
LinearPer teamTitle, done, due (date only)Webhook
Google TasksPer task listTitle, done, due (date only)Cron
Microsoft To DoPer listTitle, done, due (the person's day)Cron
GitHubOne list of assigned issuesTitle, done (close)Cron
SlackCollected messagesNone (linked when added)Message shortcut

What comes in is open, and assigned to you. Todoist brings only tasks with no assignee or assigned to you; Linear and GitHub bring only issues assigned to you. Other people's work shouldn't fill your inbox. Recurring Todoist tasks are left out: completing one moves it to the next occurrence and leaves it open, so syncing done would keep advancing it.

Slack is a source with no fields to sync. Tap "Add to ikie" on a message and it lands in the inbox with a link back to Slack. It's recorded in the same binding table as the other sources, so the task row gets the Slack logo and open link the same way.

Notion doesn't guess

A Notion database might have a checkbox called "Important" instead of "Done", or a date called "Start date" instead of "Due". At first, a lone checkbox was guessed to mean done. Then finishing a task in ikie would tick "Important" in Notion, changing someone else's data the wrong way.

So we cut the guessing (#568, #571).

  • A field whose name doesn't match done or due isn't used, even if it's the only one.
  • If there's a status select, a person picks which status counts as done.
  • If nothing matches, a person picks from existing fields, or ikie adds a done (checkbox) or due (date) field to that database.

The chosen mapping is stored per list, and the adapter follows it first.

The side that changes someone else's data doesn't guess. When unsure, let a person pick, or add a new field.

Webhook bodies aren't trusted

Syncing has to happen even when the app isn't open. So there are three paths.

  • On opening the app and after writes: runs together with calendar sync.
  • Webhooks: Notion, Todoist and Linear call ikie when something changes. ikie checks the signature, responds, then syncs that person's source.
  • Cron: runs every 5 minutes and syncs up to 20 sources, oldest first, that haven't synced in 10 minutes. Sources without webhooks rely on this.

The webhook body is used only as a signal. ikie reads as far as "whose source changed", then rereads the values from the source's API. Titles or statuses in the body are never written as-is, so webhooks arriving out of order or twice give the same result. With one sync path, webhooks, cron and app opens all ride the same engine.

Each source signs differently. All three compute HMAC-SHA256 over the raw body with a secret and compare, and Linear also rejects webhooks more than a minute old. Rewriting the body as JSON would change its bytes, so the raw text is what gets measured.

Disconnecting loses nothing

Disconnecting deletes only the token; the source, lists and bindings stay. Reconnect the same account and they attach again, so the same task doesn't come in twice (#633).

  • If a task moves to another list in the source, its binding moves with it.
  • If it disappears from the source, only the time it vanished is noted, and syncing resumes if it comes back.
  • A task deleted in ikie keeps its binding so it isn't imported again. Nothing is deleted in the source.

Connecting is account linking, not a way to sign in. Google Tasks and Microsoft To Do use the same keys as Google and Outlook sign-in but keep a separate connection, so disconnecting tasks leaves sign-in and calendars alone. Tokens are encrypted in the database.

What we learned

  • Build the port with the first source, and from the second on only adapters remain. Six of the seven added only a mapping and API calls.
  • Watch for echoes from date-only sources. A value that comes back trimmed isn't a change.
  • Don't change someone else's data on a guess. Even a lone field gets a question if its name doesn't fit.
  • A webhook is a signal. Reread values from the source, and ordering, duplicates and forgery become much less to worry about.

How calendars merge field by field is in the sync post, and how we rebuilt the inbox these tasks land in is in the inbox post.

The ikie teamikie is a calendar for your events and to-dos in one place. Here we write about the problems we run into while building it, and how we solve them.