Saharsh Tibrewala
2 November 2026 · 10 minutes · cognee, open source, python, vercel

My Vercel connector passed its tests and could still forget your data

The reviewer synced one Vercel project called web, then synced again with the name typed as wbe. My connector found no project by that name, loaded three empty tables and finished. Projects gone, deployments gone, build logs gone. No error, no warning, and my own README said that was intended.

That was one of 17 comments in the first review of my Vercel connector for Cognee, on October 20. 62 tests were green at the time. Over the next ten days there were five more reviews and 26 more inline comments. When I lined them up afterwards, almost all of them had the same shape: memory ends up wrong, and nothing tells you.

This post is about that shape. I think it is the main thing to get right in any connector that keeps an AI agent's memory in step with an outside system.

What the connector does

Cognee is a memory layer for AI agents. You add documents, it builds a knowledge graph and embeddings from them, and an agent searches that later. Connectors live in cognee-community and pull an outside source into Cognee with dlt, a Python library for loading data.

The Vercel issue, cognee#4551, was opened by a maintainer and labelled hackathon for Mergetober, a month of contributions to Cognee run with WeMakeDevs. I was assigned to it and opened cognee-community#286 on October 8. The idea is that an agent can tell you why last night's deploy failed without anyone opening the Vercel dashboard.

One sync: three Vercel API calls, dlt's rest_api source, rows rebuilt from a fixed field list, and one Cognee document per project, deployment and failed build log

It makes three calls. GET /v10/projects lists projects. GET /v7/deployments lists deployments. With build logs turned on, GET /v3/deployments/{id}/events fetches the log of each failed build. Each project, deployment and failed build log becomes one document.

Every row is rebuilt from a fixed list of fields, and that mattered on the first day. On my own account, the projects list returns env entries with their value. A connector that copied the project object whole would have put environment variables into the graph without ever calling an env endpoint.

A snapshot, not a cursor

The usual way to sync is a cursor: remember the newest thing you saw, and next time ask only for newer ones. Vercel's since filter works on creation time, so a cursor misses two things. A build that goes from BUILDING to ERROR after you read it never comes back. A deleted deployment just stops being listed, so nothing ever says it is gone.

So every run reads a fixed window in full, 30 days by default, and loads it with dlt's replace. Cognee 1.6.1 added orphan cleanup per table: anything that was in the last load and is not in this one is forgotten. An unchanged row keeps its content hash and its document id, so it is not processed again.

On my account the first live run read 16 projects, 115 deployments and 3 failed build logs in 7 requests. A second run kept every document id. Narrowing the window from 60 to 30 days forgot 10 documents and left the rest alone.

That is the whole model, and it is also where the danger is. Under replace, a load that is short by mistake looks exactly like a load where things were deleted. Any bug that makes a read come back short turns into memory that forgets, quietly. That is what the review kept finding.

Where the secrets went

Before anyone reviewed it, I ran a canary test. I made a test project with five deployments (two good, two failed builds, and one that failed on a missing root directory) and six fake environment variables. Then I synced it and searched every file the run wrote for the fake values.

Canary results: in the project object, plain values came back as clear text and sensitive ones empty. In a failed build log, a 49 character sensitive value was redacted and a 19 character one was printed in full

The project object came back with the plain values in clear text, the encrypted one as ciphertext, and the sensitive ones empty. The connector keeps none of those fields. The variables no build printed were nowhere in staging, in dlt's load files or in Cognee's data folder.

The build log was different. One failed build printed its environment. Vercel replaced the 49 character sensitive value with [REDACTED], and printed the 19 character sensitive value and the plain one as they were. Vercel only redacts sensitive values of 32 characters or more, so a build log document would carry the shorter ones into memory.

I wrote that down under the limits in the PR, and still left build logs on by default. The reviewer, goran-radonic, pushed echo STRIPE_KEY=sk_live_Abc123ShortSecret through cognee.add() and cognify() with the default settings, and it came out as node text in the graph. My README also still said "Environment variables are never ingested", which that run disproved. Build logs are now off by default. In the fifth review the example was flagged for still turning them on, since people copy examples, and now it uses the default too.

The clean up check passed. A sync with nothing changed kept all 9 document ids. I deleted one failed deployment in Vercel, and the next sync forgot its 2 documents, the deployment and its build log, and kept the other 7. With a real LLM on top, the answer named both failed deployments before the delete and only the remaining one after it.

Six reviews

Timeline of pull request 286: six reviews between October 20 and October 30, with 17, 15, 6 and 5 inline comments, and the reviewer's test runs going from 62 to 103 to 114 green

The first review listed four points to fix before merge, and said why in one line: "they're all places where memory ends up wrong without any error". The second, on October 21, still had three points open. The third and fourth, the same day, found new problems that my own fixes had created. By the sixth, on October 30, the comments were about how long a run holds a lock, and a missing return type.

Every review said how each point was checked. Goran ran each one through a real dlt pipeline and Cognee, and from the fourth review on also ran mutations: take a line out of my fix and see whether any test fails. In the fifth review, three of my guards could be removed with all 103 tests still green. That changed how I test. Every guard I add now gets a test that I have watched fail with the guard taken out.

The bug in the page cursor

The most interesting bug is easy to miss.

Two deployments created in the same millisecond. With until equal to the last created time, the second one is never listed. With until one millisecond later, it is loaded and the repeated item is dropped

Vercel lists deployments newest first, 100 to a page. Each page comes back with pagination.next, the created time of the last deployment on that page, in milliseconds. You send it back as until to get the next page, and until is exclusive: the next page only has deployments created strictly before it.

Now let two deployments share a millisecond, and let the page end between them. The first is the last item on page 1. The second has the same created time, so until leaves it out of page 2, and it is never listed. Under replace, a deployment that is never listed is forgotten on every sync.

Goran found it with the fake Vercel server in my own tests, which already modelled until as exclusive. I checked it against the real API: with until=next the last item of page 1 is not returned, and with until=next+1 it is. So the connector asks for until=next+1. Each page starts with the item the previous one ended on, and that repeat is dropped by id before anything is loaded. It costs one repeated item per page.

If 100 deployments ever share one millisecond, the cursor stops moving. The run then stops too, instead of asking for the same page forever, and memory stays as it was. The projects list pages the same way. I checked that as well rather than assume it: /v10/projects takes an exclusive until too, with next set to the last item's updatedAt.

Everything else had the same shape

Six things that went wrong without an error before the review, and what the connector does now

Once I saw the pattern, the fixes followed one rule. When the connector cannot be sure a read is complete, it stops the run, and the next run reads everything again. A stopped run leaves memory as it was. A finished run with a short read deletes things.

  • A typo in project_ids. A listed project the account does not have now stops the run, and the error says how many projects the token can see and which entries are missing. A bare string is rejected up front: project_ids="prj_web" used to be split into its letters, match nothing, and forget everything. If you really did delete a project, allow_empty=True lets one sync through to forget it. The connector then remembers that entry with the time of that run, so later runs skip it with a warning instead of stopping.
  • A token for the wrong account. A token for another team answers with a perfectly valid list of its own projects. The connector stores the accountId of the projects it saw, and a run whose projects all belong to a different account stops.
  • An expired build log. When Vercel drops an old log, the events call answers 404 or 410 while the deployment is still listed. The connector now reads the last copy back from staging and loads it again unchanged, so the document stays until the deployment goes. Only a missing table counts as nothing to keep. My first version caught every exception there, so Goran made the staging read fail with "database is locked" and showed the log being dropped. Any other staging error now stops the run.
  • Terminal escape codes. Build logs are full of colour codes. My first pattern removed an unclosed escape sequence along with everything after it, so a\x1b]0;title never ends Error: real failure came out as a, and the error line was gone.

Retries, five versions

Retries took the most rounds. Each version was wrong in a way I did not see until someone measured it.

dlt's default session retries a request up to four times and honours Retry-After up to 300 seconds. In Goran's first probe, one endpoint slept 1200 seconds. That wait happens inside Cognee's staging lock, so every other dlt ingestion in the process waits as long.

My first fix capped each wait at 60 seconds. With Retry-After: 1000 that still came to 240 seconds per request. The second stopped retrying a 429 but still waited up to 60 seconds for a 5xx, per request: six failed builds whose events call got a 503 slept 360 seconds. The third review asked whether a 5xx needed a retry at all, since the next run reads everything again anyway, so the third version had no retries.

The fifth review did the arithmetic on that. About 225 requests per run with a 0.1% failure rate means roughly one run in five fails and starts over from page 1. So the fourth version gave a 5xx or a dropped connection one retry, after at most 5 seconds. The sixth review found that this was per request, not per run: three 503s on three requests slept 5, 5 and 5 seconds. It also found that dlt retries a timeout, so a hung request was tried twice, 30 seconds each time.

The fifth version is in the PR now. dlt's own retry rules are switched off, and a small budget is passed in as dlt's retry_condition. It allows one retry for the whole run, for a 5xx or a dropped connection, after at most 5 seconds. A 429 is never retried and neither is a timeout. Retries add at most 5 seconds to a run.

Where it stands

Pull request 286 on GitHub: open, 10 commits from feature/vercel-connector into topoteretes:main

The PR is still open. It has 10 commits and 122 tests, which pass on Python 3.10 to 3.14 and on the oldest dlt it supports, 1.9.0, in a CI workflow added in the fifth round. Every review comment has a reply on its thread, with the commit that fixed it and the test that checks it.

One question from the fifth review is open on purpose. A snapshot of 30 days forgets everything older, so "what failed last quarter" has no answer. Keeping history would mean merge with a delete marker, and some other way to learn about deletions, since a deleted deployment just stops being listed. I offered to do that as a follow up once this PR is in. Until then, a longer lookback_days, or None for everything Vercel still keeps, is the way to keep last quarter.

I wrote a Prometheus connector the same month, and every point from this review that applied went into it too. Everything I opened for Mergetober has the rest.

more writing

all the writing