Git diff token counter

What it does

Paste the output of git diff, git show or git format-patch — or open a .diff or .patch file — and choose a model. The counter reads the diff into its files and hunks and shows:

• every file, most tokens first, with what the diff does to it (modified, added, deleted, renamed, copied, a mode change) and its lines added and removed;
• the tokens of each file and of each hunk, and how that number was made — exact or estimate;
• filters it suggests — lockfiles, minified files, source maps, build output, test snapshots, generated and binary files — each with the files and tokens it would take out;
• the tokens of what you keep, and whether that fits each model's context window with room for the review.

Then download or copy the diff you kept. Everything runs in your browser, in a Web Worker: a 5 MB diff is read and counted without stopping the page.

How it reads a diff

Files start at diff --git. A diff -… line (diff -u, diff -r), svn's Index: or a --- line followed by +++ starts one too, but only once a hunk or a binary line follows it — so words in a commit message such as "diff the outputs" stay part of the message. Each Binary files … differ line that diff -r writes is a file of its own. A hunk is its @@ -a,b +c,d @@ line and exactly the lines it promises, as git apply reads it — so a removed line that reads -- comment is a line, not a header. A hunk whose lines run out early is counted as it is and marked as cut off.

Names are read in git's order of trust: rename from / rename to and copy from / copy to, then the --- and +++ lines, then the diff --git line itself. Names git quotes — spaces at the ends, non-ASCII letters, quotes — are unquoted by git's own rules. Text before the first file, such as a commit message, is listed on its own and kept unless you untick it.

How it counts

Each file and each hunk is counted with the same code as the Token Counter: exactly for OpenAI's models, with OpenAI's own tokenizer in your browser; as an estimate for Anthropic, Google and xAI, ⌈characters ÷ 4⌉. The total is the kept diff counted as one text — the number the Token Counter gives if you paste that diff into it. Files counted one by one can add up to a few tokens more than that.

The suggested filters

A closed list, matched only by a file's name or by a line of the diff:

• Lockfiles, by exact name in any directory: package-lock.json, npm-shrinkwrap.json, pnpm-lock.yaml, yarn.lock, bun.lock, bun.lockb, Cargo.lock, poetry.lock, Pipfile.lock, uv.lock, go.sum, composer.lock, Gemfile.lock, Podfile.lock, packages.lock.json, flake.lock.
• Minified files: *.min.js, *.min.css.
• Source maps: *.map.
• Build output: anything under a dist/ or build/ directory.
• Test snapshots: *.snap, or anything under __snapshots__/.
• Marked as generated: a line of the diff containing @generated, or Go's // Code generated … DO NOT EDIT.
• Binary files: Binary files … differ, or a GIT binary patch.

They are suggestions. Nothing is left out until you choose a filter, and a file you tick back in is kept whatever it matched.

The export

The kept diff is the kept files exactly as you pasted them, joined in their order: line endings (CRLF too), a missing final newline and every header are kept byte for byte. Keep every file and the export is your diff, unchanged.

A browser's text field hands back every line ending as LF, so a diff with CRLF line endings — and one over a million characters, which a text field makes slow to type in — is kept on the page rather than in the field: paste it, drop it or open the file, and it is read, counted and exported exactly as it came. To use another diff, clear the one that is held first. A diff dropped into a field that already holds text is added after what the field held, on a new line: the browser does not say where in the field a drop lands, and nothing in the field is replaced. When the field's text does not end in a line break, one is put between the two, in the dropped diff's own style (CRLF if it uses CRLF); the dropped text itself is kept as it came.

A file that starts with a UTF-8 byte-order mark (U+FEFF) is read past the mark, which is kept in the export byte for byte — git itself does not remove it — and the tool says the file starts with one.

A file you open is read as UTF-8 byte for byte, a byte-order mark included. A file that is not UTF-8 is still read, with the bytes that are not UTF-8 replaced by “�”; the tool says so, and its export differs from the file there.

A diff whose lines end in a carriage return alone (CR, classic Mac) is one long line to git, and the tool says so rather than reading it as lines. A file name is drawn up to its first 300 characters; the export keeps it whole.

What it does not know

• Exact counts for Anthropic, Google and xAI, which need their APIs.
• A repository. It reads the diff you paste; it does not fetch a pull request or a branch.
• Context windows the build could not read — OpenAI's, Google's and xAI's — shown as unknown.
• What your review prompt adds. Plan the instructions with the Context window budget planner.

FAQ

Why is a lockfile so many tokens?

Lockfiles repeat package names, versions, URLs and hashes, and hashes are among the most expensive text a tokenizer meets: few of their pieces are in its vocabulary.

Is my diff sent anywhere, or put in a link?

No. It is read and counted in your browser, and a shared link carries the model, the filters you chose and the tokens kept for the review — never the diff, and never a file's name. The copied summary holds numbers only.

Sources

The token counts come from the Token Counter's own layer: OpenAI's encodings, run with the gpt-tokenizer library and tested against OpenAI's tiktoken. Context windows and output limits are Reference Data, each with its source page and date. The diff format is git's own, as git apply reads it.