reza.rosli
← All cards

Tools & ideas

TF-IDF link discovery

Deterministic similarity for connected notes

TF-IDF is a way to represent a document by the words that are frequent within it but comparatively uncommon across the wider collection. The initials stand for term frequency–inverse document frequency.

Term frequency

Term frequency counts how often a word appears in one document. A word used several times contributes more to that document’s representation than a word used once.

Inverse document frequency

Inverse document frequency reduces the weight of words that appear in many documents and increases the weight of words that distinguish a smaller part of the collection. In this studio, the weighting uses a smoothed logarithmic calculation, so every known term receives a usable weight without treating a rare word as unlimited evidence.

How the content studio uses it

For each card, the studio combines its title, description, introduction, and body. It removes HTML tags, converts the text to lowercase, and extracts words of three or more characters. The complete card collection forms the comparison corpus.

Each card becomes a TF-IDF vector. The studio measures the cosine similarity between the selected card and every eligible card: vectors pointing in a similar direction have more similarly weighted vocabulary. It excludes the selected card and cards that are already outgoing connections.

The lexical score is then supplemented by two small contextual signals: shared graph neighbours and membership in the same card group. Candidates are sorted by the combined score. “Find candidates” keeps the strongest results and creates private proposals for review.

Deterministic discovery, not AI inference

“Find candidates” does not call OpenAI, Ollama, or another generative model. The same repository content and graph produce the same ranking. It suggests where to look; it does not establish that a relationship is true.

“Ask OpenAI” and “Ask Ollama” are separate optional steps. They receive only the shortlisted candidates and approved relationship types, then return structured suggestions. Those results are validated and stored as proposals rather than written directly into a card.

What the score cannot decide

TF-IDF mainly notices shared vocabulary. It can miss related ideas expressed through different words, overvalue repeated specialist terms, and surface proximity that is interesting but not editorially meaningful. Graph and group signals add context, but they do not turn a score into evidence. A person still chooses the relationship type, checks the reason, and accepts or dismisses the proposal.

Browse the connectome

Immediate connections

Explore the cards directly connected to this one.

Connected cards

Follow links in either direction, or find cards that mention this one.

Show