Download as docx, pdf, or txt
Download as docx, pdf, or txt
You are on page 1of 1

Here's a step-by-step breakdown of how CBOW would process this:

1. Tokenization: First, we split the sentence into tokens (words). Our tokens are: ["The",
"quick", "brown", "fox", "jumps", "over", "the", "lazy", "dog"].

2. Choosing Context Window Size: Let's choose a context window size of 1 for simplicity. This
means we'll consider one word before and one word after the target word as its context.

3. Creating Training Samples: With a window size of 1, the training samples will look like this:

 Context: ["The"], Target: "quick"

 Context: ["quick", "brown"], Target: "The"

 Context: ["The", "fox"], Target: "quick"

 ... and so on for each position in the sentence.

For example, for the word "fox," the training sample would be:

 Context: ["brown", "jumps"], Target: "fox"

4. Training: In the training phase, the model tries to predict the target word ("fox" in our
example) based on the context words ("brown" and "jumps"). Initially, each word is assigned
a random vector. Through training, these vectors are adjusted to maximize the prediction
accuracy.

5. Model Output: After training, each word will have a learned vector that represents it in the
word embedding space. These vectors capture semantic relationships based on the contexts
in which words appear.

In the real world, CBOW models are trained on large corpora and with larger context windows to
capture more complex patterns and relationships in language. But this simple example gives an idea
of the core process.

You might also like