Professional Documents
Culture Documents
Cbow
Cbow
1. Tokenization: First, we split the sentence into tokens (words). Our tokens are: ["The",
"quick", "brown", "fox", "jumps", "over", "the", "lazy", "dog"].
2. Choosing Context Window Size: Let's choose a context window size of 1 for simplicity. This
means we'll consider one word before and one word after the target word as its context.
3. Creating Training Samples: With a window size of 1, the training samples will look like this:
For example, for the word "fox," the training sample would be:
4. Training: In the training phase, the model tries to predict the target word ("fox" in our
example) based on the context words ("brown" and "jumps"). Initially, each word is assigned
a random vector. Through training, these vectors are adjusted to maximize the prediction
accuracy.
5. Model Output: After training, each word will have a learned vector that represents it in the
word embedding space. These vectors capture semantic relationships based on the contexts
in which words appear.
In the real world, CBOW models are trained on large corpora and with larger context windows to
capture more complex patterns and relationships in language. But this simple example gives an idea
of the core process.