A text analytics computer program separated the phrase "I like cake" into three sections: "I," "like," and "cake." This is an example of ________ within the text preprocessing step.
A) tokenization
B) stemming
C) lemmatization
D) stop words removal

Question

Quizplus · Accepted Answer

Tokenization is the process of breaking down text into individual elements, such as words or phrases. In this example, the phrase "I like cake" is broken down into the tokens "I," "like," and "cake."

A Text Analytics Computer Program Separated the Phrase "I Like