What is the term used for the character that separates tokens?
A) tokenizer
B) delimiter
C) whitespace
D) separator

Name: What is the term used for the character that separates tokens? A) tokenizer B) delimiter C) whitespace D) separator
Brand: Quizplus
Rating: 4.8 (35 reviews)

Question

Quizplus · Accepted Answer

The term used for the character that separates tokens is the delimiter. A tokenizer is a program or function that breaks down input into individual tokens. Whitespace and separators are related but do not specifically refer to the characters used to separate tokens.

What Is the Term Used for the Character That Separates