What Are Java Tokens?
Page content

Java tokens

The tokens of a language are the basic building blocks which can be put together to construct programs. A token can be a reserved word (such as int or while), an identifier (such as b or sum), a constant (such as 25 or “Alice in Wonderland”), a delimiter (such as { or ;) or an operator (such as + or =).

For example, consider the following portion of the program we met in this article:

public class PrintSum { public static void main(String[] args) { int a, b, sum; a = 14; b = 25; sum = a + b;

Starting from the beginning, we can list the tokens (shown in bold) in order:

public - reserved word

class - reserved word

PrintSum - user identifier

{ - left brace, delimiter

public - reserved word

static - reserved word

void - reserved word

main - user identifier

( - left parenthesis, delimiter

String - class name identifier

[ - left square bracket, delimiter

] - right square bracket, delimiter

args - user identifier

) - right parenthesis, delimiter

{ - left brace, delimiter

int - reserved word

a - user identifier

, - comma, delimiter

b - user identifier

, - comma, delimiter

sum - user identifier

; - semicolon, delimiter

a - user identifier

= - equals sign, operator

14 - constant

; - semicolon, delimiter

and so on. Thus we can think of a program as a ‘stream of tokens’, which is precisely how the compiler views it. So that, as far as the compiler is concerned, the above could have been written:

public class PrintSum { public static void main(String[] args) { int a, b, sum; a = 14; b = 25; sum = a + b;

The order of the tokens is exactly the same; to the compiler, it is the same program. To the computer, only the order of the tokens is important. However, layout and spacing are important to make the program more readable to human beings.

Comments

All programming languages let you include comments in your programs. Comments can be used to remind yourself (and others) of what processing is taking place or what a particular variable is being used for. They can be used to explain or clarify any aspect of a program which may be difficult to understand by just reading the programming statements.

This is very important since the easier it is to understand a program, the more confidence you will have that it is correct. It is worth adding anything which makes a program easier to understand.

Remember that a comment (or lack of it) has absolutely no effect on how the program runs. If you remove all the comments from a program, it will run exactly the same way as with the comments.

Each language has its own way of specifying how a comment must be written. In Java, we write a comment by enclosing it within /* and */, for example:

/* This program prints a greeting */

A comment extends from /* to the next */ and may span one or more lines. The following is a valid comment:

/* This program reads characters one at a time

and counts the number of letters found */

Java also lets you use // to write one-line comments. The comment extends from // to the end of the line, for example:

a = s * s; //calculate area; store in a

In this series, we will use mainly one-line comments.

References

Reference text: Java Programming – A Beginner’s Course

Related programming references

This post is part of the series: Introduction to Java Programming

This is a series which aims to discuss and teach Java programming to the complete beginner. Absolutely no programming background is assumed.

  1. Introduction to Java Programming - An Overview
  2. Java - Data, Variable and Algorithm Explained To A Beginner
  3. Java Example: Algorithm and Program For Area of Square
  4. Java Programming For Beginners - Test, Debug, Document, Maintain
  5. JDK Java Compiler: The Java Development Kit
  6. Java Programming For Beginners - How To Compile And Run Java Programs
  7. Data Types, Constants And Variables
  8. Java Programming For Beginners - Characters and printf
  9. Java Programming For Beginners - Part 9
  10. Java Programming For Beginners - Part 10
  11. Java Programming For Beginners - Part 11
  12. Java Programming For Beginners - Part 12
  13. Java Programming For Beginners - Part 13
  14. Java Programming For Beginners - Part 14
  15. Java Programming For Beginners - Integer Data Types
  16. Java Programming for Beginners - Part 16
  17. Java Integer Arithmetic For Beginners
  18. Java Programming For Beginners - Part 18
  19. Java Programming For Beginners - Part 19
  20. Java double to int and Other Conversions