Most of the time, you are just consuming the Document Jsoup built for you, like document.select(${selector}). It would be helpful to go through W3C CSS Selector Specification to understand Jsoup’s roadmap. We have talked about Node Traverse before, what Selector will actually do is just filtering and collecting while traversing, and the key point here will be parsing and evaluating given queries.
Overview of Package
Package org.jsoup.select didn’t change much in latest version 1.12.1-SNAPSHOT, I would reference UML diagram made by Yihua Huang directly here:

Besides NodeVistor the interface, there is another similar interface NodeFilter, their methods are nearly the same:
1 | public interface NodeFilter { |
There is only one implementation in production code, which is FirstFinder, 5 other implementations are all located in test and probably reserved for the future. As for FirstFinder, it’s easy to guess that it’s created in the purpose of optimization. You don’t need to collect all the matching elements and return the first one, instead of invoking select(${selector}).get(0), Jsoup introduces another method selectFirst(${selector}) which will be useful when you just care about the first one - a frequent scenario actually.
NodeFilter: Another Traverse
I don’t want to repeat Node Traverse here again, but it will be helpful to review it in another way - the implementation of filtering. The idea and workflow are totally the same, but a little bit longer. You may compare to NodeTraversor#traverse for better understanding.
1 | public static FilterResult filter(NodeFilter filter, Node root) { |
Using FirstFinder as an example, the idea is straightforward. It keeps the evaluator and the matching result, and return a FilterResult of STOP when the first matching element was found.
1 | private static class FirstFinder implements NodeFilter { |
The Evaluators
OK. We have gone through the traverse again via review the filtering function. Now it’s time to understand how Jsoup parse a given query. We will look at Evaluator, CombiningEvaluator, and StructuralEvaluator first.
Evaluatoris a abstract class with 39 inherited subclasses which cover querying by id, name, tagName(equals or endsWith), class, attribute name, attribute value, sibling index, texts, regex match and etc. Since we have set these properties well while parsing the DOM tree, it will be very easy to implement the abstract method:1
2
3
4/**
* Test if the element meets the evaluator's requirements.
*/
public abstract boolean matches(Element root, Element element);CombiningEvaluatorcombines multiple evaluators with logic expressionAndorOr, which can process combinators like",", ">", "+", "~", " "StructuralEvaluatoris used to simulate query likediv > p > spanordiv p span.ImmediateParentwill handlediv > p > spanwhileParentwill handlediv p span. We will take a look atParentimplementation:1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25static class Parent extends StructuralEvaluator {
public Parent(Evaluator evaluator) {
this.evaluator = evaluator;
}
public boolean matches(Element root, Element element) {
if (root == element)
return false;
Element parent = element.parent();
while (true) {
if (evaluator.matches(root, parent))
return true;
if (parent == root)
break;
parent = parent.parent();
}
return false;
}
public String toString() {
return String.format(":parent%s", evaluator);
}
}And
QueryParserthat will parse a CSS selector into an Evaluator tree, handle above cases like this:1
2
3
4
5
6// for most combinators: change the current eval into an AND of the current eval and the new eval
if (combinator == '>')
currentEval = new CombiningEvaluator.And(newEval, new StructuralEvaluator.ImmediateParent(currentEval));
else if (combinator == ' ')
currentEval = new CombiningEvaluator.And(newEval, new StructuralEvaluator.Parent(currentEval));
...
QueryParser Finally
After we explored different kinds of Evaluator, it will be easy to understand the parsing process in the below:
1 | private QueryParser(String query) { |
Easier than expected, right? The implementation here is clean, elegant and easy to understand. Besides that, java doc of Selector is also impressive, you can play around with Try Jsoup with Selector syntax.
-To Be Continued-
