Part 6 – Using Java 8 BiPredicate to externalize decisions

[The code for this example is available on GitHub here.]

We have implementation specific decisions in our GrabManager class – bad idea!  We could clean that out with the Template Method pattern.  Not a bad pattern, but not  the best solution now that we have Predicates in Java 8. We really need to replace GrabManager.shouldVisit().  We need to separate implementation specific decisions from  core decisions.  Let’s work thru this decision making and coding in detail.  Here is the current shouldVisit() method:

In this method are universally applied decisions like “Have we been here before” and implementation specific decisions like, “Never leave this server.”  So how do you let a user of this class add decisions without screwing up the universal ones?


We could use the Template Method pattern.  This way you can defer certain functionality to a concrete subclass.  This is a classic pattern that’s been around since the dawn of OOP.  But should it be necessary for the consumer to subclass GrabManager to override this method?  I don’t think so.  Not for this reason at least.

The other method is to pass a new shouldVisit method into the GrabManager.  I really hate all the typing to implement this way.  But before Java 8 this is the sort of thing we had to do.  First, we need to create a Decide interface in the class and keep a copy of the object:

Then we change the shouldVisit() method to use the decide object properly:

 ….. oh my this is getting tedious! ……

Now we need to define a decide object for the GrabManager:

Oh!  That is soooo Java 7.

OK – forget you ever saw that business.  Let’s move on to using Java 8 predicates.  In a nutshell, a predicate is little more than a piece of logic that returns true or false.  The cool part about Java 8 predicates is that you can combine them with and’s and or’s.  Once you have a Predicate, you can pass it around the system just like any other object.  It does everything we did above with a nice clean syntax.  Let’s take a look at a better implementation.  First we are going to use the BiPredicate interface, because we have 2 things to test: the URL and the DEPTH.


shouldVisit is now a BiPredicate object.  When we pass it a URL and an Integer it will respond to test() with a true or false.  We use it here:

Note there is no special interface to define or superfluous syntax.  Nice and neat.  But next comes the cool part.  How do you create a BiPredicate object?  How I express the decision making process?  Is it easier to read/understand than my old method?

If this is new to you, it might not be readable.  However, once you get used to Java lambda expressions it’s a beautiful thing.  First the “->” syntax clues you in that we are working with a lambda.  On the left of it is the parameters it uses, on the right of it is the code.  In many cases you don’t need { … } around your code, but you can if you like.  You should find 3 separate lambda expressions.  They are joined together with ‘.and()’.  Finally, we pass our BiPredicate object to the GrabManager and away we go.

Final thoughts

I use this little crawler at work.  It’s fun, easy to configure, returns great results.  Let me show you another example of the predicate to reinforce the idea.  In this case we want to find all the news articles on yahoo news that reference obama in the URL:

We have successfully extracted the shouldVisit decision making from the GrabManager and exposed it to the caller.  We did that without creating a sub class or an interface or any such syntactic sugar.

Next I think it’s time we start storing our extracted information in a Graph Database, don’t you?