Hazelcast: Java Open Source In-Memory Data Grid

Hazelcast logo

At work, we’re currently using the open source version of ehcache in our apps. In our next software stack refresh we’ve talked about investigating a distributed caching framework. I think that Hazelcast might be a contender. I like that it allows you to store date in simple Lists, Sets, Maps, and Queues and handles the distributed part for you. The question we’ll need to answer at work is “how tunable is Hazelcast?”

From the website:

  • Distributed java.util.{Queue, Set, List, Map}
  • Distributed java.util.concurrency.locks.Lock
  • Distributed java.util.concurrent.ExecutorService
  • Distributed MultiMap for one to many mapping
  • Distributed Topic for publish/subscribe messaging
  • Distributed Indexing and Query support
  • Transaction support and J2EE container integration via JCA
  • Socket level encryption for secure clusters
  • Write-Through and Write-Behind persistence for maps
  • Java Client for accessing the cluster remotely
  • Dynamic HTTP session clustering
  • Support for cluster info and membership events
  • Dynamic discovery
  • Dynamic scaling
  • Dynamic partitioning with backups
  • Dynamic fail-over
  • Web-based cluster monitoring tool

Here are some code snippets from the website:

Map

import com.hazelcast.config.Config;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;

import java.util.concurrent.ConcurrentMap;

public class DistributedMap {
    public static void main(String[] args) {
        Config config = new Config();
        HazelcastInstance h = Hazelcast.newHazelcastInstance(config);
        ConcurrentMap<String, String> map = h.getMap("my-distributed-map");
        map.put("key", "value");
        map.get("key");
        //Concurrent Map methods
        map.putIfAbsent("somekey", "somevalue");
        map.replace("key", "value", "newvalue");
    }
}

MultiMap

import com.hazelcast.config.Config;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.core.MultiMap;

import java.util.Collection;

public class DistributedMultiMap {
    public static void main(String[] args) {
        Config config = new Config();
        HazelcastInstance h = Hazelcast.newHazelcastInstance(config);
        MultiMap<String, String> multiMap = h.getMultiMap("my-distributed-multimap");
        multiMap.put("key", "value1");
        multiMap.put("key", "value2");
        multiMap.put("key", "value3");

        Collection<String> values = multiMap.get("key");

        // remove specific key/value pair
        multiMap.remove("key", "value2");
    }
}

Queue

import com.hazelcast.config.Config;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;

import java.util.concurrent.BlockingQueue;
import java.util.concurrent.TimeUnit;

public class DistributedQueue {
    public static void main(String[] args) throws InterruptedException {
        Config config = new Config();
        HazelcastInstance h = Hazelcast.newHazelcastInstance(config);
        BlockingQueue<String> queue = h.getQueue("my-distributed-queue");
        queue.offer("item");
        String item = queue.poll();

        //Timed blocking Operations
        queue.offer("anotheritem", 500, TimeUnit.MILLISECONDS);
        String anotherItem = queue.poll(5, TimeUnit.SECONDS);

        //Indefinitely blocking Operations
        queue.put("yetanotheritem");
        String yetanother = queue.take();
    }
}  

Topic

import com.hazelcast.config.Config;
import com.hazelcast.core.*;

public class DistributedTopic implements MessageListener<String> {
     public static void main(String[] args) {
         Config config = new Config();
         HazelcastInstance h = Hazelcast.newHazelcastInstance(config);
         ITopic<String> topic = h.getTopic("my-distributed-topic");
         topic.addMessageListener(new DistributedTopic());
         topic.publish("Hello to distributed world");
     }

     @Override
     public void onMessage(Message<String> message) {
         System.out.println("Got message " + message.getMessageObject());
     }
}

Lock

import com.hazelcast.config.Config;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;

import java.util.concurrent.locks.Lock;

public class DistributedLock {

    public static void main(String[] args) {
        Config config = new Config();
        HazelcastInstance h = Hazelcast.newHazelcastInstance(config);
        Lock lock = h.getLock("my-distributed-lock");
        lock.lock();
        try {
            //do something here
        } finally {
            lock.unlock();
        }
    }
}

Executor Service

import com.hazelcast.config.Config;
import com.hazelcast.core.Hazelcast;
import com.hazelcast.core.HazelcastInstance;
import com.hazelcast.core.IExecutorService;
import com.hazelcast.core.Member;
import java.io.Serializable;

public class DistributedExecutorService {
    public static void main(String[] args) {
        Config config = new Config();
        HazelcastInstance h = Hazelcast.newHazelcastInstance(config);
        IExecutorService ex = h.getExecutorService("my-distributed-executor");
        ex.submit(new MessagePrinter("message to any node"));
        Member firstMember = h.getCluster().getMembers().iterator().next();
        ex.executeOnMember(new MessagePrinter("message to very first member of the cluster"), firstMember);
        ex.executeOnAllMembers(new MessagePrinter("message to all members in the cluster"));
        ex.executeOnKeyOwner(new MessagePrinter("message to the member that owns the following key"), "key");
    }

    static class MessagePrinter implements Runnable, Serializable {
        final String message;

        MessagePrinter(String message) {
            this.message = message;
        }

        @Override
        public void run() {
            System.out.println(message);
        }
    }
}

ØMQ (AKA zeromq): Open-source async socket/messaging library & concurrency framework with support for 40+ languages & most OSes

zeromq logo

I found out about this project while listening to a FLOSS Weekly episode on SaltStack.

From the website:

  • The socket library that acts as a concurrency framework.
  • Carries messages across inproc, IPC, TCP, and multicast.
  • Connect N-to-N via fanout, pubsub, pipeline, request-reply.
  • Asynch I/O for scalable multicore message-passing apps.
  • Large and active open source community.
  • 40+ languages including C, C++, Java, .NET, Python.
  • Most OSes including Linux, Windows, OS X.
  • Free software with full commercial support.

Check out the Learn the Basics page. There’s some good stuff there.

SQLAlchemy: Python SQL Toolkit and Object Relational Mapper

SQLAlchemy masthead from site

SQLAlchemy looks cool enough to make me go hunting for a Java equivalent for use on projects at work.

Take a look at the features page. I’m particularly interested in the Unit Of Work

The Unit Of Work system, a central part of SQLAlchemy’s Object Relational Mapper (ORM), organizes pending insert/update/delete operations into
queues and flushes them all in one batch. To accomplish this it performs a topological “dependency sort” of all modified items in the queue so as
to honor inter-row dependencies, and groups redundant statements together where they can sometimes be batched even further. This produces the
maximum efficiency and transaction safety, and minimizes chances of deadlocks. Modeled after Fowler’s “Unit of Work” pattern as well as
Hibernate, Java’s leading object-relational mapper.

…and Raw SQL statement mapping

SQLA’s object relational query facilities can accommodate raw SQL statements as well as plain result sets, and object instances can be generated
from these results in the same manner as any other ORM operation. Any hyper-optimized query that you or your DBA can cook up, you can run in
SQLAlchemy, and as long as it returns the expected columns within a rowset, you can get your objects from it. Statements which represent multiple
kinds of objects can be used as well, with results received as named-tuples, or with dependent objects routed into collections on parent objects.

SQLAlchemy supports…

…dialects for SQLite, Postgresql, MySQL, Oracle, MS-SQL, Firebird, Sybase and others, most of which support multiple DBAPIs. Other dialects are
published as external projects. The corresponding DB-API 2.0 implementation (or sometimes one of several available) is required to use each
particular database. View Current DBAPI Support.

Bruce Eckel is a Java guy I respect. He wrote Thinking in Java and he said this about SQLAlchemy:

SQLAlchemy is a pretty amazing design…In SQLAlchemy, you need to explicitly start a session. What’s amazing is that all the changes you make
during that session are kept in some kind of parse tree, and then when the session ends SQL is created on-the-fly to produce a single, optimal
SQL statement for that particular sequence of changes. I found this idea pretty mind-blowing.

That sounds pretty sexy to me.

Clink: Add bash command line editing to Windows cmd.exe

Screenshot of windows cmd.exe shell running Clink

I have always hated the Windows command terminal. This little extension solves some of the headaches.

From the Clink page:

Clink enhances your productivity in Microsoft Windows’ “cmd.exe”.
If you’re familiar with Bash then you will be familiar with the changes that Clink brings to “cmd.exe” (it uses the same ‘Readline’ library that
Bash uses). It is a small utility to enhance “cmd.exe”, adding more powerful command line completion, editing, and history.

  • The same line editing as Bash (from GNU’s Readline library).
  • History persistence between sessions.
  • Scriptable completion with Lua.
  • New keyboard shortcuts;
    • Paste from clipboard (Ctrl-V).
    • Incremental history search (Ctrl-R/Ctrl-S).
    • Powerful completion (TAB).
    • Undo (Ctrl-Z).
    • Automatic “cd ..” (Ctrl-Alt-U).
    • Environment variable expansion (Ctrl-Alt-E).
  • Coloured and scriptable prompt.
  • Context sensitive completion;
    • Executables (and aliases).
    • Directory commands.
    • Environment variables
    • Thirdparty tools; Git, Mercurial, SVN, Go, and P4.
  • Auto-answering of the “Terminate batch job?” prompt.

tmux: Open Source terminal multiplexer

From the tmux website:

tmux lets you switch easily between several programs in one terminal, detach them (they keep running in the background) and reattach them to a
different terminal.

tmux is intended to be a simple, modern, BSD-licensed alternative to programs such as GNU screen.

This release runs on OpenBSD, FreeBSD, NetBSD, Linux and OS X and may still
run on Solaris and AIX (although they haven’t been tested in a while).

Ack: grep-like tool written in Perl for searching source code from the command line

Ack logo

I use this all the time at work. I get people asking me all the time, “How do you get your grep results to look like that?”

From the ack website (via http://kkovacs.eu/):

  • Blazing fast: It’s fast because it only searches the stuff it makes sense
    to search.
  • Better search: Searches entire trees by default while ignoring Subversion,
    Git and other VCS directories and other files that aren’t your source code.
  • Designed for code search: Where grep is a general text search tool, ack
    is especially for the programmer searching source code. Common tasks take
    fewer keystrokes.
  • Highly portable: ack is pure Perl, so it easily runs on a Windows
    installation Perl (like Strawberry Perl) without modifications.
  • Free and open: Ack costs nothing. It’s 100% free and open source under
    Artistic License v2.0.

Ack screenshot

Speccy: PC System Information Tool

Speccy screenshot

Recently, my WIFI connection has begin to slow down in the evenings. I suspect that there is interference due to neighboring WIFI signals. “No worries, ” I thought to myself. “I’ll just bump up to the 5GHZ spectrum and leave the 2.4 GHZ proletariat below.” unfortunately, the WIFI card in my PC doesn’t support 5 GHZ. “No problem. I’ll just get a better wireless card.” Now all I had to do was figure out what kind of slots I had free.

This is where Speccy comes in. I like Speccy because it presents the information in a well organized and pretty way.

JavaScript: The Right Way

JavaScript the right way site header

From the JavaScript: The Right Way site (via Hacker News):

…a JavaScript guide intended to introduce new developers and help experienced ones to the JavaScript’s best practices.
Despite the name, this guide doesn’t mean exactly “the right way” to do JavaScript.
We just gather all the stuff from top developers and put here. Since it come from exceptional folks, we could say that it is “the right way”, or
the best way to do so.

Ready Player One: One of my favoritest books of all time

Book Cover

Synopsis from the Ready Player One website:

If you grew up in the 80s and have a single nerd bone in your body, you should at least give this book a try. I have read it twice already and I inhaled it both times. Everyone I’ve recommended this book to who read it, loved it.

At once wildly original and stuffed with irresistible nostalgia, READY PLAYER
ONE is a spectacularly genre-busting, ambitious, and charming debut—part quest
novel, part love story, and part virtual space opera set in a universe where
spell-slinging mages battle giant Japanese robots, entire planets are inspired
by Blade Runner, and flying DeLoreans achieve light speed.

It’s the year 2044, and the real world is an ugly place.

Like most of humanity, Wade Watts escapes his grim surroundings by spending his
waking hours jacked into the OASIS, a sprawling virtual utopia that lets you be
anything you want to be, a place where you can live and play and fall in love on
any of ten thousand planets.

And like most of humanity, Wade dreams of being the one to discover the ultimate
lottery ticket that lies concealed within this virtual world. For somewhere
inside this giant networked playground, OASIS creator James Halliday has hidden
a series of fiendish puzzles that will yield massive fortune—and remarkable
power—to whoever can unlock them.

For years, millions have struggled fruitlessly to attain this prize, knowing
only that Halliday’s riddles are based in the pop culture he loved—that of the
late twentieth century. And for years, millions have found in this quest another
means of escape, retreating into happy, obsessive study of Halliday’s icons.
Like many of his contemporaries, Wade is as comfortable debating the finer
points of John Hughes’s oeuvre, playing Pac-Man, or reciting Devo lyrics as he
is scrounging power to run his OASIS rig.

And then Wade stumbles upon the first puzzle.

Suddenly the whole world is watching, and thousands of competitors join the
hunt—among them certain powerful players who are willing to commit very real
murder to beat Wade to this prize. Now the only way for Wade to survive and
preserve everything he knows is to win. But to do so, he may have to leave
behind his oh-so-perfect virtual existence and face up to life—and love—in the
real world he’s always been so desperate to escape.

A world at stake.

A quest for the ultimate prize.

Are you ready?

Using the HTML 5 application cache

From the tutorial on the Mozilla Developer Network site:

HTML5 provides an application caching mechanism that lets web-based applications run offline. Developers can use the Application Cache (AppCache)
interface to specify resources that the browser should cache and make available to offline users. Applications that are cached load and work
correctly even if users click the refresh button when they are offline.

Using an application cache gives an application the following benefits:

  • Offline browsing: users can navigate a site even when they are offline.
  • Speed: cached resources are local, and therefore load faster.
  • Reduced server load: the browser only downloads resources that have changed from the server.