Today I learned... why Scala usually doesn't use null

We Scala developers use the Option monad because it's so much better to assign None to something instead of null -- it's much safer and more explicit.

And we never have to see another NullPointerException!

Fatal exception:
None.get
java.util.NoSuchElementException: None.get
at scala.None$.get(Option.scala:313)
at scala.None$.get(Option.scala:311)

Now we just have to worry about not trying to call get on an Option that's None.
This is why many people in the Scala community recommend against ever using Option.get.

It's Scala's version of the NullPointerException. But it can be avoided if we use methods like map, filter, fold, foreach, getOrElse, etc., on an Option monad... Or in the Java-style, if we really have to use get, to check if it's empty first.

But sometimes you might want it to throw an exception because it means an assumption has been violated.

Today I learned... A limitation of singletons in Scala

Singletons in Scala are pretty nice -- you can have the basic goodness of a Java singleton, but with none of the scaffolding. The static keyword is not available in Scala, so singletons are useful anytime you want a static type of behavior, but Scala singletons can extend abstract classes or traits, can be passed in, mocked, etc.

The limitation comes when you are doing unit testing and want to run multiple unit tests completely independently of each other. If your singleton has any sort of state associated with it -- not just a var or a mutable object in a val, but also something as innocuous as the current timestamp at initialization saved as a val, etc. -- then there is no way, with the possible exception of reflection, of resetting that singleton's state in-between unit tests.

These particular unit tests I'm writing are actually functional system tests -- I'm basically calling my main method with different command-line parameters once in each unit test. And since in the JVM world I can't just do something magical like shutdown and reload an AppDomain (I miss this concept from .NET), I'm stuck with state getting persisted between what are supposed to be multiple program runs, because they're all in the same JVM (of course, same as the unit test's JVM, because I'm calling them through the JVM and not through the command-line, so I can make sure they don't throw exceptions, verify side effects if desired, and so on). And the main culprit is the singleton object.

I wrote a trait that can be used to solve this problem, but unfortunately, it required refactoring all my usages of any singletons that had state, so instead of singleton.method(), it became singleton.instance.method(). Find/replace all solved this problem without any trouble for me.

The trait manages the singleton, and also requires anything extending it to have a reset method. I call the reset method for all stateful singletons in the finally block of my main method.

trait ResettableSingleton[T] {
  protected var instanceMaybe: Option[T] = None

  def isInstantiated: Boolean = instanceMaybe.nonEmpty

  /**    * Singleton instance    *    * @return Object of the class    */  def instance: T = {
    instanceMaybe synchronized {
      if (instanceMaybe.isEmpty) {
        instanceMaybe = Option(instantiate)
      }
      instanceMaybe.get
    }
  }

  /**    * Some might prefer shorter syntax, so this is just an alias for "instance"    *    * @return Instance    */  def get: T = instance

  /**    * Create an instance of the class    * This is abstract because there's no guarantee that every class has, e.g., a parameterless constructor.    *    * @return Object of the class    */  protected def instantiate: T
  /**    * Resets the singleton instance. Intended to be used if performing multiple test runs without restarting the JVM.    */  def reset(): Unit = {
    instanceMaybe = None
  }
}

Example usage:

// Take the original singleton, turn it into a class, and make the default constructor private.
class Blah private() {
  // ... Original singleton body goes here, and remains unchanged
}
/** * Resettable singleton of Blah */object Blah extends ResettableSingleton[Blah] {
/** * Create an instance of the class * @return Object of the class */ override def instantiate: Blah = {
new Blah
}
}

So I lose the syntactic sugar, but now my unit tests work. Hey, it could be worse... I could be writing Java.

Today I learned... Data-driven unit testing in Scala

I've done data-driven unit tests in C# before, but recently I learned how to do them in Scala, and it's very easy and clean.

An example unit test, using WordSpec, which follows a BDD-style syntax, is below:

"Multiplying two integers" when {
  "one is negative" should {
    "have a negative result" in {
      val positive = scala.util.Random.nextInt(10000)
      val negative = -1 * scala.util.Random.nextInt(10000)
      val result = positive * negative
      assert(result < 0, s"positive * negative should be negative, but is ${result}!")
    }
  }
}

Unit test methods can be created on-the-fly at runtime, and therefore you can take any data source and turn it into a set of unit tests. Here is an example using WordSpec, which is one of the several flavors of writing unit tests in Scala.

(1 to 100).foreach(testCaseNumber => {
  s"Test scenario $testCaseNumber" when {
    "doing something" should {
      "return success" in {
        doSomethingWithData(getDataForTest(testCaseNumber))
      }
    }
  }
}

When you run all unit tests in your class, ScalaTest will show each of these 100 data as a separate test case with separate pass/fails.

It's really slick, and if you have some common test that you want to run over and over with some slight modification that can be controlled based on some parameter, I recommend you give it a try!

Today I learned... tuple assignment in Scala

// This works
var (x, y) = try {
  (0, 0)
}

// This doesn't work... would be nice if it did!
var x: Int = 0
var y: Int = 0
(x, y) = try {
  (0, 0)
}

Today I learned... doing things the hard way when languages and OSes are glitchy

So I have some code that works just fine... most of the time. But there has been some strange stuff that doesn't make any sense and isn't easily reproducible.

Few examples I encountered recently were Scala's parallel HashMap and file renaming in Java.

Scala has a nice parallel HashMap that allows you to do gets, puts, and updates concurrently, and it handles the synchronization for you... except when it doesn't.

I put a tuple into a ParHashMap, which has no delete operations on it in my whole code, and later when I went to get that value, it wasn't there.

On another instance, when I went to put a tuple into a ParHashMap, I got an ArrayIndexOutOfBoundsException.

I replaced my ParHashMap with a single-threaded HashMap and did my own synchronization around it, and I haven't seen either problem since...


Another issue was renaming a file in Windows using Java. File.renameTo was working just fine, but all of a sudden I came across one occurrence where the file appeared to have been copied instead of renamed. So I changed my code to delete the source file after the rename, if it was still there. But then I got an exception that the source file was already in use, even though I had previously closed my writer that was operating on that file.

I ended up writing the following method to replace what should be a simple one-line file rename:

def rename(filePath: String, targetPath: String): Unit = {
    RetryHelper.retry(10, Seq(classOf[java.nio.file.FileSystemException])){
      val sourceFile = new File(filePath)
      val targetFile = new File(targetPath)
      if (!targetFile.exists) {
        Files.move(sourceFile.toPath, targetFile.toPath)
      }
      if (sourceFile.exists) {
        sourceFile.delete()
      }
    }(Thread.sleep(50))
}

That's right, the only way I could get it to work every single time, without fail (so far!) was to delete the source file after the "move" (in case it behaved as a copy for some reason), and to wrap that in a retry block that executes up to 10 times and catches FileSystemExceptions with a short sleep before retrying. Yikes!

For reference, here is my retry helper, which I discussed in a previous post but which has been revised since, as it now takes in a code block defining what to do in the exception handling block when catching one of the specified exceptions.

import scala.util.control.Exception._
// Motivation for this object is described in the following blog posts://   http://googlyadventures.blogspot.com/2015/05/what-else-i-learned-yesterday.html//   http://googlyadventures.blogspot.com/2015/06/today-i-learned-doing-things-hard-way.html
/**  * Helper methods to retry code with configurable retry count and exceptions to handle  */object RetryHelper {
  /**    * Retry any block of code up to a max number of times, optionally specifying the types of exceptions to retry    * and code to run when handling one of the retryable exception types.    *    * @param maxTries              Number of times to try before throwing the exception.    * @param exceptionTypesToRetry Types of exceptions to retry. Defaults to single-element sequence containing classOf[RuntimeException]    * @param codeToRetry           Block of code to try    * @param handlingCode          Block of code to run if there is a catchable exception    * @tparam T Return type of block of code to try    * @return Return value of block of code to try (else exception will be thrown if it failed all tries)    */  def retry[T](maxTries: Int, exceptionTypesToRetry: Seq[Class[_ <: Throwable]] = Seq(classOf[RuntimeException]))
              (codeToRetry: => T)
              (handlingCode: Throwable => Unit = _ => ()): T = {
    var result: Option[T] = None
    var left = maxTries
    while (!result.isDefined) {
      left = left - 1
      // try/catch{case...} doesn't seem to support dynamic exception types, so using handling block instead.
      handling(exceptionTypesToRetry: _*)
        .by(ex => {
          if (left <= 0) {
            throw ex
          } else {
            handlingCode(ex)
          }
        }).apply({
        result = Option(codeToRetry)
      })
    }
    result.get
  }
}

Today I learned... JavaConverters vs. JavaConversions

I have some Scala code that interops with Java code. I had previously used

import scala.collection.JavaConversions._

to support auto-conversion of collections, allowing me to call a method expecting a Java collection by passing it a Scala collection, and the other way around.

Today I refactored my code so this conversion happens explicitly.

I got rid of the JavaConversions import and instead added this:

import scala.collection.JavaConverters._

Then whenever I need code that expects a Java collection but I have a Scala collection, I call myScalaCollection.asJava, and when I do it the other way around, I call myJavaCollection.asScala.


It's less verbose and perhaps less magical, but I think it's easier to read the code and see what is going on. I'm controlling the conversion instead of letting the compiler do it for me, so I can be aware of any performance issues, etc. of possibly converting something back and forth needlessly.


There are some limitations. When you have a Scala queue, for example, a scala.collection.immutable.Queue[T], calling the implicit method .asJava converts it to a java.util.List, which is not exactly a queue! If you want a Java Queue, you'd have to new java.util.LinkedList(myScalaQueue.asJava), because LinkedList implements the Java Queue interface and takes in a java.util.List as a constructor argument.


It is also possible to implement this yourself by creating your own implicit method, e.g.:


import scala.collection.JavaConverters._
import scala.collection.Seq
import java.util.LinkedList


object ScalaSeqHelper {
  implicit class SeqImplicits(seq: Seq) {
    def asJavaLinkedList: LinkedList = {
      new LinkedList(seq.asJava)
    }
  }
}




Then if you import ScalaSeqHelper._, you can call .asJavaLinkedList on any Scala Seq (which includes the mutable and immutable Queue implementations) and get a Java LinkedList, which is also a Java Queue.

Recently I learned... Covariance and mutability

Last week I made my data generation and automated testing framework more generic.
Instead of having an engine that processed a Graph of Nodes of MySpecificData, I created an abstract engine to process over a Graph of Nodes of GenericData.

So I had an abstract method such as process(graphToProcess: Graph[Node[GenericData]])
And I had an override method such as process(graphToProcess: Graph[Node[MySpecificData]])

The compiler won't let me do this because the methods don't have the same signature, even though MySpecificData "is a" GenericData.

If you've used Java, you may tried to cast an Array<Int> to an Array<Object>, and for Arrays in Java, it allows this. The problem is you can then insert a String into this Array and then when you try and read the String from the Array<Object>, which is really an Array<Int>, you'll get a ClassCastException.

Scala will prevent this with a compilation error. Because the contents of Array can change (it's mutable), Scala forbids you from casting it to a more generic type, because it can't guarantee it type-safe at compile-time.

To say "WrapperType[SpecificWrappedType] is a WrapperType[GenericWrappedType]" requires the generic type parameter of WrapperType to be covariant in the class definition of WrapperType. If WrapperType has any mutability in it, the compiler will prevent you from declaring the generic wrapped type as covariant.

It's not always desirable to make everything immutable, unless you're a diehard Haskell geek. Sometimes in production environments, we have deeply nested data structures for which creating a new copy every time we need to do something would be an unacceptable performance hit. That is the case with my Graph[Node[NodeData]]] -- I really just need it to be mutable.

One possibility is to define a secondary type as a subtype of the original generic type, and make that type covariant, and use it in your class, but it's not very clean and makes for strange code (and I couldn't get this solution to even compile in every single situation).

The easiest thing to do is to use an annotation that will tell the Scala compiler to ignore the error due to a generic type being used in the wrong context (e.g., a covariant type used in a mutable setting).

To define a generic type as mutable, use the plus sign prefix -- e.g., "class Blah[+T <: SomeAbstraction]" defines Blah wrapped over generic type T, which is covariant and implements another type called SomeAbstraction.

Try and use it, and if you get a compiler warning related to using a covariant type in an invariant or contravariant position, just add this import:

import scala.annotation.unchecked.{uncheckedVariance => uV}

And then just put @uV after any reference to the type (except for the definition).

E.g., a method definition might pass in parameters using this type:

def someMethod(myParam1: Seq[T @uV], myParam2: (T @uV))

(If the type is by itself, wrap parentheses around it and the annotation.

And that's how you can fool the Scala compiler into allowing you to write covariant, mutable code.

The type system is there for a reason, and strong typing can be a wonderful thing. In this case, there's simply no way to prove that our code is safe at compile-time and we have to relax the compiler's default restrictions. It would be easier if there were a compiler flag or a "begin/end" kind of annotation instead of annotating every offending usage, but at least there's a way around it.

Just be smart and don't write code that isn't typesafe. Just don't try to stuff a String into an Array<Int> and you should be fine. And test your code so that that ClassCastException will occur if you happen to write bad code.

Today I learned... Windows batch loop for all files

Here is a one-liner that can be used in a Windows batch script to run a command (in this case, the GraphViz dot.exe application) once for each file in a directory (with a globbed filter).

The syntax is kind of ugly, but %%f is the filename and %%~nf is the filename without the extension.

FOR %%f IN (*.gv) DO dot -Tpng ".\%%f" -o ".\%%~nf.png"

This converts all GraphViz DOT files in a directory into PNG format.

More examples and documentation here: http://academic.evergreen.edu/projects/biophysics/technotes/program/batch.htm

Today I learned... no @SuppressWarnings in Scala!

Today I learned something annoying... Scala does not have any way to suppress warnings on individual code elements.

In Java you can use the @SuppressWarnings attribute on a code element to suppress compiler warnings for that element. .NET has something similar with the [SuppressMessage] attribute. The point of it is if you've decided the compiler warning is not a problem in your scenario, you don't want to clutter up your compiler output. Ideally, you want to be able to treat warnings as errors and fail a build if you get a single warning -- so this prevents that if there are any legitimate warnings that you're unable to suppress.

For example, if you need to use a deprecated method in some third-party API, you might have done the research and gotten the code reviewed and decided it was ok. An example that I came across today was connectionCachingEnabled in OracleDataSource. (Fortunately, I ended up not needing to use it because I realized the default value is already "disabled" which is what I wanted anyway.)

I don't have a solution, unfortunately... Just an observation, and one that others have noted as well (and the Scala community has closed this feature request with "wontfix" -- see http://stackoverflow.com/questions/3506370/is-there-an-equivalent-to-suppresswarnings-in-scala).

Today I learned... basic authenticated HTTP connections in Scala

I needed to connect to Jenkins to install (on a remote server over SSH using JSch) the latest stable trunk build of a particular project if it was newer than the version that was already installed.

I spent a lot of time fighting with various Java/Scala APIs to do HTTP connections with authentication, and many of them didn't work as I needed and/or were too complicated to use.

I also spent a lot of time trying out existing Java APIs and StackOverflow posts for how to interact with Jenkins. I discovered through trial and error that the RisingOak Jenkins API simply does not work to get a nested job.

I finally got something to work. I'm using it in this manner:

• Read contents of [jenkinsServerUrl]/job/JobName[/job/SubJobName]*/lastStableBuild/buildNumber to determine the last stable build number
• Compare to build number of known installation. If greater, ensure it's a stable trunk build (if it's the same as lastStableBuild then skip checking if stable).
• To check if a trunk build, parsing build parameter "branch" from the page [buildNumber]/injectedEnvVars under my Jenkins job sub URL
• To check if a stable build, parse the HTML of the [buildNumber] page under the Jenkins job sub URL, and check the status icon (this is brittle, but was the only/easiest way I could find).
• If not stable trunk build, decrement number until it's equal to the installed build, then stop.
• If later build found, download the archive, SFTP it up, and install it using SSH commands.

Here's the helper object that I use to make the authenticated HTTP requests:

package Helpers

import java.io.InputStream
import java.net.URL
import Helpers.InputStreamHelper.InputStreamExtensions
import org.apache.commons.codec.binary.Base64
import scala.io.Source

/** * HTTP Helper methods */object HttpHelper {
  def download(url: String, savePath: String, user: String = null, passwordOrToken: String = null, requestProperties: Map[String, String]= Map("User-Agent"->"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)")) : Unit = {
    val inputStream = getInputStreamFromUrl(url, user, passwordOrToken, requestProperties)
    try {
      inputStream.downloadToFile(savePath)
    } finally {
      inputStream.close()
    }
  }

  def getPageContentFromUrl(url: String, user: String = null, passwordOrToken: String = null, requestProperties: Map[String, String]= Map("User-Agent"->"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)")) : String = {
    val inputStream = getInputStreamFromUrl(url, user, passwordOrToken, requestProperties)
    try {
      Source.fromInputStream(inputStream).getLines().mkString("\n")
    } finally {
      inputStream.close()
    }
  }

  def getInputStreamFromUrl(url: String, user: String = null, passwordOrToken: String = null, requestProperties: Map[String, String]= Map("User-Agent"->"Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)")) : InputStream = {
    val connection = new URL(url).openConnection
    requestProperties.foreach({
      case (name, value) => connection.setRequestProperty(name, value)
    })

    if (user != null && passwordOrToken != null) {
      connection.setRequestProperty("Authorization", getHeader(user, passwordOrToken))
    }

    connection.getInputStream
  }

  def encodeCredentials(username: String, password: String): String = {
    new String(Base64.encodeBase64String((username + ":" + password).getBytes))
  }

  def getHeader(username: String, password: String): String = {
    "Basic " + encodeCredentials(username, password)
  }
}

package Helpers

import java.io.{FileOutputStream, InputStream}

/** * Created by Samer Adra on 5/13/2015. */object InputStreamHelper {
  implicit class InputStreamExtensions(val inputStream: InputStream) {
    def downloadToFile(path: String): Unit = {
      val buffer = new Array[Byte](8 * 1024)

      val outStream = new FileOutputStream(path)
      try {
        var bytesRead = 0        while ({bytesRead = inputStream.read(buffer); bytesRead != -1}) {
          outStream.write(buffer, 0, bytesRead)
        }
      } finally {
        outStream.close()
      }
    }
  }
}

Yesterday I also learned... Functional exception handling and sequence to varargs

Got this working at the end of the day yesterday and didn't get a chance to post it until now:

I experienced a network glitch doing an SFTP put, so I wanted to implement a retry wrapper that I could pass any code block to, along with the number of times to try before failing.

I wanted to pass in the types of exceptions to catch in the tries.
When I tried implementing it and passing in a variable type to the standard try/catch{case} statement (each case specifies what type(s) of exception to catch), it failed to compile.

There is an alternative way to catch exceptions in Scala -- the scala.util.control.Exception.handling() method.

handling(exceptionType1, exceptionType2, ...)
.by(ex => handleBlock)
.apply({
codeBlockToTry
})
}

In this way I was able to pass in a sequence of exception classes, cast it to _* to convert it to a variable-length parameters format (e.g., like args), and only catch the necessary types.

The parameter in my method that specifies the exception classes has a default value of a single-eleemnt sequence containing RuntimeException, and there is a type constraint indicating that all classes in the sequence must implement Throwable.

The retry method is a curried function, e.g., f(params1)(params2). The first sequence of parameters is applied, yielding a function against the second sequence of parameters.

The code is below:

import scala.util.control.Exception._
/** * Created by Samer Adra on 5/5/2015. */object RetryHelper {
/** * Retry any block of code up to a max number of times, optionally specifying the type of exception to retry. * @param maxTries Number of times to try before throwing the exception. * @param exceptionTypesToRetry Types of exception to retry. Defaults to single-element sequence containing classOf[RuntimeException] * @param codeToRetry Block of code to try * @tparam T Return type of block of code to try * @return Return value of block of code to try (else exception will be thrown if it failed all tries) */ def retry[T](maxTries: Int, exceptionTypesToRetry: Seq[Class[_ <: Throwable]] = Seq(classOf[RuntimeException]))(codeToRetry: => T): T = {
var result: Option[T] = None
var left = maxTries
while(!result.isDefined) {
left = left - 1
// try/catch{case...} doesn't seem to support dynamic exception types, so using handling block instead.
handling(exceptionTypesToRetry:_*)
.by(ex => if (left <= 0) throw ex).apply({
println("Left: " + left)
result = Some(codeToRetry)
})
}
result.get
}
}

Today I learned... DestroyJavaVM can cause JVM apps to hang instead of terminate

I have a long-running application that does some network IO (Oracle and SSH). It was taking longer than expected and it looked like it was done based on the logs. When I paused it to investigate, there was nothing active on the main thread, and all the threads were waiting.

I noticed DestroyJavaVM() in one of the threads, which I had never heard of (being new to JVM-land), and I also saw another thread that was stuck at an Oracle wait method, and a few other wait threads.

It turns out I forgot to dispose a couple of my resources before closing my application. When I fixed that, my application started terminating properly again.

Coming from C#, I've been spoiled by the using() block, which neatly cleans up any IDisposables. Really it's syntactic sugar for try{//do work}finally{disposableObject.Dispose()}.

In Scala I do the try/finally. I have read that there are ways to do something similar to C#'s using(), but they didn't seem as clean or intuitive to me, plus they'd need third-party libraries.

override def dispose(): Unit = {
  super.dispose()
  if (_dbConnection != null && !_dbConnection.isClosed) {

    _dbConnection.close()

  }

  _dbConnection = null 

I call this dispose() method in a finally block.

Today I learned... Using Java inner classes in Scala

I needed to copy some files between two remote servers. The source and destination are both HDFS so I would like to use DistCp, but right now the servers can't even ping each other so until I get that resolved I have to go through my local computer first.

I used Jsch (with a local private key specified) to establish SSH connections to each server.
I get the files from Hadoop onto the filesystem of the source server using hadoop fs -copyToLocal.
Next I SFTP the files to my local computer.
Next I SFTP the files to the destination server.
Finally I put the files in HDFS using hdfs dfs -put.

To do the SFTP to local I use the ls method of ChannelSftp (which is part of the Jsch library).
This returns a Vector of Objects, but all the objects are really of type LsEntry (so why didn't they just make it generic?). LsEntry is an inner class inside the ChannelSftp class.

My application is written in Scala, which is a JVM language, so Scala and Java code can reference each other in the same way that C# code and VB.NET code can use each other in Microsoft-land.

The problem was I couldn't cast those objects to LsEntry because the compiler kept complaining that com.jcraft.jsch.ChannelSftp.LsEntry was not valid. The funny thing is, when I would just type obj.asInstanceOf[LsEntry], the IDE knew what I was referring to because the tooltip asked if I was trying to use com.jcraft.jsch.ChannelSftp.LsEntry. When I said ok, it auto-added the import statement import com.jcraft.jsh.ChannelSftp.LsEntry, and then gave a "not found" error for the import it just added.

After much head-banging (of the against-the-wall sort, not the heavy-metal rocker sort), I found out that there is no way to include a Java inner class in a Scala import statement.

For Scala code to reference a Java inner class, the only way is to use the hash symbol. What finally worked was:

obj.asInstanceOf[ChannelSftp#LsEntry].

Apparently the reason for this is, whereas Java treats an inner class as part of the enclosing class, Scala treats it as part of an object of the enclosing class. So by default, if you declare two enclosing objects of the same type, then declare in each an object of the inner class, those two inner objects will not have the same type, because they will be under different enclosing objects. The # allows you to get an inner class defined under the class, as Java does.

I might actually use this property to restructure my program... Instead of having Graph and Node at the same level, I might make Node an inner class of Graph, thus guaranteeing that all Node operations happen in the same Graph (e.g., a Node addChild method that takes in another node would only accept another node in the same graph).

Changing this blog up a little

"Brevity is the soul of wit." So I'm going to start doing something a bit different... Instead of these long-winded posts that no one will ever read, I'll try to post more regularly and will hopefully just make short posts about something I learned (starting... next post).

This post I just want to mention that I've been working at a new job since January. I'm currently a software development engineer in test (SDET) working at FINRA.

I'm working in JVM land before, which I haven't really done since college (except for a Java-in-Visual Studio MSBuild template I built a few years ago, which hardly counts).

My project since joining FINRA has been Big Data test data generation and automated testing. I have engineered a system written in Scala that generates both the inputs and the expected outputs to a Big Data application called OATS, which takes in financial orders from broker-dealers, exchanges, etc., and links them all together so you can see the entire lifecycle of an order. I've modeled the states and their allowed transitions, and based on that am able to randomly generate data that I could expect the application would link together. I'm also leveraging this generation engine to assist us in testing of specific scenarios, which is done either by hardcoding a graph in Scala or my specifying a directory of DOT files describing a graph. The generator reads in the DOT files, transforms them into graphs of events satisfying each DOT file, fills in random data, and performs the automated testing using that input data and the generated expected outputs.

It's a lot of fun, and I've been learning a lot doing it.