Senior Developers Team Productivity X4 (from MS Research Paper)

"TDD seems to be applicable in various domains and can significantly reduce the defect density of developed software without significant productivity reduction of the development team"

• An experienced team (5 people over 10 years + 2 people under 5 years) : 155KLOC C# code (+60 test).
  

• A junior team (3 people under 10 years + 6 people under 5 years): 41 KLOC Java code (+28 test).
  

The End Of Rings Around Plain Java - A Better Concurrency Test

In my previous post, I was wondering why single thread was faster. D Andreou gave the correct explanation: as we send only 1 start message and as each node only send 1 message to the next one, there is always only 1 message being processed. So the test is optimum on 1 thread. It does not make much sense to make a multithreading benchmark on a problem that is fundamentally single threaded.

His suggestion was to simple send N start messages where N >= number of processors. In theory, the performance will become optimal with N threads then. Unfortunately this is not what happened in real life. In real life the single threaded performance is still better if you send even 16 messages on a biprocessor machine.

public static void main(String[] args) throws Exception {
    OptimizedRing ring = new OptimizedRing();
    RingNode node = ring.startRing(Integer.parseInt(args[0]));
    node.sendMessage(new StartMessage());
    node.sendMessage(new TokenMessage(node.nodeId,1));
    node.sendMessage(new TokenMessage(node.nodeId,1));
    node.sendMessage(new TokenMessage(node.nodeId,1));
    ring.executor.awaitTermination(10, TimeUnit.MINUTES);
}

My idea was that it was related to the swiching from thread to thread overhead, which is precisely what I think the original author of the test had in mind to test. I am not 100% convinced it is really what’s happening. I wanted a test that would actually be faster using N threads; so I decided to add a bit of computation at before processing each Token. Unfortunately I had the bad idea to compute Pi by Monte Carlo method to do that. Running my tests I was surprised it did not change the results, and made things worse the most computer intensive the computation was (increasing the number of monte carlo iterations). It scared me a bit wondering what the hell could be wrong there. The following class performs much worse with 2 threads compared to 1:

public class BadParallelPi {

    private static void startExecutors() throws Exception {        
        long startTime = System.currentTimeMillis();
        System.out.println(startTime);
        ExecutorService executor1 = Executors.newFixedThreadPool(1);
        executor1.execute(new Computation());
        executor1.execute(new Computation());
        executor1.shutdown();
        executor1.awaitTermination(60, TimeUnit.SECONDS);
        long delay = System.currentTimeMillis() - startTime;
        System.out.println("finished single thread in "+(delay/1000.0));
        startTime = System.currentTimeMillis();
        System.out.println(startTime);
        executor1 = Executors.newFixedThreadPool(2);
        executor1.execute(new Computation());
        executor1.execute(new Computation());
        executor1.shutdown();
        executor1.awaitTermination(60, TimeUnit.SECONDS);
        delay = System.currentTimeMillis() - startTime;
        System.out.println("finished 2 threads in "+(delay/1000.0));
    }
    
    public static class Computation implements Runnable {
        public volatile int count = 0;
         private double computePi() {
            double pi = 0;
            double x,y;
            int n = 10000000;
            for (int i=0;i<n;i++) {
                x = Math.random();
                x *= x;
                y = Math.random();
                y *= y;
                if (x+y < 1) {
                    pi +=1;
                }
            }
            pi = 4*pi/n;
            return pi;
        }
        
        public void run() {
            double pi = computePi();
            long time = System.currentTimeMillis();
            System.out.println(time+" thread "+Thread.currentThread().getId()+" pi="+pi);
            count++;
        }        
    }

    
    public static void main(String[] args) throws Exception {
        startExecutors();
    }
} 

Did you figure out why?

It took me less time with this simple code than with the original ring test to find out why. It is simply because of the Math.random call. Math.random only creates one random number generator, and it will be shared among threads. So every thread will wait at the other one at this point. Creating one random generator per thread showed 2 threads were much faster than 1, finally.

Back to the original ring test. Adding the correct way to compute Pi by Monte Carlo, I now had decent test results as long as the number of iterations is not too small. 10 iterations is enough to show a real difference between N threads and 1. Adding a small computation helps figuring out what happens behind the scene. You can also verify D Andreou claim, using only 1 start message the single threaded version is faster. If computation is too weak (for example number of Monte Carlo iteration of 0, one only measures method calls between threads (context switching), which is obviously optimal for 1 thread. Measuring Actor libraries on it is dangerous: if I write a single threaded Actor library, it will be the fastest of this test, but it certainly is not what you want to use as Actor library.

Let’s see now how Scala fares compared to the Plain Java solution, using computation:

It is interesting to compare results of 4 threads on a biprocessor with monte carlo count of 10 and 100. We see a much higher thread overhead with fewer computation. With too few computation in monte carlo, the overhead of threads is too high over 2 concurrent threads. This explains why the very simple threading architecture fares much better in the last column compared to the previous one.

Scala Actors fares much better when it is not hindered in the creation of too many threads. It seem actually very good at abstracting multithreading intricacies, while still providing near Java performance in the real world where each actor does enough computation and multithreading is important.

Object Oriented Analysis And Design with Applications Book Review

A while ago, I had a comment from someone implying I knew nothing about OO programming because I had not mentioned (and therefore read) Object Oriented Analysis And Design with Applications from G. Booch. I was intrigued by such a silly comment and decided to look at this book that was considered as the bible of OOP.

Well, I don’t find it that good! But I don’t find the bible particularly good either. I like B. Meyer Object Oriented Software Construction book much more, because it is more practical, more in touch with realities while pointing at the real important problems like:

Real systems have no top

In contrast G Booch book has too much evident concepts that don’t really make you learn anything or think things a different way. It is a good book for someone who is learning OO for the first time. It covers the subject in details, but I did not find anything in it that made me say

wow!

It is like most other book you can find on OO. Furthermore only the first parts are on OO, the rest is more a UML tutorial.

Running Rings Around Plain Java - The Killer Code

I wrote my previous post too fast. I found a very simple change that increases the speed x6!

The idea is too process messages in a ThreadPoolExecutor. As my Nodes are Runnable, I just needed to initialize a common ThreadPoolExecutor, and in a sendMessage, execute the runnable each time.

Here is the full code: public class OptimizedRing {

    private ExecutorService executor;

    public static void main(String[] args) throws Exception {
        OptimizedRing ring = new OptimizedRing();
        RingNode node = ring.startRing(Integer.parseInt(args[0]));
        node.sendMessage(new StartMessage());
    }

    public RingNode startRing(int n) {
        RingNode[] nodes = spawnNodes(n, startTimer());
        connectNodes(n, nodes);
        return nodes[0];
    }

    private Timer startTimer() {
        Timer timer = new Timer();
        new Thread(timer).start();
        return timer;
    }

    private RingNode[] spawnNodes(int n, final Timer timer) {
        System.out.println("constructing nodes");
        long start = System.currentTimeMillis();
        executor = Executors.newFixedThreadPool(4);
        RingNode[] nodes = new RingNode[n+1];
        for (int i = 0; i < n ; i++) {
            nodes[i] = new RingNode(i, timer, null);
        }
        long end = System.currentTimeMillis();
        System.out.println("Took "+(end-start)+"ms to construct "+n+" nodes");
        return nodes;
    }

    private void connectNodes(int n, RingNode[] nodes) {
        System.out.println("connecting nodes");
        nodes[n] = nodes[0];
        for (int i=0; i<n; i++) {
            nodes[i].connect(nodes[i+1]);
        }
    }

    interface Message {
        String getType();
    }

    private static class StartMessage implements Message {
        public String getType() {
            return "START";
        }
    }

    private static class StopMessage implements Message {
        public String getType() {
            return "STOP";            
        }
    }

    private static class CancelMessage implements Message {
        public String getType() {
            return "CANCEL";
        }
    }

    private static class TokenMessage implements Message {
        private int nodeId;
        private int value;

        public TokenMessage(int nodeId, int value) {
            this.nodeId = nodeId;
            this.value = value;
        }

        public String getType() {
            return "TOKEN";
        }
    }

    private class RingNode implements Runnable {
        private int nodeId;
        private Timer timer;
        private RingNode nextNode;
        private BlockingQueue<Message> queue = new LinkedBlockingQueue<Message>();
        private boolean isActive = false;
        
        public RingNode(int id, Timer timer, RingNode nextNode) {
            nodeId = id;
            this.timer = timer;
            this.nextNode = nextNode;                        
        }

        public void connect(RingNode node) {
            nextNode = node;
            isActive = true;
        }

        public void sendMessage(Message m) {
            queue.add(m);
            executor.execute(this);
        }

        public void run() {
            if (isActive) {
                try {
                    Message m = queue.take();
                    if (m instanceof StartMessage) {
                        log("Starting messages");
                        timer.sendMessage(m);
                        nextNode.sendMessage(new TokenMessage(nodeId, 0));
                    } else if (m instanceof StopMessage) {
                        log("Stopping");
                        nextNode.sendMessage(m);
                        isActive = false;
                        //
                    } else if (m instanceof TokenMessage) {
                        if (((TokenMessage)m).nodeId == nodeId) {
                            int nextValue = ((TokenMessage)m).value + 1;
                            if (nextValue % 10000 == 0) {
                                log("Around ring "+nextValue+" times");
                            }
                            if (nextValue == 1000000) {
                                timer.sendMessage(new StopMessage());
                                timer.sendMessage(new CancelMessage());
                                nextNode.sendMessage(new StopMessage());
                                isActive = false;
                            } else {
                                nextNode.sendMessage(new TokenMessage(nodeId, nextValue));
                            }

                        } else {
                            nextNode.sendMessage(m);
                        }
                    }

                } catch (InterruptedException ie) {
                    ie.printStackTrace();
                }
            }
        }

        public void log(String s) {
            System.out.println(System.currentTimeMillis()+" "+nodeId+": "+s);
        }

    }

    private static class Timer implements Runnable {
        private BlockingQueue<Message> queue = new LinkedBlockingQueue<Message>();
        private boolean timing = false;
        private long startTime;

        public void sendMessage(Message m) {
            //we don't need to change this implementation as timer is rarely called
            queue.add(m);
        }

        public void run() {
            while (true) {
                Message m;
                try {
                    m = queue.take();
                    if (m instanceof StartMessage) {
                        startTime = System.currentTimeMillis();
                        timing = true;
                    } else if (m instanceof StopMessage) {
                        long end = System.currentTimeMillis();
                        System.out.println("Start="+startTime+" Stop="+end+" Elapsed="+(end-startTime));
                        timing = false;                                        
                    } else if (m instanceof CancelMessage) {
                        break;
                    }
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        }
    }
}

Running rings around plain Java

Alex Miller has a very interesting test of Actors. He finds out Scala performance is relatively low compared to Erlang, and Kilim is very near Erlang. But Kilim code is the most difficult to read in the lot.

I thought it would be simple to just do the same test in plain Java. I wrote the code for it duplicating the scala logic using Threads instead of Actors.

public class SimpleRing {
    public static void main(String[] args) throws Exception {
        SimpleRing ring = new SimpleRing();
        RingNode node = ring.startRing(Integer.parseInt(args[0]));
        node.sendMessage(new StartMessage());
    }

    public RingNode startRing(int n) {
        RingNode[] nodes = spawnNodes(n, startTimer());
        connectNodes(n, nodes);
        return nodes[0];
    }

    private Timer startTimer() {
        Timer timer = new Timer();
        new Thread(timer).start();
        return timer;
    }


    private RingNode[] spawnNodes(int n, final Timer timer) {
        System.out.println("constructing nodes");
        long start = System.currentTimeMillis();
        RingNode[] nodes = new RingNode[n+1];

        for (int i = 0; i < n ; i++) {
            nodes[i] = new RingNode(i, timer, null);
            new Thread(nodes[i]).start(); //later use pool
        }

        long end = System.currentTimeMillis();
        System.out.println("Took "+(end-start)+"ms to construct "+n+" nodes");

        return nodes;
    }


    private void connectNodes(int n, RingNode[] nodes) {
        System.out.println("connecting nodes");
        nodes[n] = nodes[0];

        for (int i=0; i<n; i++) {
            nodes[i].connect(nodes[i+1]);
        }
    }


    interface Message {
        String getType();
    }


    private static class StartMessage implements Message {
        public String getType() {
            return "START";
        }
    }

    private static class StopMessage implements Message {
        public String getType() {
            return "STOP";            
        }
    }


    private static class CancelMessage implements Message {
        public String getType() {
            return "CANCEL";
        }
    }


    private static class TokenMessage implements Message {
        private int nodeId;
        private int value;

        public TokenMessage(int nodeId, int value) {
            this.nodeId = nodeId;
            this.value = value;
        }

        public String getType() {
            return "TOKEN";
        }
    }


    private static class RingNode implements Runnable {
        private int nodeId;
        private Timer timer;
        private RingNode nextNode;
        private BlockingQueue<Message> queue = new LinkedBlockingQueue<Message>();

        public RingNode(int id, Timer timer, RingNode nextNode) {
            nodeId = id;
            this.timer = timer;
            this.nextNode = nextNode;                        
        }


        public void connect(RingNode node) {
            nextNode = node;
        }


        public void sendMessage(Message m) {
            queue.add(m);
        }


        public void run() {
            while (true) {
                try {
                    Message m = queue.take();
                    if (m instanceof StartMessage) {
                        log("Starting messages");
                        timer.sendMessage(m);
                        nextNode.sendMessage(new TokenMessage(nodeId, 0));
                    } else if (m instanceof StopMessage) {
                        log("Stopping");
                        nextNode.sendMessage(m);
                        break;
                    } else if (m instanceof TokenMessage) {
                        if (((TokenMessage)m).nodeId == nodeId) {
                            int nextValue = ((TokenMessage)m).value + 1;
                            if (nextValue % 10000 == 0) {
                                log("Around ring "+nextValue+" times");
                            }
                            if (nextValue == 1000000) {
                                timer.sendMessage(new StopMessage());
                                timer.sendMessage(new CancelMessage());
                                nextNode.sendMessage(new StopMessage());
                                break;
                            } else {
                                nextNode.sendMessage(new TokenMessage(nodeId, nextValue));
                            }
                        } else {
                            nextNode.sendMessage(m);
                        }
                    }

                } catch (InterruptedException ie) {
                    ie.printStackTrace();
                }
            }
        }


        public void log(String s) {
            System.out.println(System.currentTimeMillis()+" "+nodeId+": "+s);
        }
    }


    private static class Timer implements Runnable {
        private BlockingQueue<Message> queue = new LinkedBlockingQueue<Message>();
        private boolean timing = false;
        private long startTime;


        public void sendMessage(Message m) {
            queue.add(m);
        }


        public void run() {
            while (true) {
                Message m;
                try {
                    m = queue.take();
                    if (m instanceof StartMessage) {
                        startTime = System.currentTimeMillis();
                        timing = true;
                    } else if (m instanceof StopMessage) {
                        long end = System.currentTimeMillis();
                        System.out.println("Start="+startTime+" Stop="+end+" Elapsed="+(end-startTime));
                        timing = false;                                        
                    } else if (m instanceof CancelMessage) {
                        break;
                    }
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
            }
        }
    }
}

• SimpleRing

  • spawn 100: 11ms
  • send 100M messages: 493073ms

• Scala Actors

  • spawn 100: 15 ms
  • send 100M messages: 270104ms

UPDATE with slight changes plain Java rocks!

More Double Troubles

408.16 - 40.82 = 367.34000000000003

• exponent = 256. Then 408.16/256 = 1.594375 = 1 + 0x9828F5C28F5C28F5C28F5C... * 2^-52
  
• We round to 52 bits, the mantissa is 0x9828F5C28F5C3 = 2676827028518339.
• As a decimal, the internal value is (2676827028518339/2^52+1) * 256 = 408.1600000000000250111042987555265426635742

• exponent = 32. Then 40.82/32= 1.275625 = 1 + 0x468F5C28F5C28F5C...*2^-52
• Rounded to 52 bits, the mantissa is 0x468F5C28F5C29 = 1241304647293993
  
• As a decimal, the internal value is (1241304647293993/2^52 + 1)*32 = 40.8200000000000002842170943040400743484497

Java Programmers Are Idiots?

My brother just sent me a funny quote. I don’t know if it is true or not:

entwickeln Sie lieber überzeugende Lösungen anstatt viele Stunden mit Coding zu verbringen? Ist Ihnen die Produktivität Ihres Teams wichtig?

Mark Driver, VP Research von Gartner, kommentierte kürzlich

“Here’s a simple equation. In terms of mental fortitude… 1 Smalltalk developer = 2.5 C++ developers 1 C++ developer = 1.5 Java developers”.

You don’t need german to understand. Of course it can not be true. How can anyone measure mental fortitude? And how does it related with productivity is another issue.

There is a famous email exchange from Linux Torvald claiming the advantages of C versus C++, here is a quote:

If you want a VCS that is written in C++, go play with Monotone. Really.They use a “real database”. They use “nice object-oriented libraries”. They use “nice C++ abstractions”. And quite frankly, as a result of all these design decisions that sound so appealing to some CS people, the end result is a horrible and unmaintainable mess.

This however is quite interesting since I am probably not the only one to have seen the disastrous effects of too enthusiastic abstraction professionals in Java. This is why many Java projects are utter crap. But again, not everybody makes this mistake. Some people know how to build good things. As the result of Java being popular, we have many crap programmers with pseudo genius theories you don’t find in C or Haskell.

On another subject, I seriously wonder why we don’t have more distributed compilation in Java as in C/C++. I am tired of seing those core doing nothing while compiling.

Double.NaN Is Evil

Double.NaN == myDouble

Double.isNaN(myDouble)

Grails Spring Union Not Surprising

Looking out at some old post. I found out I was not far from the truth in January 2008 when I stated:“In 2008 the Ruby On Rails mentality will continue to prevail. In the Java world, Grails is the most likely to benefit from it. (…) It could also be something based around Spring as their current MVC solution is not very good and very old fashioned."I don’t think I will be right with the provocative Java is dead. A post recently titled No Future In Java makes some good points about where the future of Java still is: the web applications. Grails is probably today the best contender in the Java world, far ahead from the others, and it leverages the Java developers. However I am not sure one can say that RIA is a fad, or that RIA will only be done in a super powerful browser in the future. Microsoft might be a game changer here. The real advantages of the browser application so far are: easy deployment (the killer argument IMHO), “simple” security. I would not be surprised if in the near future, Microsoft advertises a solution for RIA easily deployed, based on standard protocols (HTTP?), a bit like IBM does with Eclipse RIA, but much more ambitious.

Java Is Dead

These days, I have the feeling that Java is dead. Even if, or maybe because I have used Java so much in the past 10 years, I have this feeling.

In 1998 Java was revolutionary. It was a very simple to learn object oriented language with modern concepts and familiar syntax. Furthermore the standard library had neat features like internet networking and it could be integrated in the browser. All this at a time when the internet just started to be popular.

Today we have very few evolutions, a huge library (with lots of useful and useless stuff in). Some good stuff has been added like concurrent utils, but not many things changed overall. Open source languages like Python are much more dynamic in their library maintenance. The language does not seem to provide anything interesting when compared to the alternatives, like .NET or even with the “scripting languages” like Python. In the browser it has failed where Flash has succeeded.

Lots of things are too complicated to build in Java when compared to other languages. I feel Swing, database access (JDBC), JSP could be vastly improved to help developer productivity. Why is ORM less important in the Microsoft world? because the standard database layer of Microsoft is not as crappy as JDBC. Why don’t they have tons of web frameworks? because ASP.NET is decent, does more than JSP and does not get too much in your way at the same time. Microsoft finds the right balance between library complexity, power and developer friendlyness.

Browser applications are less popular, and desktop apps integrated to a “3-tier” architecture more popular. Java on the desktop is really weak. Give me .NET or QT anytime. There are still no big Java desktop apps on everyday people desktops except Eclipse (IBM has really done an impressive job with it). It is almost 2009 and I still have no Java app except the dev environment for my Java programmer job on my Linux desktop. I know that in my everyday job, I would be more productive with a .NET environment, just because Java sucks so much on the client side. Borland Delphi was more productive 10 years ago!

Java on the Mobile is a failure. Almost nobody uses it and is plagged with compatibility problems. However there is hope here, with Android from Google.

The only advantage of Java compared to .NET is that it is free. You have Tomcat, Glassfish for free. You can deploy on Linux. If you are a poor developer that’s quite an advantage. But most company pay for Java, they want the “security” of an IBM and they deploy on Windows machines. It does not make sense, those companies should buy the better Microsoft stack instead of IBM. And I am sure more and more will. Vista might be the big Microsoft failure, I am sure it will be fixed with Windows 7, and Microsoft dev tools are just getting better and better.

Scala, Groovy, JRuby don’t fix anything, they are just toy programming languages and are based on the JVM, on the Java libraries. In the lot, Scala does better because it has the concept of library, and they do try to build more interesting libraries than Sun. But it is too complex to be ever popular.

All the open source libraries in Java are fine but who needs to choose between 20 web frameworks, 5 loggers, etc.. There are very few really useful ones: hibernate, lucene, jmeter, junit.

If Java has no logical place in most companies, if it does not provide anything more than the alternatives, and is very weak on the desktop, what’s left to Java? the code base and the developers? That’s about it. It sounds a lot like Cobol in the early 90s. Java is dead.