Non Blocking with Traditional Java IO - On the Use of InputStream.available() and Thread.sleep()

Thu, Jun 8, 2006 Some time ago I did quite a lot of IO in Java and I yet did not see this way of reading a InputStream from a Socket:

    InputStream in=channel.getInputStream();

    channel.connect();

    byte[] tmp=new byte[1024];
    while(true){
      while(in.available()>0){
        int i=in.read(tmp, 0, 1024);
        if(i<0)break;
        System.out.print(new String(tmp, 0, i));
      }
      if(channel.isClosed()){
        System.out.println("exit-status: "+channel.getExitStatus());
        break;
      }
      try{Thread.sleep(1000);}catch(Exception ee){}
    }
    channel.disconnect();

This comes from a piece of code from an example of JSch , a good ssh client in java. A work collegue had the bad idea to remove the Thread.sleep call and was struggling with why it would randomly work.The way I would have done it is the following:

    InputStream in=channel.getInputStream();

    channel.connect();

    byte[] tmp=new byte[1024];
    int bytesRead = 0;
    while((bytesRead = in.read(tmp,0,1024) >= 0){
      System.out.print(new String(tmp, 0, bytesRead));
    }

    if(channel.isClosed()){
      System.out.println("exit-status: "+channel.getExitStatus());
      break;
    }
    channel.disconnect();

This has the advantage of being more readable and having less secret spices in it. In the first code, the call to available() is non blocking, meaning that without the Thread.sleep(), there will never be the time for the socket buffer to fill up. But is the first code more efficient or the second code?

I did a search on google to understand the interest of the first code. The only advantages I found in the first code are the possibility to interrupt the thread running the code and a finer grained control on timeouts.

There is a lengthy explanation by Doug Lea in his book "Concurrent Programming in Java". This book usually provides excellent explanations, and is a must read for anybody doing concurrent programming. But this time, about this subject, I did not find him that clear.

There is a more simple explanation in a course from San Diego State University (see last example)

A read() on an inputstream or reader blocks. Once a thread calls read() it will not respond to interrupt() (or much else) until the read is completed. This is a problem when a read could take a long time: reading from a socket or the keyboard. If the input is not forth coming, the read() could block forever.

As usual, you should not rely on all what you read on the web, as this page (SCJP Questions & Answers) testifies:

Q. When will a Thread I/O blocked?

A:

When a thread executes a read() call on an InputStream, if no byte is available. The calling Thread blocks, in other words, stops executing until a byte is available or the Thread is interrupted.

Still I am wondering if the second code would not just go into IOException (socket timeout), on timeout (adjustable with Socket.setTimeout ) and release the Thread then. Do you have an idea when the first code could be better?

Is Java Flawed - a big advantage of Python/Ruby/(your favorite interpreted language)

Fri, Jun 2, 2006

Java is supposed to be much better to build big projects, because of static type checking, and all the rigour around the language. But how many of you have seen medium sized projects taking more than 30 minutes to build.

At work, they have a standard J2EE project, with only about 50 EJBs, hundreds of JDO classes, and standard classes. Between the JDO generation, EJB generation, EJB dependencies calculations, and packaging, it takes 20 minutes. And the project is not doing that much. One can optimize to avoid dependencies calculations and it would then take about 12 minutes. But still.

With Python, Ruby or your favorite interpreted language, this would be at most a few seconds to test a new version. Now I am a fan of Java and don’t enjoy that much programming in interpreted languages, especially since modern IDE like Eclipse do so much for you.

To sum up, Java is supposed to be the right language for large scale projects, and yet it is on large scale projects that compilation is an issue. Ok, this will enforces a better separation of concerns, and will probably be beneficial in the end. But I am not sure that most people do a good separation of concerns in projects they manage. I think it is no accident if the next step in Java development involves much more runtime compilation. It somehow started with JSPs, and then with the various XML config files combined to java reflection. It is now made even more popular with annotations, see EJB 3.0.

I am wondering a bit if one day compiled languages will be only a curiosity for most enterprise projects.

Using Linux to Recover Fucked Up Windows Data

Thu, Jun 1, 2006

Recently, one of my relatives computer under Windows XP, refused to boot. There was no way of fixing it with Windows Install CD as partition table seemed corrupt to Windows. I tried everything in an 2003 Ultimate Boot CD, but nothing worked out.

Someone gave me an install cd of Ubuntu Linux, and it managed to read the data. Well sometimes only. The erratic behaviour was due to a bad ATA cable. This probably was the cause of the corruption in the first place. Anyway with a new cable, Windows was still not able to read its data. But Ubuntu Linux, now working well, was able to, without having anything to configure (except mounting the drive).

So I copied the data, and reformated the NTFS partition, reinstalled Windows, recopied the data.

After this experience, I’d recommend to any Windows user to have a spare Ubuntu Linux Live CD, just in case your Windows corrupts itself.

Java HTML Parsing Example With htmlparser

Tue, May 30, 2006 Every week, I post javablogs top 10 most read blog entries on this blog. The reason for it was that I don't look at what's happening on the week-end and this will pickup interesting stories from the weekend, and I also don't watch javablogs everyday. Overall I find it quite good to be uptodate with interesting stuff happening on javablogs.

As mentionned in an earlier post my library of choice to do the parsing is htmlparser (on sourceforge) because it's free, open source and because I am lazy and did not want to do my own. If you know a better open source library, feel free to add a comment about it, I'll be glad to hear about it. htmlparser is not the easiest library to use, there are many entry points and it's not immediately clear which one to choose. So I post here how I used it if it can save a few minutes to people having to do this task.

  private static Entry parseEntry(String content) throws ParserException
  {
    final Entry entry = new Entry();

    final NodeVisitor linkVisitor = new NodeVisitor() {
      
      @Override
      public void visitTag(Tag tag) {
        String name = tag.getTagName();

        if ("a".equalsIgnoreCase(name))
            {
              String hrefValue = tag.getAttribute("href");
              if (hrefValue != null && !hrefValue.startsWith("http://"))
              {
                if (!hrefValue.startsWith("/")) hrefValue = "/"+hrefValue;
                hrefValue = "http://javablogs.com"+hrefValue;
                //System.out.println("test, value="+hrefValue);
              }
              if (hrefValue != null)
              {
                hrefValue = hrefValue.replaceAll("&", "&amp;");
                tag.setAttribute("href", hrefValue);                
              }
            }
      }
    
    };
    
    NodeVisitor visitor = new NodeVisitor() {

      @Override
      public void visitTag(Tag tag) {        
        String name = tag.getTagName();
            if ("span".equalsIgnoreCase(name) || "div".equalsIgnoreCase(name))
            {              
              String classValue = tag.getAttribute("class");
//                LOGGER.debug("visittag name="+name+" class="+classValue+"children="+tag.getChildren().toHtml());
              if ("blogentrydetails".equals(classValue))
              {
                Pattern countPattern = Pattern.compile("Reads:\\s*([0-9]*)");
                Matcher matcher = countPattern.matcher(tag.getChildren().toHtml());
                if (matcher.find())
                {
                  String countStr = matcher.group(1);
                  entry.count = new Integer(countStr).intValue();
                }
                
              }
              else if ("blogentrysummary".equals(classValue))
              {
                try
                {
                  tag.getChildren().visitAllNodesWith(linkVisitor);
                }
                catch (ParserException pe)
                {
                  LOGGER.error(pe,pe);
                }
                entry.description = tag.getChildren().toHtml();                 
                entry.description = entry.description.replaceAll("\\s+", " ");
              }
              else if ("blogentrytitle".equals(classValue))
              {
                try
                {
                  tag.getChildren().visitAllNodesWith(linkVisitor);
                }
                catch (ParserException pe)
                {
                  LOGGER.error(pe,pe);
                }
                entry.title =tag.getChildren().toHtml(); 
                entry.title = entry.title.replaceAll("\\s+", " ");
              }              
            }
            
      }

    };
    Parser parser = new Parser(new Lexer(new Page(content,"UTF-8")));
    parser.visitAllNodesWith(visitor);
        if (entry.title != null)
        {
          return entry;
        }
        else return null;
  }

First Steps With EhCache

Thu, May 4, 2006

If you need to cache objects in your system, Ehcache is a simple cache written in Java, widely used and well tested. I will present here a short tutorial on how to use EhCache for people who don't want to look around the documentation at first, but just want to test if it works in their project and to see how easy it is to setup.

Installation

Download Ehcache from the Download link on http://ehcache.sourceforge.net. Current release is 1.2.

Unpack Ehcache with an unpacker that knows the tgz format. For unix users, it is trivial, for windows users, 7zip is a free (and open-source) unpacker. It is probably the most popular, but there are other ones like tugzip or izarc or winrar.

In your java project you need to have ehcache-1.2.jar, commons-collections-2.1.1.jar and commons-logging-1.0.4.jar (versions numbers may vary) in your classpath, those libraries are shipped with ehcache.

Cache Configuration

Write an ehcache.xml file where you describe what cache you want to use. There can be several files per project, several cache descriptions per file. I use here a persistent cache. Configuration file is well described at http://ehcache.sourceforge.net/documentation/configuration.html

</ehcache>

Code

static  {  
 //Create a CacheManager  using a specific  config file  
 cacheManager = CacheManager.create(TestClass.class.getResource( "/config/ehcache.xml"));
 cache = cacheManager.getCache("firstcache");
} 

/**
 * retrieves value  from cache if exists
 * if not  create it and add  it to cache */
public String doit(String key, String value) {
 //get an element  from cache by key
  Element e = cache.get(key);
 if (e != null)    {
   value  = (String)e.getValue();
   LOGGER.info("retrieved "+ value+" from cache ");
 }
  else   {
   value = "new value" ;
   cache.put(new Element(key, value));
 }
  return value;
}

/**
 * refresh value  for given key */
public void refresh(String key) { cache.remove(key); }

/**
 * to call eventually  when your application is  exiting  */
 public void shutdown() { cacheManager.shutdown(); }

Conclusion

Using EhCache is as simple a using a Java Map with an additional configuration file.

Null vs. Errors

Wed, Apr 26, 2006

I am not particularly a fan of JCS (Jakarta Cache System) as I find ehcache code very clean and simple. But I have to say the author has some good comments on the site:

Nulls vs. Errors

I started to support ObjectNotFoundExceptions for failed gets but the overhead and cumbersome coding needed to surround a simple get method is ridiculous. Instead the JCS return null.

For having seen too many times the ObjectNotFoundException "pattern", I can only agree!

Non Blocking with Traditional Java IO - On the Use of InputStream.available() and Thread.sleep()

Top 10 Most Read Last Week On Javablogs.com, Week 22

Is Java Flawed - a big advantage of Python/Ruby/(your favorite interpreted language)

Using Linux to Recover Fucked Up Windows Data

Java HTML Parsing Example With htmlparser

Top 10 Most Read Last Week On Javablogs.com, Week 21

Top 10 Most Read Last Week On Javablogs.com, Week 20

Top 10 Most Read Last Week On Javablogs.com, Week 19

First Steps With EhCache

Null vs. Errors