Excel Bulk Entry of Jira using Apache HttpClient & POI

Where I work, I have to regularly enter my time in JIRA using their crappy portlet interface. Because of French regulations and bad design, one can enter time for at most 1 day at a time. This is very annoying especially to enter vacation days. I decided to spend some time (took me around 2 hours - I thought it would be much more) to enter the time from a local Excel spreadsheet (via with OpenOffice), and use Java to populate JIRA.

First I had to find out what where the relevant requests. Firefox has several extensions for that, but I found Tamper Data to be the easiest to work with (hint: use copy/paste in the Tamper Data window to get the full request in a nice format).

Apache HttpClient provides an easy way to do HTTP requests and handles cookies almost automatically in Java. Here is the login phase:

List<NameValuePair> formparams = new ArrayList<NameValuePair>();
formparams.add(new BasicNameValuePair("os_username", "mouse@thecat"));
formparams.add(new BasicNameValuePair("os_password", "DEADDEAD"));
UrlEncodedFormEntity entity = new UrlEncodedFormEntity(formparams, "UTF-8");
HttpPost httppost = new HttpPost("https://jira.calypso.com/rest/gadget/1.0/login");
httppost.setEntity(entity);
DefaultHttpClient httpclient = new DefaultHttpClient();
CookieStore cookieStore = new BasicCookieStore();
httpclient.setCookieStore(cookieStore);
ResponseHandler<byte[]> handler = new ResponseHandler[]>() {
public byte[] handleResponse(HttpResponse response)
throws ClientProtocolException, IOException {
System.out.println("<-" + response.getStatusLine());
HttpEntity entity = response.getEntity();
if (entity != null) {
return EntityUtils.toByteArray(entity);
} else {
return null;
}
}
};
System.out.println("->" + httppost.getURI());
byte[] response = httpclient.execute(httppost, handler);


Then a request to our JIRA portlet looks like:


formparams = new ArrayList<NameValuePair>();
formparams.add(new BasicNameValuePair("inline", "true"));
formparams.add(new BasicNameValuePair("decorator", "dialog"));
formparams.add(new BasicNameValuePair("startDate", startDate));
formparams.add(new BasicNameValuePair("timeLogged", timeLogged));
formparams.add(new BasicNameValuePair("id", id));
formparams.add(new BasicNameValuePair("adjustEstimate", "auto"));
entity = new UrlEncodedFormEntity(formparams, "UTF-8");
httppost = new HttpPost("https://jira.calypso.com/secure/CreateWorklog.jspa");
httppost.addHeader("Referer", "https://jira.calypso.com/browse/"+ jiraCAL);
httppost.setEntity(entity);
System.out.println("->" + httppost.getURI());
response = httpclient.execute(httppost, handler);


Parsing Excel with Apache POI is a bit annoying, but I kept fixed conventions to make things simple:


InputStream inp = new FileInputStream(file);
HSSFWorkbook wb = new HSSFWorkbook(new POIFSFileSystem(inp));
List list = new ArrayList();
HSSFSheet sheet = wb.getSheetAt(0);
boolean isEmpty = false;
int i = 0;
while (!isEmpty) {
HSSFRow row = sheet.getRow(i);
if (row == null) { isEmpty=true; break;}

HSSFCell dateCell = row.getCell(0);
HSSFCell calCell = row.getCell(1);
HSSFCell idCell = row.getCell(2);
HSSFCell percentCell = row.getCell(3);
if (dateCell == null) {
isEmpty = true;
} else if (dateCell.getCellType() == HSSFCell.CELL_TYPE_NUMERIC && calCell != null){
TimeLine timeLine = new TimeLine();
timeLine.date = HSSFDateUtil.getJavaDate(dateCell.getNumericCellValue());
if (timeLine.date.after(startDate)
&& timeLine.date.before(endDate)) {
timeLine.jiraCAL = calCell.getStringCellValue();
if (timeLine.jiraCAL != null && timeLine.jiraCAL.length() > 0) {
timeLine.id = Integer.toString((int)idCell.getNumericCellValue());
timeLine.percent = Integer.toString((int)percentCell.getNumericCellValue());
list.add(timeLine);
}
}
}
i++;
}


Obviously, this is not clean code, the goal was only to do something quick and dirty to solve my immediate problem.

Keyboard Porn

I have been browsing the web, looking for a nice computer keyboard. Programming is a big part of my job, a comfortable keyboard is therefore important to my daily life.

MS Natural keyboard

I have some nostalgia for the original Microsoft natural keyboard, the white one with standard home and end keys. When it came out it looked revolutionary. I remember really improving my typing speed on it. The only minor annoyance was the heavy and loud space bar. It’s sad that MS does not make those in azerty anymore (I know qwerty is better for programming - I used it while I was working in the US, but it’s just too annoying to have a different layout from 95% of the people around). The new MS ergonomic model just looks ugly and scary with all those extra keys.

Logitech K800

At home I have a Logitech K800, very practical as I often use it from the couch and at nighttime. When I tried it, I was impressed with the key feel: the action is a bit longer than a laptop keyboard (especially the Apple ones), and smooth. But one day, my son, by keeping on hammering on it with all the force of a 2 years old, damaged it. Some keys would print many times other letters. Still after unmounting some keys, and after washing the keyboard several times and waiting a couple of weeks it started working again. But now the keys feel very mushy and not very nice to type on. This is obviously due to the strong abuse, and the fact that the back is flexible plastic probably did not help. And I wonder if a new K800 (or a Logitech Illuminated) would not just age the same way.

I looked and looked, read the forums. The hip keyboards seem to be the mechanical ones. It tempted me to get a WASD keyboard even though I never liked the feel and sound of my uncle’s IBM model M keyboard. WASD offers the combination of red cherry switches and o rings which should be significantly better for me.

MS Wired Desktop 600

Somehow despite all this research for a better keyboard, I kind of like my current work keyboard, which is just a cheap Microsoft Wired Keyboard 600. It does feel comfortable, and easy to type on. If it ages, it is just $17 to replace. I am less convinced that another keyboard would improve things significantly, I disliked the very clicky feeling of IBM model M and the old, not so comfortable (because relatively high), straight keyboards. I found also the Apple keyboards to be sadly a bit tiring with their very short action, I enjoyed their compactness but I suspect this is what made me place my hands in bad positions.

I manage to do around 70 wpm (words per minute) on the Microsoft Wired Keyboard 600, as well as on some very basic 4€ keyboard (old style slightly noisy chinese rubber dome Atlantis Land K2101), but on the Logitech K800, I only do 55 wpm. So there it is for the 80€ keyboard compared to the 4€ one. Those numbers are not very precise, I only tried a silly simple wpm website. I am still a bit curious about those MS comfort curve keyboards, they don’t have great reviews but it might be a small improvement over the basic Wired Keyboard 600.

Adaptive Quadrature for Pricing European Option with Heston

The Quantlib code to evaluate the Heston integral for European options is quite nice. It proposes Kahl & Jaeckel method as well as Gatheral method for the complex logarithm. It also contains expansions where it matters so that the resulting code is very robust. One minor issue is that it does not integrate both parts at the same time, and also does not propose Attari method for the Heston integral that is supposed to be more stable.

I was surprised to find out that out of the money, short expiry options seemed badly mispriced. In the end I discovered it was just that it required sometimes more than 3500 function evaluations to have an accuracy of 1e-6.

As this sounds a bit crazy, I thought that Jaeckel log transform was the culprit. In reality, it turned out that it was Gauss Lobatto Gander & Gautschi implementation. I tried the simplest algorithm in Espelid improved algorithms: modsim, an adaptive extrapolated Simpson method, and it was 4x faster for the same accuracy. That plus the fact that it worked out of the box (translated to Java) on my problem was impressive.

Jaeckel log transform (to change the interval from 0,+inf to 0,1) works well, and seems to offer a slight speedup (10% to 15%) for around ATM options, mid to long term for the same accuracy. Unfortunately, it can also slow down by up to 50% the convergence for more OTM options or shorter expiries. So I am not so sure about its interest vs just cutting off the integration at phi=200.

Gnome Shell more stable than Unity on Ubuntu 12.04

Regularly, the unity dock made some applications inaccessible: clicking on the app icon did not show or start the app anymore, a very annoying bug. This is quite incredible given that this version of Ubuntu is supposed to be long term support. So I decided to give one more chance to Gnome Shell. Installing it on Ubuntu 12.04 is simple with this guide.

To my surprise it is very stable so far. Earlier Gnome Shell versions were not as stable. After installing various extensions (dock especially) it is as usable as Unity for my needs. It seems more responsive as well. I am not really into the Unity new features like HUD. It sounds to me like Ubuntu is making a mistake with Unity compared to Gnome Shell.

To make an old extension support latest Gnome Shell version, it is sometimes necessary  to update  the extension metadata with what's given by gnome-shell --version. For the weather extension you can just edit using gedit:

sudo gedit /usr/share/gnome-shell/extensions/weather@gnome-shell-extensions.gnome.org/metadata.json

Google Galaxy Nexus Sound Quality Is Great

Many people are not enthusiastic of this phone sound if you read silly forums. They are wrong! the sound coming out of this thin phone is amazing, at least with high quality headphones. I find the akg q601 incredible with it: much much better than with the old ipod nano or the cowon i7.

In general most complaints i have read about the phone were wrong. The battery is ok, the size is great.

Scala Again

I am trying Scala again. Last time, several years ago, I played around with it as a web tool, combining it with a Servlet Runner like Tomcat. This time, I play around with it for some quantitative finance experiments.

Why Scala? It still seem the most advanced alternative to Java on the JVM, and the mix of functional programming and OO programming is interesting. Furthermore it goes quite far as it ships with its own library. I was curious to see if I could express some things better with Scala.

Here are my first impressions after a week:
  • I like the object keyword. It avoids the messy singleton pattern, or the classes with many static methods. I think it makes things much cleaner to not use static at all but distinguish between object & class.
  • I like the Array[Double], and especially ArrayBuffer[Double]. Finally we don't have to worry between the Double and double performance issues.
  • I was a bit annoyed by a(i) instead of a[i] but it makes sense. I wonder if there is a performance implication for arrays, hopefully not.
  • I like the real properties, automatic getter/setter: less boilerplate code, less getThis(), setThat(toto).
  • Very natural interaction with Java libraries.
  • I found a good use of case classes (to my surprise): typically an enum that can have some well defined parameters, and that you don't want to make a class (because it's not). My use case was to define boundaries of a spline.
  • I love the formatter in the scala (eclipse) IDE. Finally a formatter in eclipse that does not produce crap.
Now things I still need time to get used to:
  • member variable declared implicitly in the constructor. I first made the mistake (still?) to declare some variables twice.
  • I got hit by starting a line with a + instead of ending with a +. It is dangerous, but it certainly makes the code more consistent.
  • Performance impacts: I will need to take a look at the bytecode for some scala constructs to really understand the performance impact of some uses. For example I tend to use while loops instead of for comprehension after some scary post of the Twitter guys about for comprehension. But at first, it looks as fast as Java.
  • I wrote my code a bit fast. I am sure I could make use of more Scala features.
  • The scala IDE in eclipse 3.7.1 has known issues. I wish it was a bit more functional, but it's quite ok (search for references works, renaming works to some extent).
  • Scala unit tests: I used scala tests, but it seems a bit funny at first. Also I am not convinced by the syntax that avoid method names and prefer test("test name"). It makes it more difficult to browse the source code.
Some things they should consider:
  • Integrate directly a Log API. I just use SLF4J without any scala wrapper, but it feels like it should be part of the standard API (even if that did not work out so well for Sun).
  • Double.Epsilon is not the machine epsilon: very strange. I found out somewhere else there was the machine epsilon, don't remember where because I ended up just making a small object.
  • Unit tests should be part of the standard API.
Overall I found it quite exciting as there are definitely new ways to solve problems. It was a while since I had been excited with actual coding.

KDE 4.8 finally has a dock

KDE 4.8 finally has a dock: you just have to add the plasma icon tasks. Also the flexibility around ALT+TAB is welcome. With Krusader as file manager, Thunderbird and Firefox for email and web, it is becoming a real nice desktop, but it took a while since the very bad KDE 4.0 release.

It is easy to install under ubuntu 11.10 through the backports and seems very stable so far.

Something quite important is to tweak the fonts: use Déjà Vu Sans instead of Ubuntu fonts, use RGB subpixel rendering, use Crisp desktop effects. With those settings, KDE looks very nice. It's sad that they are not default in Kubuntu.

Update March 2013: It's been a while now that it is in the standard Ubuntu repositories and I believe installed by default, one has just to remove the task manager widget add the icon task widget:
One can also change the settings using a right click (I find useful not to highlight the windows) and it can look like:


List of companies where I have been an employee

Intern:

  • Siemens (Berlin)
  • IBM (Boeblingen)
  • Osram Sylvania (Beverly, MA)

Employee:

  • Cap Gemini (Paris) working for Alcatel
  • Silicomp (Paris) working for Alcatel Nextenso
  • C2labs / one 0 development (San Francisco, CA) working for Whenmobile, Sony Pictures, GoPix, Technorati.
  • Credit Agricole alternative (Paris)
  • Prima solutions (Paris)
  • Esearchvision (Paris)
  • Ulink (Paris)
  • Edifixio (Paris)
  • Horizon (Paris)
  • Darty (Paris)
  • Calypso (Paris)

Previous

Next