can't touch this

so been plating with the iPod touch for a few days now and am trying to type out an entry as fast a the keyboard will let me. not too bad word replacement is happening and I can type out reasonably quickly. for posts re rich text we does not work but is fine ss HTML. amy caps you see are actually word replacements.

well a good test to say the least. makes me hopeful that a lims app is not so far fetched.

Easy Gbarcode

A few posts ago I had mentioned that my Gbarcode project should really be a bit more user friendly as well as provide an easy way to create PNGs without loading memory hogging libraries like RMagick. I investigated using the Cairo libraries for this here.

Good news for every one that uses Gbarcode, and those thinking of using it, I created a small wrapper module that is more Ruby-ish and also uses Cairo to print out to PNG. The functionality is bare-bones, with a fixed height of 150 pixels and width is determined by the length of the encoded barcode. It works well for Code 128 barcodes. Grab the ruby file here.

To use this module, here is an example (note that bad encoding schemes will result in a raised excpetion, hence the begin/rescue block):

require 'gbc'
require 'gbarcode'
require 'markaby'

begin
b = GBC::Barcode.new(ARGV[0], Gbarcode::BARCODE_ANY)
puts b.ascii, b.partial, b.encoding
b.to_png("out.png")
b = Markaby::Builder.new()
b.html do
body do
img(:src => "out.png")
end
end
f = File.open("out.html","w")
f.write(b.to_s)
f.close
rescue Exception => e
puts e.message
end


ITMArT: Design: Part 1.2 "The competition revised"

During the last post, we took a look at two open-source procurement solutions out in the wild. Today I noticed that there is a third project that we should take a look at, Coupa. Coupa is a commercial venture that also provides an express edition of their product (e.g. a bit less features than the supported product).

Simply put, their mission statement is very similar to ITMArT's, namely that purchasing should be simple, intuitive and serve the needs of both the managers and users of the system, not just the managers. Taking a look at the demos, Coupa blows ITMArT out of the water feature and UI wise, while retaining some pretty advanced capabilities. There are very nice reporting modules, web forms, invoice ability, role and group based access to data and actions, very nice email integration, etc. If I where a finance guy at the University level, I would stop all in-house development right now and switch to their platform. Its that nice.

Having said that, I am not at the University level, or even departmental level. ITMArT's audience is a much smaller group, namely the medium to large laboratory, and they rarely need such things are approval hierarchies, approved vendor catalogs, etc. Those needs are usually at a higher organizational level and hence ITMArT development does not stop here.

A bit of good news is that I have lots of good ideas from the Coupa demos that I will ut on the feature request list of ITMArT. Another good bit is that Coupa express edition is a Ruby on Rails application, the same as ITMArT! So I can take advantage of some of their codebase for a few nagging concerns of mine, like authentication, group assignments/permission, and role based authorities.

ITMArT : Design : Part 1 "The competition"

So far my posts have focused on the bencher's quest (to learn code). This is mainly because it is infinitely easier to write about small and helpful scripts than about complex issues like application UI design. But since I introduced the ITMArT application, I might as well go through the design process for it. This will be a mutlipart post covering a couple of areas of the application and design process. First up will be the history and requirments of the app.


History

ITMArT began as any project does:Outlining a set of requirements and then looking to see if some existing application already does what you want to accomplish. In this case, the requirements consisted of:

1. an application that tracks purchase requests and purchase orders
2. users would often have recurring request, a searchable catalog would be ideal
   
3. new types of requests come up often so adding vendors and items should be easy and flexible
   
4. orders can only be placed by select group of users with access to funds
5. monthly report of expenditures would be great
6. email alerts of requests from request to order to receiving
   

You'd think there would be some open source ERP system out there that would have these component in it, and there are: Openbravo.com and TinyERP are just two examples. The trouble is that ERP systems generally tend to focus on the needs of the business processes and not so much on the requirements of the end user. I'll get to what that means in a minute, but suffice to say this type of focus would not have worked for us.

The ERPs

Let's first look at Openbravo, a Zope application and UI nightmare. Pop-ups all over the place, difficult navigation and difficult to use forms. Tons of click-through dialogs and switching between keyboard and mouse, essentially MSDN design taken to the web. But don't just take my word for it, demo it yourself and see.

TinyERP is much better, but still focuses on the business case. For instance their forms (and top menu) make a distinction between creating a purchase order and creating the line items for that order. Which means that the user must know ahead of time that they need to create an order request first, which means that the system is not intuitive enough and will either require documentation (which no one reads) or training (which you have to pay for). Much worse, when creating the order line items, you need to specify a ton of fields and know the purchase order ID! OK, so the AJAX search helps you out for finding the order ID, but one typo and you have a clean up on aisle 9.

The last thing that makes an ERP unsuitable would be my first point, namely the focus on business processes. Usually this is not a problem, since you can make the assumption that the community is relatively stable and business oriented. The user community we have to service is a mix of bench scientists, admin assistants and business office types. There are no resources for training and the usability expectations are high.

This means that we have to shoot for the Lowest Common Denominator in terms of the overhead placed the ordering process. Making a purchase request should be as simple as creating an email, and if it is not, then there had better be some work savings later to make up for it. Order placement in TinyERP is less than ideal, but not completely onerous as Openbravo, and the work savings are that the reporting, catalog searching and order processesing are streamlined.

The deal breaker comes in the catalog management aspect. Adding items and vendors is not intuitive or easy enough for the majority of our user base. This is where focusing on the business process of catalog management really takes a pounding on usability. The central assumption here is that catalogs come from a preferred set of vendors and are bulk updated (and not very often). The broad user community means that the vendor list will be constantly changing, that pricing will be variable, and the various catalogs will be fluid. The burden of catalog management must be passed on to the purchase requester for the application to be scalable (e.g. roled out to other departments or laboratories). Centralized management of the catalog would incur too high a cost otherwise.

Conclusion

Based on these quick reviews of existing systems, I decided that further trolling through open source projects would not be worth my time. If even a simple, easy, and good looking application like TinyERP did not fit the bill, what would? A custom application, that's what.

On the next ITMArT post, we'll dicuss the actual design choices made to fulfill the list of requirements that we had mentioned at the start.

Signs from above

So my earlier idea of using an iPhone or iPod touch as the gateway into an electronic lab notebook for tracking protocols and doing real-time data entry has been given some conflicting signals from the powers that be. One of those powers (my boss) answered my long (but not too long) email proposal with a one word reply. What was the word you ask? ... "cute" ... No signature even.

Another higher power (the internet) quickly followed that crushing monosyllabic bitch-slap with an uplifting rumour from AppleInsider that Apple is developing a PDA device based on the iPhone & Touch UI, but with a bigger screen. Bigger screen (woohoo!) and the potential to install non-Apple third party apps (highly unlikely given Apple's recent activities) would be an ideal device to use as a lab e-notebook tablet.

Oh, whom do I listen to??? Should I drop the project completely? Or should I table the project until the a new, (relatively) low-cost, and developer-friendly device pops into existence?

In the end, I decided to listen to my inner voice, you know the impatient one, and just bought myself a touch to play around with for the time being. If this mythical iPAD ever sees the light of day, I'll raise the issue again with the G-man and get him to foot the bill for what will probably be a more expensive device. Either way I win ;) Unless of course he calls my bluff ...

Cairo for barcodes

Continuing my discussion about Gbarcode 2, I tested my theory of using Cairo for creation of barcodes. It turns out that this is not so hard, but it was a bit of a lerning curve, to say the least. I used the ruby DL library to load the cairo shared libs from the system (thanks to the GD2 gem for the code hints here). A few small methods where all that were needed for bare minimum functionality: creating bars and adding text. No fancy formating here.

Next, I used gnu barcode to get layout information for a barcode, so I could test the drawing methods independently of barcode creation logic.

The result is the picture posted above. Neat huh? Code is posted below, but I think for a production gem, I'll probably not use DL, since I have to wrap the gnu C libs for actually creating barcodes from text strings anyway.

Without further ado, the test script:

require 'dl'
require 'rbconfig'

module BC
VERSION = '1.5.0'.freeze

def self.cairo_library_name
case Config::CONFIG['arch']
when /darwin/
  'libcairo.2.dylib'
when /mswin32/, /cygwin/
  'cairo.dll'
else
  'libcairo.so.2'
end
end
def self.name_for_symbol(symbol, signature)
case Config::CONFIG['arch']
when /mswin32/, /cygwin/
  sum = -4
  signature.each_byte do |char|
    sum += case char
    when ?D: 8
    else     4
    end
  end
  "#{symbol}@#{sum}"
else
  symbol.to_s
end
end

private_class_method :cairo_library_name, :name_for_symbol

LIB = DL.dlopen(cairo_library_name)
SYM = {
:cairo_image_surface_create   => 'PIII',
:cairo_create    => 'PP',
:cairo_get_target    => 'PP',
:cairo_destroy    => '0P',
:cairo_surface_destroy    => '0P',
:cairo_surface_write_to_png    => '0PS',
:cairo_set_source_rgb    => '0PDDD',
:cairo_move_to    => '0PDD',
:cairo_line_to    => '0PDD',
:cairo_set_line_width    => '0PD',
:cairo_stroke    => '0P',
:cairo_select_font_face    => '0PSII',
:cairo_set_font_size    => '0PD',
:cairo_show_text    => '0PS'
}.inject({}) { |x, (k, v)| x[k] = LIB[name_for_symbol(k, v), v]; x }

class LibraryError < rs =" SYM[:cairo_image_surface_create].call(0,w,h)" rs =" SYM[:cairo_create].call(s)"> #{s.class}] :: R[#{r} =>  #{r.class}] :: RS[#{rs} =>  #{rs.class}]"
  SYM[:cairo_set_source_rgb].call(r,0.0,0.0,0.0)
  puts "S [#{s} => #{s.class}] :: R[#{r} =>  #{r.class}] :: RS[#{rs} =>  #{rs.class}]"
  return r
end

def self.ctx w,h
  context(surface(w,h))
end

def self.add_bar ctx,x,y,w,h
  # cairo_move_to(cr,11.0,20.5);
  # cairo_line_to(cr,11.0,70.5);
  # cairo_set_line_width(cr,1.85);
  # cairo_stroke(cr);
  SYM[:cairo_move_to].call(ctx,x,y)
  SYM[:cairo_line_to].call(ctx,x,h)
  SYM[:cairo_set_line_width].call(ctx,w)
  SYM[:cairo_stroke].call(ctx)
end

def self.add_text(ctx,txt,x,y)
  # cairo_select_font_face (cr, "serif", CAIRO_FONT_SLANT_NORMAL = 0, CAIRO_FONT_WEIGHT_BOLD = 1);
  # cairo_set_font_size (cr, 12.0);
  # cairo_move_to (cr, 21.0, 90.0);
  # cairo_show_text (cr, "TEST1234");  
  SYM[:cairo_select_font_face].call(ctx,"serif",0,1)
  SYM[:cairo_set_font_size].call(ctx,12.0)
  SYM[:cairo_move_to].call(ctx,x,y)
  SYM[:cairo_show_text].call(ctx,txt)
end

def self.draw(ctx,fname)
  # surface = cairo_get_target(cr)
  # cairo_destroy(cr);
  # cairo_surface_write_to_png (surface, "hello.png");
  # cairo_surface_destroy (surface);
  r,rs = SYM[:cairo_get_target].call(ctx)
  SYM[:cairo_destroy].call(ctx);
  SYM[:cairo_surface_write_to_png].call(r,fname)
  SYM[:cairo_surface_destroy].call(r);
end
end
end

include BC
c = B.ctx 132, 100

B.add_bar(c,  11.00 , 20.00,  1.85, 70.0)
B.add_bar(c,  13.50 , 20.00,  0.85, 70.0)
B.add_bar(c,  16.50 , 20.00,  0.85, 70.0)
B.add_bar(c,  22.00 , 20.00,  1.85, 70.0)
B.add_bar(c,  25.50 , 20.00,  2.85, 70.0)
B.add_bar(c,  30.50 , 20.00,  0.85, 70.0)
B.add_bar(c,  32.50 , 20.00,  0.85, 70.0)
B.add_bar(c,  37.00 , 20.00,  1.85, 70.0)
B.add_bar(c,  39.50 , 20.00,  0.85, 70.0)
B.add_bar(c,  44.00 , 20.00,  1.85, 70.0)
B.add_bar(c,  47.50 , 20.00,  2.85, 70.0)
B.add_bar(c,  50.50 , 20.00,  0.85, 70.0)
B.add_bar(c,  55.00 , 20.00,  1.85, 70.0)
B.add_bar(c,  58.50 , 20.00,  2.85, 70.0)
B.add_bar(c,  63.50 , 20.00,  0.85, 70.0)
B.add_bar(c,  65.50 , 20.00,  0.85, 70.0)
B.add_bar(c,  68.50 , 20.00,  2.85, 70.0)
B.add_bar(c,  73.00 , 20.00,  3.85, 70.0)
B.add_bar(c,  76.50 , 20.00,  0.85, 70.0)
B.add_bar(c,  79.00 , 20.00,  1.85, 70.0)
B.add_bar(c,  83.50 , 20.00,  2.85, 70.0)
B.add_bar(c,  87.50 , 20.00,  0.85, 70.0)
B.add_bar(c,  91.50 , 20.00,  0.85, 70.0)
B.add_bar(c,  94.00 , 20.00,  1.85, 70.0)
B.add_bar(c,  99.50 , 20.00,  2.85, 70.0)
B.add_bar(c, 103.00 , 20.00,  1.85, 70.0)
B.add_bar(c, 105.50 , 20.00,  0.85, 70.0)
B.add_bar(c, 110.00 , 20.00,  1.85, 70.0)
B.add_bar(c, 115.50 , 20.00,  2.85, 70.0)
B.add_bar(c, 118.50 , 20.00,  0.85, 70.0)
B.add_bar(c, 121.00 , 20.00,  1.85, 70.0)


B.add_text(c,"TEST1234", 21.0, 90.0)
B.draw(c,"test_bc.png")

Mad plotz

A recent submission to a journal has caused us a few headaches over the past few weeks, as the editors sent the paper back to us stating that we did not meet the minimal reporting requirements for the type of experiment that was performed. Which is poetic justice in a way, since for the past few years I have been promoting the use of minimal reporting requirements and standard data formats.

I think this particular journal, however, has gone a bit too far in asking for annotated spectra for every identification in the result set. For most low-throughput experiments this is not such a big deal, but we had thousands of identifications and even wrote an algorithm to automatically assign a quality score to those identifications so that such manual validation of spectra should not be necessary.

But I digress. Instead of fighting it, we decided to give the editors what they want, annotated spectra for every hit. It turns out that this is not such a trivial thing to do. Even gathering all of the data was a tough job, since the experiment was performed many years ago on instrumentation that is nearing its end of life. A lot of file parsing and data reorganization had to be done, prior to any development effort to produce the images that the journal wanted.

A bit of background and some numbers will help us understand the enormous task we undertook. The experiment was a proteomics profile of two developmental stages in zebra fish. We used two methodologies, 2D gels and LCMS, to fractionate the samples and ran them through mass spectrometers. The 2D gels gave fewer identifications than the LCMS, but it was still a lot of data. For instance, just these results contaied of 30,000 peptide identifications! You can reduce that to about 2,000 proteins that the journal has asked for annotated spectra. Needless to say, the brute force method of taking screen shots of each spectra from the program would not work.

I wrote a few scripts and libraries to parse the raw data and the final result table to come up with the above figure. This is bringing mzXML, excel, and MGF files together with Ruby, C libraries, and the R statistical tool to produce the nice picture you see, but it took me two weeks to figure out the specifics. How on earth could a regular bencher do this?

I think the journal is in for a rude awakening once the backlash of angry rebuttals from paper submitters start flowing in. I would also like to see their reaction to the gigantic pile of spectra we are about to send them.