Radio silence: ended


Writing and drawing are slow, deliberate activities, so I tried to keep distractions to an absolute minimum, and I did everything myself. I liked this control, as it fostered a sense of craftsmanship. There was great personal satisfaction in attending to detail and quality, and I remain very proud of the standards the strip met day after day. I also liked the responsibility of knowing that, succeed or fail, it was all my own doing.

— Bill Watterson, Calvin and Hobbes

Besides writing code all day, one of the more interesting things about the job is what I get to do on the side. Technically, Janelia expects me to be physically present for nine months a year, and it gives me three months to do whatever I want — skiing in New Zealand, NIH study section, visiting some university and giving a seminar, swine flu, it all counts against my three months of “vacation time” as far as Janelia’s concerned. So. Mild-mannered software/bio geek by day; advisor to US government agencies by… uh, well… seemed like pretty much the whole summer, this year. Two big time commitments with serious committee reports to help write. One report is finished for the Department of Defense, and another is well underway for the National Academy of Sciences. That, plus three weeks in Spain for the Benasque RNA conference, plus some review work for NIH all pretty much enforced a personal vacation away from the HMMER code. Probably good, because the code was making me cross-eyed, not to mention cross. I needed a break from it.

Nonetheless, despite a few months of radio silence, things have been moving forward for HMMER3. Especially on the Grand Long Term Strategy front. HMMER now has a development team.

Michael Farrar joined the lab this summer, after a long recruitment process. Michael’s Bioinformatics paper on SIMD parallelization of Smith/Waterman is what got me on track thinking about where we were going to get the two or three orders of magnitude of speed increase in HMMER we needed to get close to BLAST speed. He’s currently working on parallelizing H3 at the threads and MPI levels for multicores and clusters. Michael expects to have both levels of parallelization in the next (and final?) beta release, 3.0b3. Having Michael in the lab means that for the first time, there’s someone else full-time responsible for the nitty gritty details of HMMER code besides me. I think he’s enjoying it, but he seems a little horrified by how much work we have to do.

Travis Wheeler joined the lab at the beginning of September as a postdoc. He’s going to be working on HMMER3 too – I’ll probably start talking about his project as it gains momentum. With Travis and Sergi Castellano, another postdoc working on HMMER3, we’ve got some depth in the research side of the project.

Bjarne Knudsen, international man of mystery, heroic cave diver, resident guru and chief scientific officer (or some such title; I didn’t quite catch it all) of CLC bio in Denmark, started looking at the H3 code at Benasque in August. We didn’t get as much time together at Benasque as we wanted (I was whisked down from the mountains by ambulance and missed half the meeting – long story, more reason for radio silence), but Bjarne kept working on the code anyway, and sent us a spiffy report on all the bazillion ways he found to optimize my code. (People who are smarter than me are annoying, but I’ve learned to deal with it. Mostly.) He just flew over and spent a week in the lab going through his modifications to the code with us in person. We’ve got his code in our Subversion now. It’s a great start and we plan to continue an open-source collaboration with CLC bio, so CLC can seamlessly integrate H3 into their tools.

These are big changes, somewhat wrenching for me personally. Suddenly our Subversion repositories are chattering with commits, with four other people working on HMMER, two on Infernal (Eric Nawrocki and Diana Kolbe), and most everyone nibbling on the underlying Easel library. My job of establishing proof of principle for the H3 codebase is clearly coming to an end, and now my job will shift towards nitpicking code contributions, dictatorially enforcing code style, writing nasty test cases that show why some clever optimization is Unsafe and Immoral, and waving my hand and saying “make it so” in an authoritative yet subtly hip manner. With the new folks joining the H3 development team, we should make progress quite a bit quicker now.

5 thoughts on “Radio silence: ended

  1. Pingback: Next Generation Sequencing » Bjarne Knudsen visits Sean Eddy to collaborate on H3

  2. glocal won’t come back anytime soon, alas. It’ll take something on the scale of a small research project to figure out E-value statistics for glocal in H3, and we won’t have time for any new research projects while H3 is stabilizing. It’s annoying me too – yesterday I was fighting the new Pfam 24 servers to visualize a zinc finger domain I knew was present, but the lack of glocal for the short domain was making it fade just under the Pfam GA threshold. It’s high priority but we’re really fighting tight timelines — H3 must be production-stable, now now now, because Pfam just rolled out the first Pfam release based on H3.

    Like

  3. Pingback: Mailund on the Internet » Blog Archive » Last week (and a bit) in the blogs

  4. Pingback: Next Generation Sequencing » High-performance computing technologies for genomics

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s