Category Archives: Biomedical Engineering

Biomedical Engineering

Five years in the lab: looking back, then forward

About this time five years ago, I was a nervous junior undergraduate studying Biomedical Engineering at Tulane University. I had just been accepted as an undergraduate member of Dr. Natalia Trayanova’s computational cardiac electrophysiology lab. The goal at that time was to complete a research project for my undergraduate thesis.


So very many things have happened since then. Here are the highlights:

  • 2002: Started learning the ropes of the lab
  • 2003: Continued to familiarize myself with the computers and code in use in the lab. The most powerful machine in our possession was an SGI with 8 processors (the Origin 300 listed here). There was almost always a wait to use those processors. Spent my summer vacation working in the lab. This was the first time I was paid to to research. Some time during this year (I think) I created the lab wiki using MoinMoin. By this time I was administering the lab computers and was sick of answering the same questions over and over. In desperation I created a wiki and started putting answers on it, referring people to the wiki when I was asked a question. The wiki is now (as of November 2007) huge, and contains basically all of the documentation of everything used in the lab, as well as gigabytes upon gigabytes of attached models, data, and images.
  • 2004: Graduated from Tulane with my Bachelor of Science in Engineering (BSE) degree. Joined the lab as a graduate student. Sometime in 2004 (I think), Tulane acquired a Linux Networx cluster, and we owned 20 nodes in that cluster.
  • 2005: Shortly after returning from my trip to Niger, Katrina struck New Orleans. The lab was scattered. Few people in the lab had access to their data. A few lab members actually snuck past armed guards to get our file servers and some workstations from our lab at Tulane. We took up residence in St.Louis, MO for two and a half months, aided by our colleagues in the labs of Drs. Yoram Rudy and Igor Efimov at Washington University. By the end of the year, we had returned to a slowly-recovering New Orleans.
  • 2006: Dr. Trayanova accepted a position as a professor at Johns Hopkins University. Almost the entire lab transfered to JHU and moved to Baltimore, MD.
  • 2007: In April, I began discussing a cluster purchase with High Performance Computing (HPC) companies. Around that time, the weather warmed up, the server room could no longer be adequately cooled, and we started limping by on 4 compute nodes. By the end of July, we had placed an order for a new cluster. We moved from Clark Hall into the newly-completed though poorly-named Computational Science and Engineering Building. In mid-November, most of our new cluster arrived, though FedEx dropped and destroyed one rack, and the cluster was not completely set up.

That brings us to the present day. Now, looking forward a little:

In the next two weeks, the cluster set-up will be completed. We will have free rein on 140 compute nodes (20 old, 120 new), all managed from one head node. The new nodes will be connected by the fastest Infiniband interconnects available on the market, and each node will have 8 GB of RAM available, with the potential to hold 64 GB each. There are four 3.0 GHz Opteron cores per node, yielding a total of 480 processors and 960 GB of RAM on the new nodes alone.

To give you some perspective on what that means, let me give you some details about the kinds of models we run. When I joined the lab, our two largest models consisted of a 4mm thick slice of the canine heart, and a very smooth, idealized model of the rabbit heart. These models are composed of 1.6 million and 0.82 million tetrahedral elements, respectively. It took something like an hour of wall clock time per millisecond of simulation time to run these models. (In other words, to get one millisecond worth of simulation data it was necessary to wait about an hour.) We could run one or two simulations at a time, at that speed.

My newest model, and currently the largest model in use in the lab, is composed of 28 million tetrahedral elements. On a cluster similar to our new one (Lonestar on TeraGrid), using 32 processors, it takes about 22 minutes of wall-clock time to simulate one millisecond in the model. Using a crude estimated unit of speed of (minutes real time / millisecond simulation time / tetrahedral element), and focusing only on the number of simulations we can run at once, not the number of CPUs required:

  • Old way: 60 minutes / 1 ms / 0.82 million tets = 73 minutes / ms sim time / million tets
  • New way: 22 minutes / 1ms / 28 million tets = .78 minutes / ms sim time / million tets

We have increased our simulation speed by almost 100 fold. We can run two to four simulations of that size at a time, vs one or two the old way. But that’s not all. We can now run bigger models. Much bigger models. We are now capable of running something the size of a dog heart (we have verified this). More importantly, we now have the technical capacity to run a model the size of the human heart, with a resolution near that of the size of a cardiac cell, and to model contraction in addition to electrical activity. It remains only to develop such models. We are prepared to store the results: the new cluster has a storage capacity of 28 TB online, with the ability to add something like 40 or 50 TB more simply by expanding the existing storage device.

In my time in the lab, I have watched our abilities expand from serial jobs with relatively small models to massively parallel jobs with the capacity to model electrical and mechanical activity in the human heart. We are just beginning a very exciting time in the lab and in the field, and what’s really killing me is that fact that there’s so much more to tell you.

But I can’t just yet.

(This post was partly inspired by a conversation with Maria and Amanda)

Cluster Update

So, the cluster is here.

The good news: I think we’ll be able to use it (albeit without Infiniband for now) by the end of the day.

The bad news: FedEx clearly dropped one entire rack. Yes. Unbelievable. About $110,000 worth of equipment. When it got here, there were plastic parts coming out from under the door of the crate. From the looks of it, it faceplanted off of a loading dock or a forklift. The tilt sensor was definitely tripped. Some of it may still be good, but it all has to be thoroughly inspected. Penguin is just building us a whole new rack. We lucked out in a way — it was the only rack with no ‘special’ components. All of the other racks have something special. One has the head node (not a big deal). One has the Infiniband switch (very expensive with long lead time to replace), and one has a fileserver with a Xyratex storage unit (also very expensive with long lead time). This rack just had compute nodes and a regular gigabit switch.

I’ll have more news and pictures once we get further along.

Finding related articles graphically

When doing a literature search, it’s a good idea to start from a few articles and then (if they are along the lines of what you are looking for) use their references and articles that reference them to expand the search.

One handy way of doing that is with the HubMed Graph Browser. You get to it by finding an article (like mine here) and then selecting the “Graph” link next to “Related” in the line of options at the bottom.

Once you load the TouchGraph, you can see the related articles, change the depth of relationships graphed, zoom in and out, and so on. It can be a nice alternative to the normal related articles list, graphically showing distance and relation.

Device handles chest compression part of CPR

Thanks to my grandma Tice for sending me this article:

The Lucas system runs on high-pressure air from either a compressed air tank or an air wall outlet in a hospital. The device is indicated for treating adults who have acute circulatory arrest – meaning they lack spontaneous breathing and pulse – as well as loss of consciousness.

Mechanical compression allows medical personnel to provide other therapies, the company said. The machine also should provide quality chest compressions for a longer period of time than a human can.

A 1995 study found that fatigue makes it difficult for even well-trained medical personnel to provide more than one minute’s worth of effective chest compressions, said Anne Devine, a Medtronic spokeswoman.

“Clearly these devices do much better compressions than humans do,” said Dr. Charles Lick, medical director of Allina Medical Transportation in Minneapolis.

Sounds kind of difficult to use, but apparently the EMTs like them enough that they’re putting them in ambulances.

Solvers for the Bidomain Equations

The thing that makes our research difficult, the main reason we need big, powerful computers, is a system of a equations known as the “Cardiac Bidomain Equations”. They are a way of representing both the insides and outsides of cells, as well as the membrane in-between, throughout a piece of tissue. A former member of our lab as well as a current one and their colleagues have just come out with a paper entitled, “Solvers for the cardiac bidomain equations”. It discusses why the system is computationally expensive, and tricks for solving it more efficiently, including multigrid methods.