Parsing multiple files - async style

One project I worked on recently involved reading in multiple Metrics logs for the Illumina iScan.  Basically, a file represented an entire scan session, where a scan consisted of a bunch of sections.  These sections included a bunch of metrics that needed to be run against a pass/fail algorithm.

Therefore, each section is graded with a pass /fail.  All of this information is organized into a hierarchical structure of many beadChips with many Sections with many section values.

Using the whole async approach, I am able to load and process approx 54 files over smbfs in seconds and then run queries against it using a web front end.

Here is the gist for populating the available metric file array.




Once we have the list of metrics files, we create beadChip objects:

for each metrics file
    new beadchip(file);

The object automatically calls its init script, which uses async to build the object's information from the log file.




Finally, here is the forEach async function used in building the beadChip: