Recent work: sindresorhus/file-type and hex skills

Yesterday my pull request for sindresorhus/file-type was accepted.  Hoorah! ?

Screen Shot 2015-11-08 at 5.34.24 PM

I’ve decided to spend more time working with OSS developers for fun and profit. Sindre Sorhus is hugely popular as a developer, in large part for the the exceedingly numerous, high-quality packages he’s submitted to and on Github.  All in all, he’s a stand-up guy who deserves the success and recognition for all the hard work he’s put in for the JS community.

Lo-and-behold, a few days ago the call for contributors was made!


I landed on this issue made for expanding sindresorhus/file-type with more compressed file types.  file-type is a Node.js module made for apps needing to identify files by their actual binary signature, similar to threatstack/libmagic for the C language.  It has  a straightforward and concise structure consisting of a series of if statements and a byte-by-byte examination of the current file stream, typically only a few bytes. Here’s an example of how it checks a file if it’s a GIF:

if (buf[0] === 0x47 && buf[1] === 0x49 && buf[2] === 0x46) {
    return {
        ext: 'gif',
        mime: 'image/gif'

Digging in…

I’ve messed with hex dumps and debugging binary files before, and this simply involved cross-referencing the first few bytes of a file with already-known signatures.

The first  task was to get the lists of file types needing signatures and grabbing the relevant data from sources around the net, mostly from the File Signatures Table, compiled by Gary Kessler.

I created a quick spreadsheet to keep things in order, too.
Screen Shot 2015-11-08 at 5.32.45 PM

To help out on quick inspections, I created tiny utility to help at first, calling it fsig


xxd $1 | head

And for inspecting temporary directories of common file types, I used this bash one-liner:

clear; for n in *; do echo $n ; xxd $n | head ; done


For extracting signatures cleanly, I ran into HexFiend for OSX.  It’s highly recommended since it displays where the cursor is by byte and allows for clean copy-paste.
Screen Shot 2015-11-08 at 4.10.52 PM

Making it work

A couple of problems I ran into was the issue of files with similar signatures.  For example, deb files are nearly identical to files made with the compress utility.  That means that care was needed when ordering the logic.

ordering logic
Care was needed in ordering the logic for comparing file signatures

To make converting the signature hex into code simple and quick, I used Vim’s handy macro recording mode.

I love Vim macros. Check out how to use them with Vim 101: A Gentle Introduction to Macros.

All set

Check outsindresorhus/file-type, and for that matter, maybe you can help with OSS by working on open issues from sindresorhus’ other projects ?

Let me know what you think on twitter  @lintuxvi!

Hackathon: NBCU2015 and Telemira

For the 2015 NBCU hackathon in Orlando, I entered, along with a friend, into the Telemundo category.  We whipped up the design and the code in a night of a rough prototype app that outlined a novel way to generated viewer interaction during broadcasts.  Enter TeleMira:

TeleMira app
Introducing: TeleMira!

Check out the code on github!

Built with Framework 7 for the easy iOS look-and-feel, it outlined a future where viewers can watch broadcasts, then during commercial breaks and credits receive time-sensitive B-reels, bloopers, and hidden content.  Clips viewed during this time could be saved and traded with friends, since only a limited amount of clips could be viewed during the clip’s countdown period.

How about some gifs to outline what’s going on?

1.  Viewers receive clips during commercial breaks or credits
2. You can watch clips by clicking on them
3. Clips you’ve watched are added to your archive of watched videos
4.  Be careful though, these time-sensitive videos will disappear if they’re not watched in time!
5.  If you miss a clip, you can check to see if your friends has it

If you haven’t yet, try out TeleMira, and tell me what you think on twitter @lintuxvi.  Also, check out the code on github!

Recent work: BDD with AngularJS and Protractor

Recently, I spent a few days getting to know the testing ecosystem around AngularJS, specifically related to Unit Testing and End-to-End testing.  Jasmine (unit-testing) and Protractor (end-to-end) are the two prescribed frameworks for those.

To get a grasp of using these tools, I built a mock application: Mailist.  Mailist’s story is that it helps send many emails easier, using a template and a CSV with recipient data.

With that in mind, I looked around for some resources online as to how BDD changes the development process.  The first article to peak my interest was Functional testing with cucumber and protractor.  Being that BDD differs from TDD in that a feature/behavior is being tested, rather than the functionality of a module/unit, bridging the conceptual gap was a little shaky.  In the middle of the Functional testing…, this graphic came up that helped to elucidate the process:

Figure 1: the BDD cycle

Take a look at the graphic: you can see that unit-testing still exists within the context of the BDD, but now it’s involved within the development of the system at large.  Red-Green-Refactor is still there, but within the greater BDD loop of focusing on the scenario.

Testing: BDD  ?  Scenarios

Whats in a Story is a great article that focuses on the scenario approach to writing features in BDD.  Boiled down, when we write out what the features will be:

1. The scenario title should say what’s different

You should be able to line up the scenarios side by side, and describe how they differ using only the title.

2. The scenario should be described in terms of Givens, Events and Outcomes

…adopt this vocabulary of “given/when/then”…

To build out the scenarios in Mailist, these rules were used.  Here’s an example scenario:

Scenario: A user loads a list and generates a queue
Given there are no emails
 And there is no template
 And there is no cached data
When the user loads a csv of data
Then the queue list should be populated
 And there should be relevant field headers

Figure 2: example scenario

Note: this closely resembles the Gherkin syntax, which I’d like to use with the Cucumber BDD framework in the future, however I’m keeping this exploration simple.

Testing: Protractor  ?  End-to-End

Looking at the chart in Figure 1, since the features we build involve multiple parts of the system, testing at the integration level makes sense.  The Angular team has done a great job providing a tool just for that: Protractor.  It’s compatible with the Jasmine syntax, and uses a convenient element selection syntax for performing actions on target elements.  For example:

var input = element(by.css('input'));

Check out the docs for more info.

Testing: Jasmine  ? Unit-Testing

Here we unit test with the popular Jasmine test framework.  For each of the scenarios, now a series of internal modules become clear.  This was the aha moment.  For each scenario, the internal requirements become clear.  And just like the chart in Figure 1, I’m back to the Red-Green-Refactor cycle, defining and testing module behavior in the typical TDD fashion.

The one thing that sticks out here with Angular is testing directives.  There are a couple of methods I’ve seen online recommended: using E2E for testing directives, or using the internal angular directive compilation.  Both are straightforward, however I was having trouble automating the file input through the directive compilation method because of browser security policies.  I’d rather not have to script launching browsers with special access policy flags, so I left those tests to Protractor.


So, in short, BDD with Angular is straightforward using Protractor and Jasmine for E2E and unit-testing.  By sticking to the chart in Figure 1, building out scenarios, and sticking with givens/events/outcomes to outline our scenarios, we have a clear idea of how our app evolves.


Jump over to Github and browse the code for Mailist.  Then take a look at these links for the tools mentioned:

For my favorite link on Angular testing, check out Year of Moo’s Full-Spectrum Testing with AngularJS and Karma.  It’s centered around Mocha, Chai, and Karma as a testing framework, but the concepts for testing each part of an Angular app is broken down here.

Let me know what you think on Twitter @lintuxvi

Baking partnerships

I‘ve been wanting to write something for myself for a while now, and being a tech blog, I should inaugurate this one with an instance of writing blessed by tech canon: fail often and fail quick.  Figuratively, you can see it like playing a poker hand or snipping a wire defusing a bomb.  As far as consequences go, they can range from emotionally draining to traumatizing (in a sort of gradient between n…n+1).  But if we’re torturing analogies here, let’s talk about baking and partnerships.

I’m mostly a coder, who dabbles in entrepreneurship, but since my passion lies in personal projects and friendships, I’ve run into a few situations where they come together.  Some of these Frankensteins have turned out well, and others not so much, but the biggest factor in determining that success has been in the partnership built.

Picking a partner: values and experience aren’t all sugar and spice

Picking a partner is a kind of baking.  You get the ingredients together, you lay them out in front of you, then crack, crack, crack, mix, mix, mix, whisk them all in a bowl and stick it in an oven.  Your project, the business, is like that oven.

Cranking the oven up reveals the wonderful possibilities, and without it your cake is just a pan of half-rendered goop, in a state of ready decomposition.  By taking the elements and putting them to the test, all the constituent elements behave together as well as the mixed ingredients were picked and placed.


Except, sometimes, we as bakers are blind.  We as bakers mislabel our ingredients, or ignore the labels altogether.  What was salt is actually sugar and what is sugar is baking soda, and what is baking soda is Ajax scrubbing powder with bleach.  Yech, and call your local poison-control center immediately.

People are the same.  You and I, we hold a few flavors for public consumption.  How we behave, the attitudes we project, our go-to methods for conflict resolution and problem-solving: we keep these as a sort of feature-set called upon as long as there’s clean, un-pressured energy to utilize before our fight-or-flight centers kick in.

Take multiple people, add a potentially big payout, and now we’ve got some heat.  You’re going to run into situations when what you thought was a Beef-Wellington ends up being just a starter-log wrapped in yolk-glazed paper-mache.  Volatile, flammable.

Some have ingredients rated for high temperatures, while others go up in flames pretty quickly.

So, big note-to-self: put the time in beforehand.

  1. ‘Date’ all potential partners, and be aware of the typical warning signs (ex. mismatched values, gut feelings, personal history)
  2. Make sure the vision is there.  No vision is like a train off the rails: crazy.
  3. Make sure there’s an end-game. Get confident about it, learn the possibilities and contingencies.
  4. Have an understanding of how each partner resolves uncertainty and conflict.  One person making decisions out of fear in a partnership is a symptom that real communication has broken down and planning is out the window.  That’s a recipe for burnout and disaster.
  5. Make sure negotiating financials isn’t akin to pulling teeth.  Someone who continually tries to negotiate you at every step is a sign that you’re not going to be working together long.

If you’re going to be spending irretrievable time in your life with others, with the intentional goal of creating something to benefit everyone involved, make sure you all balance each other’s strengths and weaknesses in a lucid and explicit way, even at the worst of times.  And that’s not easy to achieve.  But putting in the legwork beforehand will be worth it in the end.