Recent work: sindresorhus/file-type and hex skills

Yesterday my pull request for sindresorhus/file-type was accepted.  Hoorah! ?

Screen Shot 2015-11-08 at 5.34.24 PM

I’ve decided to spend more time working with OSS developers for fun and profit. Sindre Sorhus is hugely popular as a developer, in large part for the the exceedingly numerous, high-quality packages he’s submitted to NPM.org and on Github.  All in all, he’s a stand-up guy who deserves the success and recognition for all the hard work he’s put in for the JS community.

Lo-and-behold, a few days ago the call for contributors was made!

 

I landed on this issue made for expanding sindresorhus/file-type with more compressed file types.  file-type is a Node.js module made for apps needing to identify files by their actual binary signature, similar to threatstack/libmagic for the C language.  It has  a straightforward and concise structure consisting of a series of if statements and a byte-by-byte examination of the current file stream, typically only a few bytes. Here’s an example of how it checks a file if it’s a GIF:

if (buf[0] === 0x47 && buf[1] === 0x49 && buf[2] === 0x46) {
    return {
        ext: 'gif',
        mime: 'image/gif'
    };
}

Digging in…

I’ve messed with hex dumps and debugging binary files before, and this simply involved cross-referencing the first few bytes of a file with already-known signatures.

The first  task was to get the lists of file types needing signatures and grabbing the relevant data from sources around the net, mostly from the File Signatures Table, compiled by Gary Kessler.

I created a quick spreadsheet to keep things in order, too.
Screen Shot 2015-11-08 at 5.32.45 PM

To help out on quick inspections, I created tiny utility to help at first, calling it fsig

#!/bin/bash

clear
xxd $1 | head

And for inspecting temporary directories of common file types, I used this bash one-liner:

clear; for n in *; do echo $n ; xxd $n | head ; done

 

For extracting signatures cleanly, I ran into HexFiend for OSX.  It’s highly recommended since it displays where the cursor is by byte and allows for clean copy-paste.
Screen Shot 2015-11-08 at 4.10.52 PM

Making it work

A couple of problems I ran into was the issue of files with similar signatures.  For example, deb files are nearly identical to files made with the compress utility.  That means that care was needed when ordering the logic.

ordering logic
Care was needed in ordering the logic for comparing file signatures

To make converting the signature hex into code simple and quick, I used Vim’s handy macro recording mode.
macro

I love Vim macros. Check out how to use them with Vim 101: A Gentle Introduction to Macros.

All set

Check outsindresorhus/file-type, and for that matter, maybe you can help with OSS by working on open issues from sindresorhus’ other projects ?

Let me know what you think on twitter  @lintuxvi!

Hackathon: NBCU2015 and Telemira

For the 2015 NBCU hackathon in Orlando, I entered, along with a friend, into the Telemundo category.  We whipped up the design and the code in a night of a rough prototype app that outlined a novel way to generated viewer interaction during broadcasts.  Enter TeleMira:

TeleMira app
Introducing: TeleMira!

Check out the code on github!
 

Built with Framework 7 for the easy iOS look-and-feel, it outlined a future where viewers can watch broadcasts, then during commercial breaks and credits receive time-sensitive B-reels, bloopers, and hidden content.  Clips viewed during this time could be saved and traded with friends, since only a limited amount of clips could be viewed during the clip’s countdown period.

How about some gifs to outline what’s going on?

1.  Viewers receive clips during commercial breaks or credits
growl
2. You can watch clips by clicking on them
click
3. Clips you’ve watched are added to your archive of watched videos
archivo
4.  Be careful though, these time-sensitive videos will disappear if they’re not watched in time!
close
5.  If you miss a clip, you can check to see if your friends has it
amigovideos

If you haven’t yet, try out TeleMira, and tell me what you think on twitter @lintuxvi.  Also, check out the code on github!

Recent work: BDD with AngularJS and Protractor

Recently, I spent a few days getting to know the testing ecosystem around AngularJS, specifically related to Unit Testing and End-to-End testing.  Jasmine (unit-testing) and Protractor (end-to-end) are the two prescribed frameworks for those.

To get a grasp of using these tools, I built a mock application: Mailist.  Mailist’s story is that it helps send many emails easier, using a template and a CSV with recipient data.

With that in mind, I looked around for some resources online as to how BDD changes the development process.  The first article to peak my interest was Functional testing with cucumber and protractor.  Being that BDD differs from TDD in that a feature/behavior is being tested, rather than the functionality of a module/unit, bridging the conceptual gap was a little shaky.  In the middle of the Functional testing…, this graphic came up that helped to elucidate the process:

outside-in-development
Figure 1: the BDD cycle

Take a look at the graphic: you can see that unit-testing still exists within the context of the BDD, but now it’s involved within the development of the system at large.  Red-Green-Refactor is still there, but within the greater BDD loop of focusing on the scenario.

Testing: BDD  ?  Scenarios

Whats in a Story is a great article that focuses on the scenario approach to writing features in BDD.  Boiled down, when we write out what the features will be:

1. The scenario title should say what’s different

You should be able to line up the scenarios side by side, and describe how they differ using only the title.

2. The scenario should be described in terms of Givens, Events and Outcomes

…adopt this vocabulary of “given/when/then”…

To build out the scenarios in Mailist, these rules were used.  Here’s an example scenario:

Scenario: A user loads a list and generates a queue
Given there are no emails
 And there is no template
 And there is no cached data
When the user loads a csv of data
Then the queue list should be populated
 And there should be relevant field headers

Figure 2: example scenario

Note: this closely resembles the Gherkin syntax, which I’d like to use with the Cucumber BDD framework in the future, however I’m keeping this exploration simple.

Testing: Protractor  ?  End-to-End

Looking at the chart in Figure 1, since the features we build involve multiple parts of the system, testing at the integration level makes sense.  The Angular team has done a great job providing a tool just for that: Protractor.  It’s compatible with the Jasmine syntax, and uses a convenient element selection syntax for performing actions on target elements.  For example:

var input = element(by.css('input'));
input.sendKeys('123');
expect(input.getAttribute('value')).toBe('Foo123');

Check out the docs for more info.

Testing: Jasmine  ? Unit-Testing

Here we unit test with the popular Jasmine test framework.  For each of the scenarios, now a series of internal modules become clear.  This was the aha moment.  For each scenario, the internal requirements become clear.  And just like the chart in Figure 1, I’m back to the Red-Green-Refactor cycle, defining and testing module behavior in the typical TDD fashion.

The one thing that sticks out here with Angular is testing directives.  There are a couple of methods I’ve seen online recommended: using E2E for testing directives, or using the internal angular directive compilation.  Both are straightforward, however I was having trouble automating the file input through the directive compilation method because of browser security policies.  I’d rather not have to script launching browsers with special access policy flags, so I left those tests to Protractor.

Conclusion

So, in short, BDD with Angular is straightforward using Protractor and Jasmine for E2E and unit-testing.  By sticking to the chart in Figure 1, building out scenarios, and sticking with givens/events/outcomes to outline our scenarios, we have a clear idea of how our app evolves.

Links

Jump over to Github and browse the code for Mailist.  Then take a look at these links for the tools mentioned:

For my favorite link on Angular testing, check out Year of Moo’s Full-Spectrum Testing with AngularJS and Karma.  It’s centered around Mocha, Chai, and Karma as a testing framework, but the concepts for testing each part of an Angular app is broken down here.

Let me know what you think on Twitter @lintuxvi