I’ve been on a machine learning kick lately. Given a large enough dataset to train with, it’s really interesting to see what a neural network can come up with.

This week, it’s names for craft beer.

If you’re a fan of IPA beer, you’ve got names like Dang River, Yamquak, Yall in Wool, Wicked Geee, Yampy, and Oarahe Momnila Day Revenge Bass Cornationn Yerve Of Aterid Ale. Like strong pale ales? Trippel Lock, Third Maus, Third Danger, Spore of Gold and Drammnt. Stouts more your thing? Look for Sir Coffee, Shock Slate, Take Bean, Black Sink Stout, Shrump, Avidberry, or Cherry Trout Stout.

Naturally, I tried to create my own model using a Python library called Keras and a dataset of 7,500 craft beer names.

…I should leave this stuff to the professionals.

Update: Kaggle has a new tutorial teaching you how to do this exact same thing. Neat!

Test driven development has become an important process in the software engineering world. It allows coders to develop functions by first creating a series of tests that the new function must solve. The benefit of this is that once your app grows more complex and you add new functionality, you can see if any existing tests have failed, meaning that something broke (and now you know where to find it). Look no further than any popular project on Github and you’ll often see a “tests” folder.

Today, we’re going to talk about setting up tests using Tape.

Tape is an alternative to popular testing suites such as Jasmine and Mocha. Like any tool related to software engineering, there are some developers that strongly prefer Tape over other options. It’s fairly easy to setup and can easily be run in automated task runner tools such as Grunt and Gulp.

To use it as part of your project, you can install it through npm:

  npm install tape --save-dev

Once it’s been added as part of your project, you can create a new tests.js file and require the module.

For our demonstration, we’re going to write a simple test that checks if my name is Dave, plus a few additional parameters.

Start off by setting up your test.js file like so (you can name it whatever you prefer). I’ve commented the code for some additional clarity on what’s happening here.

// Require the Tape module imported from npm
var test = require('tape');

// Write your tests in the code block
test('All about Dave', function (t) {
    // The number of tests that you plan to run.
    // NOTE: If this number doesn't match up with the number
    // of tests that are run, your test will fail.
    t.plan(2);
    
    // Let's setup some variables to test
    var name = "Seymore";
    var city = "Oakland";
    var favBaseballTeam = "Athletics";

    // This test will check for my favorite baseball team.
    // The first parameter is the result, the second is
    // the value you're expecting, and the third is the message
    t.equal(favBaseballTeam, "Athletics", "Favorite baseball team should be Athletics");

    // This test will check for my name.
    // As you can probably assume, it will fail.
    t.equal(name, "Dave", "Name should be Dave");

    // This test will check if city has been set:
    if (city) {
      t.pass("City set");
    } else {
      t.fail("City not set");
    }
});

That’s it! You can run Tape from your terminal and point it to your newly created test.js file in order to run it.

We’ve been working on many projects lately that have utilized MongoDB as the primary means of database storage. I have previous experience building and using MySQL databases, so the idea of these NoSQL databases is a new concept for me.

I’m not one to shy away from new technologies, so I’ve been trying to embrace MongoDB and learn how to use it.

One of the most important things I’ve been learning is how to view the databases, collections, and records that I’ve saved in my various applications through the MongoDB command line interface.

Let’s do a quick walk through and pretend I have a database dedicated to baseball.

Once you have Mongo installed on your machine, you run the interface by typing mongo in your terminal. Now, you can bring up a list of databases by typing show databases.

How do we use a particular database? Easy! Just type use [database_name]

use baseball

Awesome! Of course, you’ll want to do more than just “use” the database. We want to see what’s inside it. This is accomplished by telling mongo to show us all collections (e.g., think of these as “tables” in a traditional SQL database).

show collections

Awesome! Now we have a collection of teams and collection of players. Well, let’s display everything within a particular collection. In this case, let’s print out all teams that we have stored in our database.

db.teams.find()

Great!

Now, let’s say you’re looking for a particular record. How do you limit your search to just one thing? Like this:

db.teams.find({team: “dodgers”})

Now, you can imagine that if we had more data, there are a lot more things that we could search for and find. It’s pretty powerful!

Anyway, this was a quick tutorial on how to use the Mongo DB CLI. I hope you found it helpful!

I recently utilized Firebase to power the database and back end of an AngularJS web application I’ve been working on.

Firebase was proved to be amazingly easy to setup. Their “5 minute quick start” sold me on the utility of this service.

How easy is it? Let’s go ahead and create ourselves a persistent chat client in about 5 minutes. I’ll assume you’ve had some previous experience with Angular, but it’s not required.

In an empty HTML page, let’s add the Firebase Javascript library in the head section of our page.

<script src='https://cdn.firebase.com/js/client/2.2.1/firebase.js'></script>

Alright, easy enough, right?

Now, you need to create a reference to your Firebase database inside your script tag. When you sign up (for a free, even!), Firebase will give you a random URL that links to your new database.

So, a reference in your script tag, like so:

  <script>
    var myData = new Firebase('https://d______.firebaseio.com/');
  </script>

Okay, okay. Not too bad.

How do you start sending data to your new Firebase DB? You just apply some Firebase methods to the reference we created about. In this case, we’ll use the set() method.

  // Store values from our input boxes here.
  var name, text; 

  // jQuery or other magic here to set input boxes
  // to the values above.

  // Save to our Firebase DB using the .set() method.
  myDataRef.set('User ' + name + ' says ' + text);

You’d likely use jQuery to set the values of a particular input box to the name and text. You can also pass in objects to the set() method like so:

  myDataRef.set({name: name, text: text});

Another option that you can utilize to store data inside Firebase is the push() method. You can utilize this method to easily create lists of objects.

  myDataRef.push({name: name, text: text});

Okay, we talked a lot about how to send messages to our database. But how do we retrieve them? We can create an event listener using the on() method, like so:

  myDataRef.on('child_added', function(snapshot) {
    var message = snapshot.val();
    displayChatMessage(message.name, message.text);
  });

And just like that, you now have a working chat client supporting many users and can share data to all clients using it. Amazing!

Sublime Text is an indispensable tool to have in your arsenal of web development goodies. There’s a nearly infinite amount of shortcuts and tricks one can use to improve their workflow.

One of my favorite shortcuts is moving either lines (or entire blocks) of code up or down a page without cutting and pasting all over the place.

Simply select the line (or multiple lines of code) that you want to move, then simply hit

CONTROL + CMD + (Up or Down arrows) on OS X
CONTROL + SHIFT + (Up or Down arrows) on Windows

I guarantee if you do this in front of your friends or family, you will look like a wizard.

Current Version: 0.1.4

Say hello to NodeEBot (pronounced as “nodey bot” or even “naughty bot”, if you prefer). It stands for Node E-books Bot.

It’s a Nodejs package for creating Twitter bots which can write their own tweets and interact with other users (by favoriting, replying, and following). This project draws heavy inspiration from the twitter_ebooks gem for Ruby.

You can see two examples of this bot in action at @daveleeeeee and @roboderp.

Installation and Usage

This project requires Nodejs v0.10.35+. If you’re looking for a place to host Nodejs projects, I’ve had success setting up a free Ubuntu virtual server through Amazon’s Web Services dashboard and installing nodejs on it.

To run, copy the project into your preferred directory and then install the required dependencies using:

npm install

You can edit various configuration settings in the bot.js file. Before you can begin you’ll need to have Twitter API credentials which can be setup right here. Once you have your consumer API key and secret as well as your access token and secret, add them to the top of the bot.js file:

// Twitter API configuration
var client = new Twitter({
  consumer_key: ‘xxxx’,
  consumer_secret: ‘xxxx’,
  access_token_key: ‘xxxx’,
  access_token_secret: ‘xxxx’
});

You’ll also need to add the Twitter username of your bot (without the @ symbol) to the config file. (This is for tracking mentions as well as making sure the bot ignores actions from itself so it doesn’t get caught in a loop).

// Your robot’s Twitter username (without the @ symbol)
// We use this to search for mentions of the robot and to prevent it from replying to itself
robotName = “xxxx”;

Once that’s done, the bot is almost ready to go. You can modify a few other settings that influence how chatty the bot is, how often it will interact with other users or use random hashtags and emojis.

In order to run the bot, I use the forever npm package. This allows us to automatically restart the server in case of a crash, as well as force restart the server in order to reload the Twitter stream (added in v 0.1.2).

Source material

The one last thing that you’ll need to do is give it some source material to generate text from. I use source material my own Twitter archive.

Right now, I haven’t implemented a way to parse the Twitter’s csv data that’s generated when you request your history. In the meantime, I’ve simply opened up the tweets.csv in a spreadsheet app, copied the contents of the ‘text’ column into a new file and used that as the source material. This script will treat each line as a separate and unique sentence.

I’ve added some basic ability to strip our Twitter usernames and URLs from the archive. That means it will treat something like:

@davely That’s great. I’ve seen something like that before. 
http://flickr.com/…

as

That’s great. I’ve seen something like that before.

Running multiple bots

If you want to run multiple bots for different Twitter accounts, copy this project into separate folders (e.g., ~/MyBot1, ~/MyBot2, ~/MyBot3, etc) and make sure you input the proper Twitter API credentials at the top of each bot.js file. Then spool up separate node instances and load up the relevant bot files.

Future things to do.

• Better modularization of our script. Right now it’s in one ginormous .js file.
• Turn it into a proper npm module.
• Better regex handling to clean up source material (e.g., links, usernames, etc
• Send direct messages back to users who DM our robot.
• Keyword ranking of our source material. (Sort of implemented but disabled right now since performance is SLOW.)
• Allow robot to reply with some content (e.g., if someone asks what it thinks about ‘baseball,’ it tries to compose a reply that mentions ‘baseball.’
• Retweet various tweets that it finds interesting based on keywords and interests.
• Let it potentially upload images or GIFs.

Changelog

v 0.1.4 (2015/05/07)

• Simple change to load and require underscore. This is going to help simplify some of my functions in future development.

v 0.1.3 (2015/04/28)

• Fixed bug that would cause bot to think that all users replying to it were found in our otherBots array and kept applying a temporary time out on replies, even if not needed.

v 0.1.2 (2015/04/27)

• Implemented a hacky fix for an issue I’m having with the Twitter Streaming API randomly dying without an error. If we’re running this with the npm package forever, let’s kill the server and restart if ever few hours.

v 0.1.1 (2015/04/19)

• Initial public release!

Other stuff

If you end up using this script in your own Twitter bots, let me know! I’d love to know how it works out for you and please let me know about any improvements or suggestions you might have.

Thanks for checking it out!

You can download the source code for nodeEbot on Github.

I’ve been playing around with Node.js recently and taking some of the neat online lessons available through Nodeschool. One of the lessons involved streaming realtime data between the client and server using Socket.io.

This sounded insanely powerful and fun, so I took a break from the rest of the lessons and started playing around with Socket.io. They have a tutorial that explains how to build a simple, realtime chat application using its library in about 20 minutes.

I’ve uploaded the results of this little project to Github and you can view it right here. This little web app is more or less a result of tutorial and I’ve added some additional stuff as I played around with it:

• Usernames!
• Number of active users!
• Display a list of all connected users!
• Check for identical usernames and prompt for different name
• Timestamps on messages!
• Sanitizing user input!
• Properly autolink URLs!
• Detect if a link to an image is pasted in and expand!
• Display welcome message on connect

All in all, it’s been a really fun learning process and has given me some ideas on how to use Node.js for future realtime applications.

Note: You can see this in action on Heroku. You’ll probably be the only one only, but hey! https://davechat.herokuapp.com/

This past weekend, I played around with jQuery, JavaScript, and Twitter’s API to create a web app that displays realtime search results for various baseball teams. I figure this is particularly important since pitchers and catchers report in about 10 days! 🙂

Anyway, the goal of this project was to rapidly build out a web app that could search and parse publicly available data (e.g., tweets!) for mentions of particular baseball teams while using various jQuery and JavaScript libraries. I could see this app having potentially wider uses beyond baseball such as returning the latest tweets from an event or hashtag. It also gave me a chance to use various libraries and frameworks as well. I’m pretty happy with how it turned out.

You can see it in action right here and the source code is available on Github.

Earlier this year, I joined a group on Meetup dedicated to creating, analyzing, and talking about digital maps. It’s organized by the fine folks at #Maptime, who have an amazing enthusiasm for all things map related. It sounded like something that’s right up my alley, considering all the time I spend thinking about maps.

Last night was project night. All sorts of folks were there, working on complicated issues such as transit, land use, green space, climate effects. It’s kind of inspiring!

I went to focus on some less lofty goals — I just wanted to get some experience playing with a JavaScript mapping library created by a company called Mapbox. My goal was to get the library up and running and then use it to display some geo-encoded data from a service I happen to use a lot: Foursquare.

It’s no secret that I really like Foursquare — mainly for keeping track of our adventures far and wide. I found a helpful PHP library for converting Foursquare (and other social networking) data into GeoJSON, which is needed by the JavaScript mapping libraries. After a couple of hours of hacking around, I got something up and running!

Pretty neat stuff! I have some ideas on how to improve this and some additional data that I want to show in the future.

In part I, I explained how I’m attempting to write a webapp that can solve simple Sudoku puzzles. Since then, I’ve made some progress. This series will track what I’ve been working on, learning, and any difficulties I’m encountering.

With a goal in mind, it was time to just dive in and go.

Step 1: Oh God, what is this even…

At first, it was kind of overwhelming. Where do I even start? How the heck am I going to even tackle any of this?

After learning the basics of Javascript loops, functions, arrays, and objects, thanks to Codecademy, I was mostly ready to go. What was my real first step? How the heck do I even manipulate data on a webpage, let alone collect and analyze it?

After a few tries, I had a simple project featuring a table (with cells individually labeled) up and running!

After some more tinkering, I was able to get the “Generate Numbers!” button to actually randomly generate the numbers 1 through 9 and fill in the cells without repeating any numbers. Sweet!

Hey, that’s kind of cool. Now we’re getting somewhere! I think I was ready to take it to the next level.

Step 2: Okay… I guess we need to make a fake Sudoku grid.

There’s probably simpler ways to go about doing this, but I ended up spending a lot of time figuring out how to build a Sudoku grid in the first place. Hey, I wanted this project to look somewhat presentable when I showed it off and (hopefully) finished it.

HTML tables were driving me crazy, so I just straight up stole this helpful thing after Googling for an answer. It looks pretty good. Pretty, pretty, pretty good.

“Wow, sir. That looks so good that I’d like to take two!”

Alright, one mission accomplished. Now onto another mission. How do I completely fill out the entire grid? I started with this, because I wanted to understand how to try to isolate certain sections of the Sudoku grid and see if I could create a series of non-duplicated random numbers.

Why did I start with randomly generating numbers? I initially thought I would just try to brute force solve some simple beginner’s level puzzles, but I later realized how ridiculous this would be. There’s a ridiculous number of possible permutations to try.

For now though, I decided to start filling out random numbers by row. This is what my initial code looked like.

[javascript]
function fillTable(row) {
for (var i = 1; i <= maxColumns; i++) {
var selectCell = row + i;
//console.log(selectCell);
document.getElementById(selectCell).innerHTML = fillCell(selectCell);
if (i == maxColumns && row != maxRow) {
i = 0;
row = nextChar(row);
} else if (i == maxColumns && row == maxRow) {
break;
}
}
}
[/javascript]

Did it work? Yes! I ended up with grid of random numbers filled out! Sure, there’s really no rhyme or reason for where numbers ended up, but it certainly looks like a completed Sudoku puzzle, right?

Step 3: Let’s put some real numbers in there

Alright, entering a bunch of random numbers into a grid is pretty fun for all of 5 minutes. (To be honest, it was pretty fun) But to take this to the next level, I really needed to have the beginning of a Sudoku puzzle in there.

A few quick Google searches later, and I found a suitable candidate! One thing I decided early on was to store all this data in an object — mostly so I could easily update it with different puzzles later on. It ended up looking like this.

[javascript]
var allCells = {
a1: "", a2: 1, a3: "", a4: 6, a5: "", a6: 4, a7: 3, a8: "", a9: 7,
b1: 3, b2: 5, b3: 6, b4: "", b5: "", b6: "", b7: "", b8: "", b9: "",
c1: "", c2: "", c3: "", c4: "", c5: 5, c6: 3, c7: 6, c8: 9, c9: "",
d1: "", d2: 8, d3: 3, d4: 2, d5: 6, d6: "", d7: 4, d8: "", d9: 9,
e1: "", e2: "", e3: "", e4: "", e5: "", e6: "", e7: "", e8: "", e9: "",
f1: 4, f2: "", f3: 5, f4: "", f5: 7, f6: 8, f7: 2, f8: 6, f9: "",
g1: "", g2: "", g3: "", g4: "", g5: "", g6: "", g7: "", g8: "", g9: "",
h1: "", h2: "", h3: "", h4: "", h5: "", h6: "", h7: 7, h8: 2, h9: 4,
i1: 7, i2: "", i3: 9, i4: 4, i5: "", i6: 2, i7: "", i8: 8, i9: ""
};
[/javascript]

On top of this, I wanted to easily differentiate the initial numbers that were created on a brand new board (so I could easily see what my script was generating, versus what was already in place on the board). I wrote a function to modify the DOM and change the font weight and background color of the starting cells.

[javascript]
function setupBoard(row) {
var row = row;
var cellValue = 0;
for (i = 0; i < maxColumns; i++) {
//console.log("ROW: " + row + (i+1));
cellValue = allCells[row+(i+1)];
document.getElementById(row + (i+1)).innerHTML = allCells[row+(i+1)];

// Just highlighting what cells we initially started with.
if (Number(cellValue) > 0) {
document.getElementById(row + (i+1)).style.fontWeight = "bold";
document.getElementById(row + (i+1)).style.backgroundColor = "#F2F2F2";
} else {
// Use this to count up total number of empty cells that we need to solve for
// The idea is that we can use this to detect if we’re stuck
emptyCells++;
}
}
}
[/javascript]

You’ll notice I have a lot of console.log() calls commented out. I liberally used these all over the place so I could make sure things were working correctly. Anyway, once all that was said and done, it ended up generating a board that looked like this.

Wow, we’re starting to chug along pretty nicely! Next time, I’ll talk about the million different functions I created in trying to solve this.