nodeEbot: A bot for Twitter that generates tweets from pseudo Markov chains

Current Version: 0.1.4

Say hello to NodeEBot (pronounced as “nodey bot” or even “naughty bot”, if you prefer). It stands for Node E-books Bot.

It’s a Nodejs package for creating Twitter bots which can write their own tweets and interact with other users (by favoriting, replying, and following). This project draws heavy inspiration from the twitter_ebooks gem for Ruby.

You can see two examples of this bot in action at @daveleeeeee and @roboderp.

Installation and Usage

This project requires Nodejs v0.10.35+. If you’re looking for a place to host Nodejs projects, I’ve had success setting up a free Ubuntu virtual server through Amazon’s Web Services dashboard and installing nodejs on it.

To run, copy the project into your preferred directory and then install the required dependencies using:

npm install

You can edit various configuration settings in the bot.js file. Before you can begin you’ll need to have Twitter API credentials which can be setup right here. Once you have your consumer API key and secret as well as your access token and secret, add them to the top of the bot.js file:

// Twitter API configuration
var client = new Twitter({
  consumer_key: ‘xxxx’,
  consumer_secret: ‘xxxx’,
  access_token_key: ‘xxxx’,
  access_token_secret: ‘xxxx’
});

You’ll also need to add the Twitter username of your bot (without the @ symbol) to the config file. (This is for tracking mentions as well as making sure the bot ignores actions from itself so it doesn’t get caught in a loop).

// Your robot’s Twitter username (without the @ symbol)
// We use this to search for mentions of the robot and to prevent it from replying to itself
robotName = “xxxx”;

Once that’s done, the bot is almost ready to go. You can modify a few other settings that influence how chatty the bot is, how often it will interact with other users or use random hashtags and emojis.

In order to run the bot, I use the forever npm package. This allows us to automatically restart the server in case of a crash, as well as force restart the server in order to reload the Twitter stream (added in v 0.1.2).

Source material

The one last thing that you’ll need to do is give it some source material to generate text from. I use source material my own Twitter archive.

Right now, I haven’t implemented a way to parse the Twitter’s csv data that’s generated when you request your history. In the meantime, I’ve simply opened up the tweets.csv in a spreadsheet app, copied the contents of the ‘text’ column into a new file and used that as the source material. This script will treat each line as a separate and unique sentence.

I’ve added some basic ability to strip our Twitter usernames and URLs from the archive. That means it will treat something like:

@davely That’s great. I’ve seen something like that before. 
http://flickr.com/…

as

That’s great. I’ve seen something like that before.

Running multiple bots

If you want to run multiple bots for different Twitter accounts, copy this project into separate folders (e.g., ~/MyBot1, ~/MyBot2, ~/MyBot3, etc) and make sure you input the proper Twitter API credentials at the top of each bot.js file. Then spool up separate node instances and load up the relevant bot files.

Future things to do.

  • Better modularization of our script. Right now it’s in one ginormous .js file.
  • Turn it into a proper npm module.
  • Better regex handling to clean up source material (e.g., links, usernames, etc
  • Send direct messages back to users who DM our robot.
  • Keyword ranking of our source material. (Sort of implemented but disabled right now since performance is SLOW.)
  • Allow robot to reply with some content (e.g., if someone asks what it thinks about ‘baseball,’ it tries to compose a reply that mentions ‘baseball.’
  • Retweet various tweets that it finds interesting based on keywords and interests.
  • Let it potentially upload images or GIFs.

Changelog

v 0.1.4 (2015/05/07)

  • Simple change to load and require underscore. This is going to help simplify some of my functions in future development.

v 0.1.3 (2015/04/28)

  • Fixed bug that would cause bot to think that all users replying to it were found in our otherBots array and kept applying a temporary time out on replies, even if not needed.

v 0.1.2 (2015/04/27)

  • Implemented a hacky fix for an issue I’m having with the Twitter Streaming API randomly dying without an error. If we’re running this with the npm package forever, let’s kill the server and restart if ever few hours.

v 0.1.1 (2015/04/19)

  • Initial public release!

Other stuff

If you end up using this script in your own Twitter bots, let me know! I’d love to know how it works out for you and please let me know about any improvements or suggestions you might have.

Thanks for checking it out!

You can download the source code for nodeEbot on Github.

Armchair managing my favorite baseball team in OOTP 16

One of my favorite computer games, Out of the Park Baseball, released its latest version just in time for the 2015 baseball season (one of the biggest features this year is the official MLB license — that means real team names and logos!). I’ll have to write up a more in-depth review later, but first I wanted to share this screenshot from a recent game.

The situation: I took over management of the Los Angeles Dodgers in 2015 (sorry, Andrew Friedman, I know you just got this job and all, but it’s time for a new sheriff) and I eventually navigated them to the 2017 World Series!

In game 2, Zack Greinke is pitching for the Dodgers, and LA ends up scoring 7 runs in the first inning! Greinke hits TWO homeruns during the game and on top of all that, Texas still came back and nearly won. Crazy!

(Click screenshot for larger view.)

2017_WS_Game2_LA_TEX

 

Anyway, the Dodgers would go on to win the 2017 World Series by sweeping Texas in 4 games. 😉

Weekend project: Building a web chat application with Node.js

screenshot

I’ve been playing around with Node.js recently and taking some of the neat online lessons available through Nodeschool. One of the lessons involved streaming realtime data between the client and server using Socket.io.

This sounded insanely powerful and fun, so I took a break from the rest of the lessons and started playing around with Socket.io. They have a tutorial that explains how to build a simple, realtime chat application using its library in about 20 minutes.

I’ve uploaded the results of this little project to Github and you can view it right here. This little web app is more or less a result of tutorial and I’ve added some additional stuff as I played around with it:

  • Usernames!
  • Number of active users!
  • Display a list of all connected users!
  • Check for identical usernames and prompt for different name
  • Timestamps on messages!
  • Sanitizing user input!
  • Properly autolink URLs!
  • Detect if a link to an image is pasted in and expand!
  • Display welcome message on connect

All in all, it’s been a really fun learning process and has given me some ideas on how to use Node.js for future realtime applications.

Note: You can see this in action on Heroku. You’ll probably be the only one only, but hey! https://davechat.herokuapp.com/

Baseball Twitter: Fun with the jQuery and Twitter’s API

screenshot_redsoxThis past weekend, I played around with jQuery, JavaScript, and Twitter’s API to create a web app that displays realtime search results for various baseball teams. I figure this is particularly important since pitchers and catchers report in about 10 days! 🙂

Anyway, the goal of this project was to rapidly build out a web app that could search and parse publicly available data (e.g., tweets!) for mentions of particular baseball teams while using various jQuery and JavaScript libraries. I could see this app having potentially wider uses beyond baseball such as returning the latest tweets from an event or hashtag. It also gave me a chance to use various libraries and frameworks as well. I’m pretty happy with how it turned out.

You can see it in action right here and the source code is available on Github.

 

Making maps: Using Mapbox to display 5,000 Foursquare check ins

Earlier this year, I joined a group on Meetup dedicated to creating, analyzing, and talking about digital maps. It’s organized by the fine folks at #Maptime, who have an amazing enthusiasm for all things map related. It sounded like something that’s right up my alley, considering all the time I spend thinking about maps.

Last night was project night. All sorts of folks were there, working on complicated issues such as transit, land use, green space, climate effects. It’s kind of inspiring!

I went to focus on some less lofty goals — I just wanted to get some experience playing with a JavaScript mapping library created by a company called Mapbox. My goal was to get the library up and running and then use it to display some geo-encoded data from a service I happen to use a lot: Foursquare.

It’s no secret that I really like Foursquare — mainly for keeping track of our adventures far and wide. I found a helpful PHP library for converting Foursquare (and other social networking) data into GeoJSON, which is needed by the JavaScript mapping libraries. After a couple of hours of hacking around, I got something up and running!

Screenshot 2015-02-04 20.06.29Pretty neat stuff! I have some ideas on how to improve this and some additional data that I want to show in the future.

 

My first half marathon

This past weekend, I completed my first official half marathon after participating in the 32nd annual Kaiser Permanente San Francisco Half Marathon! The course was fairly flat and wound its way through Golden Gate Park and along Ocean Beach. You couldn’t have asked for better better weather or scenery! Overall, it was the perfect event for a new runner like myself.

My official result? 2:17:32. That’s kind of the nice thing about your first half marathon — any time is a new personal record. I’ve been training off and on for the last few months (and indeed, I could have trained more) and was hoping to hit 2 hours.

As it turns out, I was on pace to do it until one of my calves started to cramp up exactly half way through the race, forcing me to take a stretching break and limp through the remainder of the course.

dave_schumaker_half_marathon

That’s me at mile 8! via @kerryme.

That said, it was still a really positive and fun experience and it’s something I’d like to do more of! Which is good, because I signed up for the Oakland Running Festival in March. Oh, boy!

A couple things I learned:

  • I brought my own water bottle (mainly so I could eat some of the gross energy gels that I brought with me). Overall, I wasn’t too distracted by it — I had brought a disposable bottle just in case it was annoying and I needed to chuck it. However, I skipped a bunch of water stations at the beginning of the race since I figured I had water. Big mistake! I should have taken advantage of the free water and gatorade (and used it to time when I should have consumed the gels). As it is, I ran out of water at around mile 10.
  • The gels really helped! I had the Cliff Bar variety. While they were really thick and gross to consume, I really felt that they gave me energy and I didn’t really hit a wall until the last mile and a half or so of the race. More than anything, I was distracted about my tight calf!
  • Someone else had mentioned this to me in the past, but I basically found someone who was running at a pace I was mostly comfortable with and I ended up sticking close to them for the first 6 miles. Trying to keep up with him was kind of a fun game.
  • I also discovered the “awful truth about jogging that no one ever talks about.” Oh, man is it true!

Adventures in Learning to Code: Sudoku Edition (Part II)

In part I, I explained how I’m attempting to write a webapp that can solve simple Sudoku puzzles. Since then, I’ve made some progress. This series will track what I’ve been working on, learning, and any difficulties I’m encountering.

With a goal in mind, it was time to just dive in and go.

Step 1: Oh God, what is this even…

At first, it was kind of overwhelming. Where do I even start? How the heck am I going to even tackle any of this?

After learning the basics of Javascript loops, functions, arrays, and objects, thanks to Codecademy, I was mostly ready to go. What was my real first step? How the heck do I even manipulate data on a webpage, let alone collect and analyze it?

After a few tries, I had a simple project featuring a table (with cells individually labeled) up and running!

Screenshot 2015-01-27 17.23.03

After some more tinkering, I was able to get the “Generate Numbers!” button to actually randomly generate the numbers 1 through 9 and fill in the cells without repeating any numbers. Sweet!

Screenshot 2015-01-27 17.25.58

Hey, that’s kind of cool. Now we’re getting somewhere! I think I was ready to take it to the next level.

Step 2: Okay… I guess we need to make a fake Sudoku grid.

There’s probably simpler ways to go about doing this, but I ended up spending a lot of time figuring out how to build a Sudoku grid in the first place. Hey, I wanted this project to look somewhat presentable when I showed it off and (hopefully) finished it.

HTML tables were driving me crazy, so I just straight up stole this helpful thing after Googling for an answer. It looks pretty good. Pretty, pretty, pretty good.

Screenshot 2015-01-27 17.35.02

Wow, sir. That looks so good that I’d like to take two!”

Alright, one mission accomplished. Now onto another mission. How do I completely fill out the entire grid? I started with this, because I wanted to understand how to try to isolate certain sections of the Sudoku grid and see if I could create a series of non-duplicated random numbers.

Why did I start with randomly generating numbers? I initially thought I would just try to brute force solve some simple beginner’s level puzzles, but I later realized how ridiculous this would be. There’s a ridiculous number of possible permutations to try.

For now though, I decided to start filling out random numbers by row. This is what my initial code looked like.

[javascript]
function fillTable(row) {
for (var i = 1; i <= maxColumns; i++) {
var selectCell = row + i;
//console.log(selectCell);
document.getElementById(selectCell).innerHTML = fillCell(selectCell);
if (i == maxColumns && row != maxRow) {
i = 0;
row = nextChar(row);
} else if (i == maxColumns && row == maxRow) {
break;
}
}
}
[/javascript]

Did it work? Yes! I ended up with grid of random numbers filled out! Sure, there’s really no rhyme or reason for where numbers ended up, but it certainly looks like a completed Sudoku puzzle, right?

Screenshot 2015-01-26 17.20.12

Step 3: Let’s put some real numbers in there

Alright, entering a bunch of random numbers into a grid is pretty fun for all of 5 minutes. (To be honest, it was pretty fun) But to take this to the next level, I really needed to have the beginning of a Sudoku puzzle in there.

A few quick Google searches later, and I found a suitable candidate! One thing I decided early on was to store all this data in an object — mostly so I could easily update it with different puzzles later on. It ended up looking like this.

[javascript]
var allCells = {
a1: "", a2: 1, a3: "", a4: 6, a5: "", a6: 4, a7: 3, a8: "", a9: 7,
b1: 3, b2: 5, b3: 6, b4: "", b5: "", b6: "", b7: "", b8: "", b9: "",
c1: "", c2: "", c3: "", c4: "", c5: 5, c6: 3, c7: 6, c8: 9, c9: "",
d1: "", d2: 8, d3: 3, d4: 2, d5: 6, d6: "", d7: 4, d8: "", d9: 9,
e1: "", e2: "", e3: "", e4: "", e5: "", e6: "", e7: "", e8: "", e9: "",
f1: 4, f2: "", f3: 5, f4: "", f5: 7, f6: 8, f7: 2, f8: 6, f9: "",
g1: "", g2: "", g3: "", g4: "", g5: "", g6: "", g7: "", g8: "", g9: "",
h1: "", h2: "", h3: "", h4: "", h5: "", h6: "", h7: 7, h8: 2, h9: 4,
i1: 7, i2: "", i3: 9, i4: 4, i5: "", i6: 2, i7: "", i8: 8, i9: ""
};
[/javascript]

On top of this, I wanted to easily differentiate the initial numbers that were created on a brand new board (so I could easily see what my script was generating, versus what was already in place on the board). I wrote a function to modify the DOM and change the font weight and background color of the starting cells.

[javascript]
function setupBoard(row) {
var row = row;
var cellValue = 0;
for (i = 0; i < maxColumns; i++) {
//console.log("ROW: " + row + (i+1));
cellValue = allCells[row+(i+1)];
document.getElementById(row + (i+1)).innerHTML = allCells[row+(i+1)];

// Just highlighting what cells we initially started with.
if (Number(cellValue) > 0) {
document.getElementById(row + (i+1)).style.fontWeight = "bold";
document.getElementById(row + (i+1)).style.backgroundColor = "#F2F2F2";
} else {
// Use this to count up total number of empty cells that we need to solve for
// The idea is that we can use this to detect if we’re stuck
emptyCells++;
}
}
}
[/javascript]

You’ll notice I have a lot of console.log() calls commented out. I liberally used these all over the place so I could make sure things were working correctly. Anyway, once all that was said and done, it ended up generating a board that looked like this.

Screenshot 2015-01-27 07.38.36

Wow, we’re starting to chug along pretty nicely! Next time, I’ll talk about the million different functions I created in trying to solve this.

Adventures in Learning to Code: Sudoku Edition (Part I)

Screenshot 2015-01-27 21.00.15

This series of posts will track what I’ve been working on, new things I’ve been learning, and any difficulties I’m encountering along the way. Buckle up!

One of the things I resolved to do this year was spend a bit more time learning to code. Sure, sure, I’ve dabbled in things like PHP and have used MySQL to build some pretty basic websites and tools.

Even with the little I’ve done, I’ve always enjoyed it and have found myself losing track of time while trying to solve some problem or just get this certain thing to work. It reminds me of a similar thing I’ve encountered with the addictive nature of Sid Meier’s Civilization games.

“Just one more turn…”

Late last year, I mentioned to a coworker my desire to spend a bit more time learning various techniques and theory. He suggested that if I’m serious, I should really just start trying to tackle some problems and we could go over it together.

The first thing he suggested? I should build a application to solve a Sudoku puzzle.

What??!

As tends to happen, the end of the year got a bit hectic and I put this project on the backburner. When I originally considered it, I was going to try and write it in PHP. However, I recently saw this goofy map of popular programming languages by state and thought, “I’ve never really done anything with Javascript. Maybe I should try that.”

So, that’s what I’ve been doing the last few weeks. Between taking some awesome online courses through Codecademy and Udemy, as well as reading up on various books, I’ve been trying to really get into it.

How am I doing? I’ll try to share my progress in future updates.

Improving my coffee game with technology

Nothing makes me feel ready to take on the day like my NASA mug full of fresh coffee. 

The last few months have been pretty awesome for my coffee game. Between gifts from our wedding, Christmas presents, and my birthday, I’ve started to build up quite a collection of devices to help feed my caffeine addiction.

  • Chemex
  • Aeropress
  • Variable temp electric gooseneck kettle
  • Digital scale with timer

It’s been awesome and fun experimenting with different ways to make coffee and see how it affects some of my favorite types of beans.

One of the things that I’ve recently started doing is creating a crazy Google form to track all the different coffee preparation methods I’ve been using. It looks like this.

Coffee Tracking

What sorts of additional details am I missing that I should be recording here? It’s probably getting a little bit ridiculous, but hey!

This sort of thing appeals to my sense of experimentation and adventure. I should probably put one together for my Aeropress as well. File that one under #weekendprojects.