isotope|eleven


About_jd

J.D. Warren

Software Developer

JD Warren is a Ruby on Rails developer and extensive hardware hacker. Prior to joining isotope|eleven, he was a builder focusing on remodeling apartments and rental houses. In his spare time, JD builds robots, electronic circuits, and is a technical writer with work in several online and print publications and a recent book titled "Arduino Robotics".

Blog Posts

About_jd

Postgres Search using pg_search

by: J.D. Warren

March 20th, 2013 11:52

Every now and then, I run across a gem that I end up using in nearly every project. As it turns out, several of the recent projects I have worked on have required some form of internal site content search engine. While there may be better options out there (like solr), I feel that this is the easiest way to add a search feature on your rails site using a postgres database. My main reason in trying out the postgres search gem (pg_search), was because it is extremely fast and on smaller projects I did not want to have a separate solr server running.

To install pg_search, just include the gem in your Gemfile:

gem 'pg_search'

When using pg_search, there are a multitude of options, but I will discuss the 2 main search features that I found helpful: multisearch and pg_search_scope. You can use one or both of these to get pg_search working, depending on your desired results.

multisearch

Multisearch is a single search command that when run will look at all models that have multisearch enabled and return an array of everything it finds as search objects, not the actual content object. The search objects that are returned in a multisearch include the "searchable_id" which is the actual object ID and "searchable_type" which is the object class name (ie. "Post"). From these bits of information, we can obviously look up the correct Post using its ID. This method of searching is nice because you can specify what attributes you would like to search for each model individually, so you have control over what fields are searched.

Before using multisearch you will want to run the generator to create a migration for the multisearch result documents.

rails g pg_search:migration:multisearch
rake db:migrate

Next you will want to attach pg_search to the model that you would like to search (we will do this in our Post model):

class Post < ActiveRecord::Base
  attr_accessible :body, :title
  include PgSearch
  multisearchable :against => [:body, :title]
end

Thats is for multisearch, now just call the searcher and check out the results:

results = PgSearch.multisearch("foo")

pg_search_scope

Pg_search_scope is another method of searching that acts more like a model scope that you can call to get more specific results from a given model. With this method of searching, you will add a named search scope to the model and when called, it will return a list of the actual objects that if finds. You also have options to declare what attributes should be considered more relavent than others to order the results - for instance, place results with matches in the title ('A') higher than those with matches in the body ('B') of a post. This method can be used for more granular results, but will only return matches from the model that it is called on. You can also have multiple named pg_search_scope methods in the same model, as long as they have different names.

To setup a pg_search scope, add the following to your model:

class Post < ActiveRecord::Base
  attr_accessible :body, :title

  include PgSearch
  pg_search_scope :custom_search,
    :against => {
      :title => 'A',
      :body => 'B'
    },
    :using => {
      :tsearch => {:prefix => true}
    }
end

Thats is for pg_search_scope, now just call the searcher and check out the results:

results = Post.custom_search("foo")

Thats it!

You can now run searches in your app and focus on specific models with specific attributes as well as prioritize the results to suit your needs.

To find out more about the pg_search gem, check out the github page:

https://github.com/Casecommons/pg_search

Tags:

Nathan Stott from Whiteboard IT giving a talk on Node.js:

Tags:

Ho-Sheng Hsiao giving the second talk:

Tags:

Chris Winslet from MongoHQ, giving a talk on Twilio and Mongo. There are 3 videos in this talk:

Tags:

Today I am going to walk through our recent continuous integration Traffic light notifier project that we just finished at the office. This project stemmed from my company's desire to immediately know if a developer has broken a software project, and what better way to do that than to have a huge red light flashing in your face. We connected an old salvaged traffic light fixture to our Jenkins CI-server that monitors the testing status of all of our current software projects. If all our tests are passing, the light stays green, if any test fails the light turns red to provide a visual notification of a problem. While Jenkins is running a test suite on any project, the yellow light will flash to let us know of the activity.

So how does one connect a 48" tall traffic light to a continuous integration server? With a Ruby script, an Arduino, and a few relays of course.

Traffic Light

The Ruby code will create a serial connection with the Arduino to send data, then create a web connection with the CI server to request the build status data via our CI server's built in API. A quick look through the returned data will give us a chance to see if there are any problems – if so, we'll send a signal to the Arduino to change the light status, otherwise it stays green. The Ruby script requires 3 gem dependencies to run: faraday, json, and serialport – all available from rubygems.org (eg. gem install faraday).

# Isotope11 continous integration server Traffic-light
# Ruby script to monitor json output from Jenkins CI-server, and output the status of projects to a Traffic-light. 
# If all builds are passing, the light is green. 
# If a job is currently building, the yellow light flashes. 
# If any job is failing, the red light is turned on and green turned off.
require "serialport"
require "json"
require "faraday"
require "net/http"

# create a new Serial port for the Arduino Uno which uses port /dev/ttyACM0. Older Arduinos should use /dev/ttyUSB0
sp = SerialPort.new("/dev/ttyACM0", 9600)

# wait for connection
sleep(1)

# create a new Faraday connection with the Jenkins server to read the status of each job
conn = Faraday.new('http://your_Jenkins_server_address.com')
puts 'go to loop'

loop do
  begin
    # grab the json from the jenkins api
    response = conn.get('/api/json')
    # parse the response into a list of jobs that are being monitored
    jobs = JSON.parse(response.body)["jobs"]

    # search each job to see if it contains either "anime" (building) or "red" (failing)
    should_blink = jobs.detect{|j| j["color"] =~ /anime/ }
    should_red   = jobs.detect{|j| j["color"] =~ /red/ }
  rescue
    # if no response, assume server is down – turn on Red and Yellow lights solid
    server_down = true
  end

  # check results of job colors
  if should_blink
    # something is building... flash yellow light!
    puts "Something is building... flash yellow light!"
    sp.write("1")
  else
    # nothing is building... turn yellow light Off.
    #sp.write("2")
  end

  if should_red
    # something is red... turn On red light!
    puts "Something is broken... turn On red light!"
    sp.write("3")
  else
    # nothing is red... turn On green light.
    sp.write("4")
  end

  if server_down
    sp.write("5")
  end

  # wait 5 seconds
  sleep(5)
end

# close serial data line
sp.close

The Arduino board is fitted inside of the traffic light housing, and mounts to a perforated prototyping board from Radio Shack using some male-pin headers. Above the Arduino, are two small PC mount relays capable of switching up to 1 amp at 120vac – perfect for some low wattage light bulbs. The relay coils are controlled using a 5v signal, and only consume about 90mA at that voltage level, so we can use the Arduino's onboard 5v regulator to power the relay coils. Unfortunately, we cannot simply drive the relays directly from an Arduino pin because it can only supply around 40mA per pin and the inductive switching properties present in a relay might cause damage to the Arduino. Instead, we can use 2 small N-type signal transistors (either bjt or mosfet) to interface between each relay and the Arduino output pin. Building the relay board might require some hands-on tinkering, but is a rewarding task when complete (circuit schematic file included).

Arduino mounted inside Traffic Light traffic_light schematic

The Arduino code is simple, basically listening on the serial port for 1 of about 5 signals. If the Arduino detects a recognized serial byte, it will carry out a function to control the Traffic lights - there is no extra fluff, just what is needed. If you are having trouble locating an old Traffic light, or would like to build a smaller desktop version of the notifier, you can do so with only an Arduino and a few LEDs (red, yellow, and green) - you don't even have to solder anything!

Poor man's traffic light

// Isotope11 CI-server traffic light
// Arduino Uno with 2 relays (SPDT) attached to pins 4 and 7
// isotope11.com 2-9-12

// declare variables and output pins:
int inByte; // create a variable to hold the serial input byte
long lastTx = 0; // create a “long” variable type to hold the millisecond timer value
int yellow_light = 4; // create an output variable attached to pin 4
int red_green_light = 7; // create an output variable attached to pin 7

void setup() {
  Serial.begin(9600);   // start Arduino serial monitor at 9600bps
  pinMode(yellow_light, OUTPUT); // set up pin 4 as an output
  pinMode(red_green_light, OUTPUT); // set up pin 7 as an output
}

void loop() {
  // check serial buffer
  if (Serial.available() > 0){
    inByte = Serial.read();    // read serial byte
    Serial.println(inByte);     // print serial byte
    lastTx = millis(); // set the lastTx time-stamp variable equal to the current system timer value

    // the serial bits “49” - “53” are detected when the numeric buttons “1” – “5” are pressed on the keyboard.
    switch(inByte){
    case 49:  // if serial value received is "49" (number 1), blink yellow light
      digitalWrite(yellow_light, HIGH);
      delay(1000);
      digitalWrite(yellow_light, LOW);
      break;
    case 50:  // if serial value is "50" (number 2), turn yellow light off
      digitalWrite(yellow_light, LOW);
      break;
    case 51:  // if serial value is "51" (number 3), turn red light on (green off)
      digitalWrite(red_green_light, HIGH);
      break;
    case 52:  // if serial value is "52" (number 4), turn green light on (red off)
      digitalWrite(red_green_light, LOW);
      break;
    case 53:  // if serial value is "53" (number 5), turn green and yellow lights on solid (api error)
      digitalWrite(red_green_light, LOW);
      digitalWrite(yellow_light, HIGH);  
    }    
  }
  else {
    if ((millis() - lastTx) > 10000) {
       // it has been more than 10 seconds (10000 milliseconds) since any serial information has been received
       // assume there is a break in the PC connection, and turn red and yellow lights on solid.
      digitalWrite(red_green_light, HIGH);
      digitalWrite(yellow_light, HIGH);
    }
  }
}

Repo

The github repository is here.

Parts list:

  1. an old Traffic light
  2. Arduino Uno, Radio Shack part # 276-128 - $34.99
  3. PC prototyping board, Radio Shack part #276-168 - $3.19
  4. (2) PC pin relays, Radio Shack part #275-240 - $4.69 ea
  5. (2) NPN transistors or mosfets, Radio Shack part #276-2016 - $1.19 ea
  6. (2) 10kohm resistors, Radio Shack part #271-1335 - $1.19 pk
  7. (2) 100kohm resistors, Radio Shack part #271-1347 - $1.19 pk
  8. (20) male-pin breakaway headers, Sparkfun part#PRT-00116 - $1.50
  9. (4) 8mm bolts, 1” long (with nuts) - $1.00

Tools needed:

  1. wire/wire snips
  2. solder/soldering iron
  3. drill/drill bit
Tags: