2016 twenty-four merry days of Perl Feed

Getting Drunk with Mojolicious and Memoize

Memoize - 2016-12-13

I've always wondered what Santa Claus and the elves do after the Christmas Eve's work. Perhaps they'd go get smashed afterwards with a pub crawl finding the best egg milk punch all over the world, then perhaps spend the rest of the night recovering in a nearby hotel? If they ever do though, I think I could help them a bit, being the naughty boy and all...

You see, last October, a friend and I joined a hackathon that let me play around a couple of hotel and microbrewery information APIs: the result is The Drunkery, a webapp that Santa (and everyone else) can use to (hopefully) get inebriated. For this, I used the awesome Mojolicious framework for the backend API, powering a React frontend showing a Google Map with pins to hotels in a given city, with the nearest pubs or breweries. We ran out of time trying to set up paths between the pins to visualize a pub crawl, but nevertheless we won, and hopefully we can get to improve it on the next stage :D

In this story, let me tell you about a couple of things that we learned and used for this app:

To start, writing the backend part with Mojolicious was the easiest part, but only so when much thought was put into the design of the backend's interface. I ended up with providing only two API endpoints for my partner's frontend:

use Mojolicious::Lite;
use Mojo::URL;
use Drunkery::Search;

helper search => sub {
  state $search = Drunkery::Search->new( ua => shift->ua );
};

get '/search_by_city' => sub {
  ...;
};

get '/search_by_endpoint' => sub {
  ...;
};

app->start;

These routes provide my JavaScript frontend a way to search for hotels and and breweries in either a given city, or a given geolocation endpoint, by making simple AJAX-style GET requests and receiving JSON back. Both these routes emit an array containing a city object, a list of nearby breweries, and a list of nearby hotels, and are powered via a search helper that uses the logic in Drunkery::Search.

Well, logic might not be the right word. At the time of the hackathon, the package was just as simple as this:

package Drunkery::Search;
use Mojo::Base -base;

has qw(ua);

sub fetch { shift->get(shift)->res->json }

1;

Yep! The search helper is just simply making a HTTP request to the Booking.com API, and then feeding the parsed results back unchanged to the caller.

A matter of caching

One of the problems I dealt with during this hackathon was the issue of making the Perl backend respond faster to the frontend. The first implementation I showed above simply had the backend fetch the hotel and brewery information every time it was requested: in short, we needed to have caching. I was loathe to set up another service like SQLite or Redis though, as I thought I didn't have enough time to wire those to the backend...

Enter the real star of this story, Memoize!

package Drunkery::Search;
use Memoize;

sub normalize_url { shift->to_string }

memoize( 'fetch;, NORMALIZER => 'normalize_url' );

This effectively gave me caching at nearly no cost (thanks, Higher Order Perl for reminding me!) Granted, it was imperfect (for starters, it was only an in-memory cache,) but at the time, it made sense.

As you might remember Memoize is a module that can replace any function with a function that does the same thing, but every time you call it remembers the result for any given parameters and if it's called again with those same parameters returns the same result without having to re-run the main body of the original function - i.e. if you call the function with the same URL, it doesn't have to go use the Booking.com API a second time!

The slight wrinkle is that Memoize has to be able to recognize that the arguments the function is being called with are the same - and our fetch function is being called with a URI object which will look to memoize to be a different object - and therefore different argument - each time it's called even if that URI object represents the same underlying URL each time. To address this we use a NORMALIZER function to turn our arguments - our URI object - into something that memoize can compare: We turn the URI objects into strings so that two function calls arguments for the same URL can be compared as identical.

But why? (or, what more could be done?)

I joined this hackathon on a whim, with nary an idea for what to build at all, so instead of going all-out serious, I decided to wing this the -Ofun way. Indeed, it was very funny (and easy) to build a Perl backend in such a short time, with still room for improvements.

Saving Memoized results

Fast forward to late November, where a succeeding Booking hackathon also had me use Memoize yet again for keeping API responses. This time, I tried to keep saved responses to disk by making use of DB_File:

use DB_File;
use Memoize;

sub fetch {
  my ( $c, $url ) = @_;
  $c->ua->get($url)->res->body;
}

sub normalize_url { shift; shift->to_string }

tie my %cache => 'DB_File', 'cache', O_RDWR | O_CREAT, 0666;
memoize(
  'fetch',
  NORMALIZER => 'normalize_url',
  SCALAR_CACHE => [ HASH => \%cache ],
  LIST_CACHE => 'FAULT'
);

By having memoize cache the results into a tied hash that saves any changes to the hash to file on disk the cache lasts longer than just the runtime of the demo application.

SEE ALSO

Gravatar Image This article contributed by: Zak B. Elep <zakame@cpan.org>