The simple is not so simple
During the week ending on 21 July, my study of web application development at Learners Guild had me doing things that seem numbingly simple on the surface. I developed 2 “applications”:
- One lets you look up the current temperature in a city of your choice.
- The other lets you look up movies that have some relationship to some text that you provide.
Let’s make this more concrete.
node weather Nsoatre
In about a second, you get the reply, such as:
Temperature in Kelvin: 292.59 Temperature in Celsius: 19.44 Temperature in Fahrenheit: 66.99
What my simple application won’t tell you is that it looks up the city you asked for in the Application Programming Interface (henceforth “API”) of OpenWeatherMap, and when you give it a city name it gives you back a response that may or may not have anything to do with that city. Nsoatre is a small town in Ghana, apparently unknown to OpenWeatherMap. Instead of saying it doesn’t know about Nsoatre, OpenWeatherMap gives you the temperature for Saint-Martin-de-Beauville, France. (No joke.) If you name a large Ghanaian city, such as nearby Kumasi, then you do, indeed, get data for that city.
So, why does the Guild have us Learners creating such a useless (or worse than useless) application? As I understand it, the Guild trusts us to distinguish the practical usability of a project from its value as a teacher of fundamental technologies. This exercise helped familiarize me with:
httpmodule of Node, which simplifies the task of requesting information from other computers.
- How the
get()methods (slightly) differ.
- How an information request to an API is constructed.
- How to show an API that the program has permission to get information from it.
- How to screen out invalid requests (such as
node weather, with no city).
- The splitting of an API response into “chunks” and how to reassemble the chunks before processing the response.
- How to round numbers to a particular number of decimal digits after computing them.
- Debugging code.
Maybe it’s even better for my application to give you laughably unreliable information. Then I’m not tempted to spend time trying to make its answers correct, when that would only distract me from getting the intended benefits of the exercise.
The movie application lets you look up information about movies. It’s similar to the temperature application, in some ways. You use a terminal to enter a request such as:
node movie-search Panchali
In about a second, you get the reply, such as:
Pather Panchali (1955) Apur Panchali (2013)
This application sends your search text on to the Internet Movie Database (IMDB), gets back a web page (the same one, written in HTML, that you would get if you went to the IMDB website and entered “Panchali” into the search bar), analyzes that page to extract from it just the title lines, and sends those title lines back to you.
This application seems to give more reliable answers than the temperature application does, but not perfect. If, for example, you wanted to ask about movies featuring that small Ghanaian town Nsoatre, you would get references to 2 movies: a 1957 TV series titled “Alcoa Theatre” and a 2014 feature titled “Vadelmavenepakolainen”, about a Finnish man who “has dreamed of becoming a Swede since childhood”. I haven’t found any connection between them and Nsoatre, and I don’t intend to work on that problem, so, if you want to, be my guest.
This exercise reinforced most of the topics listed above, but the movie application’s technologies differ incrementally from the temperature application’s. This one helped me become more familiar with:
- The widely used
cheerioHTML parsing library, which simplifies the task of manipulating HTML web pages.
Getting movie title lines from an IMDB search-result web page is not trivially easy. The routine that I constructed to do this, in
cheerio syntax, was:
const listCells = $('a[name=tt]') .parent() .parent() .children('table') .children('tr') .children('td.result_text');
- Find an
atag that has the name
- Then find that tag’s parent tag.
- Then find that parent tag’s parent tag (i.e. the
atag’s grandparent tag).
- Then find all of the children of that grandparent that are
- Then find all of the children of those
tabletags that are
- Then find all of the children of those tr tags that are
td(cell) tags with the class
So, because IMDB search-result pages are not optimally organized for programmatic analysis, I had an opportunity to spend a few hours learning and practicing the use of the
cheerio HTML parsing library.
Of course, I didn’t get it right the first time; this exercise, like all of them, helped me learn how to find and correct my bugs. But debugging is a huge topic; I’ll save that for another entry.