learnyounode Lesson 8 – HTTP Collect

(Last Updated On: 2017-11-05)

In this lesson we need to write a program that collects all data from an HTTP GET request and logs the number of characters and the complete string of characters received from the server.  But didn’t we just do that in the last exercise?  Not exactly.  The http module emits an events events as the request is processed.  In the previous exercise we used the on() method to act upon the first data event we received.  All other events are ignored.  Take a look at the official solution from the previous lesson again.

If you look closely you can see that we are only acting upon two events.  If we get a data event, we are logging that data to the console.  If we get an error event, we are logging the error to the console.  There is nothing in our code that allows us to act upon more than these events.  If there are more data events that might occur, we aren’t doing anything to act upon those events.

We get a couple of hints from learnyounode about how to collect all data that is sent from the server rather than just the first event.  The first hint is that we can use the ‘end’ event to determine when we have received all data.  The second hint is that we can leverage some existing node modules from npm to help us solve this problem such as ‘bl’ and ‘concat-stream’.

Official Solution

Click to show/hide solution

In the official solution, the buffer list (bl) module is used.  The response from the GET request is piped to the bl method using the .pipe() method.  The buffer list accepts a callback as an argument and exposes the data in the buffer object with the main node buffer object.  It is not necessary to use buffer list module to complete this task.  However, it is worth knowing about because the buffer list module includes a number of useful prototype methods in its API such as bl.get() which returns bytes at the specified index and bl.slice() which returns a new buffer object with bytes in the specified range.  These methods are useful for manipulating buffer streams.  To learn more about the buffer list module, refer to the module API documentation on getHub.


My Solution

If you are curious to see how this problem can be solved without using an additional node module, refer to the alternate solution below.

The alternate solution adds the response body to the body variable when a data event occurs.  When the response.on(‘end’) method encounters the end event, the body and the length of the body are logged to the console.

Lesson 9