The 2001 Perl Advent Calendar
[about] | [archives] | [contact] | [home]

On the 12th day of Advent my True Language brought to me..
POE::Component::Client::HTTP

Yesterday I talked about how POE was a great module because it allowed you to do multiple things at once in a voluntary manner. One particular module that I've used with to great effect is PoCo::Client::HTTP.

With traditional LWP you pass a UserAgent a Request for a webpage, and after a while a Response gets handed back to you. This is all very simple and straight forward. One of the things that I do with such scripts is provide RSS access to various HTML web resources. These are useful in numerous ways - not in the least that infobot is able to render XML feeds directly to IRC.

The problem with this method however is that each HTTP request is handled sequentially meaning that the script laboriously gets one page after another - a real pain if you've got slow servers. In my example where the infobot is making a HTTP request to my server it can actually time out before the last page is back and the RSS stream can be generated. With PoCo::Client::HTTP it's possible to get more than one page at once. Suddenly my script that waited for to total time it takes to get all pages only has to run for the time it takes for the longest page to return! A vast improvement.

Of course, there are other more complicated things you can do with this module than this - For example I've used it to perform a DDoS from all the spare machines in the office onto our test web cluster. Importantly it shows a good example of concurrent programming with POE that you can play around with and immediately useful and clear improvements out of the simplest script.

  • POE
  • The POE homepage
  • LWP