Using Yahoo Query Language with XPath

While developing the little script that allowed me to use Pushover to receive YouTube statistics I came across Yahoo Query Language.

Yahoo provides some very interesting services. I remember using Yahoo! Pipes a very long time ago and I was amazed by it’s simplicity. It was perfect for building data mashups that aggregate web feeds, web pages, and other services, creating Web-based apps from various sources, and publishing those apps. I learned a whole lot about scraping the web, but sadly the service was shut down on 30 September 2015.

While working on my latest Pushover script I wanted to pull some statistics from YouTube. One issue is that the YouTube API requires a developer API key which I didn’t want to depend on. I also didn’t want to rely on other external libraries for screen scraping my channel page. After looking around the web for alternative approaches I then found this fun lille project displaying YouTube statistics in real-time… but how?

After looking at the code I figured out that the script is using Yahoo Query Language and XPath to grab the view and subscriber count:

https://query.yahooapis.com/v1/public/yql?q=SELECT * FROM html where url="https://www.youtube.com/channel/UCW_wx3mNy3EiaXljR_iVVUQ/about"
AND xpath='//*[@class="about-stats"]//b'&format=xml

The code is simple and self-explanatory. If you navigate to my YouTube channel’s about page (don’t forget to subscribe) you’ll see both the subscriber and view count. Then it uses XPath to extract the bold text (b) from any HTML element that has the class “about-stats” assigned. You can also tell Yahoo to output the result in xml or json.

 

Share this:

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.