Sunday, December 8, 2013

FlickrSync - Backup Flickr Images and Data

I've since discovered that there is already FlickrSync in the Windows world, I will have to come up with a new name at some point.  

I've been using flickr for a while (see digitaltrails). Last year became concerned about Yahoo's commitment to flickr and began investigating how to routinely backup all of my flickr images and metadata.  I also wanted to be able to create a slideshow that I could carry on a usb-stick, phone, tablet, or CDROM.  I found Colm MacCarthaigh's flicktouchr, a python script that backs up all of a user's images (I think he was targeting  an Ipod-Touch - hence the name). Colm's script just gets the images, I wanted some of the metadata as well.

The flickr REST API used by flickrtouchr is very well documented at http://www.flickr.com/services/api/.  Initially I thought I'd just add some code to Flickrtouchr to make some additional API calls.   But as part of a playing around with the script I wound up changing it beyond recognition.   In the end I created a new script called flickrsync - flickr-sync - it has the following functionality (all new features unless noted):
  • Uses authentication/authorisation/signed-messages to interact with the flickr API using REST (from flickrtouchr).
  • Download all images and favorites (from flickrtouchr).
  • Download videos and determine video type from http response header.
  • Download recent updates only - n days or since last update.  This allows the script to update a previous download rather than grabbing everything all over again.
  • Download specific sets only (specified by flickr numeric ID).
  • Download photo and comment metadata (default, but optional).
  • Download set and collections metadata.
  • Optionally use copy instead of UNIX links for images that appear in more than one set  (flickrtouchr always used UNIX links). Automatically uses copy on non-UNIX systems.
  • Optionally download favourites (this was non-optional in flicktouchr).
Making a REST call to flickr API is simply a process of making an http request and decoding an XML response. This is made slightly more complex by the authentication, authorisation, and signing of requests.


Having achieved a way of routinely mirroring my data, I moved on to the task of creating a standalone slideshow.  I found Mary Lou's Responsive Image Gallery with Thumbnail Carousel and created a python script to build a slideshow by processing a modified templated version of the Gallery's HTML file.  Mary Lou's JavaScript can be easily extended to include automated slideshows with transitions.

I've recently had some time to tidy up my scripts and I've now made them available at https://github.com/digitaltrails/FlickrSync.   The README.md file contains full usage details and an example.