somedutta.ghosh@innoraft.com's picture
 
Dec 11, 2013
Booking.com exposes an API that an affliate partner can use to return hotel details as xml files. I am assuming that we already have a knowledge of using the feeds module before going ahead. In this blog I’ll only be sharing the basic API functions and how I used feeds module to parse the xml returned and create hotel/accommodation type nodes from them.

The Booking.com API functions

Below are some of the API functions and the basic relevant data that they return.
All *_id values are unique.
  1. getHotelTypes
    1. hoteltype_id
    2. name
  2. getHotels
    1. address
    2. checkin
    3. checkout
    4. city
    5. city_id
    6. hotel_id
    7. hoteltype_id
    8. is_closed
    9. latitude
    10. longitude
    11. max_persons_in_reservation
    12. max_rooms_in_reservation
    13. maxrate
    14. minrate
    15. name
    16. nr_rooms
    17. url
    18. zip
  3. getHotelDescriptionTranslations
    1. hotel_id
    2. description_id
    3. description
  4. getHotelFacilityTypes
    1. facilitytype_id
    2. hotelfacilitytype_id
    3. name
  5. getHotelFacilities
    1. facilitytype_id
    2. hotel_id
    3. hotelfacilitytype_id
  6. getHotelPhotos
    1. hotel_id
    2. url_original
  7. getCities
    1. city_id
    2. countrycode
    3. latitude
    4. longitude
    5. name
    6. nr_hotels
  8. getRooms
    1. beds -> amount
    2. beds -> type
    3. creditcard_required
    4. hotel_id
    5. max_persons
    6. max_price
    7. min_price
    8. room_id
    9. roomtype
    10. roomtype_id
    11. smoking_requested

The accommodation content type

For importing the hotel information to my site I created a content typeaccommodation with the following fields -
  1. Title
  2. Closed
  3. Name
  4. Type
  5. Area
  6. Body
  7. Images
  8. Address
    1. Latitude
    2. Longitude
    3. City
    4. Postal Code
    5. Street Location
  9. No of Rooms
  10. Room info - field collection
    1. Room ID
    2. Room Type Id
    3. Room Photos
    4. Room Type
    5. No. Of Beds
    6. Maximum Persons
    7. Maximum Price
    8. Minimum Price
  11. Maximum Price
  12. Minimum Price
  13. Check-in Time
  14. Check-out Time
  15. Web Presence - url
  16. Hotel ID
  17. City ID

The supporting modules

For importing the hotel xml data as nodes, I used the feeds module along with some other helper modules. Listed below are the modules that were used -
  1. feeds
    1. the main module that is used to create importers with file to field mapping used for the actual import
    2. https://drupal.org/project/feeds
  2. feeds_xpathparser
    1. since the data returned by the API functions are in xml I used this feeds plugin to parse the xml content
    2. https://drupal.org/project/feeds_xpathparser
  3. field_validation
    1. more on this later
    2. https://drupal.org/project/field_validation
  4. location_feeds
    1. to import location data into location type fields
    2. https://drupal.org/project/location_feeds
  5. field_collection
    1. since there can be multiple rooms with a collection of data for each I used field_collection module(non-feeds, related to content type)
    2. https://drupal.org/project/field_collection

Approach -

Now that I have defined the booking.com API functions, my content type and the modules used for this functionality we can next take a look at the approach and importers that I used.
  1. since different API calls return different data create different importers for each
    1. basic information about hotels like name, price, address, etc., hotel images, hotel description, hotel room information, etc.
  2. some functions returns ids that need to be mapped to data returned by a different API function; create importers for importing each mapped xml
    1. like hotel type and facilities
  3. for field collections, write a custom module for importing data from the xml
    1. for importing hotel rooms

The Feeds Importers

Following are the importers I used along with the fields I imported using each along with the API function used for each -
  1. Hotels Basic Info - getHotels
    1. hotel_id
    2. Area
    3. Latitude
    4. Longitude
    5. City
    6. Postal Code
    7. Street Location
    8. City ID
    9. No of Rooms
    10. URL
    11. Closed
    12. Check-in Time
    13. Check-out Time
    14. Minimum Price
    15. Maximum Price
  2. Hotels Description Info - getHotelDescriptionTranslations
    1. hotel_id
    2. Body
  3. Hotel Images - getHotelPhotos
    1. hotel_id
    2. images
  4. Hotels Facilities - getHotelFacilities + getHotelFacilityTypes
    1. hotel_id
    2. Facilities
  5. Hotels Type - getHotels + getHotelTypes
    1. hotel_id
    2. Type
Question that arises here, how do we use multiple importers to fetch different data for the same node?
For that, I have used 1 importer to create nodes while all the others to update them.
Importer Hotels Basic Info creates new nodes while Hotels Description Info, Hotel Images, Hotels Facilities, Hotels Type.
But there’s a catch here - in Settings for Node processor under importers, there is an option for ‘Update existing nodes’ but even then if any non-existing unique hotel_id is found in the xml, feeds will create a new node for it. This becomes a problem when we haven’t fetched the basic node details like title, etc but on finding a new hotel_id by say importer Hotels Description Info, a new node gets created. But we only want these other importers to update and not create nodes.
What it does is add another option, Update existing nodes only (Do not create new nodes). Problem solved!

The next hurdle in this is, the unique field by which feeds imports data. This means in the data being imported, the value of this unique field will determine whether feeds creates a new node for the entry or updates an existing one.
By default feeds allows only a few fields to be declared unique in the importer mapping like - node_id, title, etc
However, in the data returned by Booking.com the unique field is hotel_id.
How do we use a custom field hotel_id as the unique field in our feeds importer mapping?
Solution: field_validation + patch for field_validation that works with another patch for feeds.

Too many patches huh? :)
Not to worry, these patches will help us get exactly what we want from feeds! Except that you might not be able to directly apply these patches as patch -p1 < file.patchor git apply file.patch and might have to manually change the code. Not an issue, as long as you make the changes, create patch out of them and keep them for your safety.

So what are we trying to achieve with these patches?
Our intention is to make the hotel_id field unique in importer mappings. For this -
  1. use field_validation module to add validation unique to hotel_id field
    1. provides option for using unique validated fields to be used as unique mapper in feeds importers
    2. when creating nodes uses the above declared unique validated field
Now, we have feeds importers for each API function that uses the hotel_id field as unique for creation of nodes. One importer creates nodes and all others update those nodes with different data as returned by the different Booking.com functions.

Now that we have all components ready, its time for some custom coding!
Under Approach we have covered #1.
Next mapping two xmls and using this mapped xml in the importers.

Here it is important to note what fetcher has been used for the importers -
  1. Hotels Basic Info - HTTP Fetcher
  2. Hotels Description Info - HTTP Fetcher
  3. Hotel Images - HTTP Fetcher
  4. Hotels Facilities - File upload
  5. Hotels Type - File upload
As you can see the importers which uses API functions that return direct data use HTTP Fetcher and those that require data mapping use File upload.

Now lets get down to some coding.

Automate execution of feeds importers

We don’t manually want to go and import each feed individually of-course! So we create functions that does that for us -
  1. function getHotelBasicInfo($city_id = NULL, $hotel_id =NULL, $username, $password, $test=0) {}

  1. function getHotelDescription($city_id = NULL, $hotel_id =NULL, $username, $password, $test=0) {}

  1. function getHotelPhotos($city_id = NULL, $hotel_id = NULL,$username, $password, $test=0) {}


Parameters -
* $city_id: (optional)the city id for which we are fetching the hotels
* $hotel_id: (optional)the hotel id of the hotel we are fetching
* $username: the affiliate username as provided by Booking.com
* $password: the affiliate password as provided by Booking.com

The 3 functions are pretty similar, so I’ll just get into one of them -
    

  1. function getHotelBasicInfo($city_id = NULL, $hotel_id = NULL, $username, $password, $test=0) {
  if($hotel_id == NULL && $city_id == NULL) {
    drupal_set_message(‘Must provide either hotel_id or city_id’, ‘error’);
    return;
  }
  $id_str = '';
  if($city_id)
    $id_str = 'city_ids=' . $city_id;
  elseif($hotel_id)
    $id_str = 'hotel_ids=' . $hotel_id;
  
  $offset_basic_info = 0;
  fetch_basic_info:
  {
    $basic_info_url = 'https://' . $username . ':' . $password . '@distribution-xml.booking.com/xml/bookings.getHotels?languagecodes=en&' . $id_str . '&show_test=' . $test . '&rows=1000&offset=' . $offset_basic_info;
    if(simplexml_load_file($basic_info_url)) {
      $basic_info_importer_id = 'hotels_basic_info';
      trigger_hotel_info_importer($basic_info_url, $basic_info_importer_id);
      $offset_basic_info += 1000;
      goto fetch_basic_info;
    }
  }
}
    2.
We just need to change the Booking.com url and the feeds importer id for the other two functions (and nomenclature of the other vairables depending on what is being fetched) -
  1. getHotelDescription() -
    1. $description_url = 'https://' . $username . ':' . $password . '@distribution-xml.booking.com/xml/bookings.getHotelDescriptionTranslations?descriptiontype_ids=6&languagecodes=en&' . $id_str . '&show_test=' . $test . '&rows=1000&offset=' . $offset;
    2. $importer_id = 'hotels_description_info';
  2. getHotelPhotos() -
    1. $photo_url = 'https://' . $username . ':' . $password . '@distribution-xml.booking.com/xml/bookings.getHotelPhotos?' . $id_str . '&show_test=' . $test . '&rows=1000&offset=' . $offset;
    2. $importer_id = 'hotel_images';

Now that we have created functions to trigger the feeds importers directly by sending the url, next we need to take care of those xmls returned by Booking.com functions that need mapping. This is because for some values, Booking.com has a set of predifined values and ids which it returns as an xml. And these ids are referenced by hotels, hence mapping is required.

First we need a function that will create a map the data from 2 xmls and create a file -
  1. function create_mapped_xml($mapping_for, $map_from,$map_from_field, $map_to, $map_unique, $tag_unique) {}


Parameters -
* $mapping_for: the field which needs to be mapped.
* $map_from: xml file destination/url link which contains the data to be mapped against
* $map_from_field: the field in the above xml which stores the data to be retreived by mapping
* $map_to: xml file destination/url link which contains the data to be mapped to
* $map_unique: the common or unique field between the above to xmls
* $tag_unique: the unique tag in the xml for the content type accomodation

  1. Create array from first xml which contains the mapping between id and name
  2. Create array from which contains tag unique id and its corresponding names
  3. Creates xml file from array from 2.

    

  1. function create_mapped_xml($mapping_for, $map_from, $map_from_field, $map_to, $map_unique, $tag_unique) {
  $map_from_array = array();
  $map_to_array = array();

  $map_from_xml = simplexml_load_file($map_from);
  foreach ($map_from_xml->result as $result) {
    $map_from_array[intval(strip_tags($result->$map_unique->asXML()))] = strip_tags($result->$map_from_field->asXML());
  }

  $map_to_xml = simplexml_load_file($map_to);
  foreach ($map_to_xml->result as $result) {
    $map_to_array[intval(strip_tags($result->$tag_unique->asXML()))][intval(strip_tags($result->$map_unique->;asXML()))] = $map_from_array[intval(strip_tags($result->$map_unique->asXML()))];
  }

  // Create the mapped xml here
  $xml = new SimpleXMLElement('<xml/>');
  foreach ($map_to_array as $hotel_id => $hotel_info_list) {
    $result = $xml->addChild('result');
    $result->addChild("$tag_unique", "$hotel_id");
    foreach ($hotel_info_list as $info_id => $info_name) {
      $result->addChild("$map_from_field", "$info_name");
    }
  }

  // Create the directory in files folder for storing the files.
  $scheme = file_default_scheme();
  $dir_path = $scheme . '://booking_com_hotel_import';
  file_prepare_directory($dir_path, FILE_CREATE_DIRECTORY);

  Header('Content-type: text/xml');
  $xml_file_name = t('hotel') . '_' . $mapping_for . '.xml';
  file_unmanaged_save_data($xml->asXML(), $scheme . "://booking_com_hotel_import/" . $xml_file_name, FILE_EXISTS_REPLACE);
}
    2.

This function will be used for getting -
  1. hotel type
  2. hotel facilities
First lets look at hotel types. None of the Booking.com API function returns the hotel type directly along with the hotel_id. Instead it returns a predefined list of hotel types along with the hotel type id. And the function that returns the hotel basic information contains this hotel type id. Hence, we need to map these 2 xmls for which I have used the above function.
    

  1. function getHotelType($city_id = NULL, $hotel_id = NULL, $username, $password, $test=0) {}
    2.
Parameters -
* $city_id: (optional)the city id for which we are fetching the hotels
* $hotel_id: (optional)the hotel id of the hotel we are fetching
* $username: the affiliate username as provided by Booking.com
* $password: the affiliate password as provided by Booking.com

    

  1. function getHotelType($city_id = NULL, $hotel_id = NULL, $username, $password, $test=0) {
  if($hotel_id == NULL && $city_id == NULL) { 
    drupal_set_message(‘Must provide either hotel_id or city_id’, ‘error’);
    return
  }
  $id_str = '';
  if($city_id)
    $id_str = 'city_ids=' . $city_id;
  elseif($hotel_id)
    $id_str = 'hotel_ids=' . $hotel_id;

  $mapping_for = 'acc_type';
  $map_from = 'https://'.$username.':'.$password.'@distribution-xml.booking.com/xml/bookings.getHotelTypes?languagecodes=en';
  $map_from_field = 'name';
  $map_to = 'https://' . $username . ':' . $password . '@distribution-xml.booking.com/xml/bookings.getHotels?languagecodes=en&' . $id_str . '&show_test=' . $test;
  $map_unique = 'hoteltype_id';
  $tag_unique = 'hotel_id';

  // Map the 2 returned xmls and create a new xml file
  create_mapped_xml($mapping_for, $map_from, $map_from_field, $map_to, $map_unique, $tag_unique);
  $type_url = file_default_scheme()."://booking_com_hotel_import/" . '/hotel_acc_type.xml';
  $type_importer_id = 'hotels_type';

  // After creating the mapped xml file trigger the corresponding feeds importer by passing the file url.
  trigger_hotel_info_importer($type_url, $type_importer_id);
}
    2.

As you can see in the function, we create the Booking.com API url for the 2 xmls -
  1. $map_from = file_default_scheme()."://booking_com_hotel_import/" . '/hotelTypes.xml';
  2. $map_to = 'https://' . $username . ':' . $password . '@distribution-xml.booking.com/xml/bookings.getHotels?languagecodes=en&' . $id_str . '&show_test=' . $test;
Define the field we are mapping - $map_from_field = 'name';
Define the unique tag between the 2 xmls on the basis of which the mapping will be done, which in this case is the hotel type id - $map_unique = 'hoteltype_id';
And finally the unique tag/id to be used in feeds importer - $tag_unique = 'hotel_id';
We pass these values to the function create_mapped_xml() which creates and saves the mapped xml file in files folder.
Lastly, we pass this file for more specifically the url to this file to the corresponding feeds importer - Hotels Type - as we trigger it to fetch the hotel type.

Similarly for hotel facilities.
    

  1. function getHotelFacilities($city_id = NULL, $hotel_id = NULL, $username, $password, $test=0) {
  if($hotel_id == NULL && $city_id == NULL) { 
    drupal_set_message(‘Must provide either hotel_id or city_id’, ‘error’);
    return;
  }
  $id_str = '';
  if($city_id) 
    $id_str = 'city_ids=' . $city_id;
  elseif($hotel_id)
    $id_str = 'hotel_ids=' . $hotel_id;
  
  $mapping_for = 'facilities';
  $map_from = 'https://'.$username.':'.$password.'@distribution-xml.booking.com/xml/bookings.getHotelFacilityTypes?languagecodes=en';
  $map_from_field = 'name';
  $map_to = 'https://' . $username . ':' . $password . '@distribution-xml.booking.com/xml/bookings.getHotelFacilities?' . $id_str . '&show_test=' . $test;
  $map_unique = 'hotelfacilitytype_id';
  $tag_unique = 'hotel_id';

  // Map the 2 returned xmls and create a new xml file
  create_mapped_xml($mapping_for, $map_from, $map_from_field, $map_to, $map_unique, $tag_unique);
  $facility_url = file_default_scheme()."://booking_com_hotel_import/" . '/hotel_facilities.xml';
  $facility_importer_id = 'hotels_facilities';

  // After creating the mapped xml file trigger the corresponding feeds importer by passing the file url.
  trigger_hotel_info_importer($facility_url, $facility_importer_id);
}
    2.

As in fetching hotel types, we create the Booking.com API url for the 2 xmls -
  1. $map_from = file_default_scheme()."://booking_com_hotel_import/" . '/hotelFacilityTypes.xml';
  2. $map_to = 'https://' . $username . ':' . $password . '@distribution-xml.booking.com/xml/bookings.getHotelFacilities?' . $id_str . '&show_test=' . $test;
Define the field we are mapping - $map_from_field = 'name';
Define the unique tag between the 2 xmls on the basis of which the mapping will be done, which in this case is the hotel type id - $map_unique = 'hotelfacilitytype_id';
And finally the unique tag/id to be used in feeds importer - $tag_unique = 'hotel_id';
We pass these values to the function create_mapped_xml() which creates and saves the mapped xml file in files folder.
Lastly, we pass this file for more specifically the url to this file to the corresponding feeds importer - Hotels Facilities - as we trigger it to fetch the hotel facilities.

Now we have 5 functions for getting 5 sets of data for each hotel -
  1. getHotelBasicInfo($city_id = NULL, $hotel_id = NULL, $username, $password, $test=0)
  2. getHotelDescription($city_id = NULL, $hotel_id = NULL, $username, $password, $test=0)
  3. getHotelPhotos($city_id = NULL, $hotel_id = NULL, $username, $password, $test=0)
  4. getHotelType($city_id = NULL, $hotel_id = NULL, $username, $password, $test=0)
  5. function getHotelFacilities($city_id = NULL, $hotel_id = NULL, $username, $password, $test=0)

Now the final data that remains to be imported is the rooms information. What I did was a custom module that -
  1. For given city ids or hotel ids, fetches corresponding nodes into an array of hotel id and nid.
  2. Fetches node objects using nids from #1.
  3. Calls Booking.com functions getRooms and getRoomPhotos with the provided city_ids/hotel_ids to get all room information for the hotels.
  4. Stores returned data in corresponding nodes as field collections(field_acc_room_info) only if a given room doesn't already exist.
However, just to keep it a little basic in this article, lets just keep ourselves to the above 5 functions and their related data and I’ll do the rooms import in another blog.

A single process for importing Booking.com hotels to my site

So, now that we have our 5 functions lets have a single function that will call all these functions -
    

  1. function getHotelInfo($city_id, $hotel_id, $test=0) {
  $username = variable_get('hotel_booking_affiliate_user', '');
  $password = variable_get('hotel_booking_affiliate_pass', '');
  if($username != '' && $password != '') {
    getHotelBasicInfo($city_id, $hotel_id, $username, $password, $test);
    getHotelDescription($city_id, $hotel_id, $username, $password, $test);
    getHotelPhotos($city_id, $hotel_id, $username, $password, $test);
    getHotelType($city_id, $hotel_id, $username, $password, $test);
    getHotelFacilities($city_id, $hotel_id, $username, $password, $test);
  }
}
    2.

Its a very simple function that accepts city_id and hotel_id and calls the feeds importer triggering functions in return.

What we need now is -
  1. a form for storing settings for the module, which will be the Booking.com affiliate username and password, and
  2. a form from where we can set the importing in motion

I believe you can take care of the #1 form ;)
variable_get('hotel_booking_affiliate_user', '') andvariable_get('hotel_booking_affiliate_pass', '')

so I’ll directly go to the #2.

    

  1. function add_accomodation_form($form, &$form_state) {
  if(variable_get('hotel_booking_affiliate_user') == '' || variable_get('hotel_booking_affiliate_pass') == '')  
    drupal_set_message('You have not set the username password for Booking.com. Import will not work. Please visit ' . l('admin/config/content/booking_hotel_import', 'admin/config/content/booking_hotel_import'));

  $form = array();
  $options_entry_type = array(
    'automatic_city' => t('Fetch from Booking.com using City'),
    'automatic_hotel' => t('Fetch from Booking.com using Hotel ID'),
  );
  $form['entry_type'] = array(
    '#type' => 'radios',
    '#title' => t('Entry Type'),
    '#options' => $options_entry_type,
    '#attributes' => array('class' => array('entry-type')),
    '#required' => TRUE,
  );
  $form['city_id'] = array(
    '#type' => 'textfield',
    '#title' => t('City ID'),
    '#attributes' => array('class' => array('booking-hotel-id')),
  );
  $form['hotel_id'] = array(
    '#type' => 'textfield',
    '#title' => t('Hotel ID'),
    '#attributes' => array('class' => array('booking-hotel-id')),
  );
  $form['test'] = array(
    '#type' => 'checkbox',
    '#title' => t('Testing'),
  );
  $form['submit'] = array(
    '#type' => 'submit',
    '#value' => t('Add Accomodation'),
  );
  return $form;
}


function add_accomodation_form_validate($form, &$form_state) {
  $entry_type = $form_state['values']['entry_type'];
  $city_id = $form_state['values']['city_id'];
  $hotel_id = trim($form_state['values']['hotel_id']);
  if ($entry_type == 'automatic_city') {
    if ($city_id == NULL || $city_id == '') {
      form_set_error('city_id', t('Please enter the city id'));
    }
    else {
      if(!is_numeric($city_id)) {
        form_set_error(city_id', t(City ID must be a number'));
      }
      elseif(is_numeric($city_id ) && $city_id < 0) {
        form_set_error(city_id', t(City ID must be a positive number'));
      }
    }
  }
  elseif ($entry_type == 'automatic_hotel') {
    if ($hotel_id == NULL || $hotel_id == '') {
      form_set_error('hotel_id', t('Please enter the hotel id'));
    }
    else {
      if(!is_numeric($hotel_id)) {
        form_set_error('hotel_id', t('Hotel ID must be a number'));
      }
      elseif(is_numeric($hotel_id) && $hotel_id < 0) {
        form_set_error('hotel_id', t('Hotel ID must be a positive number'));
      }
    }
  }
}

function add_accomodation_form_submit($form, &amp;$form_state) {
  $entry_type = $form_state['values']['entry_type'];
  $city_id = $form_state['values']['city_id'];
  $hotel_id = trim($form_state['values']['hotel_id']);
  $test = $form_state['values']['test'];
  if ($entry_type == 'automatic_city') { 
    getHotelInfo($city_id, NULL, $test);
  }
  elseif ($entry_type == 'automatic_hotel') {
    getHotelInfo(NULL, $hotel_id, $test);
  }
}
    2.

Wrap Up!

Phew! Lets just do a quick recap of what we did -
  1. Create a content type with relevant fields
  2. Create 5 feeds importers for fetching different data per hotel
    1. Add unique validation to hotel_id field
    2. Use patches to allow feeds to use above unique field
    3. Use patch to allow only update of nodes and not creation in feeds importers
    4. One feed to create new nodes, rest to only update them
  3. A custom function that maps between 2 xmls returned by Booking.com and stores the file in files folder
  4. 5 custom functions to trigger the 5 feeds importers - either with a url to a Booking.com xml or a file
  5. Single form to trigger importing the xmls feeds as nodes, by calling the functions from #4, using either city_id or hotel_id

There you go! Single form + a content type + multiple feeds importers + some custom code + some patches(from the community, yay! ^_^), your Booking.com affiliate username and passwor and one click of a button - and you have hotels from Booking.com imported to your site.

Please let me know in the comments if you have any queries :) And all the best with this!