Import availability data for Availability Calendars from an external source with Migrate
The Availability Calendars is quite frequently used by organizations providing accomodation, or for material renting. Recently, I had to import such data from an external system (as it happens, from a Tourinsoft stream, a technology quite common in France in the tourism industry), and convert them to data intelligible to Availability Calendars, using Migrate for Drupal 7 (see my presentation on data migration with Drupal, in French). As this task proved to be harder than expected, and there is currently no documentation on the topic, I intend to expose a solution in this blog post. General knowledge about Migrate is expected, do not hesitate to refer to the official documentation, which is clearly a strength of this module.
Field Handler
Most of the time, it is enough to specify a destination and a field mapping so that Migrate imports as if magic your data into the destination entity. Sadly, here we have to deal with complex data, which Migrate will not know how to translate into something intelligible for the target field. This is the role of a field handler; we are forced to write one for the availability_calendar field type.
This class must be registered by Migrate; to that end, in your module's hook_migrate_api() in MODULE.migrate.inc, provide the following metadata:
/**
* Implements hook_migrate_api().
*/
function MODULE_migrate_api() {
$api = array(
'api' => 2,
// [...]
'field handlers' => array(
'AvailabilityCalendarFieldHandler',
),
);
return $api;
}
In Drupal 7, this class must be automatically loaded by adding this line in your .info file:
files[] = migrate_field_handlers.inc
Here are its contents:
<?php
/**
* @file
* Custom Migrate field handlers.
*
* @see https://www.drupal.org/node/1429096.
*/
/**
* Migrate availability data to availability_calendar fields.
*/
class AvailabilityCalendarFieldHandler extends MigrateFieldHandler {
/**
* Register availability_calendar type.
*/
public function __construct() {
$this->registerTypes(array('availability_calendar'));
}
/**
* Prepare data accordingly for the field API.
*/
public function prepare($entity, array $field_info, array $instance, array $values) {
module_load_include('inc', 'availability_calendar', 'availability_calendar.widget');
// Les deux lignes suivantes sont reprises de l'exemple et ne servent à
// rien dans l'état actuel des choses, ne cherchez pas.
$migration = Migration::currentMigration();
$arguments = (isset($values['arguments'])) ? $values['arguments'] : array();
$language = $this->getFieldLanguage($entity, $field_info, $arguments);
$delta = 0;
$return = array();
// Pour compatibilité avec le mode fonctionnement d'Availability Calendars.
$element = array(
'availability_states' => array(
'#options' => availability_calendar_get_states(),
),
'availability_calendar' => array(
'#settings' => $instance['widget']['settings'],
),
'#field_name' => $instance['field_name'],
'#language' => $language,
);
foreach ($values as $value) {
$element['#delta'] = $delta;
$cid = $this->prepareCalendar($value, $element, $entity);
if (!$cid) {
return;
}
$return[$language][$delta] = array(
'enabled' => 1,
'name' => '',
'cid' => $cid,
);
$delta++;
}
return $return;
}
/**
* Forcer la création de l'entité embarquant les disponibilités.
*
* L'approche « hybride » d'Availability Calendars ne permet pas de se
* contenter des seules fonctionnalités habituelles de la Field API, il faut
* en même temps forcer la création du calendrier.
*
* @see availability_calendar_field_widget_month_form_validate()
* @see availability_calendar_field_attach_submit_inc()
*/
protected function prepareCalendar($dates, $element, $entity) {
$changes = array();
$lines = explode("\n", $dates);
foreach ($lines as $line) {
if (!empty($line)) {
$change = availability_calendar_field_widget_month_form_validate_line($line, $element);
if ($change == FALSE) {
watchdog('tourinsoft', "Disponibilités d'hébergement invalides : %line", array('%line' => $line), WATCHDOG_ERROR);
return FALSE;
}
$changes[] = $change;
}
}
static $new_cid_count = 0;
$cid_unique = 'new' . ++$new_cid_count;
$field_name = $element['#field_name'];
$language_code = $element['#language'];
$delta = $element['#delta'];
$cid = availability_calendar_update_multiple_availability((int) $cid_unique, $changes);
if (!isset($entity->{$field_name}[$language_code][$delta]['cid']) || $cid != $entity->{$field_name}[$language_code][$delta]['cid']) {
// Nouveau calendrier : mettre à jour le champ.
$entity->{$field_name}[$language_code][$delta]['cid'] = $cid;
}
return $cid;
}
}
An explanation might be required:
- this class must inherit from MigrateFieldHandler, which provides the general framework to accomplish this task. This is customary when using Migrate, nothing strange up until now.
- In the constructor, we specify the field types managed by this class.
- The prepare method acts on data before they are handed to the field API. The goal is to format them as an array used by the latter (the $return array).
- We assume we previously managed (see the second part of this post) to format data as intervals of this kind (this is actually the format generated when we use the form provided by the Availability Calendars Module):
These are actually availability changes expressed in relation to the previous state of the calendar (the default state being not communicated). The state code is in the first column (columns are separated by commas); those are the states specified in the availability_calendar_state table when the module is installed. Beware that this data is dynamic (see interface at admin/config/content/availability-calendar). Here, 2 means available and 3 booked.2,2016-01-21,2016-01-23 4,2016-02-12,2016-02-27 3,2016-03-10,2016-04-24 1,2016-06-09,2016-06-11 - The difficulty lies in the fact that the author of Availability Calendars, as explained in the README, opted for a hybrid approach, which relies on a field coupled with an entity (which actually contains availability data). This is the reason why we first create a calendar entity in the prepareCalendar() method and retain its ID, which will be recorded in the field—a small API deviation. Unfortunately, there is apparently no API to create a calendar from outside an entity or node edit form, hence the duplication of some code present in availability_calendar_field_widget_month_form_validate() and availability_calendar_field_attach_submit_inc().
- Note that this field handler will not import expired intervals, due to choices made in the availability_calendar_field_widget_month_form_validate_line() function called from the prepareCalendar() method: altering this behaviour would entail more code duplication.
Integrate to the Migration Class
Once this handler is implemented, you still need to tell your migration class how to make use of it to extract the data and convert them to calendars.
In your migration class constructor:
- you have to add the source field which contains availability, in my case:
Nothing has to be changed as for the destination (a node is a node, nothing we did is going to alter that), nor the mapping (chances are we already use an ID, totally unrelated to the data we are dealing with).$ts_fields = array( // [...] 'PLANNINGETATS' => 'Accommodation availability', ); $this->source = new MigrateSourceList(new TourinsoftListJSON($json_file), new TourinsoftItemJSON($json_file), $fields); - However, we still have to map the source field with the destination field:
$this->addFieldMapping('field_ts_disponibilites', 'PLANNINGETATS');
This is all very well, but obviously such data are quite likely to be in a very different format in the source from the format expected by our field handler (see above). In my case, Tourinsoft stores data as as string containing as many characters as days in a year, each character representing availability for the corresponding day in the year (in a positional way, the first character stands for January 1, the second January 2, and so on until the last one which stands for December 31, beautifully handled, isn't it?). With Migrate, we can act in the migration class prepareRow() method:
/**
* Agir sur le format des champs à la source avant la migration.
*/
public function prepareRow($row) {
parent::prepareRow($row);
// [...]
$this->prepareAvailabilityStates($row);
}
/**
* Conversion des états de disponibilité des hébergements.
*/
protected function prepareAvailabilityStates($row) {
if (!isset($row->PLANNINGETATS)) {
return;
}
// Correspondance des codes des états des réservations, cf. table
// availability_calendar_state.
$availability_states_codes = array(
// N = non communiqué, assimilé à 1 = « Not communicated ».
'N' => 1,
// D = disponible, correspond à 2 = « Available ».
'D' => 2,
// C = complet, correspond à 3 = « Fully booked ».
'C' => 3,
);
// On assimile l'état F = fermé à C = complet.
$row->PLANNINGETATS = str_replace('F', 'C', $row->PLANNINGETATS);
// TODO : généraliser à plusieurs années quand on aura l'information.
list($year, $states) = explode(':', $row->PLANNINGETATS);
// Récupérer les intervalles de disponibilités au format Tourinsoft, par
// exemple FFF, D et F pour une chaîne de départ telle que NNNFFFDNFN.
$states_matches = array();
preg_match_all('/([^N])\1*/', $states, $states_matches, PREG_OFFSET_CAPTURE);
// Convertir les intervalles dans un format compréhensible pour le type de
// champ availability_calendar. Ce dernier prend en fait en entrée les
// changements, exprimés sous cette forme :
// 2,2016-01-21,2016-01-23
// 4,2016-02-12,2016-02-27
// 3,2016-03-10,2016-04-24
// 1,2016-06-09,2016-06-11.
$row->PLANNINGETATS = '';
foreach ($states_matches[0] as $state_match) {
$state = $state_match[0][0];
$day1 = $state_match[1] + 1;
$day2 = $day1 + strlen($state_match[0]) - 1;
if (!isset($availability_states_codes[$state])) {
// Code de disponibilité invalide.
continue;
}
$row->PLANNINGETATS .= sprintf("%d,%s,%s\n",
$availability_states_codes[$state],
date('Y-m-d', mktime(0, 0, 0, 1, $day1, (int) $year)),
date('Y-m-d', mktime(0, 0, 0, 1, $day2, (int) $year))
);
}
}
Now cross fingers and launch a migration to see where your code is going to break first!