Hacking¶
I’m not a fan of PHP. The language has many shortcomings that make it
difficult to write nicely encapsulated code. I didn’t even try but started the
project with one big index.php
file that included everything, and then
refractored some code later on into separate .php
files to prevent the file
from bloating above 1000 lines of code.
Overview¶
The file index.php
contains all the logic and structured into the following
sections (VIM can expand/collapse the corresponding code folds).
- Pre-initialization, includes : Things common to all instances.
- Set up instance : Find out form cookies or parameters which instance should be used and set up paths accordingly.
- Load configuration from
config.xls
into global$user
. See fileconfig.php
. - Login: Show the login box if the user is not currently logged in and compare
username and password to configuration options during login action. Then
load user information (access rights etc) into global
$user
. - Connect to database, load forms. The excel forms from the directory
instances/XXX/forms
are loaded and compared with the local ODK database. See file``odk_form.php``. - Actions : Do stuff, such as file upload or download. Note that so far no HTML output has been generated (if the user is logged in).
- Start HTML output
- Show the menu, depending on user’s access rights.
- Display : Generate the main output
show=overview
: Show the tabular overview over all data contained in the database. Seeoverview.php
.show=form
: Show the content of a formshow=forms
: List the contents of theinstances/XXX/forms
directory.show=admin
: Administrator view with upload/download ofconfig.xls
- Footer : Show some more information if
&test
is specified as a URL parameter and the current user hastest
rights.
The file cron.php
finally contains code that can be run without any user
interaction and is normally executed from the operating system’s job
scheduler.
Globals and Utilities¶
The following globals are noteworthy
$config
: AnExcelConfig
object – see fileconfig.php
. This object is generated from the fileinstances/XXX/config/config.xls
$user
: An array that is set to one of the values in$config->users
after successful login. With keysrights
andaccess
that describe this users access permissions.$forms
: AnOdkDirectory
object – see fileodk_form.php
. This object is generated from the filesinstances/XXX/forms/*.xls
. Every objectOdkForm
in the dictionary$forms->forms
contains information about the form as described in its.xls
file and can be used to read the data from the database. See Accessing the Database.$show
: The name of the current view (see above in Overview).$hooks
: Plugins use this object in order to install hooks.
And some utility functions
log_add($name, $message)
: Adds a message to the specified log file.alert($html, $class)
: Displays the html snippet inside an alert box with the given class (success, info, danger, error).profile_start($name)
andprofile_stop($name)
: Measure time spent for$name
(can be called multiple times, e.g. inside a recursive function). The footer displays the total time spent in every$name
(if the&test
URL parameter is specified).
Accessing the Database¶
The file odk_form.php
provides methods to access data in the ODK database
using identifiers from .xls files that were used as input to XLSForm. Please
use the file’s API doc for reference.
Developing a Plugin¶
odk_planner
comes with a simple plugin called doughnut
that adds a page
with overview plots of data in the database. This section will walk you step
by step through the creation of the doughnut plugin
that is the file plugins/doughnut.php
. The four steps outlined below
build successively on top of each other and introduce every time some new API to
give a quick an dirty introduction to the internals described above. Note that
for every step the whole source code of the plugin is included. Overwrite the
file plugins/doughnut.php
with the source listing from the different steps
to see what changes and play around with the code.
The idea of this plugin is pretty simple : Specify fields and values in the
configuration (sheet doughnut
) and let the plugin generate doughnut plots
that show the distribution of the field of interest over the whole study
population.
Step 1 : Hooking¶
The plugin needs to hook into the normal program flow at multiple points. First
it needs to include the javascript plotting library chart.js into the header of
the html page. Then it needs to change the menu (“augment the views”) with a
new menu entry doughnut
. And finally it needs to render the plots when the
view doughnut
is active.
The file plugins.php
lists some 10 different hooks together with an
explanation when they are called and the arguments. In the code below we use
the three hooks dump_headers
(to add chart.js), augment_views
(to add
the new menu point), and display
(to render a static doughnut plot).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 | <?php
function doughnut_dump_headers($args) {
echo '<script src="plugins/doughnut_chart.js"></script>';
}
function doughnut_display($args) {
global $show;
if ($show !== 'doughnut') return;
?>
<div class="row">
<br><br>
<div class="span12">
<center>
<canvas id="doughnut" style="width:100%"></canvas>
</center>
<h4 style="text-align:center">doughnut</h4>
<script type="text/javascript">
// Data copied from www.chartjs.org/docs/#doughnut-pie-chart
new Chart(document.getElementById('doughnut').getContext('2d'))
.Doughnut([{ value: 300, color:"#F7464A", highlight: "#FF5A5E", label: "Red" }, { value: 50, color: "#46BFBD", highlight: "#5AD3D1", label: "Green" }, { value: 100, color: "#FDB45C", highlight: "#FFC870", label: "Yellow" }]);
</script>
</div>
</div>
<?php
}
function doughnut_augment_views($args) {
$views =& $args['views'];
array_push($views, 'doughnut');
}
$hooks->register('dump_headers', 'doughnut_dump_headers');
$hooks->register('augment_views', 'doughnut_augment_views');
$hooks->register('display', 'doughnut_display');
|
Step 2 : Access¶
We don’t want just any user to see our plugin, but only users with the
access
right (as described in user configuration).
This is easily implemented by checking for the right in the $user
global
that contains the configuration for the currently logged in user. We add the
check before rendering the plot but also when the menu is constructed so users
without the needed access right will neither see the plugin in the menu nor be
able to access the plot by manually tweaking the URL.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | <?php
function doughnut_dump_headers($args) {
echo '<script src="plugins/doughnut_chart.js"></script>';
}
function doughnut_display($args) {
global $show, $user;
if (!in_array('data', $user['rights'])) return;
if ($show !== 'doughnut') return;
?>
<div class="row">
<br><br>
<div class="span12">
<center>
<canvas id="doughnut" style="width:100%"></canvas>
</center>
<h4 style="text-align:center">doughnut</h4>
<script type="text/javascript">
// Data copied from www.chartjs.org/docs/#doughnut-pie-chart
new Chart(document.getElementById('doughnut').getContext('2d'))
.Doughnut([{ value: 300, color:"#F7464A", highlight: "#FF5A5E", label: "Red" }, { value: 50, color: "#46BFBD", highlight: "#5AD3D1", label: "Green" }, { value: 100, color: "#FDB45C", highlight: "#FFC870", label: "Yellow" }]);
</script>
</div>
</div>
<?php
}
function doughnut_augment_views($args) {
global $user;
if (!in_array('data', $user['rights'])) return;
$views =& $args['views'];
array_push($views, 'doughnut');
}
$hooks->register('dump_headers', 'doughnut_dump_headers');
$hooks->register('augment_views', 'doughnut_augment_views');
$hooks->register('display', 'doughnut_display');
|
Step 3 : Configuration¶
This snippet iterates through the configuration key/value pairs in the lines
54-63
. Every value is parsed with the new function doughnut_config
(by the way : it’s good practise to prepend all functions in the plugin with
the plugins name to avoid name space collisions). If an error occurs during the
parsing (quite possible since Excel lets you save just about any invalid setting
imaginable) the doughnut is not rendered for that row and an error is displayed
to the user.
The function doughnut_config
expects value that specifies the field of
interest and the form this field can be found in (FORM\FIELD
). The form is
then looked up in the global associative array $forms->forms
and the
specified field is verified to exist in the $form->mapping
(read more
above). The name of the MySQL table and column that
represent the field are then returned and used in the next step to extract the
data from the MySQL database.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 | <?php
function doughnut_dump_headers($args) {
echo '<script src="plugins/doughnut_chart.js"></script>';
}
function doughnut_render($name, $config) {
$id = 'doughnut_' . $name;
?>
<div class="row">
<br><br>
<div class="span12">
<center>
<canvas id="<?php echo $id; ?>" style="width:100%"></canvas>
</center>
<h4 style="text-align:center"><?php echo $name; ?></h4>
<script type="text/javascript">
// Data copied from www.chartjs.org/docs/#doughnut-pie-chart
new Chart(document.getElementById('<?php echo $id; ?>').getContext('2d'))
.Doughnut([{ value: 300, color:"#F7464A", highlight: "#FF5A5E", label: "Red" }, { value: 50, color: "#46BFBD", highlight: "#5AD3D1", label: "Green" }, { value: 100, color: "#FDB45C", highlight: "#FFC870", label: "Yellow" }]);
</script>
</div>
</div>
<?php
}
function doughnut_config($config) {
global $forms;
// We only expect one part but allow more values for later versions.
$parts = explode(' ', $config);
if (count($parts) < 1) return 'expected "FORM\\FIELD ..."';
$form_field = explode('\\', $parts[0]);
if (count($form_field) !== 2) return 'expected "FORM\\FIELD ..."';
// $forms->forms is array that maps formid to OdkForm (see odk_form.php).
$form = @$forms->forms[$form_field[0]];
if (!$form) return 'unknown form "' . $form_field[0] . '"';
// OdkForm::mapping maps form field to table/column (see odk_form.php).
$table_column = @$form->mapping[$form_field[1]];
if (!$table_column) return 'unknown field "' . implode('\\', $form_field) . '"';
return array(
'table' => $table_column[0],
'column' => $table_column[1],
);
}
function doughnut_display($args) {
global $show, $user, $config;
if (!in_array('data', $user['rights'])) return;
if ($show !== 'doughnut') return;
// Parse doughnut config lines.
foreach($config->plugins['doughnut'] as $name=>$config_string) {
$config_or_error = doughnut_config($config_string);
if (gettype($config_or_error) === 'string') {
// Function returns string in case of parse error.
alert('Error config doughnut ' . $name . ' : ' .
$config_or_error, 'error');
} else {
doughnut_render($name, $config_or_error);
}
}
}
function doughnut_augment_views($args) {
global $user;
if (!in_array('data', $user['rights'])) return;
$views =& $args['views'];
array_push($views, 'doughnut');
}
$hooks->register('dump_headers', 'doughnut_dump_headers');
$hooks->register('augment_views', 'doughnut_augment_views');
$hooks->register('display', 'doughnut_display');
|
Step 4 : Data¶
Having all the necessary hooks, the doughnut plot, some access control and the configuration parsing in place, the only thing that need to be done is to connect the plot to the actual data from the database.
The function doughnut_query
constructs a MySQL query using the MySQL table
and column name. The query has the following form:
SELECT column AS value, COUNT(column) AS count
FROM table GROUP BY column ORDER BY column
where column
and table
will be replace with the actual values. If the
query were generated for the field LRF1\COLONY_COUNT
from the example
dataset, the following table would be the reply from the query:
value | count |
---|---|
negative | 3 |
1+ | 2 |
2+ | 3 |
3+ | 2 |
The function doughnot_json
merely translates the MySQL result to a JSON
in the format expected by chart.js. Note that we call mysql_query_
so the
query gets logged (calling the standard mysql_query
would also work fine).
If MySQL returns an error no plot is displayed but the error is shown to the
user by calling alert
(from util.php
).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 | <?php
function doughnut_dump_headers($args) {
echo '<script src="plugins/doughnut_chart.js"></script>';
}
function doughnut_query($config) {
$query = 'SELECT $column AS value, COUNT($column) AS count ' .
'FROM $table GROUP BY $column ORDER BY $column';
$query = str_replace('$table', $config['table'], $query);
$query = str_replace('$column', $config['column'], $query);
return $query;
}
function doughnut_json($curs) {
$data = array();
$colors = ['red', 'green', 'blue', 'yellow', 'magenta', 'purple', 'orange'];
$i = 0;
while($row = mysql_fetch_assoc($curs)) {
array_push($data, array(
'value' => $row['count'],
'label' => trim($row['value']),
'color' => $colors[$i++ % count($colors)]
));
}
return json_encode($data);
}
function doughnut_render($name, $config) {
global $conn;
$id = 'doughnut_' . $name;
$query = doughnut_query($config);
$curs = mysql_query_($query, $conn);
if ($curs === FALSE) {
alert('MySQL error : ' . mysql_error(), 'error');
return;
}
$json = doughnut_json($curs);
?>
<div class="row">
<br><br>
<div class="span12">
<center>
<canvas id="<?php echo $id; ?>" style="width:100%"></canvas>
</center>
<h4 style="text-align:center"><?php echo $name; ?></h4>
<script type="text/javascript">
// See www.chartjs.org/docs/#doughnut-pie-chart
new Chart(document.getElementById('<?php echo $id; ?>').getContext('2d'))
.Doughnut(<?php echo $json; ?>);
</script>
</div>
</div>
<?php
}
function doughnut_config($config) {
global $forms;
// We only expect one part but allow more values for later versions.
$parts = explode(' ', $config);
if (count($parts) < 1) return 'expected "FORM\\FIELD ..."';
$form_field = explode('\\', $parts[0]);
if (count($form_field) !== 2) return 'expected "FORM\\FIELD ..."';
// $forms->forms is array that maps formid to OdkForm (see odk_form.php).
$form = @$forms->forms[$form_field[0]];
if (!$form) return 'unknown form "' . $form_field[0] . '"';
// OdkForm::mapping maps form field to table/column (see odk_form.php).
$table_column = @$form->mapping[$form_field[1]];
if (!$table_column) return 'unknown field "' . implode('\\', $form_field) . '"';
return array(
'table' => $table_column[0],
'column' => $table_column[1],
);
}
function doughnut_display($args) {
global $show, $user, $config;
if (!in_array('data', $user['rights'])) return;
if ($show !== 'doughnut') return;
// Parse doughnut config lines.
foreach($config->plugins['doughnut'] as $name=>$config_string) {
$config_or_error = doughnut_config($config_string);
if (gettype($config_or_error) === 'string') {
// Function returns string in case of parse error.
alert('Error config doughnut ' . $name . ' : ' .
$config_or_error, 'error');
} else {
doughnut_render($name, $config_or_error);
}
}
}
function doughnut_augment_views($args) {
global $user;
if (!in_array('data', $user['rights'])) return;
$views =& $args['views'];
array_push($views, 'doughnut');
}
$hooks->register('dump_headers', 'doughnut_dump_headers');
$hooks->register('augment_views', 'doughnut_augment_views');
$hooks->register('display', 'doughnut_display');
|
Step 5 : Bucketing¶
Now compare the source from the last step with the plugin as included in the
plugins/doughnut.php
(if you overwrite it get it back from github:
doughnut.php). The configuration was extended to list possible values after
FORM\FIELD
. If these values have the form of a range of numbers (e.g.
10-20
) then the MySQL query will summarize the values of some ordinal
datapoint into the buckets thus specified.
By the way : there is also a simple test suite for this plugin. The file
test/test_doughnut.py
checks that the access restrictions work as expected,
that the configuration parsing alerts user if invalid values are specified, and
that a new plot can be added modifying the configuration.