Interactive “user guide” for FESTIVAL Experimenters

This user guide is dedicated to Experimenters discovering FESTIVAL Experimentation Portal.

Download a PDF version here: Userguide (v1.3).


Experiment Workflow

Firstly, in case the user has no account in the FESTIVAL platform, he/she must create a new account. This account will be used to access to the Experimentation Portal. As described, the provided username and password details will be later used to authenticate himself into the platform.
Once the user has an account in the platform, he/she will be able to access to the functionalities of FESTIVAL platform by two means:

  • Using directly the API to perform the experiments programmatically.
  • Using the portal to explore the resources and create the experiments.

The Experimentation Portal provides an easier way to explore resources and to create and manage the experiments, even if EaaS APIs allows the user to carry out more complex tasks for advanced experimentation; for instance, it is possible to build on top of EaaS APIs dedicated applications.
Each experiment performed using the platform is composed by several phases, as described below:

  • The first phase consists on the creation of the experiment. In this phase the user will be able to create the experiment and include its details.
  • The second phase is about the exploration of the resources in the platform. long as the user finds an appropriate resource for his/her experiment, he/she will be able to add the resource to it.
  • Once the resources are chosen, the experiment can start. This phase is important as long as the resources will be reserved (for instance, in case of the actuators and VMs) for the period of the experiment lifetime.
  • During the execution of experiment, the FESTIVAL platform provides the experimenter with several tools to support experimentation. Among these tools, it is possible to find one for the gathering of the experiment results. With the OML library, the FESTIVAL platform will allow each experimenter to gather the data he/she considers useful.
  • Additionally, the experiment details, including the experimentation period, can be modified. However, it will also depend on the rest of experimenters accessing to the platform.
  • Finally, the experiment can finish under two circumstances:
    • The experiment defined period is finishing and the user did not increase it. In this case, the FESTIVAL platform will notify the user about this at least one week before the final date.
    • The experiment is finished and the user wants to end it. In this case the user is able to stop it through the portal or EaaS APIs

Experimentation portal

Experimentation Portal is a web application dedicated to users of FESTIVAL platform, and provides them with functionalities offered by the platform itself. FESTIVAL project provides experimenters with a wide number of resources (sensors, virtual machines, open data sources etc.), and those resources are made available also through the Experimentation Portal.
From a user point of view, the portal is a web application, running in any standard web browser. Its responsive design makes it also usable from other kind of devices, for instance tablets.
As the first step, an experimenter willing to use the portal needs to create an account. Once the account is created and the user logged in, he/she can browse and search resources and create experiments. The creation of experiments allows to allocate resources for a pre-defined time period and then, when the experiment is started, the user can use them.

Click on the sections below for further details on the Experimentation Portal functioning.

Security: account creation and login

The account creation for the FESTIVAL platform is done through the Experimentation Portal. Thus, a specific webpage is dedicated for such purpose. The access to the account creation webpage can be done accessing to the Experimentation Portal main webpage, which redirects to the login webpage. Once it is loaded, in case the user does not have already an account, he/she must create a new one.
To create a new account on the Experimentation Portal, the experimenter needs to introduce the username (that will be used later to login or get the token), the password he/she wants to use, his/her full name and a contact email address. After the creation of the new account, the Experimentation Portal will show a web page to provide a public and a private key for the new user, needed to access IT resources such as Virtual Machines via an SSH connection. The credentials are based on public/private key certificates, and due to privacy and security purposes, the platform stores only the public key. Therefore, it is important to download such credentials before continuing with the use of the platform. These keys must be stored locally into the experimenter computer, making sure that they are recognised as public/private keys in the user’s favourite SSH client.
Finally, the user will be able to log into the portal using his/her credentials provided during the registration process and start exploring the platform, the available resources and create experiments.

Create experiment

The experiment creation is a straight forward process. From the left-side menu, the user can access the “Experiment” area (this page can also be reached from the home page). From there, the user access all his/her experiments and can create new one.
On the right side, a table gives an overview about the status of experiments of the user, e.g. number of running, created, stopped experiments. The experiments list gives an overview of experiment with title, dates, status and number of allocated resources. Then for each experiment, the user can access its details (and from there edit the details), “set as current” experiment or delete it.
When the user “set as current” an experiment, a label informing the user about the “current experiment” appears on top of the pages of Experimentation Portal and then the experimenter can allocate resource to the experiment itself. Experimenter can also unset the current experiment by clicking on the button “Unset”.

To create a new experiment, the user needs to click on the “Create a new experiment” button and fill the simple form, all required data are mandatory: the name of the new experiment, its description and the start date and end date of the experiment. Once created, the experiment is added in the list of experiments associated to the user, and the experimenter can starts allocating resources.
An experimenter can own at most four experiments at the same time.

Browse resources

The browse resources feature allows experimenter to navigate through the FESTIVAL resources. Indeed, in that way the experimenter accesses first the list of aggregators of FESTIVAL platform; four distinct aggregators exist: IoT, IT, Living Labs, Open Data.
Then, after having selected an aggregator, the experimenter accesses the list of testbeds which are associated to the selected aggregator.
Finally, the experimenter can access resources made available by selected testbed and he/she can require their details.

Search resources

The search resource feature allows the user to perform an advanced search on all FESTIVAL resources. Indeed, when navigating on the search page, a search form is shown on the left side with all criteria those can be used to perform a search.

As a first step, the user needs to indicate an aggregator and select testbeds on which the search should be performed. Then, the experimenter has the possibility to specify a geographical position and radius; when filling the geographical location, an auto-completion mechanism shows location matching the request. Next fields allow to obtain even more precise results by indicating keywords for title and description of resources and a regular expression. Moreover the experimenter can choose the order for obtained results.
After having browsed or searched resources, the user can access their details by clicking on the “Details” button: name, type, testbed, URL, geographical location, availability, actions to be executed on the resources, etc.

Allocate resources

To use resources, experimenters need to allocate them to an experiment previously created. In such context, the first step is to set an experiment as “current experiment”. This is possible by clicking on “set as current” button available in list of experiments of the experimenter. When an experiment is set as current, an orange button appears in the portal header to indicate which experiment is selected.
Now, when an experimenter browses or searches resources, he/she has the possibility to allocate resources to the current experiment (or to remove them if already allocated). In both cases, the current experiment is updated and the page header is also updated accordingly (to show the new number of allocated resources).

Start experiment

After the selection of all the necessary resources, the experimenter is able to start the experiment. In the page reporting the details of the experiment, the experimenter can see the status of the experiment (created) and the list of resources associated.

When the experimenter click on Start button the platform performs all the operations necessary to enable the experimenter to use resources associated to his/her experiment; this process is named “resource reservation”. At the end of this process, the experiment is running and a green alert box informs him/her that the operation is successfully completed and the status of the experiment changes from created to running.
It is important to underline that for Open Data, IT and IoT resources the reservation process is completely automatic, whereas for Living Lab resources the reservation process has to be completed via a direct interaction between the experimenter and the person in charge to manage the Living Lab. Since Living Lab activities involve people, this interaction is essential to define all the activities of a Living Lab in the experimentation.
Moreover, an additional comment it is necessary for IT resources; an IT resource associated to an experiment (i.e. virtual machine), before the starting of the experiment, is the template of a VM.


When the experiment starts, the template of the VM is replaced by the corresponding instance. Therefore, in addition to the change of the experiment status (from created to running), the experimenter can also see that the IT resource associated to the experiment now is the instance of virtual machine and not the template.

When the experiment is running, the experimenter can access to resources in different way:

  • For Open Data and IoT resources, he/she can use the information provided in detail page of the resource to retrieve data (e.g. by using the Current Data URL).
  • For Living Lab resources, since they can have different nature (services, expertise, methodologies and stakeholders) the access depends on the particular situation and it is not provided in a unique way; experimenter have to contact the Living Lab.
  • For IT resource, once the instance of the VM is created, the experimenter, from the details page of IT resource, experimenter can visualise the IP address assigned to the VM (Resource URL parameter). This information, in addition to the private key downloaded at the end of registration process, allows to access to the VM via a SSH connection.


During execution of the experiment, it is possible to use VM to run software or tools necessary to experimentation and to collect measurements related to the experiment (for instance sensor data, network data, etc.).

Access allocated IT resources (Virtual Machines)

It is possible to access VM associated to a running experiment through its Resource URL representing the public IP address of the VM itself.
To access the VM it is necessary to open an SSH console and authenticate using the personal private key downloaded after the registration.

For instance, if you use a Linux base OS you can connect by using the command ssh:

ssh -i /path/my-key-pair.pem IP_ADDRESS

The private key provided by FESTIVAL platform is in .pem format; if the program used to open the SSH console does not support this format, it is necessary to convert the key in the appropriate format. For instance, Putty http://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html uses the .ppk format. If you use Putty you can convert the .pem file in the .ppk by using the tool PuttyGen.

Once the key is converted in the .ppk format you can set the public IP address of the VM and your private key in the configuration parameters of Putty.

 

Once the ssh console is opened, to complete the login it is necessary to insert the username of the default user of the VM, that it is not the username of the experimenter. Typically, username to access the VM depends on the OS installed on it (e.g. for Ubuntu based machine the username is ubuntu).


Collect experiment results

When an experiment is running, it is possible to collect measurements and data from a VM instantiated in one of the testbeds provided by IT Aggregator; FESTIVAL platform provides an OML Server accessible from these VMs.
In order to correctly collect measurements and associate them to an experiment, it is necessary to configure an OML client appropriately. Different version of OML clients are publicly available; some examples are : OML4Py, OML4R, OML4J, OML4JS and OML4Node.
Moreover, testbeds of IT Aggregator provides templates of VMs (e.g. OMLCollectorVM – m1.small – Template(KHN)) preconfigured with a sample script based on OML4Node, an OML client for NodeJS. This script represents an example on how it is possible to interact with OML Server provided by FESTIVAL platform to collect measurements through an OML Client.
In particular, the script is organized in sections providing configuration to execute it and to access resources from which data are retrieved.

Because the script is based on a time-based job scheduler, the first section provides the possibility to configure it:

// Change this parameter to schedule the job. (E.g. the value "* /30 * * * *" will run the job every 30 minutes).
var cronTime = "* * * * * *";

In the second section it is possible to configure the resources from which retrieve the data:

var resources = [

{resourceName:"HyogoTemp", resourceUrl:"https://api.festival-project.eu/festival/eaas/experimentation/aggregators/IOT-0/testbeds/jose/resources/hyogo001_temperature-info-valueasfloat/current_data"},
{resourceName:"KyotoTemp", resourceUrl:"https://api.festival-project.eu/festival/eaas/experimentation/aggregators/IOT-0/testbeds/jose/resources/kyoto001_temperature-info-valueasfloat/current_data"},
{resourceName:"HyogoHumidity", resourceUrl:"https://api.festival-project.eu/festival/eaas/experimentation/aggregators/IOT-0/testbeds/jose/resources/hyogo001_humidity-info-valueasfloat/current_data"},
{resourceName:"KyotoHumidity", resourceUrl:"https://api.festival-project.eu/festival/eaas/experimentation/aggregators/IOT-0/testbeds/jose/resources/kyoto001_humidity-info-valueasfloat/current_data"}

];

Each element of the array resources is composed by two fields:

  • resourceName: useful to identify the resource in the script (each symbolic name must be unique).
  • resourceUrl: URL to retrieve the latest data produce by the specific resource. ResourceUrl represents the EaaS API to get the current data of a resource, so the authentication is required. For each resource this URL is provided in its detail page of the Experimentation Portal in “Current data URL” field.

In the third section of the script it is possible to setup the authentication parameters:

var userName = ‘john.doe@company.com'; // Your user name on Festival

var userPass = ‘john'; // Your user password on Festival

In the fourth section, it is possible to configure connection parameter to OML server and the schema of measurements to collect data:

// OML Configuration 

//omlAppName is the title of your measurement NOTE: blank spaces,upper case letter (A-Z)  and special chars (except "_") are not allowed due to PostgreSQL restrictions

//Please see at https://www.postgresql.org/docs/9.3/static/sql-syntax-lexical.html#SQL-SYNTAX-IDENTIFIERS

omlAppName = 'data_collection';
omlDomain = ‘experiment ID';   // Your experiment ID; blank space are not allowed
omlHost = 'festival.ckp.jp'; // Do not change it
omlPort = 1234; // Do not change it
omlCollect = 'tcp:festival.ckp.jp:1234'; // Do not change it
omlMeasurementTitle = 'hyogo_kyoto_temp_humidity';
omlMeasurementSchema = [['hyogo_temp', oml.T.double], ['kyoto_temp', oml.T.double], ['hyogo_humidity', oml.T.double], ['kyoto_humidity', oml.T.double]];

Configuration parameter of OML are:

  • omlAppName: this is the title of your measurement.
  • omlDomain: this corresponds to experiment ID.
  • omlHost: this is the host name of the OML Server.
  • omlPort: this is the TCP port on which OML Server is listening.
  • omlCollect: this is the complete URL of the OML Server.
  • omlMeasurementTitle: this is title of the table in the DB that will be created by OML to store data.
  • omlMeasurementSchema: this is the schema of the created table.

Note that omlHost, omlPort and omlCollect must not be changed, otherwise it will not be possible to collect measurements.

These configuration parameters can be taken into account as a guide to properly configure other OML clients. Each OML client provides its own APIs to define the schema of measurements to store the collected data.

Fifth section of the script contains the logic to collect data from resources and to store measurements in OML server.

// ### HERE YOU CAN GET SENSOR DATA ###

var cronJob = cron.job(cronTime, function(){
var myResults = getResourceData();
var hyogoTemp = -1;
var kyotoTemp = -1
var hyogoHumidity = -1
var kyotoHumidity = -1
var hyogoTempData = getResource(myResults, "HyogoTemp");
var kyotoTempData = getResource(myResults, "KyotoTemp");
var hyogoHumidityData = getResource(myResults, "HyogoHumidity");
var kyotoHumidityData = getResource(myResults, "KyotoHumidity");
hyogoTemp = hyogoTempData.data.dataValue;
kyotoTemp = kyotoTempData.data.dataValue;
hyogoHumidity = hyogoHumidityData.data.dataValue;
kyotoHumidity = kyotoHumidityData.data.dataValue;

// Write data on OML

oml.init({appName: omlAppName,domain: omlDomain , host: omlHost, port: omlPort, collect: omlCollect});
var mp = oml.mp(omlMeasurementTitle, omlMeasurementSchema);
mp(hyogoTemp, kyotoTemp, hyogoHumidity, kyotoHumidity);

});

the function getResourceData calls internally the methods:

  • getSecurityToken: it retrieves the token by using userName and userPass;
  • getData(token): it performs requests to EaaS APIs to retrieve all currentData defined in resources array.

Collected data area saved in temporary variables calling getResource function for each specific measurement (HyogoTemp, KyotoTemp, etc.)

Finally, Oml.init(…) initializes connection with OML server and Oml.mp (…) creates the measure point and stores data in OML server.

Measurements collected in OML Server will be available in the dashboard of the experiment provided by Experimentation Portal.

Visualize experiment results

Data collected during the execution of the experiment can be displayed on the Experimentation Portal; in the experiment details page, the experimenter can click on Dashboard button and access to the section MyExperiment Dashboard.


This section shows experimenters all the measurements defined through OML client used in one of the VMs associated to their experiments and collected by OML server.
For each measurement, the experimenter can visualize:

  • the schema associated to measurement indicating the typology of data collected;
  • the collected data by a chart line that showing the evolution of the measurement and a table showing collected values.

To visualize data of a measurement, firstly, the experimenter must define the time interval of interest and then he/she has to click on the Get Data button;


At the right of Measurements panel, the panel Schema displays the name of data collected and their type. Below the chart, a table provides experimenters with a complete visualization of collected data. The experimenter can interact with chart, highlighting a set of data or showing a specific value.
Experiment can modify the visualization of data by selecting the values to show. The experimenter selects two of the three data sets and after clicking on Redraw button, he/she can visualize the selected data sets.

Terminate experiment


Execution of an experiment terminates when one of the following two conditions is verified:

  • When an experimenter decides to stop it, by clicking on the Stop button of Experiment page.
  • When the experiment expires because it reached the end date set during its creation phase.

In both cases, when a stop command is performed, the platform proceeds with the release of resources: the process to free the resources used in the experiment. At the end of this process, the experiment is stopped and a green alert box informs the experimenter that the operation is successfully performed:
Similarly to the reservation process, even in this case, additional comments are necessary about Living Lab resources, because their release is not automated as the other resources types, but depends on the agreement between the experimenter and the Living Lab administrator.
For IT resources, the release process causes the deletion of the VM, so the instance of VM is removed from the list of the resources associated to the experiment and the template of the VM is associated again to the experiment.

API usage and Token retrieval

As described below, the FESTIVAL Experimentation Portal make use of a REST-ful API that is also available for experimentation. Therefore, all the functionalities provided by the portal are also available through the EaaS API.

The EaaS API is available in the following address, where all the methods are presented using swagger:

http://tools.festival-project.eu/swagger-ui/#/default

It is important to mention that all the methods used for the API will require the inclusion of a header with a valid token. This token can be obtained following two alternatives:

  • Login into the portal and clicking in the token button on the top (as shown in the next figure)

  • Using the Security Rest API to get the token.

This method is the most recommendable in case of long-term experiments. The method is described in the link above and using the following CURL command:

curl -X POST https://api.festival-project.eu/festival/eaas/security/token -H 'content-type: application/json' -d '{"email":"test@test.com","password": "************"}'

For more information, please consult FESTIVAL deliverable D2.5 Integrated Reusable components for EaaS.