Live Data Collection

a7

Input Live Data

This step allows for the integration of data from live online sources by means of the kSpheres platform in the context of populating newly created variables. For instance, in this particular example, such data could include: drivers’ living conditions (weather information, altitude, population density), number of traffic incidents for any given date, traffic density, car characteristics, etc. Open data sources from public available sites and police records can be used for extracting this valuable information. At this stage, appropriate text mining techniques may be implemented in order to create a suitable dataset (or extend an existing dataset). Users can control a number of parameters related to the text mining methods employed.

The user specifies a number of variables present in the dataset to be used as search terms for the detection of relevant data from online and offline sources.

a8

Dataset Examples

1. The dataset contains information about the drivers’ names. The user wishes to obtaininformation about the number of traffic accidents the drivers have been involved in. The drivers’ name and ID number variables are specified as the search term, along with the term “number of accidents”. PredMine searches in the appropriate data repositories and retrieves a number that represents the number of times each driver is mentioned in traffic accident reports.

2. The dataset contains information about the drivers’ names and place of residence. The user wishes to obtain information about the number of traffic accidents the drivers have been involved in, but only near their place of residence. In this case, in addition to the drivers’ names and ID numbers, their place of residence is entered as a search term. PredMine returns the number of times each driver is mentioned in traffic accident reports, provided that the accidents occurred in the municipality the driver resides in (or, alternatively, within a user-specified distance from the driver’s place of residence).

3. The user wishes to discover how accident-prone each area a driver resides in is. The user enters “area of residence” and the term “number of accidents” as the search terms. PredMine retrieves the number of times a traffic accident has occurred in each of the areas drivers reside in. Subsequently, the user can refine the values by imposing a recency criterion (e.g. by specifying thathe is interested only in traffic accidents that occurred during the last two months).

4. The dataset contains information about the drivers’ names and place of residence. The user wishes to obtain information about the population density and the altitude of the place that the driver lives since the actuarial department of the insurance company believes that a strong correlation exists between these place characteristics and the accidents. The user enters “area of residence” and the term “population density” and “altitude” as the search terms. PredMine returns the population density and the altitude of the drivers’ place of residence accessing open and live data from the internet.

Top