Think about the vision of autonomous cars on the roads. It will be a reality in twenty years or so.
Sensor fusion means combining data from disparate sources to give more accurate information about the environment than if the sources are used individually. In a car, a camera, radar and other devices collect information and together assist the car to move.
Sensor fusion is on our and our clients' agenda, too. In one of the experiments, we cooperated with the Finnish Transport and Communications Agency (Traficom), who plans and maintains traffic flows and ensures traffic safety and smooth functioning of the transport system.
The key question was to identify when sensor fusion is superior compared to a single source of data in transport surveillance. In this case, single source of data refers to video data generated by traditional camera surveillance the agency uses, and sensor fusion means the combination of video data and point cloud data from liDAR sensors. To get an answer, we needed another goal; to evaluate how modern AI solutions can recognize anomalies in traffic flow.
One might wonder what liDAR measurement technology is. It is based on lighting a target with laser light impulses at a rapid speed. When the light hits the target, it bounces back to the measurement device. The measurement device creates a 3D presentation, namely a point cloud, of the target based on the time of the light to bounce back and the wavelength of the bounced light.
In our experiment, liDAR was seen as an interesting technology due to two things. It is cost-effective as the prices of LiDAR measurement devices are going down. To add up, the information liDAR technology produces is easily interpreted compared to video data. LiDAR measurement is also very reliable during the night in contrast to video technology.
To find answers to the Traficom experiment questions, we needed two things; a both video and liDAR data as well as a machine learning solution to interpret the data.
In addition to camera surveillance, we installed liDAR sensors in a tunnel and started to receive traffic flow data. The idea was to catch disturbances such as stalled cars, people and animals on the road, and feed this data in a machine learning based solution. We had collected data from the normal traffic flow and taught the system to recognize traffic anomalies.
Even with a short piloting time, we were able to create an accurate solution that can detect subtle traffic flow anomalies whose root cause such as stopped vehicle is not visible in the video or liDar.
When analysing the pilot experiment more deeply, ideas for further study arise. In our pilot the standardized lighting and weather conditions in tunnels were good enough to produce video data. In varying lighting and weather conditions liDAR is presumably more beneficial compared to video.
The amount of anomalies in traffic flow was small in this pilot, but in a larger data set it could reveal cases in which it would be easier to observe anomalies with liDAR data only. The parameters of the sensor fusion model could then be optimized.
Due to the limited resolution, it is critical how the liDAR sensors are placed. By varying the installation, it is possible to improve the information content of data.
Traficom's ability to use sensor fusion with the quick development of machine learning technology and the descending costs of cloud computing techniques opens up new opportunities. When data is gathered to a common cloud, solutions can be developed incrementally, and various data sets can be combined with sensor fusion and other techniques as well. For example, combining video and liDAR, weather, accident and traffic jam and dynamic road condition data is the key to get an understanding of the traffic situation real-time, do predictive and automated surveillance and various business analyses.
Same technologies can also be applied in other businesses such as police, border control and security services and autonomous vehicle control. In the end, sensor fusion is all about cost-efficient and agile solutions and a safer world – together with you.
Ari’s passion lies at the cross section of artificial intelligence and data science, data driven process development and deployment as well as business intelligence and leadership. Based on his experience, a successful data driven business transformation requires that all these domains are developed in sync.