Intelligent Sensing and Perception

Research goals


Intelligent Sensing and Perception

The ISP group develops methodologies and technologies for designing Intelligent Sensing and Perception systems endowed with Intelligent abilities with the aim of improving the quality of life in any physical environment: industries, competitive farms, modern societies, smart infrastructures, etc.

Within a physical world in a continuous evolution, innovation is imperative. Intelligent Sensing and Perception Systems represent key enabling technologies for shaping this innovation in any physical environment with the aim of improving the quality of life or, in other words, of creating a smart future. The objective of the ISP research group is to develop methodologies and technologies for designing advanced perception systems endowed with intelligent abilities. Based on multisensor systems, such as sensor networks and/or visual systems, together with computational intelligence, ISP systems support people in making decisions in different application fields, or in planning and controlling automatic devices. The main issues are related to the acquisition, processing, analysis, fusion and interpretation of multisensor data. Relying on machine learning algorithms, ISP systems are able to learn and accumulate knowledge from historical data in order to support intelligent decision making in complex and highly uncertain physical environments.

Research activities

Research activities address the following issues:

Data Acquisition: Perception of the real world involves several factors depending on the nature of the observed phenomena. Static or dynamic events require different acquisition modalities. Therefore, different aspects must be considered and managed: scale, resolution, acquisition speed, field of view, performance needs, accuracy, etc. Furthermore, multisensor systems are needed when different points of view are required such as in case of complex, large and cluttered environments. As a consequence, a variety of data are obtained such as: mono-dimensional signals, images, depth values, point clouds, thermographic signals, video sequences, and so on.

Data Processing: After acquisition, data must be processed in order to extract useful information. Raw data could drive inaccurate decisions as consistency, uniqueness, reliability, relevance, completeness are not guaranteed. Data processing or data manipulation is, therefore, a fundamental step that involves different aspects depending on the type of data, their dimension and their quality. The aim is to design and develop methodologies for registration, normalization, analysis and fusion of multisource and multimodal data. One important process in this context is the extraction of relevant features for use in model construction and in the subsequent decision making phase. Feature selection methodologies are developed in order to extract relevant information, to reduce dimensionality, to simplify models, to enhance generalization. Data mining problems are also considered to manage large amount of data and discover interesting patterns that can be used for further analysis and exploration.

Decision Making: The final stage includes the design and development of methodologies using Machine Learning and Artificial Intelligence algorithms, to generate data interpretation or to support decisions. ISP systems can operate autonomously or in cooperation with operators to drive better decisions and to control problems.

The application fields are numerous and disparate and include, but are not limited to:

High resolution systems are used to inspect materials for quality control in order to check the production process and prevent damages in the final products.

Diagnostics: :

Transport: development and real- time analysis of sensory data (images, 3D point cloud) acquired by mobile platforms.

Industrial: development of systems for the acquisition and real-time interpretation of high-precision 3D point cloud, for quality control in industrial production lines (e.g. the tire industry, tools construction industry, automotive sector).

Aerospace: development of systems for the acquisition and interpretation of sensory data for non-destructive testing of composite materials used in the aerospace industry. In particular, techniques for pre-processing and processing of thermal images and ultrasonic signals have been developed for the realization of automatic systems for defect detection.

Autonomous Navigation of Mobile Robotic Platforms:

Industrial Mobile Vehicles: design and development of navigation systems for autonomous vehicles operating in both warehouses and not accessible areas for different purposes such as inspection and surveillance.

Service and Field Robotics: multi-sensor processing algorithms, also using alternative sensing techniques, for ambient awareness of robotic vehicles working in dynamic semi-structured and unstructured environments.

Robotic networks: design and development of novel estimation and cooperative perception strategies for robotic networks, also integrated with IoT devices, to perform tasks, such as cooperative mapping, target tracking, and environmental monitoring.

Drones:: design and development of external localization systems to support drones in indoor environments. These systems are based on the analysis of images, 3D data, sound data, lasers etc.

Surveillance: Development of vision systems for solving problems related to the surveillance of environments. Different issues are involved: Activity/Gesture recognition; People Detection and Trajectory Analysis; People Re-identification; Active Assisted Living; Monitoring of dynamic events; etc.

Smart cameras are used for surveillance and security issues such as: observing people behaviors to improve the quality of life or to detect anomalous actions; analyzing the interactions between people and machines in all the contexts where technologies can support the human operators.

Sensor and robot networks are used to develop cooperative perception strategies.

Advanced multi-sensor perception systems are developed for ambient awareness of robotic vehicles.

Main research projects

Agricultural Interoperability and Analysis System (ATLAS), H2020 (Grant No.857125)
BEhavioral integrated System for diagnosis, support and monItoring of neuro-Degenerative diseasEs (POR Puglia FESR-FSE 2014-2020 (Id. YJGRA7)
Electronic Shopping & Home delivery of Edible goods with Low environmental Footprint, POR Puglia FESR-FSE 2014-2020 (Id. OSW3NO1)
Simultaneous Safety and Surveying for Collaborative Agricultural Vehicles, FP7 ERA-NET ICT-AGRI 2 (2016-2018)
Defects, Damages and Repairing Techniques in Productive Processes of Large Structures of Composite Materials, National Funding (2014-2016)

key words

Advanced Perception, Vision Systems, Real-Time Signal Processing, Multisensor Data Fusion, Mobile Robotics, Sensor and Robot Networks, Decision Making, Design and Control of Intelligent Devices.


Dr. Tiziana D'Orazio - Bari
ISP Group responsible
tiziana.dorazio [at]