Teaching robots to follow a moving object is more difficult than you think, requiring complex algorithms and a different way of thinking.

Take a look around. What do you see? Most of us have two eyes and we use those eyes to collect light that reflects off the objects around us. The eyes convert that light into electrical signals that are processed by our brain. This builds a representation of the world and we use that to navigate during our everyday lives. Even robots that are the most like humans in appearance, however, don’t see the world the way we do.

Instead, algorithms recognize features in images collected by a robot’s sensors and cameras. The software may create a very basic map of the environment and learn to recognize patterns to help the robot understand its surroundings. This means that robots are being programmed by humans to see things the human thinks the robot will need to see. While this has many very successful examples, no robot is capable of navigating the world using just vision for static recognition.

If you spot a bird outside, you can watch that bird fly through the sky until it lands or disappears from view. This is visual object tracking, and it’s a simple task for humans: spot the object and follow it. For robots, it’s much more difficult.

To improve visual object tracking in robotic applications, Dr. Sajid Javed, Assistant Professor, Dr. Jorge Dias, Professor, Dr. Lakmal Seneviratne, Professor, and Dr. Naoufel Werghi, Professor, all from the Department of Electrical Engineering and Computer Science at Khalifa University, collaborated with Dr. Arif Mahmood from Information Technology University, Pakistan, to develop an AI algorithm that is both highly accurate and quick when detecting and tracking a generic object. Their proposed solution was published in.

“Visual object tracking is a fundamental and challenging task in many high-level vision and robotics applications,” Dr. Javed explains. “Typically, the difficulties lie in developing detection algorithms that can handle blurred images from fast motion, ignore background clutter and deal with significant scale and light variations.”

Object tracking is an application of deep learning where a program takes an initial set of object detections and follows them as they move around frames in a video. The algorithms allow the robot to automatically identify an object in a video and interpret it as a set of trajectories to predict where it will end up.

The first step in tracking an object is to detect it. The research team’s solution narrows a search area down and instructs the robot to find all object instances of one or more pre-determined object classes. The algorithm is trained on a series of examples of these object classes to learn what it is looking for, regardless of the object’s scale, location or pose and despite any partial occlusions or poor lighting conditions.

Once the object has been identified, it needs to be followed. Robots can do this by continuously re-identifying the object in subsequent images, but for visual object tracking to be useful in robotic applications, interpreting the object as a set of trajectories with high accuracy is required.

Algorithms for tracking objects need to accurately perform detections and localize objects of interest in the least amount of time possible. This is especially imperative for real-time object tracking models.

“Discriminative correlation filters (DCF) are well suited to object tracking because of their impressive performance in terms of speed and accuracy,” Dr. Javed says. “In most DCF methods, an online correlation filter is trained from the region of interest in the current frame and then employed to track the target object in subsequent frames.”

High detection accuracy and fast processing speed are difficult to combine: More accurate tracking tasks often require longer processing times, while quicker responses are more prone to errors. In the research team’s solution, accuracy and speed are achieved by constructing a spatiotemporal graph that models and predicts where an object is likely to appear based on its previous identified location. Out of a series of possible trajectories, the most probable is selected by the DCF, which filters the background noise and any other distractions.

To evaluate their algorithm, the team tested it on six challenging benchmark datasets and compared it with 33 existing state-of-the-art trackers. Their results were excellent, achieving higher accuracy than existing trackers on many tests and ranking among the top three for the remaining tests.

As mobile robots and autonomous machines are increasingly deployed, object detection systems are becoming more important. Although great progress is being made, we are still far from achieving human-level performance, but solutions like this are a vital step towards that level of performance. 

Jade Sterling
Science Writer
28 November 2021

The post A New Solution for Visual Object Tracking in Robotics appeared first on Khalifa University.

The features of a cell nucleus can reveal much about the health of a cell, but finding the nucleus among the background noise of a tissue sample can be laborious and time-consuming when done manually. Khalifa University researchers are exploring how AI can be leveraged to speed up the process of detecting a cell’s nucleus.

Known as the cell’s ‘command center’, the nucleus is a large organelle that stores the cell’s DNA. The nucleus controls all of the cell’s activities, such as growth and metabolism, using the DNA’s genetic information.

Pathologists use features of the cell nucleus to distinguish benign from malignant cells. They examine tissue samples for cells with increased nuclear size and irregularities in the nuclear membrane or abnormal distribution of chromatin, a substance within the chromosomes in the nucleus.

Artificial intelligence techniques are being investigated to streamline the process of nuclei detection. However, these techniques struggle when the tissue samples are noisy or when nuclei appear crowded.

Dr. Naoufel Werghi, Associate Professor, has collaborated with Dr. Sajid Javed, Assistant Professor, and Dr. Jorge Dias, Professor, to develop a machine learning solution that can sift through the background noise and more easily identify individual nuclei. Their results were published in.

In an ideal world, a sample of tissue would contain only that tissue, but most samples contain a rich mix of several other types. Most normal epithelial cells have nuclei which are round to oval-shaped, most lymphoid cells have round nuclei, while stromal cells have ovoid to spindle-shaped nuclei. Any machine learning algorithm would need to be trained on the specific nucleus shape for the nucleus it is tasked to detect, adding a further layer of training required if the algorithm were to be used in other applications.

“Nucleus detection is a challenging task because of the nuclear clutter and diverse shapes and sizes,” said Dr. Werghi. “Additionally, computation challenges arise because the images analyzed are multi-gigapixel images, and could contain billions of pixels and tens of thousands of cell nuclei.”

A number of potential methods have been reported for automatic detection of cell nuclei, including deep learning methods to train a convolutional neural network (CNN) to generate probability maps of where the cell nuclei are present.

Existing approaches are promising, but they require a significant amount of training data and expensive platforms to cater to the high computational requirements. The model proposed by the research team can be trained using much smaller training datasets that can be executed on a typical desktop computer.

“Our solution uses correlation filters to sort through the data and identify nuclei,” explained Dr. Werghi. “Compared to end-to-end deep learning in previous methods, correlation filters are computationally effective and require significantly less training data. The correlation filters are also flexible and can detect complex and irregular-shaped nuclei without requiring handcrafted features.”

These correlation filters help the algorithm to better discriminate different nuclear components from the non-nuclear regions and also to discern each nucleus from the remaining nuclei where they are clustered.

Constraints are placed on the spatial structure of the nucleus and its local contextual information in the correlation filter framework to handle varying nuclei shapes, texture, and clutter. The first considers the spatial structure of the nucleus, while the second discriminates between the nucleus and the non-nucleus region. Both of these help to reduce the noise in any given sample.

The team plans to explore the strength of correlation filters in analysis classifying cell nuclei and tissue phenotyping problems, such as cancer detection.

This research was funded by Khalifa University Center for Autonomous Robotic Systems (KUCARS).

Jade Sterling
Science Writer
5 July 2021

The post Sifting Through the Noise to Find the Nucleus with Artificial Intelligence appeared first on Khalifa University.

Khalifa University PhD student Yusra Alkendi shared her research in robotics and autonomous systems during a technical session on Sunday, 4 April, at the Dubai Electricity & Water Authority (DEWA) R&D Center, the research arm of the Mohammed Bin Rashid Al Maktoum (MBR) Solar Park, the world’s largest single site solar park located in Dubai.

During her presentation to the DEWA R&D Center researchers, Yusra presented some of the major research projects being carried out at the KU Center of Robotics and Autonomous Systems (KUCARS). 

She presented the center’s research activities under the theme of “Robotic Systems for Critical Infrastructure Exploration, Inspection and Maintenance,” which includes projects to improve robotic system localization, mapping, and navigation; 3D reconstruction and coverage path planning; and systems identification and control.

She also shared the center’s activities under the theme “Automated Systems for Assembly, Manufacturing, and Transportation,” which include projects to enhance robotics grasping and manipulation and robotics drilling and assembly.

Yusra’s research thesis, titled “Towards More Robust Autonomous Perception and Navigation Systems in Extreme Environments” is of particular interest to the DEWA R&D Center. She is studying how to improve persistent surveillance of critical infrastructure using single to multiple unmanned robotic systems; how multi-unmanned aerial vehicle (UAV) systems can be leveraged for improved maintenance, surveillance, and security tasks in solar power plants; and how to optimize aerial-ground robotics system capable of operating autonomously to perform maintenance operations with minimal human intervention.

Yusra is supervised by Dr. Yahya Zweiri, Associate Professor of Aerospace Engineering, and Dr. Lakmal Seneviratne, Professor of Mechanical Engineering and Director of KUCARS.

About KUCARS, Dr. Seneviratne said: “Robotics is a powerful technology poised to have a disruptive societal impact. KUCARS brings together a critical mass of researchers, with state of the art labs, to address some of the cutting edge R&D challenges in robotics. KUCARS has a strong international reputation and has the vision to be amongst the top robotics labs in the world.” 

The DEWA R&D Center brings together 39 researchers and focuses on four research areas, including producing electricity from solar power and other clean energy technologies; the integration of smart grids; energy efficiency, and water. Since its establishment, the Center has hosted over 24 technical sessions in order to share knowledge and promote creativity and development.

Erica Solomon
Senior Publication Specialist
23 May 2021

The post PhD Student Shares Research at DEWA R&D Center appeared first on Khalifa University.