Skip to content

system_description.md

Latest commit

 

History

History
265 lines (201 loc) · 12.7 KB

system_description.md

File metadata and controls

265 lines (201 loc) · 12.7 KB
Raw

System description

This exemplar builds a sensor fusion based perception pipeline. The components of the system are shown below and further described in the following.

For each node in the system, we can inject faults. This can be done by setting a parameter of the respective node to a value that leads to a symptom that can be detected.

system_overview

Note: In the following restarting refers to a lifecycle deactivate and activate while a redeploy refers to a lifecycle finalize followed by a ros2 launch of the respective node.

Camera node

The camera node is responsible for providing RGB images and is able to artificially degrade the incoming image from the rosbag by shifting it towards a darker spectrum.

Use the parameter

  • image_degradation to set the degradiation from 0.0=no degradation to 1.0=maximum degradation. Can be resolved by using image enhancement. An internal timer will reset the image degradation to 0.
  • camera_driver_issue to drop images, can be resolved by restarting the node
  • hardware_disconnect to drop images, can only be resolved by redeploying the node

Depth node

The depth node is responsible to publish the depth image for the current RGB image and can artificially degrades the incoming depth by adding noise to simulate unreasonable data at higher altitudes.

Use the parameter depth_degradation to set the degradiation from 0.0=no degradation to 1.0=maximum degradation.

Image enhancement node

The image enhancement node reverses the image degradation introduced by the camera. It doesn't check if the image was originally good, therefore as soon as the RGB images from the camera are not degraded anymore, the image enhancement will decrease the quality of the image again.

Sensor fusion node

The sensor fusion node synchronizes the data streams from the camera, depth and enhancement.

Use the parameter

  • topic_camera_input to set the rgb input to either /rgb_camera or /rgb_enhanced
  • modality to set the fusion to 0=fusion, 1=rgb or 2=depth.
  • sync_issue to drop images, can be resolved by restarting the node
  • memory_leakage to drop images, can only be resolved by redeploying the node

Segmentation node

The Segmentation node takes the fused data as input and outputs a predicted segmentation. Use the parameter

  • cuda_out_of_memory to drop images, can be resolved by restarting the node
  • gpu_failure to drop images, can only be resolved by redeploying the node

Supplementary information experiments

Evaluator

This node will compare the ground truth segmentation with the segmentation generated by the system. The results are directly logged via a logger that writes the results into a csv.

Scenario executor

This node will introduce faults to the system. The faults introduced are described in scenario files. The scenario executor will choose the next scenario that has not been executed yet, i.e., which can not be found in the log_dump folder.

We use a set of 50 scenario files (/ros_ws/src/experiment_setup/resources/scenarios) each introducing three faults at the same time. Therefore, every managing system is evaluated on identical scenarios. This keeps the results comparable and repeatable.

Evaluation of log files

To calculate the results in our table, for each table there are scripts in the experiment_setup folder To retrieve our log files you can get them from the same server as above:

Extending the artifact

Different image sequences

SUNSET is based on the SynDrone dataset and uses a subset of images from Town 01. If you want to use different images or completely different datasets in your setup, you can create a new ROS bag for this with this script

In case you use a different dataset, you might also want to change the segmentation model since the models used in SUNSET are trained from scratch on the SynDrone dataset. Therefore adapt the implementation in this class or add your own implementation and just replace the model in the segmentation node

New ROS Nodes

Since the SynDrone dataset allows also to perform object detection and LiDAR segmentation, SUNSET should be easily expandable with these nodes. Every node in our artifact implements the interfaces so that a managing system can adapt the respective node. These nodes inherit from a common base class which itself inherits from ROS2 lifecycle nodes and extends the functionality so that every node provides interfaces for:

  • Reparametrization (based on ROS2 parameters)
  • Change of communication (based on ROS2 parameters)
  • Activating / Deactivating (based on ROS2 lifecycle nodes)
  • Redeploy (based on ROS2 lifecycle nodes)

Reparametrization: Changing the ROS parameter of a node. This will change also the value of a node's class member. Therefore, you do not have to check the value of a ROS parameter yourself when using it inside of your algorithm

Change of Communication: Changing the ROS parameter describing which topic a certain subscription should listen to. Changing the value of this parameter will destroy an existing subscription and create a new one. For an example refer to parameter configuration.

Activating / Deactivating: While activating, the base class creates all communication channels described in the respective config. While deactivating, the base class destroys all communication channels.

Redeploy: To redeploy, the node should receive a lifecycle transition SHUTDOWN. The process will be exited cleanly and the managing system is responsible to relaunch the node with the respective launch script that is needed for every ROS node present in the system.

Based on this base class, new nodes for object detection and LiDAR segmentation can be added to SUNSET. For examples how to use the base class to extend our artifact, follow the instructions in the next section.

Communication configuration

An example on how to configure the communication for your node is shown in here

The following is a more extensive example showcasing a configuration for each communication possibility supported by the base class.

from python_base_class.node_config import CommunicationTypes
from std_msgs.msg import String
from std_srvs.srv import SetBool

comm_types = [
    {
        "comm_type": CommunicationTypes.PUBLISHER,
        "name": "/example_ns2/example_name",
        "msg_type": String,
        "time_period": 0.5,
        "callback": "publisher_callback",
        "always_on": False
    },
    {
        "comm_type": CommunicationTypes.PUBLISHER,
        "name": "/example_ns2/publisher_wo_callback",
        "msg_type": String,
        "always_on": False
    },
    {
        "comm_type": CommunicationTypes.SUBSCRIPTION,
        "name": "/example_ns/example_name",
        "msg_type": String,
        "callback": "subscription_callback",
        "always_on": False,
        "param": "topic_camera_input",
    },
    {
        "comm_type": CommunicationTypes.SERVICE,
        "name": "/example_ns/service",
        "srv_type": SetBool,
        "callback": "service_callback",
        "always_on": False
    },
    {
        "comm_type": CommunicationTypes.SERVICE_CLIENT,
        "name": "/example_ns/service_2",
        "srv_type": SetBool,
        "always_on": False
    },
]

This example dictionary also specifies exactly how a config has to look like. Each key is necessary and will also be validated by the base class. If there is a typo or a key missing, the base class will throw an error. If configuring a publisher, you can choose between using a timer or publishing the data in whatever way you prefer as seen in the example above. In case you need a callback for the communication way, the name of the callback in the configuration has to match exactly the name of the function that you define in your node. If you want that your communication is always available (ONLY IN RARE CASES!), you can set the always_on parameter to True.

Parameter configuration

For parameters there is a central config file that is built like parameter files are usually used in ROS2 (see ROS2 documentation)

Imagine the params.yaml file looks like this:

camera_node:
  ros__parameters:
    param1: 2.3

depth_node:
  ros__parameters:
    param2: 1

sensor_fusion:
  ros__parameters:
    topic_camera_input: "rgb_camera"

There are two types of parameters:

  • describing a value that you can use in your algorithms that should be adaptable, e.g. a confidence threshold in your object detection model. If you adapt a parameter here, the respective class member variable will be changed to the new value, i.e. the name of the parameter has to match the name of the member variable
  • describing a communication interface, e.g. which topic to listen to if you want to have camera images. If you adapt this kind of parameter, the respective communication interface will be changed. To which communication interface a parameter refers, is defined in the communication configuration with the key "param".

Currently the difference between these two parameters is detected by the name, i.e. if the substring "topic" is in the parameter name, it will be handled as communication change, otherwise only the class member variable will be changed.

You are implementing the camera_node e.g. like this:

import rclpy
from typing import List
import rclpy.parameter
from sensor_msgs.msg import Image

from python_base_class.engel_base_class import ENGELBaseClass
from managed_subsystem.config.camera_config import comm_types

from ament_index_python.packages import get_package_share_directory
import os


class CameraNode(ENGELBaseClass):
    def __init__(
        self,
        comm_types: List,
        node_name: str = "camera_node",
        param_file: str = "params.yaml",
    ):
        config_file = os.path.join(
            get_package_share_directory("managed_subsystem"), "config", param_file
        )
        super().__init__(node_name, comm_types, config_file)

        
        self.param1 = None # instantiating a class member variable that will be filled with value by the parameters
        self.topic_test_input = None

        self.trigger_configure()
        self.trigger_activate()
        if self.validate_parameters() is False:
            self.logger.warn(
                f"Not all parameters are initialized correctly. Every parameter in \
                    the params.yaml file has to be a class member of {self.get_name()}"
            )
        
    def publisher_callback(self):
        msg = Image()
        publisher = self.get_comm_object("/camera_raw")
        msg.header.frame_id = "/camera_raw"
        msg.header.stamp = self.get_clock().now().to_msg()
        publisher.publish(msg)

    def subscription_callback(self, msg: Image):
        pass

with your communication configuration looking like this:

from python_base_class.node_config import CommunicationTypes
from sensor_msgs.msg import Image

comm_types = [
    {
        "comm_type": CommunicationTypes.PUBLISHER,
        "name": "/camera_raw",
        "msg_type": Image,
        "time_period": 0.5,
        "callback": "publisher_callback",
        "always_on": False,
    },
    {
        "comm_type": CommunicationTypes.SUBSCRIPTION,
        "name": "/test_img",
        "msg_type": Image,
        "callback": "subscription_callback",
        "always_on": False,
        'param': 'topic_test_input',
    }
]

The following things have to be considered now:

  • the name of the node (specified in your CameraNode class constructor) has to match one entry in the params.yaml file that describes the parameters used for this node.
  • there has to be a class member variable with the same name as the parameter
  • The handling of the parameters happens in the base class
  • At the end of the constructor you can validate the parameters with the self.validate_parameters function. This will log a warning if there is something wrong.